AI Image UnCrop: Convert Portrait to Landscape Programmatically in one click
Tutorial using Stable Diffusion SDXL 1.0 API from Segmind.com
Introduction
In the world of digital images, a common challenge many content creators, graphic designers, and social media enthusiasts face is the limitation imposed by the orientation of their images.
This constraint can restrict the usability of high-quality visuals, especially in scenarios where the composition’s context is paramount, such as website headers, banners, panoramic views, digital marketing content, or thematic photography collections.
What if this barrier no longer existed, and with just a click, a portrait could seamlessly transform into an engaging landscape image?
In this blog post, we’ll see how to use AI to uncrop an image with just a click of a button. This allows us to expand random stock photos in portrait mode to wide landscape images or murals. We will use Segmind’s Stable Diffusion Outpainting API and combine it with OpenCV’s image stitching algorithm to achieve this. Let’s get started!
Step 1: The Theory
Let’s begin with an image in portrait form.
We fix our canvas for outpainting at 1024 X 1024 pixels. We take our image in portrait mode and then resize it such that the maximum dimension is 1024 pixels which is usually the height for vertical images.
We shall then do outpainting with X_offset as 0 and Y_offset as 0. This means on the canvas we place the image to the left and leave space on the right to do outpainting. Then we get the following outpainted image.
Now our goal is to extend the image to the left and right as well to make it wide and landscape.
In order to achieve this we are going to create two images by cropping the original image, one that captures the first 3/4th and another that captures the last 3/4th of the image.
Since our original image is 1024 by 1024, this will create two images of width 768 and height 1024. Now we place each of them on a new 1024 by 1024 canvas, but this time we extend the first cropped image on the left. So we give an X offset of 256 pixels and do outpainting to get a 1024x1024 image. The second cropped image is extended on the right so no X offset is necessary and with outpainting it will generate a 1024x1024 version with the image extended.
We finally have two images with a good overlap in the middle that can be combined to make a wide landscape image.
We use OpenCV Stitcher class and get a final image that is in landscape mode combining the two images. So we started with a portrait image and finally ended up with a wide landscape image all programmatically achieved using AI.
Step 2: Signup for services and get the API Keys
To get started, you’ll need to sign up for Segmind. We are going to use Segmind’s outpainting API to achieve this.
Get Segmind’s API Key
Go to Segmind.com and Log in/Sign up
Click on the top right to go to the console.
Once in the console, click on the “API keys” tab and “Create New API Key”.
You get a few free credits daily but for interrupted usage, you can go to “billing” and “add credits” by paying with your card.
If you want to know how much cost each API call incurs for Segmind, you can go to the corresponding model’s pricing tab and see it. An example of SD outpainting pricing is shown here.
Step 3: The Code
The Google Colab notebook containing the full code can be found here.
Install the necessary Python libraries and enter your API keys from the above step when prompted!
!pip install ipyplot
from getpass import getpass
segmindkey = getpass('Enter the segmind API key: ')Let’s define some useful functions.
So save_cropped_image takes in the parameter for crop which is crop_last_quarter or crop_first_quarter. So basically you are going to ake the image and then get 3/4th of the image starting from the first or 3/4th of the image leaving out the first quarter. So depending on what you pass, you will save one of those images.
The function resize_image is useful if your original image dimensions are different than 1024 by 1024, we will resize to adjust the maximum size to 1024 while maintaining the aspect ratio.
The function get_image is the main function that does outpainting. It takes all parameters like x_offset, y_offset and returns an outpainted image based on our prompt.
import requests
import base64
import random
import io
from PIL import Image
def save_cropped_image(image, filename, crop):
"""Crop and save the image based on the crop parameter."""
threequarter_width = int(image.width * 0.75)
onequarter_width = int(image.width * 0.25)
if crop == "crop_last_quarter":
cropped_img = image.crop((0, 0, threequarter_width, image.height))
elif crop == "crop_first_quarter":
cropped_img = image.crop((onequarter_width, 0, image.width, image.height))
else:
cropped_img = image
cropped_img.save(filename)
return cropped_img
def toB64(file_path, img_width, img_height):
img = Image.open(file_path)
img = resize_image(img, img_width, img_height)
buffered = io.BytesIO()
img.save(buffered, format="JPEG")
return base64.b64encode(buffered.getvalue()).decode('utf-8')
def resize_image(img, img_width, img_height):
"""Resize an image maintaining its aspect ratio."""
aspect = img.width / img.height
if aspect > 1:
# Width is greater than height
new_width = min(img.width, img_width)
new_height = int(new_width / aspect)
else:
# Height is greater than width or equal
new_height = min(img.height, img_height)
new_width = int(new_height * aspect)
return img.resize((new_width, new_height),Image.Resampling.LANCZOS)
def get_image(filename,
segmindkey,
prompt="",
negative_prompt="ugly, deformed, out of frame",
scheduler="DDIM",
num_inference_steps=30,
img_width=1024,
img_height=1024,
scale=1.0,
strength=1,
offset_x=0,
offset_y=0,
guidance_scale=7.5,
mask_expand=0,
seed=None):
if seed is None:
seed = random.randint(1,1000000)
print('seed ',seed)
headers = {
"x-api-key": segmindkey,
"Content-Type": "application/json"
}
data = {
"image": toB64(filename, img_width, img_height),
"prompt": prompt,
"negative_prompt": negative_prompt,
"scheduler": scheduler,
"num_inference_steps": num_inference_steps,
"img_width": img_width,
"img_height": img_height,
"scale": scale,
"strength": strength,
"offset_x": offset_x,
"offset_y": offset_y,
"guidance_scale": guidance_scale,
"mask_expand": mask_expand,
"seed": seed
}
response = requests.post(
"https://api.segmind.com/v1/sd1.5-outpaint",
headers=headers,
json=data
)
print (response)
image_data = response.content
image = Image.open(io.BytesIO(image_data))
return image
For this tutorial, you can experiment with vertical images available on the Stock Photo website, Pixabay!
A few examples of images used in this tutorial are these:
https://pixabay.com/illustrations/bear-child-bird-trees-woods-7858736/
https://pixabay.com/photos/spa-soaps-soap-cubes-lifestyle-8227623/
https://pixabay.com/photos/hot-air-balloons-bagan-sunset-1807521/
Download a sample image from the links above. I am using the image from the first link (bear-child..).
After your first can to get_image you get an initial outpainted image with output dimensions 1024 X 1024. Note that if the dimensions are square you just get the same image resized with no outpainting as it is already square.
import io
import ipyplot
# https://pixabay.com/illustrations/bear-child-bird-trees-woods-7858736/
filename = 'bear.jpg'
prompt = "tall trees with birds"
image = get_image(filename,segmindkey=segmindkey,prompt=prompt,scale=1.0,offset_x=0,offset_y=0)
ipyplot.plot_images([image],img_width = 500)Initial image:
Now as mentioned earlier we will generate two crops (first 3/4th and last 3/4th) of the image to expand further.
def generate_cropped_filename(filename, crop_type):
base_name, file_extension = filename.rsplit('.', 1) # Split the filename into base_name and extension
return "{}_{}.{}".format(base_name, crop_type, file_extension)
image_last_quarter_cropped = generate_cropped_filename(filename, "crop_last_quarter")
image_first_quarter_cropped = generate_cropped_filename(filename, "crop_first_quarter")
img_last_quarter_cropped = save_cropped_image(image, image_last_quarter_cropped, crop="crop_last_quarter")
img_first_quarter_cropped = save_cropped_image(image, image_first_quarter_cropped, crop="crop_first_quarter")
print(image_last_quarter_cropped)
print(image_first_quarter_cropped)
ipyplot.plot_images([img_last_quarter_cropped,img_first_quarter_cropped],img_width = 500)
Now we will use a prompt to expand the first image on the left and the second image on the right.
prompt = "tall trees"
image1 = get_image(image_last_quarter_cropped,segmindkey=segmindkey,prompt=prompt,scale=1.0,offset_x=256,offset_y=0)
image2 = get_image(image_first_quarter_cropped,segmindkey=segmindkey,prompt=prompt,scale=1.0,offset_x=0,offset_y=0)
ipyplot.plot_images([image1,image2],img_width = 500)
Now let’s merge these two images using OpenCV.
import cv2
import numpy as np
from google.colab.patches import cv2_imshow
def generate_stitched_filename(filename, suffix="_stitched"):
base_name, file_extension = filename.rsplit('.', 1) # Split the filename into base_name and extension
return "{}{}{}.{}".format(base_name, suffix, "", file_extension)
def stitch_images(images):
# Create a Stitcher instance
stitcher = cv2.Stitcher_create()
# Use the stitch method to stitch the images
status, stitched_img = stitcher.stitch(images)
# Check if the stitching was successful
if status == cv2.Stitcher_OK:
return stitched_img
else:
print("Error during stitching, error code: ", status)
return None
all_images=[image,image1,image2]
# Assuming all_images contains all your generated images, but you must convert them to OpenCV format first.
all_images_cv = [np.array(im)[:, :, ::-1] for im in all_images] # Convert PIL images to OpenCV format
stitched_img = stitch_images(all_images_cv)
output_path = generate_stitched_filename(filename) # Define your desired output path
cv2.imwrite(output_path, stitched_img)
# cv2_imshow(stitched_img)
stitched_img_rgb = cv2.cvtColor(stitched_img, cv2.COLOR_BGR2RGB)
ipyplot.plot_images([stitched_img_rgb],img_width = 800)We get the final wide landscape image -
In this tutorial, we explored an innovative use case of AI image outpainting in transforming portrait images into landscape format leveraging the stable diffusion outpainting API from Segmind.
The programmatic image uncropping technique holds immense potential across various sectors. For marketers and advertisers, this capability means adapting imagery to different digital ad formats without losing essential content or compromising on aesthetics.
For responsive web design, it offers a way to create images for various screen sizes and resolutions from a base image. Furthermore, social media influencers and content creators can effortlessly repurpose visuals for multiple platforms, ensuring their content remains consistent and engaging regardless of the medium.
Happy AI exploration and if you loved the content, feel free to follow me on Twitter for daily AI content!
