Lab 4: Combining Pictures

Introduction

The objectives of this lab are to become more familiar with techniques for blending images and swapping the background of an image.

Lab Entrance Assignment

1. Type in (or copy and paste) the swapBack and chromakey functions from the notes on Conditional Statements.
2. In order to use these functions, you will need to have some pictures to use. The chromakey function modifies a picture that has a subject in front of a green screen by replacing all of the green pixels with the color from the correpsonding pixels in another picture. This makes it appear that the subject is somewhere other than in front of a green screen. (Note for Spring 2020: We did not get to visit the TV studio on K's campus to take pictures in front of the green screen, so there are some pictures of a gorilla in front of the green screen that you may use. Details are below.)
   The swapBack function works by modifying a picture with a subject in front of some background to replace the background pixels with the color from corresponding pixels in another picture. It gives the effect of the subject being somewhere other than where the original picture was taken. (Note for Spring 2020: We did not get a chance to take pictures around campus to use for this exercise. There are some pictures you can use, or you may want to take your own. Details are below.)

Part 1: Swapping Backgrounds

Chromakey

1. Take a look at the chromakey function that you copied from the reading notes. Notice that the if statement uses a definition of "green" to be where the sum of the red values and blue values is less than the green value. Start with this definition of green, although you may end up adjusting it later.
2. If you have something that would work as a green background, feel free to take a photo of something in front of that green background and then take a photo of something else to use as a new background. If you use your own photos, you should resize them to something smaller than 700x700, and they need to be the same size before using them with these functions in JES. There are a few photos of a stuffed gorilla in front of the green screen in the TV studio at K that are avaiable on Kit for you to use. Under the "File" tab, you should find a folder titled "Gorilla on Green". You may save any of these files to work with. The "Extra Backgrounds" folder is just that, extra backgrounds that you can use. The images in both of these folders have been resized to be 600x400 pixels to make them easier to use in JES.
3. Once you have a picture of something in front of a green screen and another one with some other background (both the same size), run the chromakey function with these two pictures. What does the result look like? If it's not very good, try changing the condition in the if statement to better match the RGB values of the pixels that look green. One variation that has worked in the past was to pick a maximum threshold for red and a maximum threshold for blue, based on the distribution of colors in your picture, rather than comparing the sum of the red and blue to the amount of green. You may need to experiment a bit before you find values that work well for you. Keep editing this condition until you get something fairly satisfying, but keep in mind, we are not professionals, and the goal is not perfection; it is to understand what is happening.
4. Save a resulting picture to submit with this lab. (Use the Help menu to bring up an explanation of writePictureTo if you have forgotten how to use it.)

SwapBack

In this technique, a background in a picture is replaced with a different background.

1. Take a look at the swapBack function that you copied from the reading notes. First note that this function takes 3 parameters (3 images of the same size): the first is a subject in front of some background, the second is that same background without the subject, and the third is a new background.

2. Notice that, in order to determine whether a pixel is part of the subject or part of the background, the if statement computes the distance between the colors of two pixels, one from the new picture (a duplicate of the original), and the corresponding one from the background picture. If that distance is small, then it is assumed to be part of the background, so the color of the pixel in the new picture gets changed to be the color of the corresponding pixel of the new background.

3. Choose 3 pictures to pass into this function. There are a few pictures on Kit that you can use, or you can at least use these as a model to show you what to do. In Kit, under the "File" tab, you will see a "SwapBack Pictures" folder. There are 6 images in that folder; 4 of them are 400x600 pixels, the other 2 are 600x400 pixels. They are meant to be used in pairs as the first two parameters in the swapBack function. If you would like to create your own pictures, you should also do them in pairs. You will need to be careful to get the exact background without your subject - I would recommend using a tripod and lining up the crosshairs on your camera with something so that your camera stays still. Lighting can also be tricky (which might be evident in the pictures provided).

4. Run the swapBack function with the 3 pictures you've chosen. What happens? Probably not much. The value 15 in the inequality is actually very small. Your images probably have more variations due to lighting and shadows that is not easy to recognize. Try changing that 15 to something bigger, like 100, or 150, or even bigger. Experiment with that value until you find something somewhat satisfying and save your resulting image. Remember, we're not experts, so you're not expected to have perfect results. Sometimes we can get unexpected, kind of cool, results when our code doesn't do what we thought it would.

Part 2: Blending Images

1. The following function blends two images. It overlays one picture with another. The result is that you can see both images at once. If the images are different heights, the function determines the minimum height and only combines the pictures for that much height. The overlap amount specifies how many pixels wide the overlap area should be. If you want to completely overlay one picture with another, the overlap amount would be the entire width of one of the pictures.

   def blendPictures(pict1, pict2, overlapAmt):
     width1 = getWidth(pict1)
     height1 = getHeight(pict1)
     width2 = getWidth(pict2)
     height2 = getHeight(pict2)
   
     # Set up width and height for new canvas
     newWidth = width1 + width2 - overlapAmt
     newHeight = min(height1, height2)
   
     # Create the canvas to hold the blended pictures
     newCanvas = makeEmptyPicture(newWidth, newHeight)
   
     # Copy the first picture up to the overlap section
     for x in range(width1 - overlapAmt):
       for y in range(newHeight):
         color = getColor(getPixel(pict1, x, y))
         setColor(getPixel(newCanvas, x, y), color)
   
     # Copy the blended section
     # 50% pict1 and 50% pict2
     pict2_x = 0
     for pict1_x in range(width1 - overlapAmt, width1):
       for y in range(newHeight):
         pixel1 = getPixel(pict1, pict1_x, y)
         pixel2 = getPixel(pict2, pict2_x, y)
         newRed = 0.50 * getRed(pixel1) + 0.50 * getRed(pixel2)
         newGreen = 0.50 * getGreen(pixel1) + 0.50 * getGreen(pixel2)
         newBlue = 0.50 * getBlue(pixel1) + 0.50 * getBlue(pixel2)
         color = makeColor(newRed, newGreen, newBlue)
         setColor(getPixel(newCanvas, pict1_x, y), color)
       pict2_x = pict2_x + 1
   
     # Copy the remaining section of pict2
     targetX = width1
     for x in range(overlapAmt, width2):
       for y in range(newHeight):
         color = getColor(getPixel(pict2, x, y))
         setColor(getPixel(newCanvas, targetX, y), color)
       targetX = targetX + 1
   
     # Return the new canvas
     return newCanvas
   

   Run this function with several different images and overlap amounts. Are the results what you expected? Save one set of blended images to submit with this lab.

Submit your results

1. Submit the file containing your functions on Kit. Be sure you have included documentation at the top of the file with your name, date, and lab description.

2. Submit 3 newly created images on Kit - one that was obtained from chromakey, one that was obtained from swapBack, and one that was obtained from blendPictures.