Day 96 - Facing Image Region Missing On Your Photo? Let's Check The Image On Missing Pixel Filler - IT Consultant - SAP, Artificial Intelligence and Machine Learning

We have old or new photos with missing image regions on the corner, and then we would go to the Photoshop tool to fix it.

It is a tedious job and costs you some dollars to fix the photos which you love.

Today, Artificial Intelligence can fix your image wherever image missing the region for you.

We will check Missing Pixel Filler how it works and fix missing image regions.

What is Missing Pixel Filler?

Missing Pixel Filler is a python package that, given images that may contain missing data regions (like satellite imagery with swath gaps), returns these images with the regions filled. These regions of missing data are filled using a dynamic method that incorporates randomly chosen, non-null neighboring pixels. The missing data filling algorithms can be changed according to user preferences.

Source: Missing Pixel Filler

What are functions available in Missing Pixel Filter?

Filling Method 1

fill_swath_with_random_rgb(img) – Selects pixel values from a random Gaussian distribution to fill missing pixel values.

Filling Method 2

fill_swath_with_random_pixel_from_image_new(img) – Fills swath gap with a random pixel from a non-missing portion of data.
get_random_pixel_from_image(x_arr, y_arr) – Selects other pixel values from non-missing portions of the image.

Filling Method 3

fill_swath_with_neighboring_pixel(img) – Fills “dynamic” system to fill swath, with nearest pixels having a higher probability of selection.
get_neighboring_pixel() – Selects pixel values from non-missing portions of the image to fill missing pixel values with probability inversely proportional to distance.

Let’s check the code in Google Colab and check the code here.

Cited:

@article{cao2020swathgaps,
title={Reducing Effects of Swath Gaps in Unsupervised Machine Learning},
author={Chen, Sarah, and Cao, Esther and Koul, Anirudh and Ganju, Siddha and Praveen, Satyarth and Kasam, Meher Anand},
journal={Committee on Space Research Machine Learning for Space Sciences Workshop},
year={2021}
}

Copy to Clipboard

# Updated:
# input: img (numpy array)
# output: image_with_swath (numpy array), swath parameters
img = []
x = len(img)
y = len(img)

def generate_swath_top_left(img):
  swath_img = img.copy()
  swath_img[10:100, 10:100] = 0
  return swath_img

def generate_swath_bottom_left(img):
  swath_img = img.copy()
  swath_img[130:220, 10:100] = 0
  return swath_img

def generate_swath_top_right(img):
  swath_img = img.copy()
  swath_img[10:100, 130:220] = 0
  return swath_img

def generate_swath_bottom_right(img):
  swath_img = img.copy()
  swath_img[130:220, 130:220] = 0
  return swath_img

def generate_random_swath(img):
  # resize image to 224 by 224
  image_with_swath = cv2.resize(img, dsize=(224, 224), interpolation=cv2.INTER_CUBIC)
  img_area = 224 * 224

rand = random.randint(1, 4)

print(rand)

img_with_swath = img
  if rand == 1:
    img_with_swath = generate_swath_top_left(img_with_swath)
  elif rand == 2:
    img_with_swath = generate_swath_top_right(img_with_swath)
  elif rand == 3:
    img_with_swath = generate_swath_bottom_left(img_with_swath)
  elif rand == 4:
    img_with_swath = generate_swath_bottom_right(img_with_swath)

return img_with_swath

Copy to Clipboard

# Dynamically tries finding non empty points in neighbourhood
# When it fails few times, it increases neighbourhood size automatically
def get_neighboring_pixel(img, x, y):
  x_rand, y_rand = 0,0

max_num_tries = 30
  max_tries_per_neighbourhood = 3
  neighbourhood_size_increment = 10
  current_window_size = 10
  total_tries = 0
  for _ in range(math.ceil(max_num_tries/max_tries_per_neighbourhood)):
    for _ in range(max_tries_per_neighbourhood):
      min_x = max(0, x-current_window_size)
      max_x = min(224, x+current_window_size)
      min_y = max(0, y-current_window_size)
      max_y = min(224, y+current_window_size)
      x_rand = random.randint(min_x, max_x-1)
      y_rand = random.randint(min_y, max_y-1)
      total_tries += 1
      if not(img[x_rand][y_rand][0]==0 and img[x_rand][y_rand][1]==0 and img[x_rand][y_rand][2]==0):
        return x_rand, y_rand
    current_window_size += neighbourhood_size_increment

return x_rand, y_rand
    
def fill_swath_with_neighboring_pixel(img):
  img_with_neighbor_filled = img.copy()
  (x_swath, y_swath, z_swath) = np.where(img == [0, 0, 0])

for i in range(len(x_swath)):
    x_rand, y_rand = get_neighboring_pixel(img, x_swath[i], y_swath[i])
    img_with_neighbor_filled[x_swath[i]][y_swath[i]] = img[x_rand][y_rand]
  return img_with_neighbor_filled

Copy to Clipboard

Copy to Clipboard

Few things that need to note down here:

Code is getting img variable missing, and I fixed the code.
The image requires the same dimension on all sides. If I am uploading a different size, then I am getting an error “value error: 3. empty range for randrange() (621,224, -397)… E-g 224 x 224 size would be better, and you need to add a code to resize it.
The code is considering the good region as a missing region on the image. We can see on the 1st image the issue.

I have raised issue no 2 to the developer to fix the code on Google Colab. I have added a code for the first and second; however, the third issue needs to check in a different image, or the developer needs to comment on it.