Improve accuracy of scaling down image

Question

I'm using Python 2.7 and PIL (pillow)

I have a script that takes fairly rough images of a maze and makes a cleaner, smaller image. Sample input and output:

Which was generated from this image:

In that case, the script didn't work perfectly, but worked pretty well.

However, another image of the same maze produced this result:

That's less good.

I'm generating the images displayed side-by-side by looking at average values for each square on the 16x16 grid, then deciding if the square represents mostly black or mostly white pixels. However, since the perspective transformation isn't perfect, the squares aren't always lined up.

Are there any algorithms that would help with accuracy? Any way to look at squares of the grid that aren't perfectly square chunks?

A piece of my code:

#This image is already transformed and thresholded, like the first half of my side-by-side images
thresh = Image.open('Thresholded_Image.jpg')
pixsize = thresh.size[0]/16
segments = []
for i in range(16):
    for j in range(16):
        box = (j*pixsize,i*pixsize,(j+1)*pixsize,(i+1)*pixsize)
        segments.append(thresh.crop(box))
def blackWhite(image):
    '''Return `True` if the image is mostly white, else `False`'''
    l=image.convert('L').load()
    w,h=image.size
    lums=sum([[l[x,y] for x in range(w)] for y in range(h)],[])
    return sum(lums)/float(len(lums))>127
whites = []
for y in range(16):
    for x in range(16):
        seg = segments[16*y+x]
        if blackWhite(seg):
            whites.append((x,y))

maze = Image.new('L',(16,16))
l=maze.load()
for w in whites:
    x,y=w
    l[x,y] = 255

Answer 1

_{(As requested, reposting comment as answer.)}

Consider weighting pixels that are near the center of the square you're evaluating more heavily, and those towards the edges less - that will help combat small misalignment. You could also try to locate the corners and then adjust the image so that the corners form a perfect square to help combat skew.