Screenshot comparison

I think it would be great to have screenshots taken during testruns and to automatically compare them with another testrun? Ideally, this should be done by just showing the actually diffing parts of the image.

I've played with http://www.imagemagick.org/, a great open-source tool for image manipulation with excellent command-line support.

Assume we have two screenshots taken at different times. (I've used the captureScreenshot command and this script.)
Klick images to enlarge.

infoq First shot Second shot

theserverside First shot Second shot

wikipedia First shot Second shot

First, we need to adjust the sizes of the images:

convert -layers merge molyScreenshot1.png molyScreenshot2.png result1.png

Now we can compare the results

compare molyScreenshot1.png result1.png result2.png

Which gives us:

As one can see, the screenshots get compared on a pixel by pixel base. This causes problems already with dynamic content, but it becomes a mess when the layout changes slightly:

(Note that the difference is just the additional blue header at the head of the page)

First shot Second shot

but the comparison looks like this:

The developement of more advanced image comparison using Fast Fourier Transform in ImageMagick has not yet been finished, volunteers wanted: http://www.imagemagick.org/discourse-server/viewtopic.php?f=1&t=13329&start=0

This paper about localizing subimages seems also interesting, if you have the mathematical background (I do not): http://werner.yellowcouch.org/Papers/subimg/index.html#515

Do you know another free and simple cross-platform tool for doing this?

If yes, please drop me a line to or use comment section below.

Trying to locate this subimage inside the big one shown above using function IsImageSimilar from ImageMagick worked for me:

$ /usr/bin/time -v ./locate_subimage2 1_molyScreenshot-456.png subimage-1.png
717 460
Command being timed: "./locate_subimage2 1_molyScreenshot-456.png subimage-1.png"
User time (seconds): 20.84
System time (seconds): 0.05
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:20.91
Average shared text size (kbytes): 0
Average unshared data size (kbytes): 0
Average stack size (kbytes): 0
Average total size (kbytes): 0
Maximum resident set size (kbytes): 0
Average resident set size (kbytes): 0
Major (requiring I/O) page faults: 0
Minor (reclaiming a frame) page faults: 5016
Voluntary context switches: 2
Involuntary context switches: 44
Swaps: 0
File system inputs: 16
File system outputs: 0
Socket messages sent: 0
Socket messages received: 0!result-nytimes3.png|thumbnail!
Note that the location 717, 460 was found correctly! However, it's just one subimage and it took > 20 seconds to find its location.
Thus I'm still in need for a performant routine to compare two browser screenshots completely.

As a side note, comparing a screenshot taken in Internet Explorer 8 with a screenshot taken in Firefox 3.1 looks like this:

Fonts are rendered quite different and screencapture tool is unregistered