testingweb-applicationsautomated-testsapplitools

Is testing web app with images comparision a good approach?

I was wondering if testing web application based on comparing images of a screenshot is used in the industry and if it's a good approach.

Scenario:
- model image is taken by hand
- test compares only selected parts of an image
- test is written in selenium for example
- test has always the same data to work on
- test is always running on the same screen (the same resolution)
- test will compare images after some steps (for example: if the user account page looks good after registration - we have always the same data for test)

Does this have some advantages? Is it a 'stable' approach to testing web apps? Could it be useful as the last step of (for example) selenium test to verify the results? What do you think about it?

I did a search for that topic but couldn't find any good resources.

Solution

  • There are solutions for this kind of testing, for example, there is Applitools.

    It is based on taking a baseline screenshot, and then the subsequent screenshots take the diff from the original image. There are 4 different comparison levels:

    1. Exact (MatchLevel.EXACT) - pixel-to-pixel comparison
    2. Strict (MatchLevel.STRICT) - Strict compares everything including content (text), fonts, layout, colors and position of each of the elements but knows to ignore rendering changes that are not visible to the human
    3. Content (MatchLevel.CONTENT) - Content works in a similar way to Strict except for the fact that it ignores colors
    4. Layout (MatchLevel.LAYOUT) - compares the layouts (i.e. structure) of the baseline and actual images. It ignores the content, color and other style changes between the pages.

    A big advantage, in my opinion, is that this kind of testing can catch unexpected bugs (visual or otherwise) and in one go (you don't need to write multiple assertions, you just compare screenshots). You can write scripts with less code, as well.

    Possible downsides are: you cannot handle dynamic content, sometimes it is impossible to take a screenshot and, since there are screenshots, test execution (working with img files) can be longer.

    NOTE: I'm not involved with Applitools, but they have a site with many tutorial courses.