Additions to the VRT collection

In my last post, I wrote about the first Visual Regression Test I made for the test site. I have now written a couple more, which had a few differences and additions that I think make more sense of the way Gemini tests can be written. Not much more, mind. I’ve also uploaded the project to a github repo so you can look at the files directly if you like

I’ve made two new tests, one for the nav bar and one for the Saitama cube. The first simply finds the navbar, hovers over it and then clicks on it, and takes screenshots of the header section before and after clicking. This validates that clicking on the navbar takes you to the correct page. The second test clicks on a radio and then waits the duration of the animation and takes a screenshot of the cube to check that it is showing the correct face.

This post is a simple descriptor of how I wrote those tests.

Navbar Test

Link to test file here.
The bulk of this test is the same as the first vrt I wrote. All I needed to do was change the name of the suite, set it to find the navbar instead of the header, and add a ‘clicked’ state that includes a wait time:

In the first line, the first parameter inside the gemini.suite function is ‘navbar’. When I first made this test, I forgot to change this from ‘header’ and so when it was run, it overwrote the screenshots I had for the original test. Therefore, I learned that this piece of text identifies what the test is and which set of screenshots belong to it!

The url and captured area both stayed the same, as the header is a good enough area to screenshot to check that the page has changed.

In order to perform actions on the navbar, I had to change the find line so that it pointed to the navigation bar rather than the header. If I just wanted to take screenshots of something without interacting with it at all, the test would not require the before(function… part, and would only need to have .capture(‘plain’).

On capturing the ‘hovered’ and ‘clicked’ states, it’s worth mentioning that I only pointed the test to a.nav-about, instead of to the individual link I wanted it to find. This is because when you use the actions.mouseMove() method, it puts the mouse over the exact center of whatever element it’s pointed to. That happens to put it over the link I want for this test. However, if I were to make a change to the navbar then this might change.

A new method introduced in this test is actions.wait():

The above piece of code runs a function that clicks on the navbar, then waits for 500 milliseconds (0.5 seconds) before taking the screenshot called ‘clicked’.

See below the two of the screenshots for this test. The top one is ‘clicked’ and the bottom one is ‘hovered’. You can see that the header is different in the top one, showing that the page has changed.

Saitama cube test

Link to test file here.
Again, I’ve changed as little as possible about the test, since I think it’s good to learn incrementally. It’s better to build something slowly and know what change broke it, than to start afresh and have no idea why it’s throwing errors. This test captures the container that holds the Saitama cube, then clicks on a radio to change the face, waits the duration of the animation and then captures a new screenshot.

I changed the url to ‘localhost:3000/about.html’, as I don’t want to navigate to this page inside the test. I could have added this onto the end of the nav one, but I have kept them separate for the purpose of demo’ing.

The test captures the ‘.cubeContainer’ div. which is the area that includes the cube and radios. I’ve also added a line that sets the tolerance to 3.5. It took a bit of digging to find out what this value actually means. By default it’s set to 2.3, and there’s no real reason to change this except to demonstrate that you can. The tolerance is how much difference between new and ‘golden’ images is acceptable. Sometimes you can have tests fail because everything has shifted minutely to the left, or a text box is 1px lower than it was the last time it was loaded. If you don’t want to see this kind of stuff, then you’d need to up the number if you dare. As for what value to use, it would be easier to experiment than to try and understand – the number comes from a colour difference equation called CIEDE2000, you can read the wiki article here if you’re interested. Most of the time, the default value is fine.

While the nav find method only looked for the navbar as a whole, I wanted this one to find a particular radio and select it. When writing the selector name, make sure you use a CSS Selector format, since Selenium hates us all so refuses other formats. The simpler, the better. You can find a list of how to select elements of different kinds at w3 here. In this example, I used ‘input.cube-right’. This means that the type of element it input. If I just left it at that, it would find the first element matching that description and select that. Adding ‘.cube-right’ to this says that it should look for an input element with the class name ‘cube-right’.

Above is a screenshot from the html file for the about page, showing that each radio (input) is inside a label (where the text next to the radio selector lives), and they are all together inside a paragraph named ‘radioGroup’. When I first made this, they didn’t have individual class names, so I added them in to make the test work. There were already a bunch of identifiers, none of which could be selected using Selenium.

This is the advantage of having the power to change both the product code and the test code. If this was a product in a company then that would unlikely be the case, which is why it’s considered best to write your tests alongside your product instead of afterwards. If a big change needs to happen to the product in order to make it testable, then it’s better to find this out early on rather than later when it’s finished and developers have moved on to other projects.

The last part of the test simply captures the screenshots as usual. When the radio is clicked, the cube rotates to show the correct face in an animation that takes exactly 1 second. Therefore, I instructed the test to wait 1005 milliseconds before capturing the second screenshot.

What next?

It’s pretty much Christmas now, so I won’t be doing much over that period. My next challenge will be in creating a VRT that works with iFrames. iFrames are basically webpages within webpages, allowing you to inject your html into a page without either document affecting the styling of the other. It’s commonly used in the advertising industry, but can be a big pain in the butt since many testing tools are simply unable to detect the elements inside the iFrame. This includes gemini and cypress, the two testing frameworks I’ve worked with so far.

In order to find out if this is doable, I plan on putting an iFrame on the third page of the site (Recipes), and from there I’ll either be able to write a gemini test that can select elements inside the iFrame, or I will need to build a test without gemini, probably using Puppeteer and maybe something like Differencify. This is what I’m looking to achieve going into next year.

In other news, I was offered a scholarship from the awesome Ministry of Testing this week, paying my ticket to TestBash Brighton in April! This is super exciting, probably the most exciting thing that has happened since I got my job as a QA, and I look forward to writing up about it as it happens. 😀

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.