How I Made My Life Easier by Letting QA Write E2E Tests?

Lately, I have worked as a web developer with a company that deals with logistics. Their business is to supervise the transport of commodities from point A to point B. The application I worked on was intended for the suppliers of this company to track their merchandise. Each persona would connect to the application and perform tasks such as declaring the dispatch or the reception of the products.

The people who worked on the previous projects followed a V-model lifecycle. These projects were delivered with many bugs after a few years. Developers spent a couple more years trying to fix the application. Therefore, the quality of the application is one of their main focuses. We were using an agile methodology and after each sprint, all the application was tested to ensure that no bug or regression appeared. QA (Quality assurance) was responsible for the quality of the application.

Testing the application - how to do it effectively?

The QA was manually testing the application. Running the full set of tests would take half a day, so the tests were performed every other week. It was not frequent enough to catch all the bugs and meanwhile, QA could not focus on their other tasks. It was not satisfying enough for the client and the QA,that is why the developer team suggested implementing end-to-end (e2e) testing.

But this idea was not perfect: in typical e2e testing, developers are writing the e2e tests. While they are implementing those tests, they are not developing features, which is detrimental for the client. On top of that, the QA, which is responsible for the quality of the application, does not understand what the e2e tests are testing.

Our challenge was finding a solution where QA would not have to test the application manually but without making testing the developers’ responsibility.

What if QA could write tests?

If the QA can write the e2e tests, the tests and the quality of the application are directly managed by the QA. The QA ensures thus the high quality of the application. It is pretty fast for the QA, because they write the test once, and it is then run automatically. And no need to wait two weeks now to run the tests anymore, they can be run every day.

Which means developers can fix bugs faster, for they might have created the bug recently and the code is still in their mind.

This was nice and good, but we had one major issue, for our QA did not have a developer background, yet popular e2e test frameworks, such as Cypress, are asking for knowledge in Javascript.

Writing tests without coding (or very little)

A common test framework, called Cucumber, can be used without development skills and both developers and the QA had already used it. Through its language (Gherkin), it allows team members to write tests in a human-readable language.

Cucumber can be then connected to many common testing tools, and it can be connected to Cypress thanks to cypress-cucumber-preprocessor. To develop tests, all you need is a grammar of test sentences, for example, “When I log in”. You still have to code, but your sentences are reusable, so when the grammar is done right, developers are not needed for every test case.

Our process was as follows. First, the QA would decide which test cases were to be tested automatically (critical test cases that are very tedious to run manually). The developers would then translate from Gherkin to Cypress by creating the Gherkin grammar that the QA can use.

You’ll find an example of a feature written in Gherkin by the QA and the steps definitions in Cypress (as the developer will code them). You can retrieve this example here.

# Scenario in Cucumber/Gherkin
Feature: Cypress click
    Scenario: Click and navigate
        Given I'm on the "/" page
        When I click on "Querying"
        Then I should have "querying" in the URL

// Translation of the Gherkin steps to Cypres
import { Given, When, Then } from "cypress-cucumber-preprocessor/steps";

const baseUrl = "https://example.cypress.io";
Given("I'm on the {string} page", (url) => {
  cy.visit(baseUrl + url);
});

When("I click on {string}", (label) => {
  cy.get(".home-list").contains(label).click();
});

Then("I should have {string} in the URL", (url) => {
  cy.url().should("include", url);
});

Make our cucumber tests run

Our team was using Xray, a popular plugin that powers up Jira by providing tools to manage tests. As a result, the QA had to write all its test cases in XRay, which allows the QA to write manual tests or cucumber tests. In addition, Xray comes with an API that developers can use to get test cases or send tests results. Each morning, the tests were automatically pulled from XRay, run and then the results were sent to XRay.

Example of a test case in XRay

How was this solution for us?

With this solution, developers do as little code as possible. They only need to translate the Gherkin sentences into Cypress tests, but each sentence is reusable, so the QA can write new tests without the need of development.

The QA has a complete picture of the quality of the application. They decide which test cases should be executed each day and they know immediately if a test fails.

This process is not perfect and it still requires some maintenance from the developers, but no more than for regular automated tests. It even requires less time from developers as they are not responsible for keeping the test cases up-to-date.

And if you want to have your QA writing the e2e tests?

First, train your QA for using Cucumber. It is easy to understand, even with no development skills. Feel free to check the comprehensive Cucumber documentation.

Next, you need to decide where the QA should write their tests. Requiring QA to use git may be excessive. Xray is a good tool as it is designed to be used with Cucumber, but it can be any tool that your QA uses to write its test sets. Ideally, the tool should support Gherkin syntax and have an API to let developers run tests automatically.