Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How much is too much assertions in automation testing?

I am given the task to build a test suit using testcafe and, as I write tests I stumble upon one particular question “how much assertions is too much?”. Basically, after the tests are done, a report is generated. Looking at the report it is not intuitive. For example, If an element is not found on the webpage, I’ll see something like:

>Selector('tads') does not exist in the DOM. 

This forces me to go through the test manually to verify what failed.

According to the testcafe documentation, you can add an optional message to the assertion. as seen here

As of right now, I have assertions with some messaging in a few places. Would it be wise to have an assertion(with a concise error message) after every click or every action? (i.e. click the login button, do an assertion to see if the login modal appears. Now login, assert that the login modal disappears)

The code would look something like this:

await t.click(this.loginButton);
await t.expect(this.loginButton.exists).ok("I don’t see the login button");

await signup.newUserSignUp();
await t.expect(this.loginButton.exists).notOk("The login modal didn’t disappear"); 

any feedback would be awesome.

like image 334
Roosevelt H Avatar asked Feb 16 '19 16:02

Roosevelt H


Video Answer


1 Answers

TestCafe assertions can be used either to assert what is expected in a test, but also can be used as a waiting mechanism to ensure an element is ready before acting on it.

This means you may end up with many assertions inside a single test.

Personally, I write each test in a BDD style like this:

fixture("Feature <feature-id>: description of the feature ");

test("Scenario <scenario-id>: description of the scenario", async (t) => {
  // Given 
  await t
   .<any action>
   .<any action>
   ...

  // When
  await t
   .<any action>

  // Then
  await t
   .expect ... 
   .expect ...
   ...

});

In the Given and the When section you can use a t.expect() but only as a waiting mechanism.

I also never put a message in the .ok() or .notOk() because when a test fails I always have to go through the test manually to verify what failed.

So structuring your tests in a BDD style will help you to switch from this question : how much assertions is too much? to this question : how much tests is too much?

like image 86
hdorgeval Avatar answered Oct 28 '22 12:10

hdorgeval