How to make a list of failed specs using jasmine custom reporter to post to slack?

Question

I am trying to work on a custom jasmine reporter and get a list of all the failed specs in the specDone function:

specDone: function(result) {
    if(result.status == 'failed') {          
        failedExpectations.push(result.fullName);
        console.log(failedExpectations);         
    }       
}

where failedExpectations will store an entire list of the failed specs and i need to access this in the afterLaunch function in the protractor config file. But due to the fact that the config file loads everytime a new spec runs it basically gets overwritten and scoping is such that I cannot access it in the afterLaunch function, that is where I am making the call to the slack api. Is there a way to achieve this?

This is what i have it based on : http://jasmine.github.io/2.1/custom_reporter.html

Answer 1

I think the best way is to post the results asynchronously after each spec (*or every "it" and "describe") using @slack/web-api. This way you don't have to worry about overwriting. Basically you "collect" all the results during the test run and send it before the next suite starts. Keep in mind all of this should be done as a class.

First you prepare your you '@slack/web-api', so install it (https://www.npmjs.com/package/@slack/web-api).

npm i -D '@slack/web-api'

Then import it in your reporter:

import { WebClient } from '@slack/web-api';

And initialize it with your token. (https://slack.com/intl/en-pl/help/articles/215770388-Create-and-regenerate-API-tokens):

this.channel = yourSlackChannel;
this.slackApp = new WebClient(yourAuthToken);

Don't forget to invite your slack app to the channel. Then prepare your result "interface" according to your needs and possibilities. For example:

this.results = {
      title: '',
      status: '',
      color: '',
      successTests: [],
      fails: [],
    };

Then prepare a method / function for posting your results:

 postResultOnSlack = (res) => {
try {
  this.slackApp.chat.postMessage({
    text: `Suit name: ${res.title}`,
    icon_emoji: ':clipboard:',
    attachments: [
      {
        color: res.color,
        fields: [
          {
            title: 'Successful tests:',
            value: ` ${res.successTests}`,
            short: false
          },
          {
            title: 'Failed tests:',
            value: ` ${res.fails}`,
            short: false
          },
        ]
      }
    ],
    channel: this.channel
  });
  console.log('Message posted!');
} catch (error) {
  console.log(error);
}

When you got all of this ready it's time to "collect" your results. So on every 'suitStart' remember to "clear" the results:

  suiteStarted(result) {
    this.results.title = result.fullName;
    this.results.status = '';
    this.results.color = '';
    this.results.successTests = [];
    this.results.fails = [];
  }

Then collect success and failed tests:

 onSpecDone(result) {
    this.results.status = result.status

 // here you can push result messages or whole stack or do both:
    this.results.successTests.push(`${test.passedExpectations}`);

    for(var i = 0; i < result.failedExpectations.length; i++) {
    this.results.fails.push(test.failedExpectations[i].message);
    }

 // I'm not sure what is the type of status but I guess it's like this:
    result.status==1 ? this.results.color = #DC143C : this.results.color = #048a04;
    }

And finally send them:

suiteDone() {
      this.postResultOnSlack(this.results);
    }

NOTE: It is just a draft based on reporter of mine. I just wanted to show you the flow. I was looking at Jasmine custom reporter but this was based on WDIO custom reporter based on 'spec reporter'. They are all very similar but you probably have to adjust it. The main point is to collect the results during the test and send them after each part of test run. *You can look up this explanation: https://webdriver.io/docs/customreporter.html I highly recommend this framework, you can use it with Jasmine on top.