I just started using nightwatch with browserstack and I'm noticing that when we get a failed test, nightwatch registers the failure, but browserstack does not. sample test I am using. Also I am using free trial version of BrowserStack.
My question is:
From BrowserStack doc:
REST API
It is possible to mark tests as either a pass or a fail, using the following snippet:
var request = require("request");
request({
uri: "https://user:[email protected]/automate/sessions/<session-id>.json",
method: "PUT",
form: {
"status": "completed",
"reason":""
}
});
The two potential values for status can either be completed or error. Optionally, a reason can also be passed.
My questions are:
A session on BrowserStack has only three types of statuses: Completed, Error or Timeout. Selenium (and hence, BrowserStack) does not have a way of understanding, if a test has passed or failed. Its by the multiple assertions in your tests that appear on your console, that you infer if a test has passed / failed. These assertions however, do not reach BrowserStack. As you rightly identified, you can use the REST-API, to change the status of the session to 'Error', if you see a failure in your console.
I would suggest fetching the session ID of the test as the test is being executed, since fetching the session ID after the test execution is a lengthy process. In Nightwatch, you can fetch session ID as follows:
browser.session(function(session) {
console.log(session.sessionId);
});
If you came here searching for a solution in Python, you could use
requests.put(
"https://api.browserstack.com/automate/sessions/{}.json".format(driver.session_id),
auth=(USERNAME, ACCESS_KEY),
json={"status": "failed", "reason": "test failed"})
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With