Can Angular scenario e2e tests integrate an external resource workflow

Question

How do you use Karma Runner (ex-Testacular) e2e scenario runner to integrate an offsite/out-of-app/external-resource workflow in an e2e test?

Basic scenario:

1. App has a button.
2. Click button to send data to service Provider X (eg. submit to http://service/submit or whatever).
3. Provider X requires a click/submit/action at this URL.
4. Provider X then sends something back to our App.

I'm not asking "how do you stub Provider X". I'm asking how do you actually end-to-end test this -- so our tests can actually catch 3rd-party service API breaking changes. I'm basically looking to be able to drive an out-of-app website, similar to how you're able to with something like Selenium's Webdriver.

Currently my problem is using element( ref ).click() to attempt step 2, it doesn't appear that it ever loads the required site, so no DOM to act on, no way to complete the e2e.

Answer 1

It may not be possible to execute this scenario with Karma. I'll give my understanding of how Karma works, how it compares to Webdriver, and what would would be necessary to implement this scenario.

First of all, it is important to keep in mind that Karma and angular-scenario are not the same thing. Karma is a generic shell for running browser-based tests. It launches browser processes, starts node to serve up a web page containing a test harness, runs the tests by sending commands over a websocket, and then collects the results. It is agnostic with respect to the actual harness/framework that runs the tests. It has adapters for the Angular scenario runner (used for Angular e2e tests), Jasmine (commonly used for Angular unit tests) and other popular JavaScript test frameworks.

Test runners such as Jasmine execute in the same window context as the code under test, which is why it is useful for unit tests. The Angular scenario runner, in contrast, has an arms-length relationship to the system under test. It loads the entire application in an iframe, and then for the most part simulates user events and tests assertions on visibly exposed DOM elements.

Because the scenario runner interacts with the application across an iframe boundary, it is constrained by the browser's same-domain policy. It would not be possible to serve up the scenario runner on http://localhost:8000, then load http://yourapp in an iframe, and then trigger events inside that iframe.

One way around this is to simply host the scenario runner on the same server as your application. Here is a simple example of that:

http://plnkr.co/edit/rfqpSq?p=preview

Cramming a test runner into the application is not very satisfying, though. Karma gives you a way around this with a built-in HTTP proxy. You can configure Karma to proxy any number of different servers so that they all appear to originate from the same origin. Simply add them to the proxies section of the configuration file:

proxies = {
  '/': 'http://yourapp/',
  '/service/': 'http://service/'
} 

This allows your e2e tests to cross domains, but the catch is that when a transfer of domain occurs via a form submission or redirect, it has to be to the proxied URL. Otherwise the scenario runner will lose control of the iframe and will not be able to continue the test.

Note that Selenium Webdriver does not have this limitation. It operates above the level of an individual browser window, so it is not constrained by the same domain policy.

So let's look at your specific scenario. In terms of actual HTTP requests and responses, I am assuming this is the flow that you want to exercise:

1. Load your application from http://yoursite/yourapp.html
2. User submits form that looks like:

<form method="POST" action="http://service/remotesubmission">
  ...your form...
  <input type="submit">
</form>

1. Provider X handles remote POST and then redirects to a form at :

<form method="POST" action="http://service/nextsubmission">
  ...provider's form...
  <input type="submit">
</form>

1. Provider X handles form POST and then redirects back to your appp.

If this is the actual scenario, then I think that the only way you could make this work with Karma and the angular scenario runner would be to:

1. Conditionalize the action URL in your form so that it points to the proxied URL for the external service when under test.
2. Ensure that the redirect and action URLs in the provider's flow stay within the proxied paths. I don't think it's currently possible, but perhaps it would be possible to patch Karma with a way to set up substitution patterns for proxied content.