Angular Tests break at random: "Uncaught TypeError: You provided 'undefined' where a stream was expected."

Question

We have a medium sized angular app with currently about 700 unit tests. A few weeks ago, perfectly fine tests started to break. Even stranger: running the tests twice can yield to different results, i.e. different tests may break. In the console, we always find the error :

Uncaught TypeError: You provided 'undefined' where a stream was expected.

But the stack trace gives no hint to where the root of the error is actually located (see end of this post). The stack trace shows a connection to the mergeMap operator, but it turns out that we use this operator no where in our app and nowhere in our tests.

I stepped through all spec files and let them run on their own (with fdescribe). Every single spec file passes without errors. Running them all together leads to the described breakage.

Of course my guess was that we were facing an async problem so I took the effort to go through all the tests and wrap each one of them in an async environment. I also checked that every subscription gets unsubscribed at some point - this was the case for our app but not always for our tests. However, the error still persists.

It's a big issue for our project. Any advice is very welcome. Maybe somebody knows a way to locate the part of our tests that is causing the problem?

We now use jasmine 3.3.0, karma v3.1.4 and Angular 7.1.3. We did the update of jasmine and angular a week ago because we hoped to get rid of the problem. Only one thing changed: before the update, tests didn't break at random but at a fixed number of tests (in our case, 639 Tests would cause a test to break, 638, 640, 641... etc would pass; 648 would break again). I assume it has something to do with the random seed that jasmine is now using.

Here is the full stack trace:

<!-- language: lang-none -->
Uncaught TypeError: You provided 'undefined' where a stream was expected. You can provide an Observable, Promise, Array, or Iterable.
    at subscribeTo (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/util/subscribeTo.js:41)
    at subscribeToResult (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/util/subscribeToResult.js:11)
    at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapSubscriber._innerSub (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:74)
    at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapSubscriber._tryNext (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:68)
    at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapSubscriber._next (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:51)
    at MergeMapSubscriber.push../node_modules/rxjs/_esm5/internal/Subscriber.js.Subscriber.next (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/Subscriber.js:54)
    at Observable._subscribe (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/util/subscribeToArray.js:5)
    at Observable.push../node_modules/rxjs/_esm5/internal/Observable.js.Observable._trySubscribe (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/Observable.js:43)
    at Observable.push../node_modules/rxjs/_esm5/internal/Observable.js.Observable.subscribe (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/Observable.js:29)
    at MergeMapOperator.push../node_modules/rxjs/_esm5/internal/operators/mergeMap.js.MergeMapOperator.call (:9876/_karma_webpack_/webpack:/node_modules/rxjs/_esm5/internal/operators/mergeMap.js:29)
    at ____________________Elapsed_3_ms__At__Thu_Dec_27_2018_10_03_35_GMT_0100__Mitteleurop_ische_Normalzeit_ ()
    at Object.onScheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone-testing.js:108)
    at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.scheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:401)
    at Object.onScheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:297)
    at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.scheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:401)
    at Zone.push../node_modules/zone.js/dist/zone.js.Zone.scheduleTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:232)
    at Zone.push../node_modules/zone.js/dist/zone.js.Zone.scheduleMacroTask (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:255)
    at scheduleMacroTaskWithCurrentZone (:9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:1114)
    at :9876/_karma_webpack_/webpack:/node_modules/zone.js/dist/zone.js:2090

Answer 1

Oooof, sounds like things have turned flaky. We had a run in with random breaking of unit tests recently. Have you been updating your Angular and Karma versions consistently?

What we ran into is that the way unit tests are setup by default (by the Angular CLI) has changed, and that older tests were not running the proper async ways.

The error you are seeing does differ from what we saw, but I'm certain this is an avenue worth exploring to remove any flakiness introduced by the unit tests setup.

As taken from https://angular.io/guide/testing#calling-compilecomponents

describe('BannerComponent', () => {
    let component: BannerComponent
    let fixture: ComponentFixture<BannerComponent>

    beforeEach(async(() => {
      TestBed.configureTestingModule({
        declarations: [ BannerComponent ],
      }).compileComponents();  // compile template and css
    }));

    beforeEach(() => {
        fixture = TestBed.createComponent(BannerComponent)
        component = fixture.componentInstance
        fixture.detectChanges()
    })

    it('should create', () => {
        expect(component).toBeTruthy()
    })

Extra attention for the first beforeEach() which has an async() => {} in there, and a required .compileComponent().

The second beforeEach() is to define and populate the component variable within the shared context of the describe().

I hope this helps you figure out what is causing the flakiness. As the iterator issue stemming from RxJS seems to be pointing towards a test that is relying on state being set by a previous test, where it receives an input in the form of an Observable. If this Observable is set or defined later than the tests execution, you may be running into issues like the one you're describing.