What is "callback hell" and how and why does RX solve it?

Question

1) What is a "callback hell" for someone who does not know javascript and node.js ?

This other question has some examples of Javascript callback hell: How to avoid long nesting of asynchronous functions in Node.js

The problem in Javascript is that the only way to "freeze" a computation and have the "rest of it" execute latter (asynchronously) is to put "the rest of it" inside a callback.

For example, say I want to run code that looks like this:

x = getData();
y = getMoreData(x);
z = getMoreData(y);
...

What happens if now I want to make the getData functions asynchronous, meaning that I get a chance to run some other code while I am waiting for them to return their values? In Javascript, the only way would be to rewrite everything that touches an async computation using continuation passing style:

getData(function(x){
    getMoreData(x, function(y){
        getMoreData(y, function(z){ 
            ...
        });
    });
});

I don't think I need to convince anyone that this version is uglier than the previous one. :-)

2) When (in what kind of settings) does the "callback hell problem" occur?

When you have lots of callback functions in your code! It gets harder to work with them the more of them you have in your code and it gets particularly bad when you need to do loops, try-catch blocks and things like that.

For example, as far as I know, in JavaScript the only way to execute a series of asynchronous functions where one is run after the previous returns is using a recursive function. You can't use a for loop.

// we would like to write the following
for(var i=0; i<10; i++){
    doSomething(i);
}
blah();

Instead, we might need to end up writing:

function loop(i, onDone){
    if(i >= 10){
        onDone()
    }else{
        doSomething(i, function(){
            loop(i+1, onDone);
        });
     }
}
loop(0, function(){
    blah();
});

//ugh!

The number of questions we get here on StackOverflow asking how to do this kind of thing is a testament to how confusing it is :)

3) Why does it occur ?

It occurs because in JavaScript the only way to delay a computation so that it runs after the asynchronous call returns is to put the delayed code inside a callback function. You cannot delay code that was written in traditional synchronous style so you end up with nested callbacks everywhere.

4) Or can "callback hell" occur also in a single threaded application?

Asynchronous programming has to do with concurrency while a single-thread has to do with parallelism. The two concepts are actually not the same thing.

You can still have concurrent code in a single threaded context. In fact, JavaScript, the queen of callback hell, is single threaded.

What is the difference between concurrency and parallelism?

5) could you please also show how RX solves the "callback hell problem" on that simple example.

I don't know anything about RX in particular, but usually this problem gets solved by adding native support for asynchronous computation in the programming language. The implementations can vary and include: async, generators, coroutines, and callcc.

In Python we can implement that previous loop example with something along the lines of:

def myLoop():
    for i in range(10):
        doSomething(i)
        yield

myGen = myLoop()

This is not the full code but the idea is that the "yield" pauses our for loop until someone calls myGen.next(). The important thing is that we could still write the code using a for loop, without needing to turn out logic "inside out" like we had to do in that recursive loop function.

Just answer the question: could you please also show how RX solves the "callback hell problem" on that simple example?

The magic is flatMap. We can write the following code in Rx for @hugomg's example:

def getData() = Observable[X]
getData().flatMap(x -> Observable[Y])
         .flatMap(y -> Observable[Z])
         .map(z -> ...)...

It's like you are writing some synchronous FP codes, but actually you can make them asynchronous by Scheduler.

To address the question of how Rx solves callback hell:

First let's describe callback hell again.

Imagine a case were we must do http to get three resources - person, planet and galaxy. Our objective is to find the galaxy the person lives in. First we must get the person, then the planet, then the galaxy. That's three callbacks for three asynchronous operations.

getPerson(person => { 
   getPlanet(person, (planet) => {
       getGalaxy(planet, (galaxy) => {
           console.log(galaxy);
       });
   });
});

Each callback is nested. Each inner callback is dependent on its parent. This leads to the "pyramid of doom" style of callback hell. The code looks like a > sign.

To solve this in RxJs you could do something like so:

getPerson()
  .map(person => getPlanet(person))
  .map(planet => getGalaxy(planet))
  .mergeAll()
  .subscribe(galaxy => console.log(galaxy));

With the mergeMap AKA flatMap operator you could make it more succinct:

getPerson()
  .mergeMap(person => getPlanet(person))
  .mergeMap(planet => getGalaxy(planet))
  .subscribe(galaxy => console.log(galaxy));

As you can see, the code is flattened and contains a single chain of method calls. We have no "pyramid of doom".

Hence, callback hell is avoided.

In case you were wondering, promises are another way to avoid callback hell, but promises are eager, not lazy like observables and (generally speaking) you cannot cancel them as easily.

Callback hell is any code where the use of function callbacks in async code becomes obscure or difficult to follow. Generally, when there is more than one level of indirection, code using callbacks can become harder to follow, harder to refactor, and harder to test. A code smell is multiple levels of indentation due to passing multiple layers of function literals.

This often happens when behaviour has dependencies, i.e. when A must happen before B must happen before C. Then you get code like this:

a({
    parameter : someParameter,
    callback : function() {
        b({
             parameter : someOtherParameter,
             callback : function({
                 c(yetAnotherParameter)
        })
    }
});

If you have lots of behavioural dependencies in your code like this, it can get troublesome fast. Especially if it branches...

a({
    parameter : someParameter,
    callback : function(status) {
        if (status == states.SUCCESS) {
          b(function(status) {
              if (status == states.SUCCESS) {
                 c(function(status){
                     if (status == states.SUCCESS) {
                         // Not an exaggeration. I have seen
                         // code that looks like this regularly.
                     }
                 });
              }
          });
        } elseif (status == states.PENDING {
          ...
        }
    }
});

This won't do. How can we make asynchronous code execute in a determined order without having to pass all these callbacks around?

RX is short for 'reactive extensions'. I haven't used it, but Googling suggests it's an event-based framework, which makes sense. Events are a common pattern to make code execute in order without creating brittle coupling. You can make C listen to the event 'bFinished' which only happens after B is called listening to 'aFinished'. You can then easily add extra steps or extend this kind of behaviour, and can easily test that your code executes in order by merely broadcasting events in your test case.

Call back hell means you are inside of a callback of inside another callback and it goes to nth call until your needs not fullfiled.

Let's understand through an example of fake ajax call by using set timeout API, lets assume we have a recipe API, we need to download all recipe.

<body>
    <script>
        function getRecipe(){
            setTimeout(()=>{
                const recipeId = [83938, 73838, 7638];
                console.log(recipeId);
            }, 1500);
        }
        getRecipe();
    </script>
</body>

In the above example after 1.5 sec when timer expires inside code of call back will execute, in other words, through our fake ajax call all recipe will downloaded from the server. Now we need to download a particular recipe data.

<body>
    <script>
        function getRecipe(){
            setTimeout(()=>{
                const recipeId = [83938, 73838, 7638];
                console.log(recipeId);
                setTimeout(id=>{
                    const recipe = {title:'Fresh Apple Juice', publisher:'Suru'};
                    console.log(`${id}: ${recipe.title}`);
                }, 1500, recipeId[2])
            }, 1500);
        }
        getRecipe();
    </script>
</body>

To download a particular recipe data we wrote code inside of our first callback and passed recipe Id.

Now let's say we need to download all the recipes of the same publisher of the recipe which id is 7638.

<body>
    <script>
        function getRecipe(){
            setTimeout(()=>{
                const recipeId = [83938, 73838, 7638];
                console.log(recipeId);
                setTimeout(id=>{
                    const recipe = {title:'Fresh Apple Juice', publisher:'Suru'};
                    console.log(`${id}: ${recipe.title}`);
                    setTimeout(publisher=>{
                        const recipe2 = {title:'Fresh Apple Pie', publisher:'Suru'};
                        console.log(recipe2);
                    }, 1500, recipe.publisher);
                }, 1500, recipeId[2])
            }, 1500);
        }
        getRecipe();
    </script>
</body>

To full-fill our needs which is to download all the recipes of publisher name suru, we wrote code inside of our second call back. It is clear we wrote a callback chain which is called callback hell.

If you want to avoid callback hell, you can use Promise, which is js es6 feature, each promise takes a callback which is called when a promise is full-filled. promise callback has two options either it is resolved or reject. Suppose your API call is successful you can call resolve and pass data through the resolve, you can get this data by using then(). But if your API failed you can use reject, use catch to catch the error. Remember a promise always use then for resolve and catch for reject

Let's solve the previous callback hell problem using a promise.

<body>
    <script>

        const getIds = new Promise((resolve, reject)=>{
            setTimeout(()=>{
                const downloadSuccessfull = true;
                const recipeId = [83938, 73838, 7638];
                if(downloadSuccessfull){
                    resolve(recipeId);
                }else{
                    reject('download failed 404');
                }
            }, 1500);
        });

        getIds.then(IDs=>{
            console.log(IDs);
        }).catch(error=>{
            console.log(error);
        });
    </script>
</body>

Now download particular recipe:

<body>
    <script>
        const getIds = new Promise((resolve, reject)=>{
            setTimeout(()=>{
                const downloadSuccessfull = true;
                const recipeId = [83938, 73838, 7638];
                if(downloadSuccessfull){
                    resolve(recipeId);
                }else{
                    reject('download failed 404');
                }
            }, 1500);
        });

        const getRecipe = recID => {
            return new Promise((resolve, reject)=>{
                setTimeout(id => {
                    const downloadSuccessfull = true;
                    if (downloadSuccessfull){
                        const recipe = {title:'Fresh Apple Juice', publisher:'Suru'};
                        resolve(`${id}: ${recipe.title}`);
                    }else{
                        reject(`${id}: recipe download failed 404`);
                    }

                }, 1500, recID)
            })
        }
        getIds.then(IDs=>{
            console.log(IDs);
            return getRecipe(IDs[2]);
        }).
        then(recipe =>{
            console.log(recipe);
        })
        .catch(error=>{
            console.log(error);
        });
    </script>
</body>

Now we can write another method call allRecipeOfAPublisher like getRecipe which will also return a promise, and we can write another then() to receive resolve promise for allRecipeOfAPublisher, I hope at this point you can do it by yourself.

So we learned how to construct and consumed promises, now let's make consuming a promise easier by using async/await which is introduced in es8.

<body>
    <script>

        const getIds = new Promise((resolve, reject)=>{
            setTimeout(()=>{
                const downloadSuccessfull = true;
                const recipeId = [83938, 73838, 7638];
                if(downloadSuccessfull){
                    resolve(recipeId);
                }else{
                    reject('download failed 404');
                }
            }, 1500);
        });

        const getRecipe = recID => {
            return new Promise((resolve, reject)=>{
                setTimeout(id => {
                    const downloadSuccessfull = true;
                    if (downloadSuccessfull){
                        const recipe = {title:'Fresh Apple Juice', publisher:'Suru'};
                        resolve(`${id}: ${recipe.title}`);
                    }else{
                        reject(`${id}: recipe download failed 404`);
                    }

                }, 1500, recID)
            })
        }

        async function getRecipesAw(){
            const IDs = await getIds;
            console.log(IDs);
            const recipe = await getRecipe(IDs[2]);
            console.log(recipe);
        }

        getRecipesAw();
    </script>
</body>

In the above example, we used an async function because it will run in the background, inside async function we used await keyword before each method which returns or is a promise because to wait on that position until that promise fulfilled, in other words in the bellow codes until getIds completed resolved or reject program will stop executing codes bellow that line when IDs returned then we again called getRecipe() function with a id and waited by using await keyword until data returned. So this is how finally we recovered from the callback hell.

  async function getRecipesAw(){
            const IDs = await getIds;
            console.log(IDs);
            const recipe = await getRecipe(IDs[2]);
            console.log(recipe);
        }

To use await we will need a async function, we can return a promise so use then for resolve promise and cath for reject promise

from the above example:

 async function getRecipesAw(){
            const IDs = await getIds;
            const recipe = await getRecipe(IDs[2]);
            return recipe;
        }

        getRecipesAw().then(result=>{
            console.log(result);
        }).catch(error=>{
            console.log(error);
        });