Bot Framework messes up dialog state

Question

I'm currently making a chatbot with Microsoft's Bot Framework. In my flow I have a final dialog that lets the user know, that they are participating in the competition. There is also an error-handling method for unknown input. The two methods are seen here:

[Serializable]
public class ConcertCityDialog : AbstractBasicDialog<DialogResult>
{
    private static FacebookService FacebookService => new FacebookService(new FacebookClient());

    [LuisIntent("ConcertCity")]
    public async Task ConcertCityIntent(IDialogContext context, LuisResult result)
    {
        var fbAccount = await FacebookService.GetAccountAsync(context.Activity.From.Id);

        var selectedCityName = result.Entities.FirstOrDefault()?.Entity;

        concert_city selectedCity;
        using (var concertCityService = new ConcertCityService())
        {
            selectedCity = concertCityService.FindConcertCity(selectedCityName);
        }

        if (selectedCity == null)
        {
            await NoneIntent(context, result);
            return;
        }

        user_interaction latestInteraction;
        using (var userService = new MessengerUserService())
        {
            var user = userService.FindByFacebookIdIncludeInteractions(context.Activity.From.Id);
            latestInteraction = user.user_interaction.MaxBy(e => e.created_at);
        }

        latestInteraction.preferred_city_id = selectedCity.id;
        latestInteraction.gif_created = true;

        using (var userInteractionService = new UserInteractionService())
        {
            userInteractionService.UpdateUserInteraction(latestInteraction);
        }

        var shareIntroReply = context.MakeMessage();
        shareIntroReply.Text = "Great choice! You are now participating in the competition. If you dare then pass your message \uD83D\uDE0E";

        await context.PostAsync(shareIntroReply);

        var reply = await MessageUtility.MakeShareMessageCard(context, fbAccount, latestInteraction, false);

        await context.PostAsync(reply);

        context.Done(DialogResult.Done);
    }

    [LuisIntent("")]
    [LuisIntent("None")]
    public async Task NoneIntent(IDialogContext context, LuisResult result)
    {
        messenger_user user;
        using (var userService = new MessengerUserService())
        {
            user = userService.FindByFacebookId(context.Activity.From.Id);
        }

        var phrase = CreateMisunderstoodPhrase(user, result.Query);

        using (var misunderstoodPhraseService = new MisunderstoodPhraseService())
        {
            misunderstoodPhraseService.CreatePhrase(phrase);
        }

        List<concert_city> concertCities;
        using (var concertCityService = new ConcertCityService())
        {
            concertCities = concertCityService.GetUpcomingConcertCities().ToList();
        }

        // Prompt city
        var reply = context.MakeMessage();
        reply.Text = "I'm not sure what you mean \uD83E\uDD14<br/>Which Grøn Koncert would you like to attend?";

        reply.SuggestedActions = new SuggestedActions
        {
            Actions = concertCities.Select(e => MessageUtility.MakeQuickAnswer(e.name)).ToList()
        };

        await context.PostAsync(reply);

        context.Wait(MessageReceived);
    }

    protected override void OnDeserializedCustom(StreamingContext context)
    {
    }
}

And here is the AbstractBasicDialog implementation:

[Serializable]
public abstract class AbstractBasicDialog<T> : LuisDialog<T>
{
    protected AbstractBasicDialog() : base(new LuisService(new LuisModelAttribute(
        ConfigurationManager.AppSettings["LuisAppId"],
        ConfigurationManager.AppSettings["LuisAPIKey"],
        domain: ConfigurationManager.AppSettings["LuisAPIHostName"])))
    {
    }

    [LuisIntent("Cancel")]
    public virtual async Task CancelIntent(IDialogContext context, LuisResult result)
    {
        var randomQuotes = new List<string>
        {
            "If you say so, I'll leave you alone for now",
            "alright then, I'll leave you alone",
            "Okay then, I won't bother you anymore"
        };

        await context.PostAsync(MessageUtility.RandAnswer(randomQuotes));

        context.Done(DialogResult.Cancel);
    }

    [LuisIntent("Start")]
    public virtual async Task StartIntent(IDialogContext context, LuisResult result)
    {
        context.Done(DialogResult.Restart);
    }

    [LuisIntent("CustomerSupport")]
    public async Task CustomerSupportIntent(IDialogContext context, LuisResult result)
    {
        using (var userService = new MessengerUserService())
        {
            var user = userService.FindByFacebookId(context.Activity.From.Id);
            if (user != null)
            {
                user.receiving_support = true;
                userService.UpdateUser(user);
            }
        }

        await context.PostAsync("I'll let customer service know, that you want to talk to them. They will get back to you within 24 hours.<br/>If at any time you want to return to me, and start passing a message, just type \"Stop customer support\".");

        context.Call(new CustomerSupportDialog(), ResumeAfterCustomerSupport);
    }

    private async Task ResumeAfterCustomerSupport(IDialogContext context, IAwaitable<DialogResult> result)
    {
        context.Done(await result);
    }

    protected misunderstood_phrase CreateMisunderstoodPhrase(messenger_user user, string phrase)
    {
        return new misunderstood_phrase
        {
            phrase = phrase,
            dialog = GetType().Name,
            messenger_user_id = user.id
        };
    }

    [OnDeserialized]
    private void OnDeserialized(StreamingContext context)
    {
        OnDeserializedCustom(context);
    }

    protected abstract void OnDeserializedCustom(StreamingContext context);
}

The call chain starts at this dialog:

[Serializable]
public class BasicLuisDialog : LuisDialog<DialogResult>
{
    private static FacebookService FacebookService => new FacebookService(new FacebookClient());

    public BasicLuisDialog() : base(new LuisService(new LuisModelAttribute(
        ConfigurationManager.AppSettings["LuisAppId"],
        ConfigurationManager.AppSettings["LuisAPIKey"],
        domain: ConfigurationManager.AppSettings["LuisAPIHostName"])))
    {
    }

    [LuisIntent("")]
    [LuisIntent("None")]
    public async Task NoneIntent(IDialogContext context, LuisResult result)
    {
        var facebookAccount = await FacebookService.GetAccountAsync(context.Activity.From.Id);

        RegisterUser(facebookAccount, null, out var user);

        var phrase = CreateMisunderstoodPhrase(user, result.Query);
        using (var misunderstoodPhraseService = new MisunderstoodPhraseService())
        {
            misunderstoodPhraseService.CreatePhrase(phrase);
        }

        var reply = context.MakeMessage();
        reply.SuggestedActions = new SuggestedActions
        {
            Actions = new List<CardAction>
            {
                new CardAction { Title = "Get started", Type = ActionTypes.ImBack, Value = "Get started" },
                new CardAction { Title = "Customer support", Type = ActionTypes.ImBack, Value = "Customer support" }
            }
        };

        var name = string.IsNullOrEmpty(facebookAccount.FirstName) ? "" : $"{facebookAccount.FirstName} ";
        reply.Text = $"Hm, I'm not sure what you mean {name} \uD83E\uDD14 Here are some ways you can interact with me:";

        await context.PostAsync(reply);
        context.Wait(MessageReceived);
    }

    [LuisIntent("Greeting")]
    [LuisIntent("Positive")]
    [LuisIntent("Start")]
    public async Task GreetingIntent(IDialogContext context, LuisResult result)
    {
        var rnd = new Random();
        var facebookAccount = await FacebookService.GetAccountAsync(context.Activity.From.Id);

        // Initial Greeting
        var greetings = new List<string>
        {
            "Well hello there",
            "Hi there"
        };

        if (!string.IsNullOrEmpty(facebookAccount.FirstName))
        {
            greetings.Add("Hi {0}");
            greetings.Add("Hello {0}");
            greetings.Add("Welcome {0}");
        }

        if (facebookAccount.Gender == "male")
            greetings.Add("Hey handsome");
        else if (facebookAccount.Gender == "female")
            greetings.Add("Hi gorgeous");

        var randIndex = rnd.Next(greetings.Count);

        var greeting = string.Format(greetings[randIndex], facebookAccount.FirstName);

        await context.PostAsync(greeting);

        await MessageUtility.StartTyping(context, 300);

        country country;
        using (var countryService = new CountryService())
        {
            country = countryService.FindCountry(facebookAccount.Locale);
        }
        var userHasCountry = RegisterUser(facebookAccount, country, out var user);

        // If user contry not found prompt for answer
        if (!userHasCountry)
        {
            var countryReply = context.MakeMessage();
            countryReply.Text = "You are hard to keep track of - where are you from?";
            countryReply.SuggestedActions = new SuggestedActions
            {
                Actions = new List<CardAction>
                {
                    MessageUtility.MakeQuickAnswer("Denmark"),
                    MessageUtility.MakeQuickAnswer("Norway"),
                    MessageUtility.MakeQuickAnswer("Sweden"),
                    MessageUtility.MakeQuickAnswer("Other")
                }
            };

            await context.PostAsync(countryReply);

            context.Call(new CountryDialog(), AfterCountryDialog);
        }
        else
        {
            await FunPrompt(context, country);
        }
    }

    private async Task AfterCountryDialog(IDialogContext countryContext, IAwaitable<country> countryAwaitable)
    {
        var country = await countryAwaitable;

        var facebookAccount = await FacebookService.GetAccountAsync(countryContext.Activity.From.Id);

        using (var userService = new MessengerUserService())
        {
            var user = userService.FindByFacebookId(facebookAccount.Id);

            user.country = country;
            userService.UpdateUser(user);
        }

        var reply = countryContext.MakeMessage();
        reply.Text = "That's cool \uD83D\uDE0E";

        await countryContext.PostAsync(reply);

        await MessageUtility.StartTyping(countryContext, 350);

        await FunPrompt(countryContext, country);
    }

    private async Task FunPrompt(IDialogContext context, country country)
    {
        if (country?.name == "norway" && DateTime.Now < new DateTime(2018, 8, 13))
        {
            var reply = context.MakeMessage();
            reply.Text = "Unfortunately the competition isn't open in Norway yet. You can still talk to customer support if you want to";
            reply.SuggestedActions = new SuggestedActions
            {
                Actions = new List<CardAction>
                {
                    MessageUtility.MakeQuickAnswer("Customer support")
                }
            };

            await context.PostAsync(reply);

            context.Wait(MessageReceived);
        }
        else if ((country?.name == "denmark" && DateTime.Now >= new DateTime(2018, 7, 29)) ||
                 (country?.name == "norway" && DateTime.Now >= new DateTime(2018, 10, 21)))
        {
            var reply = context.MakeMessage();
            reply.Text = "The competition has ended. You can still talk to customer support if you want to";
            reply.SuggestedActions = new SuggestedActions
            {
                Actions = new List<CardAction>
                {
                    MessageUtility.MakeQuickAnswer("Customer support")
                }
            };

            await context.PostAsync(reply);

            context.Wait(MessageReceived);
        }
        else
        {
            await context.PostAsync("Are you up for some fun?");

            context.Call(new IntroductionDialog(), ResumeAfterDialog);
        }
    }

    [LuisIntent("CustomerSupport")]
    public async Task CustomerSupportIntent(IDialogContext context, LuisResult result)
    {
        using (var userService = new MessengerUserService())
        {
            var user = userService.FindByFacebookId(context.Activity.From.Id);
            if (user != null)
            {
                user.receiving_support = true;
                userService.UpdateUser(user);
            }
        }

        await context.PostAsync("I'll let customer support know, that you want to talk to them. They should be messaging you shortly.<br/>You can end your conversation with customer support at any time by typing \"Stop customer support\".");

        context.Call(new CustomerSupportDialog(), ResumeAfterDialog);
    }

    private async Task ResumeAfterDialog(IDialogContext context, IAwaitable<DialogResult> result)
    {
        var resultState = await result;
        if (resultState == DialogResult.Restart)
            await GreetingIntent(context, null);
        else if (resultState == DialogResult.CustomerSupport)
            await ResumeAfterCustomerSupport(context);
        else if (resultState == DialogResult.Done || resultState == DialogResult.Cancel)
            context.Done(resultState);
        else
            context.Wait(MessageReceived);
    }

    private async Task ResumeAfterCustomerSupport(IDialogContext context)
    {
        using (var userService = new MessengerUserService())
        {
            var user = userService.FindByFacebookId(context.Activity.From.Id);
            if (user != null)
            {
                user.receiving_support = false;
                userService.UpdateUser(user);
            }
        }

        await context.PostAsync("I hope you got the help you needed. Would you like to pass a message to a friend?");

        context.Call(new IntroductionDialog(), ResumeAfterDialog);
    }

    private bool RegisterUser(FacebookAccount fbAccount, country country, out messenger_user user)
    {
        if (string.IsNullOrEmpty(fbAccount?.Id))
        {
            user = null;
            return false;
        }

        using (var userService = new MessengerUserService())
        {
            user = userService.FindByFacebookId(fbAccount.Id);

            if (user != null)
                return user.country != null;

            user = new messenger_user
            {
                id = fbAccount.Id,
                country = country
            };

            userService.CreateUser(user);

            return user.country != null;
        }
    }

    protected misunderstood_phrase CreateMisunderstoodPhrase(messenger_user user, string phrase)
    {
        return new misunderstood_phrase
        {
            phrase = phrase,
            dialog = GetType().Name,
            messenger_user_id = user.id
        };
    }
}

This works most of the time. The user is told that their registration was a success and the flow exits with the context.Done() call. Sometimes however the chatbot doesn't register the dialog as being exited, as seen here:

As you can see the chatbot is still in the same Dialog even though I have called the Done() method. This is a general problem in my chatbot, as it happens sometimes in all my dialogs.

Do you have any input as to what could be wrong?

EDIT: When debugging this I've added breakpoints every time it calls context.Call. When my issue arises it stops hitting these breakpoints afterwards. Could this be a side-effect of some DI or something? This is my DI code:

Conversation.UpdateContainer(builder =>
{
    builder.RegisterModule(new DialogModule());
    builder.RegisterModule(new ReflectionSurrogateModule());
    builder.RegisterModule(new DialogModule_MakeRoot());
    builder.RegisterModule(new AzureModule(Assembly.GetExecutingAssembly()));

    var store = new TableBotDataStore(ConfigurationManager.ConnectionStrings["StorageConnectionString"].ConnectionString);

    builder.Register(c => store)
        .Keyed<IBotDataStore<BotData>>(AzureModule.Key_DataStore)
        .AsSelf()
        .SingleInstance();

    builder.Register(c => new CachingBotDataStore(store,
            CachingBotDataStoreConsistencyPolicy
                .ETagBasedConsistency))
        .As<IBotDataStore<BotData>>()
        .AsSelf()
        .InstancePerLifetimeScope();

    builder.RegisterType<BasicLuisDialog>().As<LuisDialog<DialogResult>>().InstancePerDependency();
});

Answer 1

I think I finally found the problem. In my code I had implemented a helper method in a static class that would send a typing response and wait a certain amount of time. Seeing as the context was passed into this static method it seems that this was causing some issues.

After changing the method to an extension method of the LuisDialog I no longer have this issue.

I would appreciate if anyone can expand on why this might have been a problem.

EDIT: The method in question:

public static async Task StartTyping(IDialogContext context, int sleep)
{
    var typingMsg = context.MakeMessage();
    typingMsg.Type = ActivityTypes.Typing;

    await context.PostAsync(typingMsg);
    await Task.Delay(sleep);
}

Answer 2

I faced a very similar issue and while moving the typing sending into a base class from a static helper class as Frederik did help to highly reduce the number of times the problem occured, the final solution was this: https://github.com/Microsoft/BotBuilder/issues/4477

In short, I had to downgrade the bot-related NuGet packages (Microsoft.Bot.Builder, Microsoft.Bot.Builder.History, Microsoft.Bot.Connector) to 3.13.1 and the issue disappeared.

Answer 3

since in [LuisIntent("ConcertCity")] you are using context.Done() so the current dialog gets exit from the stack. This is why the next message is being handled by the previous dialog or the message controller where the 'None' intent is being called and you are getting this response

reply.Text = "I'm not sure what you mean \uD83E\uDD14<br/>Which Grøn Koncert would you like to attend?";

You should not do context.Done() every places, this should only be called when you have to go to the previous dialog on the stack.