AWS Logging not working

Question

I created a very simple Console app which is supposed to log the messages to the AWS Logs but although the app runs I can't find any log on AWS. I think publishing the app code does not make sense: I presume it's ok and it does not throw any exception. I think the problem is located in the AWS settings. This is what I did in AWS:

• created some role , not sure why but did it almost close to what aws poor and messy documentation says. So the role is created, not exactly as it was supposed in the "documentation" but it contains the required permissions for the logs. Why I created it? - I don't have a clue - my app does not use it!

• Created the Log Group - ok, this parameter is what I put into the config of my app

• Not sure I need t create the log stream, but ok, I created it, but when I click on it it says "No events found." and "It appears you have not installed a CloudWatch Logs agent .." Why do I need some agent? what is it? how to install? - absolutely not clear and pointing to the poor aws "documentation" is useless.

I guess these are the major things done in the AWS but..still no result - nothing works, I cant see the logs.

Searched for the answer in google, youtube, etc - no result. Found some code which is similar to mine but it's no enought - it seems there are some settings required to be done on AWS.

What's wrong?

Answer 1

You have two options:

1. Write log files to disk and use CloudWatch Agent to submit these logs to CloudWatch: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/QuickStartWindows2016.html

With this option you don't need to configure anything related to AWS in the program, but you have to install and configure the Agent.

2. Use AWS.Logger NuGet package and configure it to send the logs to CloudWatch, in this case you don't need to use the Agent: https://github.com/aws/aws-logging-dotnet/tree/master/samples/AspNetCore

With this option you must create AWS API user with CloudWatch Log writing permission and put this user credentials into AWS.Logger configuration. Show the configuring code you used if you need an advice on this.

Answer 2

I had a similar problem, which turned out to be more config-related.

Firstly, make sure that you have AWS Toolkit for Visual Studio installed and set up with the appropriate user. I use an IAM User with the correct policy permissions to read and write Cloudwatch logs.

Here's a copy of my basic console test that works correctly:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Configuration;
using System;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;


namespace Runner
{
    class Program
    {
        public static async Task Main(string[] args)
        {
            var services = ConfigureServices(new ServiceCollection())
                .BuildServiceProvider();
            await services.GetService<App>().RunAsync();
        }

        private static IServiceCollection ConfigureServices(IServiceCollection services)
        {
            var configuration = ConfigurationFactory.GetConfiguration();

            services
                .AddSingleton(configuration)
                .AddLogging(builder =>
                {
                    var config = configuration.GetSection("Logging");
                    builder
                        .AddConfiguration(configuration.GetSection("Logging"))
                        .AddConsole()
                        .AddDebug()
                        .AddAWSProvider(configuration.GetAWSLoggingConfigSection().Config);
                })

            // add app
            services.AddTransient<App>();

            return services;
        }
    }

    public class App
    {
        private ILogger<App> Logger;

        public App(ILogger<App> logger)
        {
            Logger = logger;
        }

        public async Task RunAsync()
        {
            try
            {
                Logger.LogTrace("LogTrace", "{\"Test\":1}");
                Logger.LogInformation("LogInformation", "{\"Test\":2}");
                Logger.LogWarning("LogWarning", "{\"Test\":3}");
                Logger.LogDebug("LogDebug", "{\"Test\":4}");
                Logger.LogError("LogError", "{\"Test\":5}");
                Logger.LogCritical("LogCritical", "{\"Test\":6}");
                Thread.Sleep(3000);
                Debugger.Break();
            }
            catch (Exception ex)
            {
                throw;
            }
        }
    }
}

And my appsettings.json file is:

{
  "Logging": {
    "Region": "eu-west-1",
    "LogGroup": "/dev/runner",
    "IncludeLogLevel": true,
    "IncludeCategory": true,
    "IncludeNewline": true,
    "IncludeException": true,
    "IncludeEventId": false,
    "IncludeScopes": false,
    "LogLevel": {
      "Default": "Debug",
      "System": "Information",
      "Microsoft": "Information"
    },
    "Console": {
      "LogLevel": {
        "Default": "Error",
        "System": "Information",
        "Microsoft": "Information"
      }
    },
    "Debug": {
      "LogLevel": {
        "Default": "Trace",
        "System": "Information",
        "Microsoft": "Information"
      }
    }
  }
}

The Thread.Sleep is to allow the console logger to catch up with itself - if you just break you often don't see anything.

Similarly, if you quit the program executing at the breakpoint the AWS logger won't flush its buffers to Cloudwatch (it will just create the logstream and leave it empty), so let the program run to completion to populate the logstream itself.