How to debug or see output for a .NET Core app deployed in Elastic Beanstalk?

Question

Specifically I am interested in being able to see output via their interface. For instance, is there anyway to get the ILogger.LogFoo methods or Console.WriteLine to show up in either the Monitoring or Logs tabs or anywhere else?

I am trying to stay away from proprietary AWS ways of seeing output in case the app needs to be deployed elsewhere.

Answer 1

There's no way to make a Windows-based application's logs show up in the Elastic Beanstalk interface. I specifically asked AWS support this a few months back and received the following reply telling me to use CloudWatch instead:

Hello,

The beanstalk logs uses it's own logic to collect the logs. In the linux verison of it it is possible to simply add the file to the path of the logging, but in windows the logs are collected individually since they use a different type of logging driver to collect the logs, therefore it is not possible to simply add files to a path and see them show up in the beanstalk logging.

what you can do instead of using the built in beanstalk logging is use cloudformation [sic] logs instead.

http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html

with this you should be able to set up a logger that will allow you to send logs directly to amazon cloudwatch logs.

Best regards,

elijah f. Amazon Web Services

It is probably prudent that you want to avoid making all of your logging code be tightly coupled to - as you put it in the question - "proprietary AWS ways of seeing output". However, it's possible to use CloudWatch with only loose coupling by using the ASP.NET Core framework's standard logging together with the AWS.Logger.AspNetCore logging provider that integrates ASP.NET Core's built-in logger with CloudWatch.

Based upon a glance over the commit in which I added this feature to my current company's codebase, I think that in order to get it working I had to make the following changes, mostly based upon the official ASP.NET Core Cloudwatch integration example at https://github.com/aws/aws-logging-dotnet/tree/master/samples/AspNetCore/WebSample:

1. Run

   Install-Package AWS.Logger.Core
   Install-Package AWS.Logger.AspNetCore
   

2. Add

   loggerFactory.AddAWSProvider(Configuration.GetAWSLoggingConfigSection());
   

   to my application's

   public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
   

   method in Startup.cs (the same place that the template application contains calls to loggerFactory.AddConsole(Configuration.GetSection("Logging")); and loggerFactory.AddDebug();.

3. Add an AWS.Logging section to the root of my appsettings.config file:

   {
     "AWS.Logging": {
       "Region": "us-west-2",
       "LogGroup": "put-whatever-name-you-like-here",
       "LogLevel": {
         "Default": "Debug",
         "System": "Information",
         "Microsoft": "Information"
       }
     },
     ...
   }
   

4. Per the instructions at http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html#AWSHowTo.cloudwatchlogs.permissions, create a custom policy at https://console.aws.amazon.com/iam/home#/policies with the following content:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "logs:CreateLogGroup",
           "logs:CreateLogStream",
           "logs:GetLogEvents",
           "logs:PutLogEvents",
           "logs:DescribeLogGroups",
           "logs:DescribeLogStreams",
           "logs:PutRetentionPolicy"
         ],
         "Resource": [
           "arn:aws:logs:us-west-2:*:*"
         ]
       }
     ]
   }
   

   and then attach it to the the aws-elasticbeanstalk-ec2-role at https://console.aws.amazon.com/iam/home#/roles/aws-elasticbeanstalk-ec2-role. (Note that us-west-2 in the JSON above is specific to my application, since that's the region that it's deployed in, and that aws-elasticbeanstalk-ec2-role is the default role for Elastic Beanstalk instances - you may need to attach the policy to a different role if you're bringing up instances with a different role.)

Once all that is done, your ILoggers should log to CloudWatch. If you go to https://console.aws.amazon.com/cloudwatch/home, click "Logs" in the sidebar, and then click on the log group whose name matches the LogGroup setting in your appsettings file, you should be able to see your logs.