How to set AWS credentials with .net Core

Question

I have to debug an existing project about .net Core and AWS. Our project runs well on our AWS instance but we can't run the project in local.

Firstly we got the AmazonServiceException: Unable to find credentials, but now we have the message: AmazonClientException: No RegionEndpoint or ServiceURL configured. I think it's better.

Our configuration: In our application we have 3 appsettings.{env.EnvironmentName}.json (Development, Local and Production). We know by default VS use the development file. In our development appsettings file we have no AWS object but in the local appsettings file we have only that:

"AWS": {
      "Region": "ap-southeast-2"
}

We don't have any web.config or other json config file.

We tried to create a credential file as:

[name of my IAM profile]
aws_access_key_id=accesskey
aws_secret_access_key=secretkey
region=ap-southeast-2

But we didn't find how we can use it.

We also try to run the project with the dotnet core run command and to specify some environment variables as :

export AWS_Region=ap-southeast-2
export AWS_ACCESS_KEY_ID=id
export AWS_SECRET_ACCESS_KEY=secret
export AWS_SESSION_TOKEN=token
export 
AWS_CREDENTIAL_FILE=/Users/user/Developer/exemple/nameproject/G$

But same error.

Program.cs file:

var host = new WebHostBuilder()
    .UseKestrel(options => options.AddServerHeader = false)
    .UseContentRoot(Directory.GetCurrentDirectory())
    .UseIISIntegration()
    .UseStartup<Startup>()
    .UseUrls("http://*:9000")
    .CaptureStartupErrors(true)
    .Build();

host.Run();

Startup file (first function):

public Startup(IHostingEnvironment env) {
        // Configuration override https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration
        var builder = new ConfigurationBuilder()
            .SetBasePath(env.ContentRootPath)
            .AddJsonFile($"appsettings.{env.EnvironmentName}.json", false, true)
            .AddEnvironmentVariables();

        Configuration = builder.Build();

        // Shared startup configurator
        CommonStartup = new CommonStartup(env, Configuration);
    }

Here is our question: Where or how are configured the credentials of our project?

Thank you in advance for your answer.

Answer 1

In AWS, you can create an IAM role and configure it to only have access to the resources it needs (S3 read/write, SES etc). You can then attach this role to the EC2 instance.

If you're using the AWS SDK, use something like:

services.AddDefaultAWSOptions(Configuration.GetAWSOptions());
services.AddAWSService<IAmazonS3>();

It will automatically take care of the permissions for you.

For local development, you probably want to use a credentials file. In your appsettings.Local.json you would have an setting similar to this:

"AWS": {
  "Profile": "myprofilename",
  "Region": "eu-west-1",
  "ProfilesLocation": "C:\\Credentials.txt"
}

You might want to store the credentials file outside of your project so you don't accidentally check it in to source control.

Credentials.txt would look like:

[myprofilename]
aws_access_key_id=MY_ACCESS_KEY
aws_secret_access_key=MY_ACCESS_SECRET

Setting the Environment

You probably want the code sitting on the server to be identical for each environment - it makes deployment and other tasks so much easier. You can achieve this by using the Parameter Store to store all your configuration for each AWS environment.

The way I go about this is to use a "tag" on the EC2 instance to specify the environment name. I then use the tag to get correct configuration from the Parameter Store.

In your case, the tag would either be environment=development or environment=production

The parameter names/keys in the store should match the property names in your JSON appsettings file that you want to override.

They would look similar to:

/development/ConnectionStrings/Database
/development/MySettingGroup/MySetting
/production/ConnectionStrings/Database
/production/MySettingGroup/MySetting

I've added some code to github that checks for the tag and parameters etc - if it's running locally it defaults to an environment name of "LocalDevelopment" (that's the convention I use - so you would need to change that to "Local") and loads the correct appsettings file.

https://github.com/secretorange/aws-aspnetcore-environment-startup

The files you would need to use in your project are here:

https://github.com/secretorange/aws-aspnetcore-environment-startup/tree/master/AWSBoot/Boot

Using the BootHelper your Startup code would look similar to this:

public static IWebHost BuildWebHost()
{
    // ===================================
    // Get the boot config from the server
    // ===================================
    var bootConfig = Task.Run(() => BootHelper.GetConfig()).Result;

    var webHost = new WebHostBuilder()
        .UseKestrel()
        .UseContentRoot(Directory.GetCurrentDirectory())
        .ConfigureAppConfiguration((context, config) =>
        {
            // !!! IMPORTANT !!!
            // Set the environment from boot config
            context.HostingEnvironment.EnvironmentName = bootConfig.Environment;

            config.AddJsonFile("appsettings.json", optional: true)
                    .AddJsonFile($"appsettings.{context.HostingEnvironment.EnvironmentName}.json", optional: true);

            // !!! IMPORTANT !!!
            // If there are any parameters from the server
            // then we'll use them to override anything in the JSON files
            config.AddInMemoryCollection(bootConfig.Parameters);
        })
        .UseIISIntegration()
        .UseStartup<Startup>()
        .Build();

    return webHost;
}

The IAM Role(s) in AWS will need policies attached to grant access to tags and parameters etc. They will look similar to:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
              "ec2:DescribeInstances",
              "tag:GetResources",
              "tag:GetTagValues",
              "tag:GetTagKeys"
          ],
          "Resource": "*"
      }
  ]
}


{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": "ssm:GetParametersByPath",
          "Resource": [
              "arn:aws:ssm:YOUR_ARN_HERE:parameter/development",
              "arn:aws:ssm:YOUR_ARN_HERE:parameter/development/*"
          ]
      }
  ]
}

It might seem like a bit of a hassle to setup - but once it's up an running it means you can easily keep all your secrets out of source control and also create a new environment (Staging for example) simply by creating a new tag and parameters. No code changes required.

If you prefer, you can keep some config in appsettings.Production.json - it just means that if you want to create a new environment, you'll need to create a new JSON file and deploy new code etc. It's perhaps a bit cleaner if all environment information is in AWS (i.e. Parameter Store).