How to Encrypt and upload data using selling-partner-api in Amzon using .net

Question

I want to use selling-partner-api-docs for .Net . I found one reference from below url but that is use Java example coding : https://github.com/amzn/selling-partner-api-docs/blob/main/guides/use-case-guides/feeds-api-use-case-guide-2020-09-04.md#step-2-encrypt-and-upload-the-feed-data

But i want to use .Net coding can any one suggest of .Net coding of below java coding part

    import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.nio.charset.StandardCharsets;

import com.amazon.spapi.documents.UploadHelper;
import com.amazon.spapi.documents.UploadSpecification;
import com.amazon.spapi.documents.exception.CryptoException;
import com.amazon.spapi.documents.exception.HttpResponseException;
import com.amazon.spapi.documents.impl.AESCryptoStreamFactory;

/* We want to maintain encryption at rest, so do not write unencrypted data to disk.  This is bad:
InputStream source = new FileInputStream(new File("/path/to/myFeed.xml"));

Instead, if your data can fit in memory, you can create an InputStream from a String (see encryptAndUpload_fromString()).
Otherwise, you can pipe data into an InputStream using Piped streams (see encryptAndUpload_fromPipedInputStream()).
 */
public class UploadExample {
  private final UploadHelper uploadHelper = new UploadHelper.Builder().build();

  // key, initializationVector, and url are returned by the createFeedDocument operation.
  public void encryptAndUpload_fromString(String key, String initializationVector, String url) {
    AESCryptoStreamFactory aesCryptoStreamFactory =
      new AESCryptoStreamFactory.Builder(key, initializationVector)
      .build();

    // This contentType must be the same value that was provided to createFeedDocument.
    String contentType = String.format("text/plain; charset=%s", StandardCharsets.UTF_8);

    // The character set must be the same one that is specified in contentType.
    try
      (InputStream source = new ByteArrayInputStream("my feed data".getBytes(StandardCharsets.UTF_8))) {
        UploadSpecification uploadSpec =
          new UploadSpecification.Builder(contentType, aesCryptoStreamFactory, source, url)
          .build();

        uploadHelper.upload(uploadSpec);
      }
    catch (CryptoException | HttpResponseException | IOException e) {
      // Handle exception.
    }
  }

  // key, initializationVector, and url are returned from createFeedDocument.
  public void encryptAndUpload_fromPipedInputStream(String key, String initializationVector, String url) {
    AESCryptoStreamFactory aesCryptoStreamFactory =
      new AESCryptoStreamFactory.Builder(key, initializationVector)
      .build();

    // This contentType must be the same value that was provided to createFeedDocument.
    String contentType = String.format("text/plain; charset=%s", StandardCharsets.UTF_8);

    try
      (PipedInputStream source = new PipedInputStream()) {
        new Thread(
          new Runnable() {
          public void run() {
            try
              (PipedOutputStream feedContents = new PipedOutputStream(source)) {
                // The character set must be the same one that is specified in contentType.
                feedContents.write("my feed data\n".getBytes(StandardCharsets.UTF_8));
                feedContents.write("more feed data".getBytes(StandardCharsets.UTF_8));
              }
            catch (IOException e) {
              // Handle exception.
            }
          }
        }).start();

        UploadSpecification uploadSpec =
          new UploadSpecification.Builder(contentType, aesCryptoStreamFactory, source, url)
          .build();

        uploadHelper.upload(uploadSpec);
      }
    catch (CryptoException | HttpResponseException | IOException e) {
    
    }
  }
}

Edit -----------------------------------------------------

This is what I have tried and this is what I have got.

STEP 1

REQUEST URL: https://sellingpartnerapi-na.amazon.com/feeds/2020-09-04/documents REQUEST BODY:{"contentType":"text/plain;charset=utf-8"} Request Headers = {Host: sellingpartnerapi-na.amazon.com x-amz-date: 20210203T120516Z Authorization: AWS4-HMAC-SHA256 Credential=XXXXXXXX/20210203/us-east-1/execute-api/aws4_request, SignedHeaders=host;x-amz-date, Signature=XXXX

The Credential and Signature are created on the partner portal.

RESPONSE STEP 1

{"payload": {"encryptionDetails":{"standard":"AES","initializationVector":"TTAVo5bUDNfuk7KPzgm+ow==", "key":"GrpKm3UIvxiM5xUTlzaCC9xJFORMX41chAKUk0G6Cbg="}, "feedDocumentId":"amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3", "url":"https://tortuga-prod-na.s3-external-1.amazonaws.com/%2FNinetyDays/amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210203T114111Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=300&X-Amz-Credential=AKIA5U6MO6RANYPNEUPL%2F20210203%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1fd8b69523c06d76664c22c4093be5e8adc187436f7119aa9d4b51302cc8ae84"}}

STEP 2: In step 2 I am using the URL coming from the first Step Response but it is not getting me result.

REQUEST URL:

https://tortuga-prod-na.s3-external-1.amazonaws.com/%2FNinetyDays/amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20210203T114111Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=300&X-Amz-Credential=AKIA5U6MO6RANYPNEUPL%2F20210203%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1fd8b69523c06d76664c22c4093be5e8adc187436f7119aa9d4b51302cc8ae84

See the Signature and the Credential here are coming different than one we have got from the response of Step 1

RESPONSE FROM STEP 2

<?xml version="1.0" encoding="UTF-8"?>

-<Error>

<Code>SignatureDoesNotMatch</Code>

<Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>

<AWSAccessKeyId>AKIA5U6MO6RANYPNEUPL</AWSAccessKeyId>

<StringToSign>AWS4-HMAC-SHA256 20210203T114111Z 20210203/us-east-1/s3/aws4_request 057d93b83f8254c64b8ffccdfb885b79e5d96c0d2045c27732fc42ae722e335e</StringToSign>

<SignatureProvided>1fd8b69523c06d76664c22c4093be5e8adc187436f7119aa9d4b51302cc8ae84</SignatureProvided>

<StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 32 31 30 32 30 33 54 31 31 34 31 31 31 5a 0a 32 30 32 31 30 32 30 33 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 30 35 37 64 39 33 62 38 33 66 38 32 35 34 63 36 34 62 38 66 66 63 63 64 66 62 38 38 35 62 37 39 65 35 64 39 36 63 30 64 32 30 34 35 63 32 37 37 33 32 66 63 34 32 61 65 37 32 32 65 33 33 35 65</StringToSignBytes>

<CanonicalRequest>PUT //NinetyDays/amzn1.tortuga.3.9968967c-048c-4e8b-a6c1-ffd764f005d4.T508PJ0OCPKJ3 X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA5U6MO6RANYPNEUPL%2F20210203%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210203T114111Z&X-Amz-Expires=300&X-Amz-SignedHeaders=content-type%3Bhost content-type:text/plain; charset=utf-8 host:tortuga-prod-na.s3-external-1.amazonaws.com content-type;host UNSIGNED-PAYLOAD</CanonicalRequest>

<CanonicalRequestBytes>50 55 54 0a 2f 2f 4e 69 6e 65 74 79 44 61 79 73 2f 61 6d 7a 6e 31 2e 74 6f 72 74 75 67 61 2e 33 2e 39 39 36 38 39 36 37 63 2d 30 34 38 63 2d 34 65 38 62 2d 61 36 63 31 2d 66 66 64 37 36 34 66 30 30 35 64 34 2e 54 35 30 38 50 4a 30 4f 43 50 4b 4a 33 0a 58 2d 41 6d 7a 2d 41 6c 67 6f 72 69 74 68 6d 3d 41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 26 58 2d 41 6d 7a 2d 43 72 65 64 65 6e 74 69 61 6c 3d 41 4b 49 41 35 55 36 4d 4f 36 52 41 4e 59 50 4e 45 55 50 4c 25 32 46 32 30 32 31 30 32 30 33 25 32 46 75 73 2d 65 61 73 74 2d 31 25 32 46 73 33 25 32 46 61 77 73 34 5f 72 65 71 75 65 73 74 26 58 2d 41 6d 7a 2d 44 61 74 65 3d 32 30 32 31 30 32 30 33 54 31 31 34 31 31 31 5a 26 58 2d 41 6d 7a 2d 45 78 70 69 72 65 73 3d 33 30 30 26 58 2d 41 6d 7a 2d 53 69 67 6e 65 64 48 65 61 64 65 72 73 3d 63 6f 6e 74 65 6e 74 2d 74 79 70 65 25 33 42 68 6f 73 74 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3a 74 65 78 74 2f 70 6c 61 69 6e 3b 20 63 68 61 72 73 65 74 3d 75 74 66 2d 38 0a 68 6f 73 74 3a 74 6f 72 74 75 67 61 2d 70 72 6f 64 2d 6e 61 2e 73 33 2d 65 78 74 65 72 6e 61 6c 2d 31 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 0a 0a 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3b 68 6f 73 74 0a 55 4e 53 49 47 4e 45 44 2d 50 41 59 4c 4f 41 44</CanonicalRequestBytes>

<RequestId>48A2CCE3EFA66E89</RequestId>

<HostId>hiZxZwoTgGG4PBvGLchnKV94AA57zzGqnHh5BbTCIAt1ubD47O+8uQMClkDDBoJBgiXgVb57TRE=</HostId>

</Error>

Answer 1

Hopefully this helps some c sharpers out there for Step #2 https://github.com/amzn/selling-partner-api-docs/blob/main/guides/use-case-guides/feeds-api-use-case-guide-2020-09-04.md#step-2-encrypt-and-upload-the-feed-data

Assuming you have received an OK response for Step #1

The response for Step #1 will look something like this:

{
  "payload":
  {
    "feedDocumentId":"amzn1.tortuga.3.920614b0-fc4c-4393-b0d9-fff175300000.T29XK4YL08B2VM",
    "url":"https://tortuga-prod-na.s3.amazonaws.com/%2FNinetyDays/amzn1.tortuga.3.920614b0-fc4c-4393-b0d9-fff175300000.T29XK4YL08B2VM?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20200919T035824Z&X-Amz-SignedHeaders=<headers>&X-Amz-Expires=300&X-Amz-Credential=<credential>&X-Amz-Signature=<signature>",
    "encryptionDetails":
    {
      "standard":"AES",
      "initializationVector":"kF3bZt0FSv6JQEimfEJD8g==",
      "key":"5EZo/P06OGF0UAy8QuOnMIaQbkAvYBru6EGsFvK8wJ2="
    }
}

Convert EncryptionDetails.Key and EncryptionDetails.InitializationVector to bytes and read your flat file (feed) into a string variable...

var key = Convert.FromBase64String(createFeedDocumentResponse.Payload.EncryptionDetails.Key);
var iv = Convert.FromBase64String(createFeedDocumentResponse.Payload.EncryptionDetails.InitializationVector);
string feedData = File.ReadAllText(@"C:\temp\AmazonFlatFileTest.txt");

Encrypt the feed using AES w/ the key and iv variables above...

Here's an encrypt and decrypt function I took from Microsoft and altered slightly (credit to https://docs.microsoft.com/en-us/dotnet/api/system.security.cryptography.aescryptoserviceprovider?view=net-5.0)...

private byte[] EncryptStringToBytes_Aes(string plainText, byte[] key, byte[] initializationVector)
{
  // Check arguments.
  if (plainText == null || plainText.Length <= 0)
    throw new ArgumentNullException("plainText");
  if (key == null || key.Length <= 0)
    throw new ArgumentNullException("Key");
  if (initializationVector == null || initializationVector.Length <= 0)
    throw new ArgumentNullException("initializationVector");
  byte[] encrypted;

  // Create an Aes object
  // with the specified key and IV.
  using (Aes aesAlg = Aes.Create())
  {
    aesAlg.Key = key;
    aesAlg.IV = initializationVector;

    // Create an encryptor to perform the stream transform.
    ICryptoTransform encryptor = aesAlg.CreateEncryptor(aesAlg.Key, aesAlg.IV);

    // Create the streams used for encryption.
    using (MemoryStream msEncrypt = new MemoryStream())
    {
      using (CryptoStream csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
      {
        using (StreamWriter swEncrypt = new StreamWriter(csEncrypt, Encoding.UTF8))
        {
          //Write all data to the stream.
          swEncrypt.Write(plainText);
        }
        encrypted = msEncrypt.ToArray();
      }
    }
  }

  // Return the encrypted bytes from the memory stream.
  return encrypted;
}

And the decrypt function... You will need this in Step #5 decrypt and process the results.

EDIT: It's noticed for larger inputs, Amazon will use GZIP compression with their response (Payload.CompressionAlgorithm). Included 'compressionAlgorithm' parameter to the DecryptStringFromBytes_Aes method.

private string DecryptStringFromBytes_Aes(byte[] cipherText, byte[] Key, byte[] initializationVector, string compressionAlgorithm)
{
  // Validate Compression Algorithm
  var isGzip = string.Equals(compressionAlgorithm, "GZIP", StringComparison.OrdinalIgnoreCase);
  var compressionAlgorithmValid = compressionAlgorithm == null || isGzip;

  if (!compressionAlgorithmValid)
  {
    throw new InvalidOperationException($"Unexpected CompressionAlgorithm encounted. compressionAlgorithm = {compressionAlgorithm}");
  }

  // Check arguments.
  if (cipherText == null || cipherText.Length <= 0)
    throw new ArgumentNullException("cipherText");
  if (Key == null || Key.Length <= 0)
    throw new ArgumentNullException("Key");
  if (initializationVector == null || initializationVector.Length <= 0)
    throw new ArgumentNullException("IV");

  // Declare the string used to hold
  // the decrypted text.
  string plaintext = null;

  // Create an Aes object
  // with the specified key and IV.
  using (Aes aesAlg = Aes.Create())
  {
    aesAlg.Key = Key;
    aesAlg.IV = initializationVector;

    // Create a decryptor to perform the stream transform.
    ICryptoTransform decryptor = aesAlg.CreateDecryptor(aesAlg.Key, aesAlg.IV);

    // Create the streams used for decryption.
    using (MemoryStream msDecrypt = new MemoryStream(cipherText))
    {
      using (CryptoStream csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read))
      {
        if (isGzip)
        {
          using (var decompressedFileStream = new MemoryStream())
          {

            using (GZipStream decompressionStream = new GZipStream(csDecrypt, CompressionMode.Decompress))
            {
              decompressionStream.CopyTo(decompressedFileStream);
              decompressedFileStream.Position = 0;

              using (var writer = new StreamReader(decompressedFileStream))
              {
                plaintext = writer.ReadToEnd();
              }
            }

          }
        }
        else
        {
          using (StreamReader srDecrypt = new StreamReader(csDecrypt, Encoding.UTF8))
          {
            // Read the decrypted bytes from the decrypting stream
            // and place them in a string.
            plaintext = srDecrypt.ReadToEnd();
          }
        }
      }
    }
  }

  return plaintext;
}

With the returned result from EncryptStringToBytes_Aes, you can now upload to S3 url (provided to you in Payload from Step #1). Here is an example using the library Rest Sharp (which some Amazon employee seems to like judging by the reference in Amazon's C# client example).

NB: The Content Type must match the content type in your CreateFeedDocument request (Step #1).

Here is a function you can use to upload to S3...

private async Task UploadFile(byte[] bytes, string url)
{
  var contentType = "text/plain; charset=utf-8"; // this should be the same as what was used in Step #1 (in the CreateFeedDocument API request)

  RestClient restClient = new RestClient(url);
  IRestRequest restRequest = new RestRequest(Method.PUT);
  restRequest.AddParameter(contentType, bytes, ParameterType.RequestBody);

  var response = await restClient.ExecuteAsync(restRequest);

  if (!response.IsSuccessful)
  {
    // your error logic
  }

  // success. Move to Step #3
}

Happy days...