How to mock - reading file from s3

Question

I am new to writing unit tests. I am trying to read a JSON file stored in S3 and I am getting an "Argument passed to when() is not a mock!" and "profile file cannot be null" error.

This is what I have tried so far Retrieving Object Using JAVA:

private void amazonS3Read() {
    String clientRegion = "us-east-1";
    String bucketName = "version";
    String key = "version.txt";
    S3Object fullObject = null;
    try {
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
            .withRegion(clientRegion)
            .withCredentials(new ProfileCredentialsProvider())
            .build();
        fullObject = s3Client.getObject(new GetObjectRequest(bucketName, key));
        S3ObjectInputStream s3is = fullObject.getObjectContent();
        json = returnStringFromInputStream(s3is);
        fullObject.close();
        s3is.close();
    } catch (AmazonServiceException e) {
        // The call was transmitted successfully, but Amazon S3 couldn't process
        // it, so it returned an error response.
        e.printStackTrace();
    } catch (SdkClientException e) {
        // Amazon S3 couldn't be contacted for a response, or the client
        // couldn't parse the response from Amazon S3.
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }
    //Do some operations with the data
}

Test File

 @Test
 public void amazonS3ReadTest() throws Exception {
     String bucket = "version";
     String keyName = "version.json";
     InputStream inputStream = null;
     S3Object s3Object = Mockito.mock(S3Object.class);
     GetObjectRequest getObjectRequest = Mockito.mock(GetObjectRequest.class);

     getObjectRequest = new GetObjectRequest(bucket, keyName);

     AmazonS3 client = Mockito.mock(AmazonS3.class);
     Mockito.doNothing().when(AmazonS3ClientBuilder.standard());
     client = AmazonS3ClientBuilder.standard()
         .withRegion(clientRegion)
         .withCredentials(new ProfileCredentialsProvider())
         .build();

     Mockito.doReturn(s3Object).when(client).getObject(getObjectRequest);
     s3Object = client.getObject(getObjectRequest);

     Mockito.doReturn(inputStream).when(s3Object).getObjectContent();
     inputStream = s3Object.getObjectContent();
     //performing other operations
 }

Getting two different exceptions:

Argument passed to when() is not a mock! Example of correct stubbing: doThrow(new RuntimeException()).when(mock).someMethod();

org.mockito.exceptions.misusing.NotAMockException: 
Argument passed to when() is not a mock!
Example of correct stubbing: 

OR

profile file cannot be null

java.lang.IllegalArgumentException: profile file cannot be null
at com.amazonaws.util.ValidationUtils.assertNotNull(ValidationUtils.java:37)
at com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:142)
at com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:133)
at com.amazonaws.auth.profile.ProfilesConfigFile.<init>(ProfilesConfigFile.java:100)
at com.amazonaws.auth.profile.ProfileCredentialsProvider.getCredentials(ProfileCredentialsProvider.java:135)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1184)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.runBeforeRequestHandlers(AmazonHttpClient.java:774)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:726)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4443)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4390)
at com.amazonaws.services.s3.AmazonS3Client.getObject(AmazonS3Client.java:1427)

What am I doing wrong and how do I fix this?

Answer 1

Your approach looks wrong.

• You want to mock dependencies and invocations of a private method :amazonS3Read() and you seem to want to unit test that method.
  We don't unit test private methods of a class but we test the class from its API (application programming interface), that is public/protected method.
• Your unit test is a series of mock recording: most of it is a description via Mockito of what your private method does. I have even a hard time to identify the no mocked part.... What do you assert here ? That you invoke 4 methods on some mocks ? Unfortunately, it asserts nothing in terms of result/behavior. You can add incorrect invocations between the invoked methods and the test will stay green because you don't test a result that you can assert with the assertEquals(...) idiom.
  It doesn't mean that mocking a method is not acceptable but when your test is mainly mocking, something is wrong and we can trust in its result.

I would advise you two things :

• write an unit test that focuses on asserting the logic that you performed : computation/transformation/transmitted value and so for... don't focus on chaining methods.

• write some integration tests with some light and simple S3 compatible servers that will give you a real feedback in terms of behavior assertion. Side effects may be tested in this way.
  You have for example Riak, MinIo or still Localstack.

---

To be more concrete, here is a refactor approach to improve things.
If the amazonS3Read() private method has to be unitary tested, you should probably move it into a specific class for example MyAwsClient and make it a public method.

Then the idea is to make amazonS3Read() as clear as possible in terms of responsibility.
Its logic could be summarized such as :

1) Get some identifier information to pass to the S3 services.
Which means defined a method with the parameters :

public Result amazonS3Read(String clientRegion, String bucketName, String key) {...}

2) Apply all fine grained S3 functions to get the S3ObjectInputStream object.
We could gather all of these in a specific method of a class AmazonS3Facade :

S3ObjectInputStream s3is = amazonS3Facade.getObjectContent(clientRegion, bucketName, key);

3) Do your logic that is process the returned S3ObjectInputStream and return a result

json = returnStringFromInputStream(s3is); 
// ...   
return result;

How to test that now ?

Simply enough.
With JUnit 5 :

@ExtendWith(MockitoExtension.class)
public MyAwsClientTest{

    MyAwsClient myAwsClient;

    @Mock 
    AmazonS3Facade amazonS3FacadeMock;        

    @Before
    void before(){
        myAwsClient = new MyAwsClient(amazonS3FacadeMock);
    }

    @Test
    void amazonS3Read(){

        // given
        String clientRegion = "us-east-1";
        String bucketName = "version";
        String key = "version.txt";

       S3ObjectInputStream s3IsFromMock = ... // provide a stream with a real content. We rely on it to perform the assertion
       Mockito.when(amazonS3FacadeMock.getObjectContent(clientRegion, bucketName, key))
              .thenReturn(s3IsFromMock);

       // when    
       Result result = myAwsClient.amazonS3Read(clientRegion, bucketName, key);

      // assert result content.
      Assertions.assertEquals(...);
    }
}

What are the advantages ?

• the class implementation is readable and maintainable because it focuses on your functional processing.
• the whole S3 logic was moved into a single place AmazonS3Facade (Single Responsibility principle/modularity).
• thanks to that, the test implementation is now readable and maintainable
• the test tests really the logic that you perform (instead of verifying a series of invocations on multiple mocks).

Note that unitary testing AmazonS3Facade has few/no value since that is only a series of invocation to S3 components, impossible to assert in terms of returned result and so very brittle.
But writing an integration test for that with a simple and lightweight S3 compatible server as these quoted early makes really sense.