How to get the Aws cloudwatch logs using java

Question

We are working on AWS Lambda and Cloudwatch logs. Now, I want fetch all the log events from the Cloudwatch logs without using logStreamName by Java.

Since we are generating the logstream in dynamic way, am not sure how to fetch all the logs from the Cloudwatch log group.

I know, If we have logstream name, then we can use the below code

ClientConfiguration clientConfig = getClientConfig();

AWSLogsClientBuilder builder = AWSLogsClientBuilder.standard();

AWSLogs logsClient= builder.withCredentials(new AWSStaticCredentialsProvider(new ProfileCredentialsProvider(profile).getCredentials())).withRegion(Regions.AP_SOUTHEAST_2).withClientConfiguration(clientConfig).build();

GetLogEventsRequest request = new GetLogEventsRequest()
                        .withStartTime(1531231200000L)
                        .withEndTime(1531576800000L)
                        .withLogGroupName("FlowLogs_GroupName")
                        .withLogStreamName("eni-xxxxx");

GetLogEventsResult result = logsClient.getLogEvents(request);

result.getEvents().forEach(outputLogEvent -> {
          System.out.println(outputLogEvent.getMessage());
}); 

Since am new to this AWS, can anyone please help me with some code samples?

Answer 1

You can use DescribeLogStreamsRequest to get log stream name. Hope this would help

  public static void main( String[] args )
        {
            ClientConfiguration clientConfig = new ClientConfiguration();

            AWSLogsClientBuilder builder = AWSLogsClientBuilder.standard();

            AWSLogs logsClient = builder.withCredentials( new AWSStaticCredentialsProvider( new ProfileCredentialsProvider().getCredentials() ) )
                    .withRegion( Regions.AP_SOUTHEAST_2 )
                    .withClientConfiguration( clientConfig ).build();

            DescribeLogStreamsRequest describeLogStreamsRequest = new DescribeLogStreamsRequest().withLogGroupName( "FlowLogs_GroupName"  );
            DescribeLogStreamsResult describeLogStreamsResult = logsClient.describeLogStreams( describeLogStreamsRequest );

            for ( LogStream logStream : describeLogStreamsResult.getLogStreams() )
            {
                GetLogEventsRequest getLogEventsRequest = new GetLogEventsRequest()
                        .withStartTime( 1531231200000L )
                        .withEndTime( 1531576800000L )
                        .withLogGroupName( "FlowLogs_GroupName" )
                        .withLogStreamName( logStream.getLogStreamName() );

                GetLogEventsResult result = logsClient.getLogEvents( getLogEventsRequest );

                result.getEvents().forEach( outputLogEvent -> {
                    System.out.println( outputLogEvent.getMessage() );
                } );

            }
        }

Answer 2

In addition to https://stackoverflow.com/a/54074465/11090297

If you need filtered logs Stream data then you can use FilterLogEventsRequest

Here is an example of it:

 {
    ClientConfiguration clientConfig = new ClientConfiguration();


  AWSLogsClientBuilder builder = AWSLogsClientBuilder.standard();

    AWSLogs logsClient = builder.withCredentials((new ClasspathPropertiesFileCredentialsProvider("aws.properties")))
        .withRegion(Regions.US_EAST_1)
        .withClientConfiguration(clientConfig).build();

    DescribeLogStreamsRequest describeLogStreamsRequest = new DescribeLogStreamsRequest().withLogGroupName("/aws/audit");
    DescribeLogStreamsResult describeLogStreamsResult = logsClient.describeLogStreams(describeLogStreamsRequest);

    for (LogStream logStream : describeLogStreamsResult.getLogStreams()) {

// Add FilterPattern which will only fetch logs required
      FilterLogEventsRequest filterLogEventsRequest =  new FilterLogEventsRequest().withLogGroupName("/aws/audit")
         .withLogStreamNames(Arrays.asList(logStream.getLogStreamName())).withFilterPattern("vayuj");
      FilterLogEventsResult result = logsClient.filterLogEvents(filterLogEventsRequest);

      result.getEvents().forEach(outputLogEvent -> {
        System.out.println(outputLogEvent.getMessage());
      });

    }
  }