Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Too many LOGS getting generated for Hystrix-AMQP

so I added the dependency for Hystrix-AMQP to my service and the log file is going crazy it just keep on logging metrics stuff. I need that jar to actually use it with turbine-AMQP.

here is what i have in my gradle for hystrix:-

compile         ("org.springframework.cloud:spring-cloud-starter-hystrix:1.0.6.RELEASE")
compile         ('org.springframework.cloud:spring-cloud-starter-bus-amqp:1.0.6.RELEASE')
compile         ('org.springframework.cloud:spring-cloud-netflix-hystrix-amqp:1.0.7.RELEASE')
compile         ('com.netflix.hystrix:hystrix-javanica:1.5.2')

This is what keeps on generating in my logs it just keeps on running:-

   2016-05-03 13:49:14.698 INFO [LogMessage=Starting span: MilliSpan(begin=1462308554698, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=21825112-0c71-4c6a-a9ca-51b11a21e4e5, parents=[], spanId=053946b5-7287-41f4-8579-d048655f41ea, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:14.698 INFO  [LogMessage=Continued span: MilliSpan(begin=1462308554698, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=21825112-0c71-4c6a-a9ca-51b11a21e4e5, parents=[], spanId=053946b5-7287-41f4-8579-d048655f41ea, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:14.698 INFO  [LogMessage=Stopped span: MilliSpan(begin=1462308554698, end=1462308554698, name=execution(HystrixStreamTask.sendMetrics()), traceId=21825112-0c71-4c6a-a9ca-51b11a21e4e5, parents=[], spanId=053946b5-7287-41f4-8579-d048655f41ea, remote=false, annotations={/messaging/headers/id=e1cc5042-1a5c-e3f9-6f3c-de936d1aa959, /messaging/headers/timestamp=1462308554698, /messaging/payload/type=java.lang.String, /messaging/payload/size=592}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:14.698 INFO  [LogMessage=Starting span: MilliSpan(begin=1462308554698, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=6cc342bb-9693-493a-8fa8-8a17c2ff06c3, parents=[], spanId=10cdee69-22f8-43ab-883f-3e09b29ab6fb, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:14.699 INFO [LogMessage=Continued span: MilliSpan(begin=1462308554698, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=6cc342bb-9693-493a-8fa8-8a17c2ff06c3, parents=[], spanId=10cdee69-22f8-43ab-883f-3e09b29ab6fb, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:14.699 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308554698, end=1462308554699, name=execution(HystrixStreamTask.gatherMetrics()), traceId=6cc342bb-9693-493a-8fa8-8a17c2ff06c3, parents=[], spanId=10cdee69-22f8-43ab-883f-3e09b29ab6fb, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.199 INFO [LogMessage=Starting span: MilliSpan(begin=1462308555199, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=4c5fd89d-f5d2-45bf-a727-32a3e1b57c35, parents=[], spanId=c7e254b5-63de-4a00-9d91-1584edd9650c, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.199 INFO [LogMessage=Starting span: MilliSpan(begin=1462308555199, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=896e5143-fa67-4fd1-b97e-517282780e94, parents=[], spanId=ee1d4f95-30c1-4784-8366-58c3a73c3565, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.199 INFO [LogMessage=Continued span: MilliSpan(begin=1462308555199, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=4c5fd89d-f5d2-45bf-a727-32a3e1b57c35, parents=[], spanId=c7e254b5-63de-4a00-9d91-1584edd9650c, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.199 INFO [LogMessage=Continued span: MilliSpan(begin=1462308555199, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=896e5143-fa67-4fd1-b97e-517282780e94, parents=[], spanId=ee1d4f95-30c1-4784-8366-58c3a73c3565, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.199 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308555199, end=1462308555199, name=execution(HystrixStreamTask.gatherMetrics()), traceId=896e5143-fa67-4fd1-b97e-517282780e94, parents=[], spanId=ee1d4f95-30c1-4784-8366-58c3a73c3565, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.200 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308555199, end=1462308555200, name=execution(HystrixStreamTask.sendMetrics()), traceId=4c5fd89d-f5d2-45bf-a727-32a3e1b57c35, parents=[], spanId=c7e254b5-63de-4a00-9d91-1584edd9650c, remote=false, annotations={/messaging/headers/id=4cd7f1c7-0aba-63c3-fddd-aeb5ea9b21e5, /messaging/headers/timestamp=1462308555200, /messaging/payload/type=java.lang.String, /messaging/payload/size=592}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.698 INFO [LogMessage=Starting span: MilliSpan(begin=1462308555698, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=eb214d13-6e16-4b76-abd7-1217e4932e93, parents=[], spanId=4d9790b0-745a-46a9-99b8-f5d23a570113, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.698 INFO [LogMessage=Continued span: MilliSpan(begin=1462308555698, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=eb214d13-6e16-4b76-abd7-1217e4932e93, parents=[], spanId=4d9790b0-745a-46a9-99b8-f5d23a570113, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.699 INFO [LogMessage=Starting span: MilliSpan(begin=1462308555699, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=8651f8c4-6de8-4a6a-a28e-7d248d6a6ac3, parents=[], spanId=3fa8d40b-e14f-4209-a10b-adc47d03cccd, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.699 INFO [LogMessage=Continued span: MilliSpan(begin=1462308555699, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=8651f8c4-6de8-4a6a-a28e-7d248d6a6ac3, parents=[], spanId=3fa8d40b-e14f-4209-a10b-adc47d03cccd, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.699 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308555698, end=1462308555699, name=execution(HystrixStreamTask.sendMetrics()), traceId=eb214d13-6e16-4b76-abd7-1217e4932e93, parents=[], spanId=4d9790b0-745a-46a9-99b8-f5d23a570113, remote=false, annotations={/messaging/headers/id=a59cf028-1b92-1a67-c4f2-651fb0f488e0, /messaging/headers/timestamp=1462308555699, /messaging/payload/type=java.lang.String, /messaging/payload/size=592}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:15.700 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308555699, end=1462308555700, name=execution(HystrixStreamTask.gatherMetrics()), traceId=8651f8c4-6de8-4a6a-a28e-7d248d6a6ac3, parents=[], spanId=3fa8d40b-e14f-4209-a10b-adc47d03cccd, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.197 INFO [LogMessage=Starting span: MilliSpan(begin=1462308556197, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=9599a694-1d0d-40c1-82d5-9c4af32d71d2, parents=[], spanId=0e2b6691-a5ef-40f2-b9ff-00203075f1f0, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.197 INFO [LogMessage=Continued span: MilliSpan(begin=1462308556197, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=9599a694-1d0d-40c1-82d5-9c4af32d71d2, parents=[], spanId=0e2b6691-a5ef-40f2-b9ff-00203075f1f0, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.198 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308556197, end=1462308556198, name=execution(HystrixStreamTask.sendMetrics()), traceId=9599a694-1d0d-40c1-82d5-9c4af32d71d2, parents=[], spanId=0e2b6691-a5ef-40f2-b9ff-00203075f1f0, remote=false, annotations={/messaging/headers/id=78c358eb-547c-9c3f-9b4c-2cd971da2198, /messaging/headers/timestamp=1462308556198, /messaging/payload/type=java.lang.String, /messaging/payload/size=592}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.199 INFO [LogMessage=Starting span: MilliSpan(begin=1462308556198, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=bed72154-51a5-4637-b0a8-7b1e5ba352f3, parents=[], spanId=107b9f6d-e97c-4f67-a196-8725293dc1f4, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.199 INFO [LogMessage=Continued span: MilliSpan(begin=1462308556198, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=bed72154-51a5-4637-b0a8-7b1e5ba352f3, parents=[], spanId=107b9f6d-e97c-4f67-a196-8725293dc1f4, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.199 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308556198, end=1462308556199, name=execution(HystrixStreamTask.gatherMetrics()), traceId=bed72154-51a5-4637-b0a8-7b1e5ba352f3, parents=[], spanId=107b9f6d-e97c-4f67-a196-8725293dc1f4, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.698 INFO [LogMessage=Starting span: MilliSpan(begin=1462308556698, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=0d5c02ca-91ad-432a-8b6b-ed460e7fbe20, parents=[], spanId=fbf0674d-2610-455a-8cd8-d3602ee50d55, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.698 INFO [LogMessage=Starting span: MilliSpan(begin=1462308556698, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=fec97646-fd36-40f2-8708-0c34b96fb32d, parents=[], spanId=12442026-67da-4f8b-9584-8d1fa188b88c, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.698 INFO [LogMessage=Continued span: MilliSpan(begin=1462308556698, end=0, name=execution(HystrixStreamTask.sendMetrics()), traceId=0d5c02ca-91ad-432a-8b6b-ed460e7fbe20, parents=[], spanId=fbf0674d-2610-455a-8cd8-d3602ee50d55, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.698 INFO [LogMessage=Continued span: MilliSpan(begin=1462308556698, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=fec97646-fd36-40f2-8708-0c34b96fb32d, parents=[], spanId=12442026-67da-4f8b-9584-8d1fa188b88c, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.698 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308556698, end=1462308556698, name=execution(HystrixStreamTask.gatherMetrics()), traceId=fec97646-fd36-40f2-8708-0c34b96fb32d, parents=[], spanId=12442026-67da-4f8b-9584-8d1fa188b88c, remote=false, annotations={}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:16.699 INFO [LogMessage=Stopped span: MilliSpan(begin=1462308556698, end=1462308556699, name=execution(HystrixStreamTask.sendMetrics()), traceId=0d5c02ca-91ad-432a-8b6b-ed460e7fbe20, parents=[], spanId=fbf0674d-2610-455a-8cd8-d3602ee50d55, remote=false, annotations={/messaging/headers/id=92af5087-ff24-e8b9-94dc-863b1dd17ba2, /messaging/headers/timestamp=1462308556699, /messaging/payload/type=java.lang.String, /messaging/payload/size=592}, processId=null, timelineAnnotations=[])]
2016-05-03 13:49:17.198 INFO [LogMessage=Starting span: MilliSpan(begin=1462308557198, end=0, name=execution(HystrixStreamTask.gatherMetrics()), traceId=e389098b-c606-478f-9b55-b8a7c4cceed5, parents=[], spanId=8fbc4ffd-6e42-4489-baa2-69d42a562c96, remote=false, annotations={}, processId=null, timelineAnnotations=[])]

Here is what i have in my application.yml

hystrix: 
  command.default.execution.isolation.strategy: SEMAPHORE
  command.default.execution.isolation.thread.timeoutInMilliseconds: 60000
  command.default.execution.timeout.enabled: false
  command.default.fallback.enabled: false
  threadpool.default.coreSize: 20

How can i stop the service from generating all the logs repetitively. Is there some sort of property that I can set in application.yml or if there is anything i can do.

UPDATE:

It looks like the issue is caused because of sleuth based on the issue here- https://github.com/spring-cloud/spring-cloud-sleuth/issues/49

I added spring.sleuth.log.slf4j.enabled: false but the problem it removes all the trace iDs and span ids so adding this does fix the issue but my application logs will be messed up.

Any suggestions.

like image 388
Grinish Nepal Avatar asked May 03 '16 21:05

Grinish Nepal


2 Answers

Check out the latest version of Sleuth - now this problem should not be present anymore.

like image 94
Marcin Grzejszczak Avatar answered Nov 15 '22 14:11

Marcin Grzejszczak


Try also disabling spring.sleuth.integration:

spring.sleuth.integration.enabled=false
like image 30
Tomasz Dzieniak Avatar answered Nov 15 '22 14:11

Tomasz Dzieniak