Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Where does this ClientAbortException come from?

I have a Spring-Boot application, which constantly shows this error in its logs:

[xec-104] ERROR Exception Processing ErrorPage[errorCode=0, location=/error] org.apache.catalina.connector.ClientAbortException: java.io.IOException: Connection reset by peer at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:340) at org.apache.catalina.connector.OutputBuffer.flush(OutputBuffer.java:303) at org.apache.catalina.connector.CoyoteOutputStream.flush(CoyoteOutputStream.j ava:109) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.flush(UTF8JsonGenerator.j ava:1048) at com.fasterxml.jackson.databind.ObjectWriter.writeValue(ObjectWriter.java:95 3) at org.springframework.http.converter.json.AbstractJackson2HttpMessageConverte r.writeInternal(AbstractJackson2HttpMessageConverter.java:285) at org.springframework.http.converter.AbstractGenericHttpMessageConverter.writ e(AbstractGenericHttpMessageConverter.java:100) at org.springframework.web.servlet.mvc.method.annotation.AbstractMessageConver terMethodProcessor.writeWithMessageConverters(AbstractMessageConverterMethod Processor.java:231) at org.springframework.web.servlet.mvc.method.annotation.HttpEntityMethodProce ssor.handleReturnValue(HttpEntityMethodProcessor.java:203) at org.springframework.web.method.support.HandlerMethodReturnValueHandlerCompo site.handleReturnValue(HandlerMethodReturnValueHandlerComposite.java:81) at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandl erMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:132) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandler Adapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandler Adapter.handleInternal(RequestMappingHandlerAdapter.java:738) at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.han dle(AbstractHandlerMethodAdapter.java:85) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServ let.java:963) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServl et.java:897) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkSe rvlet.java:970) at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.jav a:861) at javax.servlet.http.HttpServlet.service(HttpServlet.java:622) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.j ava:846) at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applicatio nFilterChain.java:230) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterC hain.java:165) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher .java:726) at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDi spatcher.java:471) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatc her.java:394) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatche r.java:311) at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:39 5) at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:25 4) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:17 7) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79 ) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.jav a:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:784) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.jav a:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtoc ol.java:802) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.ja va:1410) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java :49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1 142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java: 617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.j ava:61) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93) at sun.nio.ch.IOUtil.write(IOUtil.java:65) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471) at org.apache.tomcat.util.net.NioChannel.write(NioChannel.java:134) at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.ja va:101) at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:157) at org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper.doWrite(NioEndpoint .java:1221) at org.apache.tomcat.util.net.SocketWrapperBase.flushBlocking(SocketWrapperBas e.java:451) at org.apache.tomcat.util.net.SocketWrapperBase.flush(SocketWrapperBase.java:4 41) at org.apache.coyote.http11.Http11OutputBuffer.flushBuffer(Http11OutputBuffer. java:514) at org.apache.coyote.http11.Http11OutputBuffer.flush(Http11OutputBuffer.java:2 43) at org.apache.coyote.http11.Http11Processor.flush(Http11Processor.java:1495) at org.apache.coyote.AbstractProcessor.action(AbstractProcessor.java:284) at org.apache.coyote.Response.action(Response.java:167) at org.apache.catalina.connector.OutputBuffer.doFlush(OutputBuffer.java:336) ... 41 common frames omitted

As you can see, the stack trace does not reveal anything from my application. This exception happens purely in the infrastructure. What does it mean and how can I reproduce that? Also, I don't get why there's Jackson in the stack trace. I have no where any JSON output in my application.

At the end I want to understand if my application has an issue. If so, fix that. If not, I don't want to have this in my log as an ERROR.

Edit: As requested, this is the pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>net.mixclouddownload</groupId>
	<artifactId>downloader</artifactId>
	<version>0.0.1-SNAPSHOT</version>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>1.4.3.RELEASE</version>
	</parent>

	<properties>
		<java-version>1.8</java-version>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-actuator</artifactId>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-devtools</artifactId>
			<optional>true</optional>
		</dependency>
		<dependency>
			<groupId>org.springframework.retry</groupId>
			<artifactId>spring-retry</artifactId>
		</dependency>
		<dependency>
			<groupId>org.aspectj</groupId>
			<artifactId>aspectjweaver</artifactId>
		</dependency>
		<dependency>
			<groupId>net.bull.javamelody</groupId>
			<artifactId>javamelody-core</artifactId>
			<version>1.63.0</version>
		</dependency>
		<dependency>
			<groupId>net.sf.jtidy</groupId>
			<artifactId>jtidy</artifactId>
			<version>r938</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-thymeleaf</artifactId>
		</dependency>
		<dependency>
			<groupId>org.webjars</groupId>
			<artifactId>bootstrap</artifactId>
			<version>4.0.0-alpha.6</version>
		</dependency>
		<dependency>
			<groupId>org.webjars</groupId>
			<artifactId>font-awesome</artifactId>
			<version>4.7.0</version>
		</dependency>
		<dependency>
			<groupId>commons-io</groupId>
			<artifactId>commons-io</artifactId>
			<version>2.5</version>
		</dependency>
		<dependency>
			<groupId>com.ibm.icu</groupId>
			<artifactId>icu4j</artifactId>
			<version>58.2</version>
		</dependency>
		<dependency>
			<groupId>org.webjars.bower</groupId>
			<artifactId>bootstrap-social</artifactId>
			<version>5.0.0</version>
		</dependency>
		<dependency>
			<groupId>org.apache.httpcomponents</groupId>
			<artifactId>httpclient</artifactId>
		</dependency>
		<dependency>
			<groupId>org.twitter4j</groupId>
			<artifactId>twitter4j-core</artifactId>
			<version>4.0.6</version>
		</dependency>
		<dependency>
			<groupId>org.facebook4j</groupId>
			<artifactId>facebook4j-core</artifactId>
			<version>2.4.8</version>
		</dependency>
		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
			<version>21.0</version>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-cache</artifactId>
		</dependency>
		<dependency>
			<groupId>javax.cache</groupId>
			<artifactId>cache-api</artifactId>
		</dependency>
		<dependency>
			<groupId>ca.pjer</groupId>
			<artifactId>logback-awslogs-appender</artifactId>
			<version>0.1.0</version>
		</dependency>
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
			<version>3.5</version>
		</dependency>

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.mockito</groupId>
			<artifactId>mockito-core</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
		</dependency>
	</dependencies>
	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-compiler-plugin</artifactId>
				<configuration>
					<source>${java-version}</source>
					<target>${java-version}</target>
					<compilerArgument>-Xlint:all</compilerArgument>
					<showWarnings>true</showWarnings>
					<showDeprecation>true</showDeprecation>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
				<configuration>
					<executable>true</executable>
				</configuration>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-failsafe-plugin</artifactId>
				<executions>
					<execution>
						<goals>
							<goal>integration-test</goal>
							<goal>verify</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>
like image 463
Jessy James Avatar asked Jan 19 '17 22:01

Jessy James


1 Answers

This is caused because the client is closing the connection before Tomcat has the chance to write out the response.

The main perpetrator of this exception tends to be "Internet Explorer" because of how it handles resource connections (like downloading JavaScript, css, icons etc); if it decides it no longer needs the resource, it closes the connection before the server has a chance to write out to it. I also see it more when you are using Angularjs, or other SPA frameworks, in IE. You will know it is caused by IE because the request resource path with be to a resource file (as described previously).

The second (or equally?) most common culprit could be users refreshing the page multiple times before it has a chance to finish loading, or the user navigating away before the page finishes loading.

The third culprit, and less common but WAY more infuriating, is when you are calling services on your own network. You will see a "ClientAbortException" on the called service and a "SocketException: connection reset" on the client service. The cause is normally because of badly configured firewalls. I talked with my Systems Administrator, and she said that the last time it happened to me it was because the replication traffic was too high during peak usage so the firewall could not return the packet in time. The solution was to throttle traffic from specific sources (such as replication) during peak usage of other essential applications. I suppose you could duplicate this error by simulating heavy usage and traffic through the firewall.

like image 51
Pytry Avatar answered Nov 03 '22 19:11

Pytry