HttpClient not Serializable Exception

Question

I am trying to implement a basic 1 spout - 1 bolt Storm Topology. I have a Storm Bolt to make an HTTP Request using the Apache HttpClient (4.3.1). But, I get the following exception when I run it:

[main] ERROR org.apache.zookeeper.server.NIOServerCnxn  - Thread Thread[main,5,main] died
java.lang.RuntimeException: java.io.NotSerializableException: org.apache.http.impl.client.InternalHttpClient
    at backtype.storm.utils.Utils.serialize(Utils.java:56)
    at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:89)
    at app.storm.StormTopology.main(StormTopology.java:26)
Caused by: java.io.NotSerializableException: org.apache.http.impl.client.InternalHttpClient
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
    at backtype.storm.utils.Utils.serialize(Utils.java:52)
    ... 2 more 

Anyone else seen this? I know Storm itself uses the HTTP Client (4.1.1) internally. I tried to replace the internal library with the 4.3.1 version and I got another error:

Exception in thread "main" java.lang.NoSuchFieldError: INSTANCE
    at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:52)
    at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<init>(DefaultHttpRequestWriterFactory.java:56)
    at org.apache.http.impl.io.DefaultHttpRequestWriterFactory.<clinit>(DefaultHttpRequestWriterFactory.java:46)
    at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<init>(ManagedHttpClientConnectionFactory.java:72)
    at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<init>(ManagedHttpClientConnectionFactory.java:84)
    at org.apache.http.impl.conn.ManagedHttpClientConnectionFactory.<clinit>(ManagedHttpClientConnectionFactory.java:59)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$InternalConnectionFactory.<init>(PoolingHttpClientConnectionManager.java:487)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:147)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:136)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.<init>(PoolingHttpClientConnectionManager.java:112)
    at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:727)
    at org.apache.http.impl.client.HttpClients.createDefault(HttpClients.java:58)
    at app.storm.bolts.DataFetcherBolt.<init>(DataFetcherBolt.java:34)
    at app.storm.StormTopology.main(StormTopology.java:18)

Answer 1

I got my answer from the Google Forum for the Storm project. Here is the link to that post:

https://groups.google.com/forum/#!topic/storm-user/vdfamKeR9Lk

The fix is to instantiate the HttpClient instance in the prepare method of the Bolt class.