I am trying to wrap a Java class to be called from Python using thrift's TFileTransport. I tried using two protocols TJSON and TBinary but I keep getting this exception
org.apache.thrift.transport.TTransportException: FileTransport error: bad event size
at org.apache.thrift.transport.TFileTransport.readEvent(TFileTransport.java:327)
at org.apache.thrift.transport.TFileTransport.read(TFileTransport.java:468)
at org.apache.thrift.transport.TFileTransport.readAll(TFileTransport.java:439)
at org.apache.thrift.protocol.TJSONProtocol$LookaheadReader.read(TJSONProtocol.java:263)
at org.apache.thrift.protocol.TJSONProtocol.readJSONSyntaxChar(TJSONProtocol.java:320)
at org.apache.thrift.protocol.TJSONProtocol.readJSONArrayStart(TJSONProtocol.java:784)
at org.apache.thrift.protocol.TJSONProtocol.readMessageBegin(TJSONProtocol.java:795)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
at org.apache.thrift.transport.TFileProcessor.processUntil(TFileProcessor.java:69)
at org.apache.thrift.transport.TFileProcessor.processChunk(TFileProcessor.java:102)
at org.apache.thrift.transport.TFileProcessor.processChunk(TFileProcessor.java:111)
at org.apache.thrift.transport.TFileProcessor.processChunk(TFileProcessor.java:118)
at com.netflix.suro.client.SendToPyServer.startThriftServer(SendToPyServer.java:51)
at com.netflix.suro.client.SendToPyServer.main(SendToPyServer.java:67)
This is how my Python client looks:
def __init__(self):
self.outFile=open("../../ThriftFile.in","a")
self.transport = TTransport.TFileObjectTransport(self.outFile)
self.protocol = TJSONProtocol.TJSONProtocol(self.transport)
self.client = sendPyInterface.Client(self.protocol)
self.transport.open()
def send(self,routingKey, message):
self.transport.write(pickle.dumps(self.client.send_send(routingKey, message)))
def configClient(self,configurationDict):
self.transport.write(pickle.dumps(self.client.send_ClientConfig(configurationDict)))
if __name__ == "__main__":
SuroClient=SuroPyClient()
configurationDict={"ClientConfig.LB_TYPE":"static","ClientConfig.LB_SERVER":"localhost:7101"}
SuroClient.configClient(configurationDict)
SuroClient.send("routingKey", "testMessage")
and this is my server:
public static void startThriftServer(SendPyInterface.Processor processor) {
try {
File input = new File("src/main/java/com/netflix/suro/client/ThriftFile.in");
if(!input.exists()){
input.createNewFile();
}
File output = new File("src/main/java/com/netflix/suro/client/ThriftFile.out");
if(!output.exists()){
output.createNewFile();
}
TFileTransport inputFileTransport = new TFileTransport(input.getAbsolutePath(), true);
TFileTransport outputFileTransport = new TFileTransport(output.getAbsolutePath(), false);
System.out.println(input.getAbsolutePath());
System.out.println(input.length());
inputFileTransport.open();
outputFileTransport.open();
System.out.println(inputFileTransport.getBytesRemainingInBuffer());
inputFileTransport.setTailPolicy(tailPolicy.WAIT_FOREVER);
System.out.println("Wait ...");
System.out.println(inputFileTransport.getBuffer());
TFileProcessor fProcessor =
new TFileProcessor(processor, new TJSONProtocol.Factory(), inputFileTransport, outputFileTransport);
try {
fProcessor.processChunk();
} catch (TTransportException e) {
e.printStackTrace();
}
System.out.println("File Thrift service started ...");
} catch (Exception e) {
e.printStackTrace();
}
I have followed the example from this thread: Example on how to use TFileTransport in Thrift (Client/Server)
and from this post: http://theprogrammersguideto.com/thrift/blog/code/chapter-3-moving-bytes-with-transports/
According to the docs in the old Thrift wiki, the Java and C++ TFileTransport is actually a framed transport, which writes an additional 4-byte header preceding the data. Seems the names are a bit misleading here.
In contrast, the Python TFileObjectTransport used is just a simple wrapper. This means, that the transports are not compatible, hence the error message.
So instead of TFileTransport
, the TSimpleFileTransport
should be used with Java.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With