Protobuf file reading only some field

Question

I have more then 50 fields that is continiously writing in the .proto file but my query is that

1) if I need to read only 10 fields then how can this be achieve.

2) If I need to read partial data from the particular field then how can I achieve.

this should be done without loading all the data from the .proto file.

Thanks for your concern.

Answer 1

This is not really possible with Protobufs. In theory you could write a streaming parser that might be able to extract part of the message without parsing the whole thing, but it would only work if the fields you needed happened to be located towards the front of the message, since you'd at least have to parse through everything before the fields you want. In any case, none of the standard protobuf implementations provide an easy way to do streaming parses, because this isn't the way protobuf is designed to be used. Some third-party implementations, such as upb, might help.

On the other hand, Cap'n Proto, an alternative to Protocol Buffers, does support reading just one field out of a large file, without having to parse the fields before it. It does this by placing fields at fixed offsets and taking advantage of mmap() for large files.

Disclosure: I am the author of both Cap'n Proto and Protocol Buffers v2 (the version open sourced by Google).