Disclaimers: I am not a c++ programmer, please save me from myself.
I'm trying to create a PutObjectRequest in c++ using the AWS SDK.
I have a 'uint8_t*' (in Java-land from whence I hail we call this a byte[], I believe on planet c++ this is a buffer), and I need to get it into an Aws::IOStream somehow.
All of the examples show the data coming directly from the filesystem.
I've seen a couple of similar-ish (but not really) questions with answers that point to another third party library called Boost, but surely this is a common usecase? Why would I need a third party library to do something that should just be possible using the AWS SDK?:
"I have data, I want to put it up on S3. No it's not in the filesystem, yes I created it in memory."
uint8_t* buf; //<-- How do I get this...
...
Aws::S3::Model::PutObjectRequest object_request;
object_request.WithBucket(output_bucket).WithKey(key_name);
object_request.SetBody(data); //<-- ...into here
I really appreciate any help or pointers (no pun intended) here.
Update
I've tried everything in the comments, and this:
std::shared_ptr<Aws::IOStream> objectStream = Aws::MakeShared<Aws::StringStream>("PutObjectInputStream");
*objectStream << data;
objectStream->flush();
object_request.SetBody(objectStream);
and this:
std::shared_ptr<Aws::IOStream> objectStream = Aws::MakeShared<Aws::StringStream>("PutObjectInputStream");
std::istringstream is((char*) data);
*objectStream << is.rdbuf();
objectStream->flush();
object_request.SetBody(objectStream);
which compile, but each only uploads 2 bytes of data.
Other thing I've tried that don't compile are:
auto input_data = Aws::MakeShared<Aws::IOStream>("PutObjectInputStream", std::istringstream((char*) data), std::ios_base::in | std::ios_base::binary);
object_request.SetBody(input_data);
and
object_request.SetBody(std::make_shared<std::istringstream>( std::istringstream( (char*) spn ) ));
and these ones creates the object on S3, but with 0 bytes:
std::shared_ptr<Aws::IOStream> objectStream = Aws::MakeShared<Aws::StringStream>("PutObjectInputStream");
objectStream->rdbuf()->pubsetbuf(static_cast<char*>(reinterpret_cast<char*>(data)), length);
objectStream->rdbuf()->pubseekpos(length);
objectStream->seekg(0);
object_request.SetBody(objectStream);
std::shared_ptr<Aws::IOStream> objectStream = Aws::MakeShared<Aws::StringStream>("PutObjectInputStream");
objectStream->rdbuf()->pubsetbuf(reinterpret_cast<char*>(data), length);
objectStream->rdbuf()->pubseekpos(length);
objectStream->seekg(0);
object_request.SetBody(objectStream);
To upload folders and files to an S3 bucketSign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/ . In the Buckets list, choose the name of the bucket that you want to upload your folders or files to. Choose Upload.
Individual Amazon S3 objects can range in size from a minimum of 0 bytes to a maximum of 5 TB. The largest object that can be uploaded in a single PUT is 5 GB. For objects larger than 100 MB, customers should consider using the Multipart Upload capability.
Instead of using the Amazon S3 console, try uploading the file using the AWS Command Line Interface (AWS CLI) or an AWS SDK. Note: If you use the Amazon S3 console, the maximum file size for uploads is 160 GB. To upload a file that is larger than 160 GB, use the AWS CLI, AWS SDK, or Amazon S3 REST API.
There are three ways in which you can upload a file to amazon S3.
And after hours of hacking, here is the answer:
Aws::S3::Model::PutObjectRequest object_request;
object_request.WithBucket(output_bucket).WithKey(key_name);
auto data = Aws::MakeShared<Aws::StringStream>("PutObjectInputStream", std::stringstream::in | std::stringstream::out | std::stringstream::binary);
data->write(reinterpret_cast<char*>(buffer), length);
object_request.SetBody(data);
Thanks to Ben Voigt for pointing out that when you make_shared
(or MakeShared
in my case), you're not actually passing it the data at that point. You're just telling it what T
you're making shared.
Also helped by nbubis' answer to this question: const char * to std::basic_iostream
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With