Upload to S3 with progress in plain Ruby script

Question

This question is related to this one: Tracking Upload Progress of File to S3 Using Ruby aws-sdk,

However since there is no clear solution to this I was wondering if there's a better/easier way (if one exists) of getting file upload progress with S3 using Ruby in 2018?

In my current setup I'm basically creating a new Resource, fetch my bucket and call upload_file but I haven't yet found any options for passing blocks which would help in yielding some sort of progress.

...
@connection = Aws::S3::Resource.new
@s3_bucket = @connection.bucket(bucket)
@s3_bucket.object(path).upload_file(data, {acl: 'public-read'})
...

Is there a way to do this using the newest sdk-for-ruby v3?

Any help (or even better a small example) would be great.

Answer 1

The example Trevor gives in https://stackoverflow.com/a/12147709/153886 is not hacky from what I can see - just wiring things together. The SDK simply does not provide a feature for passing progress details on all operations. Plus, Trevor is the maintainer of the Ruby SDK at AWS so I trust his judgement.

Expanding on his example

bar = ProgressBar.create(:title => "Uploading action", :starting_at => 0, :total => file.size)
obj = s3.buckets['my-bucket'].objects['object-key']
obj.write(:content_length => file.size) do |writable, n_bytes|
  writable.write(file.read(n_bytes))
  bar.progress += n_bytes
end

If you want to have a progress block right in the upload_file method I believe you will need to open a PR to the SDK. It is not that strange that is not the case for Ruby (or for any other runtime) because, for example, there could be an optimisation in the HTTP client library that uses IO.copy_stream from your source body argument to the destination socket, which does not relay progress anywhere.