S3 Batch Operations: stuck on Preparing using JSON inventory of billions

Question

I've created an S3 Batch Operation using an S3 inventory JSON file that's pointing to a few billion objects in my S3 bucket.

The operation has been stuck on "Preparing" status for 24 hours now.

What are the preparation times to expect in these kinds of volumes?

Would preparation time shorten if instead of providing it with the JSON manifest I'll join all the inventory CSVs into one uber-CSV?

I've used awscli to create the request like so:

aws s3control create-job \
    --region ... \
    --account-id ... \
    --operation '{"S3PutObjectCopy":{"TargetResource":"arn:aws:s3:::some-bucket","MetadataDirective":"COPY"}}' \
    --manifest '{"Spec":{"Format":"S3InventoryReport_CSV_20161130"},"Location":{"ObjectArn":"arn:aws:s3:::path_to_manifest/manifest.json","ETag":"..."}}' \
    --report '{"Bucket":"arn:aws:s3:::some-bucket","Prefix":"reports", "Format":"Report_CSV_20180820", "Enabled":true, "ReportScope":"AllTasks"}' \
    --priority 42 \
    --role-arn ... \
    --client-request-token $(uuidgen) \
    --description "Batch request"

Answer 1

After ~4 days the tasks completed the preparation phase and were ready to be ran