403 error when trying to backup AWS S3 database to heroku postgres

Question

I'm trying to deploy my Django app on Heroku. I've created a Heroku Postgres database, and I want to import an existing PostgreSQL database into it from AWS S3. I followed the tutorial in the Heroku docs to the letter. I created a Heroku Postgres database and promoted it to primary db. I followed Heroku's AWS S3 tutorial to set all the config vars so heroku could access my S3 bucket. But when I run heroku pg:backups restore 'https://s3.amazonaws.com/<bucket name>/<database name>' DATABASE_URL, I get an error message saying "An error occurred and your backup didn't finish."

Here is the full output of heroku pg:backups info :

=== Backup info: r012
Database:    BACKUP
Started:     2015-07-13 13:43:30 +0000
Finished:    2015-07-13 13:43:30 +0000
Status:      Failed
Type:        Manual
Backup Size: 0.00B
=== Backup Logs
2015-07-13 13:43:30 +0000: 2015/07/13 13:43:30 aborting: could not write to output stream: Expected HTTP Status 200, received: "403 Forbidden"
2015-07-13 13:43:30 +0000: pg_restore: [custom archiver] could not read from input file: end of file
2015-07-13 13:43:30 +0000: waiting for restore to complete
2015-07-13 13:43:30 +0000: restore done
2015-07-13 13:43:30 +0000: waiting for download to complete
2015-07-13 13:43:30 +0000: download done

Naturally, since this is a 403 error I assumed something was wrong with my authentication. I've run heroku config and triple-checked all my config variables. The AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and DATABASE_URL environmental variables are all correct, and match what I see on AWS and in the Heroku Postgres GUI.

Why am I seeing this error message?

Answer 1

Well, I feel dumb, but the reason for the issues was that I was forgetting to add ".dump" to the end of the filename in my AWS url.

Answer 2

Same Error Pulling From AWS into Heroku

Heroku will sometimes refuse to import a backup/dump file if permissions are not set properly on the dump file inside the AWS bucket. (If Heroku can't access the file, it can't upload it.)

You can find the permission settings under either the Actions > Properties button or by clicking on the file and then clicking the button "Properties" on the right hand side.

One easy fix is to temporarily make the dump file public, do your heroku restore, and then lock down or delete the dump file.