Amazon Redshift error: Mandatory url not present in manifest file

Question

I have the following manifest file

{
  "entries": [
    {
      "url": "s3://gcmregtokenvalidation/registrations_invalid_token/registration_master_2017-07-25T10:47:40.049+05:30.gz",
      "mandatory": true
    },
    {
      "url": "s3://gcmregtokenvalidation/registrations_invalid_token/registration_master_2017-07-25T10:47:59.441+05:30.gz",
      "mandatory": true
    },
    {
      "url": "s3://gcmregtokenvalidation/registrations_invalid_token/registration_master_2017-07-25T10:48:19.297+05:30.gz",
      "mandatory": true
    },
    {
      "url": "s3://gcmregtokenvalidation/registrations_invalid_token/registration_master_2017-07-25T10:48:40.277+05:30.gz",
      "mandatory": true
    }
  ]
}

and following copy command

COPY registrations_invalid_token
FROM  's3://locationnew/redshiftloader-
manifest/registrations_invalid_token_25_07_2017_11:55:11.manifest' 
CREDENTIALS 'aws_access_key_id=XXX;aws_secret_access_key=XXX'
manifest gzip delimiter ',' dateformat 'auto'
FILLRECORD IGNOREBLANKLINES TRIMBLANKS TRUNCATECOLUMNS
MAXERROR 100000 region 'ap-south-1';

I got the following exception

Amazon Invalid operation: Mandatory url is not present in manifest file. Details:
----------------------------------------------- error: Mandatory url is not present in manifest file. code: 8001 context:
Manifest file location=s3://locationnew/redshiftloader-manifest/registrations_invalid_token_25_07_2017_11:55:11.manifest url=s3://gcmregtokenvalidation/registrations_invalid_token/registration_master_2017-07-25T10:47:40.049+05:30.gz query: 1475619 location: s3_utility.cpp:337 process:
padbmaster [pid=32720]
-----------------------------------------------;

Why I am getting such an error?

Answer 1

The Using a Manifest to Specify Data Files documentation says:

The optional mandatory flag specifies whether COPY should return an error if the file is not found. The default of mandatory is false. Regardless of any mandatory settings, COPY will terminate if no files are found.

The Mandatory url is not present in manifest file error is badly worded, but it might be saying that the specified file could not be found on Amazon S3.

You could test it with:

aws s3 ls s3://gcmregtokenvalidation/registrations_invalid_token/registration_master_2017-07-25T10:47:40.049+05:30.gz

Also, try setting mandatory to false to see if the error disappears.