CKAN : Upload to datastore failed; Resource too large to download

Question

When i try to upload a large csv file to CKAN datastore it fails and shows the following message

Error: Resource too large to download: 5158278929 > max (10485760).

I changed the maximum in megabytes a resources upload to

ckan.max_resource_size = 5120

in

/etc/ckan/production.ini

What else do i need to change to upload a large csv to ckan.

Screenshot:

Answer 1

That error message comes from the DataPusher, not from CKAN itself: https://github.com/ckan/datapusher/blob/master/datapusher/jobs.py#L250. Unfortunately it looks like the DataPusher's maximum file size is hard-coded to 10MB: https://github.com/ckan/datapusher/blob/master/datapusher/jobs.py#L28. Pushing larger files into the DataStore is not supported.

Two possible workarounds might be:

1. Use the DataStore API to add the data yourself.

2. Change the MAX_CONTENT_LENGTH on the line in the DataPusher source code that I linked to above, to something bigger.