Where does CKAN store the files pushed to datastore/filestore?

Question

I am wondering where CKAN stores the files uploaded to the datastore/filestore.

I just set up the environment, and I tried to search on /var/lib/ckan/default/storage, but I can find only the icons. The CSV files that I uploaded in the tool are not there.

I searched in /usr/lib/ckan, but the files are not there. I could not find the CSV files anywhere else.

Where are these files being stored?

Answer 1

FileStore

By default, resource files are saved in the file system.

Inside CKAN .ini configuration file, the property ckan.storage_path points to the base folder where files are stored, for example:

ckan.storage_path = /var/lib/ckan

That folder contains two more folders:

• storage contains the uploaded group / organization images
• resources stores the CSV (and PDF, XLS, ...) resource files, under two levels of folders

In order to find a specific resource file, look at its GUID:

• the first three characters of the GUID are the name of the first-level folder
• the following three characters of the GUID are the name of the second-level folder
• the remaining of the GUID (- characters included) is the file name

For example, if a resource GUID is 3F2504E0-4F89-11D3-9A0C-0305E82C3301, it is stored on the file system as /var/lib/ckan/resources/3F2/504/E0-4F89-11D3-9A0C-0305E82C3301

DataStore

If you configured the DataStore, CSV files are also stored in a database for easier querying.

Installing the DataStore involves adding in CKAN .ini configuration files something like this:

ckan.datastore.write_url = postgresql://ckan_default:pass@localhost/datastore_default
ckan.datastore.read_url = postgresql://datastore_default:pass@localhost/datastore_default

which state the database users, server and database name.

The data of each resource in the DataStore is contained in a database table whose name is the resource GUID.