Git Large File Storage with Google Cloud Storage

Question

I am part of the project, where we use git repository hosted on google cloud source repository. Right now we use google cloud storage to store raw and processed data.

Everyone involved in the project downloads the data and places it locally in the ./data folder, which is .gitignore-ed.

I would prefer to use git LFS instead, but it is required that if the data has to be stored somewhere externally, it may only be GCS.

Is it possible to configure git LFS, Google Cloud Source Repository and Google Cloud Storage, with git LFS using pointers to GCS data files (instead of some other external server), while keeping full security provided by Google, with account authentication, SSH etc.?

Answer 1

I don't know of a git LFS implementation for GCS, but there are a few for S3, and GCS works fine with many applications designed for use with S3. It might be worth trying out one of the S3 implementations of git LFS to see if they work with GCS. If not, they could probably be modified to work reasonably quickly.