Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a limit on the number of files in Google Cloud Storage ( GCS )?

I believe there should not be any limit but just wanted to confirm (as no mention in official docs):

  1. Is there a limit on the number of files in Google Cloud Storage (GCS)?
  2. Is there a performance impact (in access and write operation) if I have a very large number of files in GCS?
  3. Is there a limit on file name length (since I could use the filename to create pseudo directory structure)?
like image 526
gsinha Avatar asked Feb 15 '15 11:02

gsinha


1 Answers

Re (3): per https://cloud.google.com/storage/docs/bucket-naming, bucket names are limited to 222 characters (and with several other limitations); per https://cloud.google.com/storage/docs/naming-objects, object names are limited to 1024 characters (when utf8-encoded), with one mandatory limitation ("must not contain Carriage Return or Line Feed characters") and several "strongly recommended" conventions (no control characters, avoid certain punctuation characters).

Re (1) and (2), to the best of my knowledge there are no limitations on numbers of objects you can store in GCS, nor performance implications depending on such numbers. Google's online docs do specifically say "any amount of data".

However, if you need a firm commitment for a Project of Unusual Size (many petabytes, not the mere terabytes mentioned at https://cloud.google.com/storage/docs/overview) you may be best advised to get such a commitment "officially", by contacting Sales at https://cloud.google.com/contact/ .

http://googlecloudplatform.blogspot.com/2013/11/justdevelopit-migrates-petabytes-of-data-to-google-cloud-storage.html specifically interviews a customer using Cloud Storage for "over 10 petabytes [[growing]] at a rate of 800 terabytes a month", so, at least up to such orders of magnitude, you should definitely be fine.

like image 140
Alex Martelli Avatar answered Oct 27 '22 13:10

Alex Martelli