Wildcards in GCS bucket Java client api

Question

Using Wildcards in file name i am trying to read files from GCS bucket. in gsutil command line wildcards is working in specifying file names.

but in java client api

GcsFilename filename = new GcsFilename(BUCKETNAME, "big*");

it is searching for file named "big*" instead of file starting with big . please help me how i can use Wildcards in GCSFilename.

Thanks in advance.

Answer 1

Wildcard characters are a feature of gsutil, but they're not an inherent part of the Google Cloud Storage API. You can, however, handle this the same way that gsutil does.

If you want to find the name of every object that begins with a certain prefix, Google Cloud Storage's APIs provide a list method with a "prefix" argument. Only objects matching the prefix will be returned. This doesn't work for arbitrary regular expressions, but it will work for your example.

The documentation for the list method goes into more detail.

Answer 2

As Brandon Yarbrough mentioned, GcsFilename represent a name of a single GCS Object, which could include any valid UTF-8 character [excluding a few such as \r \n but including '*' though not recommended). see https://developers.google.com/storage/docs/bucketnaming#objectnames for more info. GAE GCS client does not support listing yet (though that is planned to be added), so for now you can use the GCS XML or JSON API directly (using urlfetch) or use the Java GCS api client, https://developers.google.com/api-client-library/java/apis/storage/v1 See example for the latter option:

public class ListServlet extends HttpServlet {

  public static final List<String> OAUTH_SCOPES =
      ImmutableList.of("https://www.googleapis.com/auth/devstorage.read_write");

  @Override
  protected void doPost(HttpServletRequest req, HttpServletResponse resp)
      throws ServletException, IOException {
    try {
      String bucket = req.getParameter("bucket");
      AppIdentityCredential cred = new AppIdentityCredential(OAUTH_SCOPES);
      Storage storage = new Storage.Builder(new UrlFetchTransport(), new JacksonFactory(), cred)
        .setApplicationName(SystemProperty.applicationId.get()).build();
      Objects.List list = storage.objects().list(bucket);
      for (StorageObject o : list.execute().getItems()) {
        resp.getWriter().println(o.getName() + " -> " + o);
      }
    } catch (Exception ex) {
      throw new ServletException(ex);
    }
  }
}