Google Cloud SDK is trying to access gslib.USER_AGENT, which does not exist

Question

After a recent update, my gsutil commands are failing with this exception:

gsutil cp /Users/myself/my-project/client/libs/image-content-2020-6-23.jar gs://my-bucket/content
Traceback (most recent call last):
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gsutil", line 21, in <module>
    gsutil.RunMain()
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gsutil.py", line 123, in RunMain
    sys.exit(gslib.__main__.main())
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 429, in main
    return _RunNamedCommandAndHandleExceptions(
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 767, in _RunNamedCommandAndHandleExceptions
    _HandleUnknownFailure(e)
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 625, in _RunNamedCommandAndHandleExceptions
    return command_runner.RunNamedCommand(command_name,
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand
    return_code = command_inst.RunCommand()
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 1205, in RunCommand
    self.Apply(_CopyFuncWrapper,
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1485, in Apply
    caller_id = self._SetUpPerCallerState()
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/command.py", line 1360, in _SetUpPerCallerState
    class_map[caller_id] = cls
  File "<string>", line 2, in __setitem__
  File "/usr/local/Cellar/[email protected]/3.8.3/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/managers.py", line 850, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError: 
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/Cellar/[email protected]/3.8.3/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/managers.py", line 243, in serve_client
    request = recv()
  File "/usr/local/Cellar/[email protected]/3.8.3/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 251, in recv
    return _ForkingPickler.loads(buf.getbuffer())
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/commands/cp.py", line 30, in <module>
    from gslib.command import Command
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/command.py", line 50, in <module>
    from gslib.cloud_api_delegator import CloudApiDelegator
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/cloud_api_delegator.py", line 26, in <module>
    from gslib.cs_api_map import ApiMapConstants
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/cs_api_map.py", line 23, in <module>
    from gslib.gcs_json_api import GcsJsonApi
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/gcs_json_api.py", line 72, in <module>
    from gslib.third_party.storage_apitools import storage_v1_client as apitools_client
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/third_party/storage_apitools/storage_v1_client.py", line 26, in <module>
    class StorageV1(base_api.BaseApiClient):
  File "/Users/myself/google-cloud-sdk/platform/gsutil/gslib/third_party/storage_apitools/storage_v1_client.py", line 38, in StorageV1
    _USER_AGENT += gslib.USER_AGENT
AttributeError: module 'gslib' has no attribute 'USER_AGENT'
---------------------------------------------------------------------------

Here is my gcloud version:

$ gcloud version
Google Cloud SDK 297.0.1
bq 2.0.58
core 2020.06.17
gsutil 4.51

Here is the version of Python being used by gcloud:

$ printenv CLOUDSDK_PYTHON
/usr/local/Cellar/[email protected]/3.8.3/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python

$ /usr/local/Cellar/[email protected]/3.8.3/Frameworks/Python.framework/Versions/3.8/Resources/Python.app/Contents/MacOS/Python -V
Python 3.8.3

I brew reinstalled python@3, reinstalled the google cloud SDK, and nothing is working.

For now, I have commented out this line in the offending file:

File: google-cloud-sdk/platform/gsutil/gslib/third_party/storage_apitools/storage_v1_client.py

  _VERSION = u'v1'
  _CLIENT_ID = 'nomatter'
  _CLIENT_SECRET = 'nomatter'
  _USER_AGENT = 'apitools Python/%s' % platform.python_version()
  # I commented out this line here:
  #_USER_AGENT += gslib.USER_AGENT

  _CLIENT_CLASS_NAME = u'StorageV1'
  _URL_VERSION = u'v1'
  _API_KEY = None

This has temporarily worked around the problem and my gsutil cp commands are now succeeding.

Is this a Google Cloud SDK bug, or an issue with my system?

Answer 1

This issue has been flagged in the gsutil library, and a fix has been merged. It is being released in gsutil version 302.0.0, scheduled to be released on July 21, 2020.

If you need to get around this issue immediately, you can roll back to version 297.0.1 using the following command:

gcloud components update --version 297.0.1

Answer 2

I had the same issue and needed to get working again so used:

gcloud components restore

to roll back to the previous version and all is working again.

Answer 3

Edit:

A issue has been raised on the Issue Tracker, I recommend you to go there and add more information if possible: AttributeError: module 'gslib' has no attribute 'USER_AGENT'