Bitbucket Pipelines: gcloud crashed (UnicodeDecodeError)

Question

Edit: I want to note that this issue doesn't happen if I deploy manually with cloud sdk on my computer. Only with pipelines

Edit again: I added a verbosity=debug to my gcloud app deploy. Here is what's generated:

Do you want to continue (Y/n)?  
DEBUG: No bucket specified, retrieving default bucket.
DEBUG: Using bucket [gs://staging.PROJECT.appspot.com].
DEBUG: Service [appengineflex.googleapis.com] is already enabled for project [PROJECT]
Beginning deployment of service [default]...
INFO: Need Dockerfile to be generated for runtime php
Building and pushing image for service [default]
DEBUG: 'ascii' codec can't decode byte 0xe2 in position 26: ordinal not in range(128)

Traceback (most recent call last):
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 845, in Execute
    resources = calliope_command.Run(cli=self, args=args)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 767, in Run
    resources = command_instance.Run(args)
  File "/usr/lib/google-cloud-sdk/lib/surface/app/deploy.py", line 87, in Run
    parallel_build=False)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 594, in RunDeploy
    flex_image_build_option=flex_image_build_option)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 393, in Deploy
    flex_image_build_option)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 283, in _PossiblyBuildAndPush
    self.deploy_options.parallel_build)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/deploy_command_util.py", line 427, in BuildAndPushDockerImage
    skip_files=service.parsed.skip_files.regex)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/cloud_build.py", line 151, in UploadSource
    skip_files)
  File "/usr/lib/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/cloud_build.py", line 122, in _GetIncludedPaths
    paths = docker.utils.exclude_paths(root, exclude)
  File "/usr/bin/../lib/google-cloud-sdk/lib/third_party/docker/docker/utils/utils.py", line 111, in exclude_paths
    all_paths = get_paths(root)
  File "/usr/bin/../lib/google-cloud-sdk/lib/third_party/docker/docker/utils/utils.py", line 135, in get_paths
    for parent, dirs, files in os.walk(root, followlinks=False):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 286, in walk
    if isdir(join(top, name)):
  File "/usr/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 26: ordinal not in range(128)
ERROR: gcloud crashed (UnicodeDecodeError): 'ascii' codec can't decode byte 0xe2 in position 26: ordinal not in range(128)

I'm trying to deploy to gcloud using bitbucket pipelines.

I didn't have this strange error initially until I was finished editing certain files (i'm deploying a wordpress site using app engine).

However, even after reverting my repository back to when my build was successful, it ends up failing again with `ERROR: gcloud crashed (UnicodeDecodeError): 'ascii' codec can't decode byte 0xe2 in position 26: ordinal not in range(128)

This makes no sense to me. It was working fine. Is there a way to expand on this error or find what file is making this occur?

The following is after the build is triggered in pipelines:

+ gcloud app deploy
Services to deploy:
descriptor:      [/opt/atlassian/pipelines/agent/build/app.yaml]
source:          [/opt/atlassian/pipelines/agent/build]
target project:  [project]
target service:  [default]
target version:  [20180731t175825]
target url:      [https://site.appspot.com]
Do you want to continue (Y/n)?  
Beginning deployment of service [default]...
Building and pushing image for service [default]
ERROR: gcloud crashed (UnicodeDecodeError): 'ascii' codec can't decode byte 0xe2 in position 26: ordinal not in range(128)
If you would like to report this issue, please run the following command:
  gcloud feedback
To check gcloud for common problems, please run the following command:
  gcloud info --run-diagnostics

This is my bitbucket-pipelines.yml

image: php:7.1.1

pipelines:
  branches:
   master:
    - step:
        # set GCLOUD_PROJECT environment variablee to your project ID
        # set GCLOUD_API_KEYFILE environment variable to base64-encoded keyfile as described here: https://confluence.atlassian.com/x/dm2xNQ
        name: Deploy to GCloud
        deployment: production   # set to test, staging or production
        # trigger: manual  # uncomment to have a manual step
        image: google/cloud-sdk:latest
        caches:
          - composer
        script:
          - echo $GCLOUD_API_KEYFILE | base64 --decode --ignore-garbage > ./gcloud-api-key.json
          - gcloud auth activate-service-account --key-file gcloud-api-key.json
          - gcloud config set project $GCLOUD_PROJECT
          - gcloud app deploy app.yaml

app.yaml

runtime: php
env: flex

handlers:
- url: /(.*\.(htm|html|css|js))$
  static_files: wordpress/\1
  upload: wordpress/.*\.(htm|html|css|js)$
  application_readable: true

- url: /wp-content/(.*\.(ico|jpg|jpeg|png|gif|woff|ttf|otf|eot|svg))$
  static_files: wordpress/wp-content/\1
  upload: wordpress/wp-content/.*\.(ico|jpg|jpeg|png|gif|woff|ttf|otf|eot|svg)$
  application_readable: true

- url: /(.*\.(ico|jpg|jpeg|png|gif|woff|ttf|otf|eot|svg))$
  static_files: wordpress/\1
  upload: wordpress/.*\.(ico|jpg|jpeg|png|gif|woff|ttf|otf|eot|svg)$
  application_readable: true

- url: /wp-includes/images/media/(.*\.(ico|jpg|jpeg|png|gif|woff|ttf|otf|eot|svg))$
  static_files: wordpress/wp-includes/images/media/\1
  upload: wordpress/wp-includes/images/media/.*\.(ico|jpg|jpeg|png|gif|woff|ttf|otf|eot|svg)$
  application_readable: true

- url: /wp-admin/(.+)
  script: wordpress/wp-admin/\1
  secure: always

- url: /wp-admin/
  script: wordpress/wp-admin/index.php
  secure: always

- url: /wp-login.php
  script: wordpress/wp-login.php
  secure: always

- url: /wp-cron.php
  script: wordpress/wp-cron.php
  login: admin

- url: /xmlrpc.php
  script: wordpress/xmlrpc.php

- url: /wp-(.+).php
  script: wordpress/wp-\1.php

- url: /(.+)?/?
  script: wordpress/index.php

beta_settings:
  cloud_sql_instances: project:us-central1:match

runtime_config:
  document_root: wordpress
  skip_lockdown_document_root: true

env_variables:
  WHITELIST_FUNCTIONS: escapeshellarg,escapeshellcmd,exec,pclose,popen,shell_exec,phpversion,php_uname

Answer 1

The problem is caused by a non-ascii character in a path that gcloud is trying to process.

The message UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 26: ordinal not in range(128) is a Python exception, indicating a problem trying to encode the byte 0xe2 as ascii.

Running gcloud --verbosity=debug .... generates a Python traceback.

The end of the traceback

  File "/usr/bin/../lib/google-cloud-sdk/lib/third_party/docker/docker/utils/utils.py", line 135, in get_paths
    for parent, dirs, files in os.walk(root, followlinks=False):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 296, in walk
    for x in walk(new_path, topdown, onerror, followlinks):
  File "/usr/lib/python2.7/os.py", line 286, in walk
    if isdir(join(top, name)):
  File "/usr/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b

shows that the Python code is walking a filesystem hierarchy (that's what os.walk does) and that the problematic operation is building up a path ( this is what path += '/' + b is doing).

Given this information, the next step is to inspect paths relating to the deployment for non-ascii characters, which the OP has been able to do successfully using the techniques in this superuser answer.