Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

google-cloud-sdk installation fails on python syntax error

on a "Linux supermicro 4.15.0-72-generic #81-Ubuntu SMP Tue Nov 26 12:20:02 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux" I see this happening when doing "sudo apt-get install google-cloud-sdk"

> Reading package lists... Done Building dependency tree        Reading
> state information... Done Suggested packages:  
> google-cloud-sdk-app-engine-java google-cloud-sdk-app-engine-python
> google-cloud-sdk-pubsub-emulator google-cloud-sdk-bigtable-emulator
> google-cloud-sdk-datastore-emulator kubectl The following NEW packages
> will be installed:   google-cloud-sdk 0 upgraded, 1 newly installed, 0
> to remove and 13 not upgraded. Need to get 0 B/25.0 MB of archives.
> After this operation, 194 MB of additional disk space will be used.
> Selecting previously unselected package google-cloud-sdk. (Reading
> database ... 267451 files and directories currently installed.)
> Preparing to unpack .../google-cloud-sdk_274.0.0-0_all.deb ...
> Unpacking google-cloud-sdk (274.0.0-0) ... Setting up google-cloud-sdk
> (274.0.0-0) ... Compiling
> platform/bq/third_party/yaml/lib3/__init__.py ...   File
> "platform/bq/third_party/yaml/lib3/__init__.py", line 284
>     class YAMLObject(metaclass=YAMLObjectMetaclass):
>                               ^ SyntaxError: invalid syntax

Google is not my friend, I can not find back what is the cause of this. This happens after a "apt upgrade", so I have all the latest packages (python,...)

like image 398
Karel Goderis Avatar asked Dec 18 '19 13:12

Karel Goderis


2 Answers

Python3

I was experiencing the same YAMLObjectMetaclass syntax error and resolved the issue by configuring gcloud, gsutil and bq to use Python 3 by setting the following environment variables:

ENV CLOUDSDK_PYTHON=python3
ENV CLOUDSDK_GSUTIL_PYTHON=python3
ENV CLOUDSDK_BQ_PYTHON=python3

see: gcloud topic startup

After setting the 3 environment variables above I was able to successfully install gcloud sdk 274.0.1-0.

Dockerfile code block:

FROM ubuntu:1810
#...removed section for brevity
# install google cloud platform sdk with python3
# see: https://cloud.google.com/sdk/gcloud/reference/topic/startup
ENV CLOUDSDK_PYTHON=python3
# gsutil now supports Python 3.5 and up in addition to Python 2.7.x.
# To use a different interpreter for gsutil than for the other Python tools,
# you can set the desired interpreter by setting
# the CLOUDSDK_GSUTIL_PYTHON environment variable.
ENV CLOUDSDK_GSUTIL_PYTHON=python3
# bq now supports Python 3.5 and up in addition to Python 2.7.x.
# To use a different interpreter for bq than for the other Python tools,
# you can set the desired interpreter by setting the
# CLOUDSDK_BQ_PYTHON environment variable.
ENV CLOUDSDK_BQ_PYTHON=python3
RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" && \
    echo "deb http://packages.cloud.google.com/apt ${CLOUD_SDK_REPO} main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && \
    apt-get update -y && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    google-cloud-sdk \
    kubectl \
    google-cloud-sdk-app-engine-grpc \
    google-cloud-sdk-app-engine-go \
    google-cloud-sdk-cloud-build-local \
    google-cloud-sdk-app-engine-python \
    google-cloud-sdk-cbt \
    google-cloud-sdk-app-engine-python-extras \
    google-cloud-sdk-app-engine-java

Build output:

2972  ---> 6fa2385d7961
2973 Step 31/59 : ENV CLOUDSDK_PYTHON=python3
2974  ---> Running in 57ff376529c3
2975 Removing intermediate container 57ff376529c3
2976  ---> d99387b3fed7
2977 Step 32/59 : ENV CLOUDSDK_GSUTIL_PYTHON=python3
2978  ---> Running in 5e8170bbdfdd
2979 Removing intermediate container 5e8170bbdfdd
2980  ---> 358dde5de9c4
2981 Step 33/59 : ENV CLOUDSDK_BQ_PYTHON=python3
2982  ---> Running in 0f69d9a9b714
2983 Removing intermediate container 0f69d9a9b714
2984  ---> 76a0a37d12ec
2985 Step 34/59 : RUN export CLOUD_SDK_REPO="cloud-sdk-$(lsb_release -c -s)" &&     echo "deb http://packages.cloud.google.com/apt ${CLOUD_SDK_REPO} main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list &&     curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - &&     apt-get update -y && DEBIAN_FRONTEND=noninteractive apt-get install -y     google-cloud-sdk     kubectl     google-cloud-sdk-app-engine-grpc     google-cloud-sdk-app-engine-go     google-cloud-sdk-cloud-build-local     google-cloud-sdk-app-engine-python     google-cloud-sdk-cbt     google-cloud-sdk-app-engine-python-extras     google-cloud-sdk-app-engine-java
2986  ---> Running in 746a47f450dc
2987 deb http://packages.cloud.google.com/apt cloud-sdk-cosmic main
2988   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2989                                  Dload  Upload   Total   Spent    Left  Speed
2990 100   659  100   659    0     0   2768      0 --:--:-- --:--:-- --:--:--  2941
2991 Warning: apt-key output should not be parsed (stdout is not a terminal)
2992 OK
2993 Get:1 http://packages.cloud.google.com/apt cloud-sdk-cosmic InRelease [6,372 B]
2994 Hit:2 https://download.docker.com/linux/ubuntu cosmic InRelease
2995 Hit:3 http://security.ubuntu.com/ubuntu cosmic-security InRelease
2996 Hit:4 http://archive.ubuntu.com/ubuntu cosmic InRelease
2997 Hit:5 http://archive.ubuntu.com/ubuntu cosmic-updates InRelease
2998 Ign:6 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 Packages
2999 Hit:7 http://archive.ubuntu.com/ubuntu cosmic-backports InRelease
3000 Get:6 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 Packages [40.9 kB]
3001 Fetched 47.3 kB in 1s (53.7 kB/s)
3002 Reading package lists...
3003 Reading package lists...
3004 Building dependency tree...
3005 Reading state information...
3006 The following additional packages will be installed:
3007   python-crcmod
3008 Suggested packages:
3009   google-cloud-sdk-pubsub-emulator google-cloud-sdk-bigtable-emulator
3010   google-cloud-sdk-datastore-emulator
3011 The following NEW packages will be installed:
3012   google-cloud-sdk google-cloud-sdk-app-engine-go
3013   google-cloud-sdk-app-engine-grpc google-cloud-sdk-app-engine-java
3014   google-cloud-sdk-app-engine-python google-cloud-sdk-app-engine-python-extras
3015   google-cloud-sdk-cbt google-cloud-sdk-cloud-build-local kubectl
3016   python-crcmod
3017 0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.
3018 Need to get 119 MB of archives.
3019 After this operation, 548 MB of additional disk space will be used.
3020 Get:1 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk all 274.0.1-0 [25.0 MB]
3021 Get:2 http://archive.ubuntu.com/ubuntu cosmic/universe amd64 python-crcmod amd64 1.7+dfsg-1build1 [17.8 kB]
3022 Get:3 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-app-engine-python all 274.0.1-0 [4,278 kB]
3023 Get:4 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-app-engine-go amd64 274.0.1-0 [2,296 kB]
3024 Get:5 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-app-engine-grpc amd64 274.0.1-0 [1,558 kB]
3025 Get:6 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-app-engine-java all 274.0.1-0 [56.5 MB]
3026 Get:7 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-app-engine-python-extras all 274.0.1-0 [14.9 MB]
3027 Get:8 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-cbt amd64 274.0.1-0 [3,387 kB]
3028 Get:9 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 google-cloud-sdk-cloud-build-local amd64 274.0.1-0 [2,771 kB]
3029 Get:10 http://packages.cloud.google.com/apt cloud-sdk-cosmic/main amd64 kubectl amd64 1.17.0-00 [8,742 kB]
3030 debconf: delaying package configuration, since apt-utils is not installed
3031 Fetched 119 MB in 10s (11.7 MB/s)
3032 Selecting previously unselected package python-crcmod.
3033 (Reading database ... 41146 files and directories currently installed.)
3034 Preparing to unpack .../0-python-crcmod_1.7+dfsg-1build1_amd64.deb ...
3035 Unpacking python-crcmod (1.7+dfsg-1build1) ...
3036 Selecting previously unselected package google-cloud-sdk.
3037 Preparing to unpack .../1-google-cloud-sdk_274.0.1-0_all.deb ...
3038 Unpacking google-cloud-sdk (274.0.1-0) ...
3039 Selecting previously unselected package google-cloud-sdk-app-engine-python.
3040 Preparing to unpack .../2-google-cloud-sdk-app-engine-python_274.0.1-0_all.deb ...
3041 Unpacking google-cloud-sdk-app-engine-python (274.0.1-0) ...
3042 Selecting previously unselected package google-cloud-sdk-app-engine-go.
3043 Preparing to unpack .../3-google-cloud-sdk-app-engine-go_274.0.1-0_amd64.deb ...
3044 Unpacking google-cloud-sdk-app-engine-go (274.0.1-0) ...
3045 Selecting previously unselected package google-cloud-sdk-app-engine-grpc.
3046 Preparing to unpack .../4-google-cloud-sdk-app-engine-grpc_274.0.1-0_amd64.deb ...
3047 Unpacking google-cloud-sdk-app-engine-grpc (274.0.1-0) ...
3048 Selecting previously unselected package google-cloud-sdk-app-engine-java.
3049 Preparing to unpack .../5-google-cloud-sdk-app-engine-java_274.0.1-0_all.deb ...
3050 Unpacking google-cloud-sdk-app-engine-java (274.0.1-0) ...
3051 Selecting previously unselected package google-cloud-sdk-app-engine-python-extras.
3052 Preparing to unpack .../6-google-cloud-sdk-app-engine-python-extras_274.0.1-0_all.deb ...
3053 Unpacking google-cloud-sdk-app-engine-python-extras (274.0.1-0) ...
3054 Selecting previously unselected package google-cloud-sdk-cbt.
3055 Preparing to unpack .../7-google-cloud-sdk-cbt_274.0.1-0_amd64.deb ...
3056 Unpacking google-cloud-sdk-cbt (274.0.1-0) ...
3057 Selecting previously unselected package google-cloud-sdk-cloud-build-local.
3058 Preparing to unpack .../8-google-cloud-sdk-cloud-build-local_274.0.1-0_amd64.deb ...
3059 Unpacking google-cloud-sdk-cloud-build-local (274.0.1-0) ...
3060 Selecting previously unselected package kubectl.
3061 Preparing to unpack .../9-kubectl_1.17.0-00_amd64.deb ...
3062 Unpacking kubectl (1.17.0-00) ...
3063 Setting up python-crcmod (1.7+dfsg-1build1) ...
3064 Setting up kubectl (1.17.0-00) ...
3065 Setting up google-cloud-sdk (274.0.1-0) ...
3066 Setting up google-cloud-sdk-app-engine-python (274.0.1-0) ...
3067 Setting up google-cloud-sdk-app-engine-grpc (274.0.1-0) ...
3068 Setting up google-cloud-sdk-cloud-build-local (274.0.1-0) ...
3069 Setting up google-cloud-sdk-cbt (274.0.1-0) ...
3070 Processing triggers for google-cloud-sdk (274.0.1-0) ...
3071 Setting up google-cloud-sdk-app-engine-java (274.0.1-0) ...
3072 Setting up google-cloud-sdk-app-engine-go (274.0.1-0) ...
3073 Setting up google-cloud-sdk-app-engine-python-extras (274.0.1-0) ...
3074 Processing triggers for google-cloud-sdk (274.0.1-0) ...
3075 Removing intermediate container 746a47f450dc
3076  ---> c128e79ea8f1
3077 Step 35/59 : RUN apt-get update --fix-missing && DEBIAN_FRONTEND=noninteractive apt-get upgrade -y && apt-get dist-upgrade -y
3078  ---> Running in 0b74e8d18d1b
3079 Hit:1 http://packages.cloud.google.com/apt cloud-sdk-cosmic InRelease
3080 Hit:2 http://archive.ubuntu.com/ubuntu cosmic InRelease
3081 Hit:3 https://download.docker.com/linux/ubuntu cosmic InRelease
3082 Hit:4 http://security.ubuntu.com/ubuntu cosmic-security InRelease
3083 Hit:5 http://archive.ubuntu.com/ubuntu cosmic-updates InRelease
3084 Hit:6 http://archive.ubuntu.com/ubuntu cosmic-backports InRelease
3085 Reading package lists...
3086 Reading package lists...
3087 Building dependency tree...
3088 Reading state information...
3089 Calculating upgrade...
3090 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
3091 Reading package lists...
3092 Building dependency tree...
3093 Reading state information...
3094 Calculating upgrade...
3095 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
3096 Removing intermediate container 0b74e8d18d1b
3097  ---> 85cca2b48996
3098 Step 36/59 : WORKDIR ${DOCKER_IMAGE_USER_HOME}
3099  ---> Running in 63c9f8ce5e1d
like image 53
Brent Dorsey Avatar answered Sep 26 '22 19:09

Brent Dorsey


This is a known issue that started happening with the release of the 274.0.0 Cloud SDK version. The installation tends to go through even though the error is thrown (check the Cloud SDK version with gcloud info). Engineering is aware, and they're looking into it.

The current workaround is to install a previous version of the Cloud SDK, such as 272.0.0 with sudo apt-get install google-cloud-sdk=272.0.0-0.

Update:

The issue has been resolved with the release of the Cloud SDK version 275.0.0 - installing the Cloud SDK with Linux based package tools should not throw the aforementioned error anymore. If you're still having issues, please open an Issue Tracker thread under the Cloud SDK component explaining the problem you're encountering.

like image 29
Maxim Avatar answered Sep 23 '22 19:09

Maxim