Your application has authenticated using end user credentials from the Google Cloud SDK which are not supported by the translate.googleapis.com

Question

After reading similar questions, such as:

i am using gcp service account but when calling dialogue flow api its giving error :

and

Why is Google Cloud API trying to connect as an end-user?

and applying the suggested solutions I am still getting the error:

    Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
  "code" : 403,
  "errors" : [ {
    "domain" : "usageLimits",
    "message" : "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the translate.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.",
    "reason" : "rateLimitExceeded"
  } ],
  "message" : "Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the translate.googleapis.com. We recommend that most server applications use service accounts instead. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/.",
  "status" : "PERMISSION_DENIED"

My pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>seller</groupId>
    <artifactId>home.digest</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>war</packaging>

    <name>home.digest Maven Webapp</name>
    <!-- FIXME change it to the project's website -->
    <url>http://maven.apache.org</url>
    <repositories>
        <repository>
            <id>prime-repo</id>
            <name>Prime Repo</name>
            <url>http://repository.primefaces.org</url>
        </repository>
    </repositories>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.11</version>
            <scope>test</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>3.141.59</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.10</version>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.6</version>
        </dependency>
        <dependency>
            <groupId>org.primefaces</groupId>
            <artifactId>primefaces</artifactId>
            <version>6.2</version>
        </dependency>
        <dependency>
            <groupId>org.jboss.logging</groupId>
            <artifactId>jboss-logging</artifactId>
            <version>3.0.1.GA</version>
            <scope>compile</scope>
            <optional>true</optional>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.hibernate.javax.persistence/hibernate-jpa-2.1-api -->
        <dependency>
            <groupId>org.hibernate.javax.persistence</groupId>
            <artifactId>hibernate-jpa-2.1-api</artifactId>
            <version>1.0.2.Final</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.hibernate/hibernate-core -->
        <dependency>
            <groupId>org.hibernate</groupId>
            <artifactId>hibernate-core</artifactId>
            <version>4.3.6.Final</version>
             <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/javax.enterprise/cdi-api -->
        <dependency>
            <groupId>javax.enterprise</groupId>
            <artifactId>cdi-api</artifactId>
            <version>2.0.SP1</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.jboss.as</groupId>
            <artifactId>jboss-as-web</artifactId>
            <version>7.1.1.Final</version>
            <scope>compile</scope>
            <optional>true</optional>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.jboss.spec.javax.ejb/jboss-ejb-api_3.2_spec -->
        <dependency>
            <groupId>org.jboss.spec.javax.ejb</groupId>
            <artifactId>jboss-ejb-api_3.2_spec</artifactId>
            <version>1.0.2.Final</version>
            <scope>provided</scope>
        </dependency>

        <!-- https://mvnrepository.com/artifact/com.google.cloud/google-cloud-translate -->
        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>google-cloud-translate</artifactId>
            <version>1.79.0</version>
        </dependency>
        <dependency>
            <!-- jsoup HTML parser library @ https://jsoup.org/ -->
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.11.3</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.8.1</version>
        </dependency>
    </dependencies>

    <build>
        <finalName>home.digest</finalName>
        <pluginManagement><!-- lock down plugins versions to avoid using Maven 
                defaults (may be moved to parent pom) -->
            <plugins>
                <plugin>
                    <artifactId>maven-clean-plugin</artifactId>
                    <version>3.1.0</version>
                </plugin>
                <!-- see http://maven.apache.org/ref/current/maven-core/default-bindings.html#Plugin_bindings_for_war_packaging -->
                <plugin>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>3.0.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <version>3.8.0</version>
                </plugin>
                <plugin>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.22.1</version>
                </plugin>
                <plugin>
                    <artifactId>maven-war-plugin</artifactId>
                    <version>3.2.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-install-plugin</artifactId>
                    <version>2.5.2</version>
                </plugin>
                <plugin>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <version>2.8.2</version>
                </plugin>
            </plugins>
        </pluginManagement>
    </build>
</project>

My code:

com.google.cloud.translate.Translate translate = TranslateOptions.getDefaultInstance().getService();
        String translatexText = "EMPTY";
        try {
            Translation translation = translate.translate("Guten Tag", Translate.TranslateOption.sourceLanguage("de"),
                    Translate.TranslateOption.targetLanguage("bg"),
                    // Use "base" for standard edition, "nmt" for the
                    // premium model.
                    Translate.TranslateOption.model("nmt"));

            translatexText = translation.getTranslatedText();
        } catch (Exception e) {
            Logger.getLogger(TestServlet.class).error(e.getMessage(), e);
        }

        System.out.println(translatexText);

Answer 1

This error message is caused by using User Credentials when you setup the Cloud SDK. Typically this is done using the command gcloud auth login.

There are several methods to solve this problem. Each method uses a Service Account.

Method 1:

Create a service account and set up the Cloud SDK to use the service account.

Example command:

gcloud auth activate-service-account [email protected] --key-file=/fullpath/service-account.json

Method 2:

Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to your service account JSON file.

set GOOGLE_APPLICATION_CREDENTIALS=/fullpath/service-account.json

Method 3:

Specify the service account when creating your Java SDK clients.

This link shows examples of specifying a service account file:

Setting Up Authentication for Server to Server Production Applications

Answer 2

In general, running the code from GCP and using a service account is best practice. Not exporting service account keys is also a great security practice. So the question is already answered.

In this additional answer, I want to share my insights where the error message in the question's title occurred, what is happening, and how to setup your machine to do some local development without exporting service account keys (though, exporting short-lived service account keys from a throwaway serviceaccount in a test project may both be simpler and better practice for testing stuff).

I'd like to use the sheets API as an example. The sheets API requires at least the oauth2 scope https://www.googleapis.com/auth/spreadsheets.readonly to read from a spreadsheet.

My example Golang program was

package main
 
import (
    "context"
    "fmt"
    "log"
 
    "google.golang.org/api/option"
    "google.golang.org/api/sheets/v4"
)
 
func main() {
    ctx := context.Background()
 
    // using default authentication, whatever the environment provides. See https://cloud.google.com/docs/authentication#environment-service-accounts
    srv, err := sheets.NewService(ctx, option.WithScopes(sheets.SpreadsheetsReadonlyScope))
    if err != nil {
        log.Fatalf("Unable to retrieve Sheets client: %v", err)
    }
 
    // A sample spreadsheet:
    // https://docs.google.com/spreadsheets/d/1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms/edit
    spreadsheetId := "1BxiMVs0XRA5nFMdKvBdBZjgmUUqptlbs74OgvE2upms"
    readRange := "Class Data!A2:E"
    resp, err := srv.Spreadsheets.Values.Get(spreadsheetId, readRange).Do()
    if err != nil {
        log.Fatalf("Unable to retrieve data from sheet: %v", err)
    }
 
    if len(resp.Values) == 0 {
        fmt.Println("No data found.")
    } else {
        fmt.Println("got some results, ...")
    }
}

I was trying to read a spreadsheet from the Google Cloud Shell (the GCP built-in web cloud terminal) and the full error message that brought me here was:

googleapi: Error 403: Your application has authenticated using end user credentials from the Google Cloud SDK or Google Cloud Shell which are not supported by the sheets.googleapis.com. We recommend configuring the billing/quota_project setting in gcloud or using a service account through the auth/impersonate_service_account setting. For more information about service accounts and how to use them in your application, see https://cloud.google.com/docs/authentication/., accessNotConfigured

To access sheets with you user account via Google Cloud SDK, we need Application Default Credentials with the corresponding oauth scopes.

$ gcloud auth application-default login --scopes=https://www.googleapis.com/auth/spreadsheets.readonly,openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform

I set the scopes to the default ones and added the spreadsheets scope. When following the browser-based authentication flow, I got asked to allow

Google Auth Library to:

View and manage your data across Google Cloud Platform services

View your Google Spreadsheets

On my computer, the code is now working.

By default, when acquiring the Application Default Credentials without specifying --scopes, you will only have permissions to View and manage your data across Google Cloud Platform services and will not be allowed to talk to the sheets API. Without specifying the --scopes, when talking to the sheets API, the expected error message is googleapi: Error 403: Request had insufficient authentication scopes.. This is indeed what I can observe when running the code on my computer.

Yet, when running the code on Google Cloud Shell, I get the error message from the question's title, i.e. Your application has authenticated using end user credentials .... Also running gcloud auth application-default login with the corresponding --scopes does not change this behavior on Google Cloud Shell. This is because when the code is run on Google Cloud Shell, the Application Default Credentials are not used.

I wrote some small debugging code to uncover how the Google API oauth2 library is finding its default credentials:

package main
 
import (
    "context"
    "fmt"
 
    "golang.org/x/oauth2/google"
)
 
func main() {
    ctx := context.Background()
    creds, err := google.FindDefaultCredentials(ctx, "https://www.googleapis.com/auth/spreadsheets.readonly")
    if err != nil {
        panic(fmt.Sprintf("google.FindDefaultCredentials(): %v", err))
    }
    fmt.Printf("creds uses credentials file? %#v\n", creds.JSON != nil)
    t, err := creds.TokenSource.Token()
    if err != nil {
        panic(fmt.Sprintf("Token(): %v", err))
    }
    fmt.Printf("token: %#v\n", t)
}

On my computer, after running

$ gcloud auth application-default login

my debugging code prints:

creds uses credentials file? true
token: &oauth2.Token{AccessToken:"...", ..., raw:map[string]interface {}{"access_token":"...", "scope":"https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/accounts.reauth https://www.googleapis.com/auth/cloud-platform openid", "token_type":"Bearer"}}

On my computer, after running

$ gcloud auth application-default login --scopes=https://www.googleapis.com/auth/spreadsheets.readonly,openid,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/cloud-platform

my debugging code prints:

creds uses credentials file? true
token: &oauth2.Token{AccessToken:"...", ..., raw:map[string]interface {}{"access_token":"...", "scope":"https://www.googleapis.com/auth/spreadsheets.readonly https://www.googleapis.com/auth/cloud-platform https://www.googleapis.com/auth/accounts.reauth openid https://www.googleapis.com/auth/userinfo.email", "token_type":"Bearer"}}

We now have the https://www.googleapis.com/auth/spreadsheets.readonly scope included.

Whatever I try, when running my debugging code on Google Cloud Shell, I always get

creds uses credentials file? false
token: &oauth2.Token{AccessToken:"...", ..., raw:map[string]interface {}{"oauth2.google.serviceAccount":"default", "oauth2.google.tokenSource":"compute-metadata"}}

This means that the Application Default Credentials are not used on Google Cloud Shell. This is because the Application Default Credentials on Google Cloud Shell are not written to the default location $HOME/.config/gcloud/application_default_credentials.json and google.FindDefaultCredentials() thus tries to authenticate via the GCP metadata server.