Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Google BigQuery with .NET documentation/ samples

I require to query data using Google BigQuery API. But I am struggling to find .NET Samples, and there was no documentation included with the binary (Google.Apis.Bigquery.dll). Can anybody provide me with sample usage for .NET?

like image 576
user1399937 Avatar asked Sep 16 '12 01:09

user1399937


People also ask

Is there an IDE for BigQuery?

IDE (Integrated Development Environment) for BigQuery. Easily run queries, visualize your schema, auto-complete SQL queries, integrate with other databases, build CRUD applications on top of your BigQuery data, and so much more.

When should you not use BigQuery?

However, despite its unique advantages and powerful features, BigQuery is not a silver bullet. It is not recommended to use it on data that changes too often and, due to its storage location bound to Google's own services and processing limitations it's best not to use it as a primary data storage.

Can BigQuery be used for ETL?

BigQuery is a serverless, scalable cloud-based data warehouse provided by Google Cloud Platform. It is a fully managed warehouse that allows users to perform ETL on the data with the help of SQL queries.


1 Answers

Here's a working sample, based in part off of Michael's response:

using DotNetOpenAuth.OAuth2;
using Google.Apis.Authentication.OAuth2;
using Google.Apis.Authentication.OAuth2.DotNetOpenAuth;

using Google.Apis.Bigquery.v2;
using Google.Apis.Bigquery.v2.Data;

using Google.Apis.Util;
using System;
using System.Diagnostics;
using System.Collections.Generic;

namespace BigQueryConsole
{
    public class BigQueryConsole
    {
        // Put your client ID and secret here (from https://developers.google.com/console)
        // Use the installed app flow here.
        // Client ID looks like "9999999.apps.googleusercontent.com"
        static string clientId = "YOURCLIENTID";  
        static string clientSecret = "YOURSECRET";

        // Project ID is in the URL of your project on the APIs Console
        // Project ID looks like "999999";
        static string projectId = "YOURPROJECTID";

        // Query in SQL-like form
        static string query = "SELECT state, count(*) from [publicdata:samples.natality] GROUP BY state ORDER BY state ASC";

        public static void Main(string[] args)
        {
            // Register an authenticator.
            var provider = new NativeApplicationClient(GoogleAuthenticationServer.Description);

            provider.ClientIdentifier = clientId;
            provider.ClientSecret = clientSecret;

            // Initiate an OAuth 2.0 flow to get an access token

            var auth = new OAuth2Authenticator<NativeApplicationClient>(provider, GetAuthorization);

            // Create the service.
            var service = new BigqueryService(auth);
            JobsResource j = service.Jobs;
            QueryRequest qr = new QueryRequest();
            qr.Query = query;

            QueryResponse response = j.Query(qr, projectId).Fetch();
            foreach (TableRow row in response.Rows)
            {
                List<string> list = new List<string>();
                foreach (TableRow.FData field in row.F)
                {
                    list.Add(field.V);
                }
                Console.WriteLine(String.Join("\t", list));
            }
            Console.WriteLine("\nPress enter to exit");
            Console.ReadLine();
        }

        private static IAuthorizationState GetAuthorization(NativeApplicationClient arg)
        {
            // Get the auth URL:
            IAuthorizationState state = new AuthorizationState(new[] {  BigqueryService.Scopes.Bigquery.GetStringValue() });
            state.Callback = new Uri(NativeApplicationClient.OutOfBandCallbackUrl);
            Uri authUri = arg.RequestUserAuthorization(state);

            // Request authorization from the user (by opening a browser window):
            Process.Start(authUri.ToString());
            Console.Write("  Authorization Code: ");
            string authCode = Console.ReadLine();
            Console.WriteLine();

            // Retrieve the access token by using the authorization code:
            return arg.ProcessUserAuthorization(authCode, state);
        }
    }
}

This uses synchronous queries. For asyncronous queries, the code would be slightly different.

You'll need to reference both the BigQuery service dll (under 'Services' directory in the binary download), plus the other dlls in the 'Lib' directory. Binary release is here: http://code.google.com/p/google-api-dotnet-client/wiki/Downloads#Latest_Stable_Release

The .NET code is going to be very similar to the Java library code (they're generated off of the same API description): https://developers.google.com/bigquery/docs/developers_guide#batchqueries

We'll get more samples out there soon, but hopefully this helps in the meantime.

like image 158
Ryan Boyd Avatar answered Oct 29 '22 06:10

Ryan Boyd