How to avoid header while exporting BigQuery table in to Google Storage

Question

I have developed below code which is helping to export BigQuery table in to Google storage bucket. I want to merge files into single file with out header, so that next processes will use file with out any issue.

    def export_bq_table_to_gcs(self, table_name):
        client = bigquery.Client(project=project_name)

        print("Exporting table {}".format(table_name))
        dataset_ref = client.dataset(dataset_name,
                                     project=project_name)
        dataset = bigquery.Dataset(dataset_ref)
        table_ref = dataset.table(table_name)
        size_bytes = client.get_table(table_ref).num_bytes

        # For tables bigger than 1GB uses Google auto split, otherwise export is forced in a single file.
        if size_bytes > 10 ** 9:
            destination_uris = [
                      'gs://{}/{}{}*.csv'.format(bucket_name,
                                       f'{table_name}_temp', uid)]
        else:
            destination_uris = [
                      'gs://{}/{}{}.csv'.format(bucket_name,
                                      f'{table_name}_temp', uid)]

        extract_job = client.extract_table(table_ref, destination_uris)  # API request
        result = extract_job.result()  # Waits for job to complete.

        if result.state != 'DONE' or result.errors:
            raise Exception('Failed extract job {} for table {}'.format(result.job_id, table_name))
        else:
            print('BQ table(s) export completed successfully')
        storage_client = storage.Client(project=gs_project_name)
        bucket = storage_client.get_bucket(gs_bucket_name)
        blob_list = bucket.list_blobs(prefix=f'{table_name}_temp')
        print('Merging shard files into single file')
        bucket.blob(f'{table_name}.csv').compose(blob_list)

Can you please help me to find a way to skip header.

Thanks,

Raghunath.

Answer 1

We can avoid header by using jobConfig to set the print_header parameter to False. Sample code

job_config = bigquery.job.ExtractJobConfig(print_header=False)
extract_job = client.extract_table(table_ref, destination_uris,
                                   job_config=job_config)

Thanks