import pyarrow not working <- error is "ValueError: The pyarrow library is not installed, please install pyarrow to use the to_arrow() function."

Question

I have tried installing it in the terminal and in juypter lab and it says that it has been successfully installed but when I run df = query_job.to_dataframe() I keep getting the error " ValueError: The pyarrow library is not installed, please install pyarrow to use the to_arrow() function.". I have no idea how to fix this. Any advice? I am trying to access data from google data studio ultimately with the code,

from google.cloud import bigquery
import pandas
import numpy
import pyarrow
bigquery_client = bigquery.Client()
import os 
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] ='full file path here.json'
QUERY = """
SELECT * 
FROM `warehouse`
LIMIT 100
"""
query_job = bigquery_client.query(QUERY)
df = query_job.to_dataframe()

Answer 1

I had the same issue. Fixed after the following:

pip install --upgrade 'google-cloud-bigquery[bqstorage,pandas]'

Source: https://cloud.google.com/bigquery/docs/bigquery-storage-python-pandas

Answer 2

I got the same error message ModuleNotFoundError: No module named 'pyarrow' when testing your Python code. This behavior disappeared after installing the pyarrow dependency with pip install pyarrow.

Edit: It worked for me once I restarted the kernel after running pip install pyarrow

Answer 3

I had the same issue because I had pyarrow 2.0, however you will need version 1.0.1 . Try running this line: pip install pandas-gbq==0.14.0