Can't install pyarrow on OSX / Python 3.9: is this me or an incompatible package?

Question

I'm trying to install pyarrow with pip3 on OSX 11.0.1, and getting error messages.

I'm using Python 3.9 and not sure if that is the problem.

Here is the error summary:

  ERROR: Command errored out with exit status 1:
   command: /usr/local/opt/[email protected]/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip install --ignore-installed --no-user --prefix /private/var/folders/mk/2fgx_1s96zjd1r9xzhs2ht_00000gn/T/pip-build-env-pev1z3i2/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'cython >= 0.29' 'numpy==1.14.5; python_version<'"'"'3.7'"'"'' 'numpy==1.16.0; python_version>='"'"'3.7'"'"'' setuptools setuptools_scm wheel
       cwd: None

Here is the complete (very long) error output, I'm not sure how to read this to find the source of the problem: https://pastebin.com/RQcEuwbz

It seems like perhaps pyarrow had problems with Python 3.9, but these should now have been fixed? I'm a bit stumped about what to do next.

UPDATE: Ah, found a PR to make pyarrow work with Python 3.9 that hasn't been merged yet - so I guess the pip version of pyarrow doesn't work with Python 3.9 and that is the problem. It looks like the PR hasn't been merged because it doesn't work on Windows, but that doesn't matter to me since I'm on OSX. Is there a way I could tell pip to install pyarrow from the version in the PR?

Answer 1

Update 2021: Pyarrow has had wheels for Python3.9 since Feb/2021, Pyarrow Version 3.0.0

See Pyarrow Github

I just had to upgrade the pip version.

(Edit queue of the previous answer is full so I am posting an answer myself)

Answer 2

As of today Dec. 16 2020, neither fastparquet nor pyarrow have wheels for Python 3.9

This will hopefully change soon.