Scrapy on M1 Mac: MemoryError: Cannot allocate write+execute memory for ffi.callback()

Question

I'm new to scrapy, and recently started using it on the M1 MacBook Air. I've encountered an issue.

For example, when I try to do something like this:

scrapy shell bbc.com

It would return me: MemoryError: Cannot allocate write+execute memory for ffi.callback(). You might be running on a system that prevents this. For more information, see https://cffi.readthedocs.io/en/latest/using.html#callbacks

It is worth noting that if I do the exact same thing on Windows, everything works. Also, when I use scrapy shell on websites like http://quotes.toscrape.com/ or HTTP websites on the M1 Mac, it also works. Seems like a HTTPS issue but I'm really not sure.

Any help will be greatly appreciated.

Answer 1

conda uninstall --force cffi
pip install cffi

https://github.com/snowflakedb/snowflake-connector-python/issues/1205