Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to use Python lxml on Google App Engine?

Can I use Python lxml on Google App Engine? (or do I have to use Beautiful Soup?)

I have started using Beautiful Soup but it seems slow. I am just starting to play with the idea of "screen scraping" data from other websites to create some sort of "mash-up".

like image 625
Nick Perkins Avatar asked Dec 12 '09 21:12

Nick Perkins


People also ask

Is Python used in App Engine?

App Engine offers you a choice between two Python language environments.

Which programming environment is used for Google App Engine?

Google App Engine provides four possible runtime environments for applications, one for each of four programming languages: Java, Python, PHP, and Go.

Is Google App Engine part of GCP?

App Engine is part of Google Cloud Platform, a suite of services for running scalable applications, performing large amounts of computational work, and storing, using, and analyzing large amounts of data.


2 Answers

EDIT: The lxml library is now supported.


Short answer: you can't.

From AppEngine's docs: "Application code written for the Python environment must be written exclusively in Python. Extensions written in the C language are not supported"

Now whether you must use BeautifulSoup or not, it really depends on how you plan to use it. Update the question with details and I'll try to update here with other suggestions, if possible.

like image 166
Caio Romão Avatar answered Sep 19 '22 15:09

Caio Romão


To add to Caio's revised answer, you can use lxml and this is how to do it:

In your app.yaml file, add the following:

libraries:
- name: lxml
  version: "2.3"   # or "latest"

Now import lxml will work in your Python script.

Here is the link to the official documentation.

like image 22
Jim Pivarski Avatar answered Sep 20 '22 15:09

Jim Pivarski