Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dynamically getting a list of standard library python packages names

I'd like to get a list of names of all standard lib packages.

By this, I mean those listed on I thought of parsing https://docs.python.org/3/library/: strings name for which

__import__(name)

sys.builtin_module_names looked promising, but that's not it.

I thought of parsing ~/.pyenv/versions/3.8.6/lib/python3.8/ or https://docs.python.org/3/library/, but surely there's a better way!

Addendum

For those who need an X for a Y: I'm statically navigating the imports of packages to analyze them -- namely, to see what third party packages are used, what standard libs, with what frequency, etc.

like image 797
thorwhalen Avatar asked Apr 20 '21 18:04

thorwhalen


People also ask

What packages are in the Python standard library?

Python standard library. The Python Standard Library contains the exact syntax, semantics, and tokens of Python. It contains built-in modules that provide access to basic system functionality like I/O and some other core modules. Most of the Python Libraries are written in the C programming language.

Is requests in Python standard library?

The Python Requests module is a popular alternative to the standard library's HTTP handling (e.g. urllib2).

Is NumPy standard library?

It is a third-party library (i.e. it is not part of Python's standard library) that facilitates numerical computing in Python by providing users with a versatile N-dimensional array object for storing data, and powerful mathematical functions for operating on those arrays of numbers.

What is the name of the common library that is available with all Python distributions?

The Python Standard Library is a collection of exact syntax, token, and semantics of Python. It comes bundled with core Python distribution.

What are the different libraries in Python?

This Python Library Tutorial, we will discuss Python Standard library and different libraries offered by Python Programming Language: Matplotlib, scipy, numpy, etc. So, let’s start the Python Libraries Tutorial.

How do I list all installed packages in Python?

How to List Installed Python Packages The Pip, Pipenv, Anaconda Navigator, and Conda Package Managers can all be used to list installed Python packages. You can also use the ActiveState Platform’s command line interface (CLI), the State Tool to list all installed packages using a simple “state packages” command.

What is a package in Python?

A package is a library that can be installed using a package manager like rubygems or npm. The Python Standard Library is a collection of exact syntax, token, and semantics of Python. It comes bundled with core Python distribution. We mentioned this when we began with an introduction.

What is the Python standard library?

The Python Standard Library is a collection of exact syntax, token, and semantics of Python. It comes bundled with core Python distribution. We mentioned this when we began with an introduction. It is written in C, and handles functionality like I/O and other core modules. All this functionality together makes Python the language it is.


1 Answers

I will post my solution here, but will wait to see if there's better before accepting it as the answer.

Install unbox and do this:

from unbox import builtin_module_names

This should give you a set a names for the python version of your environment (2.7 and 3.5-3.9 supported).

To get these, I parsed the list out of https://docs.python.org/{version}/library/ html pages and filtered out those that were not importable (from 3.8). You can verify that all the names are importable by doing:

for name in builtin_module_names:
    _ = importlib.import_module(name)

These names are contained in the package's data folder (as .csv files) and can be found here on github.

Note that you won't find all modules there -- only those that are (1) documented on said page and importable. For example, easter eggs such as this and antigravity won't be listed. You can find them in the larger scanned_standard_lib_names set, obtained by scanning local files:

from unbox import scanned_standard_lib_names
assert scanned_standard_lib_names.issuperset({'this', 'antigravity'})
like image 177
thorwhalen Avatar answered Oct 21 '22 10:10

thorwhalen