Python docstrings to GitHub README.md

Question

How do I transcode Python documentation strings to a GitHub readme.md file?

Even though it seems like something everyone does, I cannot seem to get a decent solution and I am assuming it should be easy, so it seems unlikely folks are going throw two converters…

What I have tried

pydoc Actually simple. The output of pydoc is manpages (groff format for Unix systems). Which is a dead end as man to md is not a thing. Via HTML, pydoc3 -w + pandoc, utterly munges the docstrings to bits.

Custom code There seems to be lots of short custom code, but for the few I tried the output does not seem to be as good as that pydoc, which has a summary, adds inherited methods and lists some attributes.

mkdocs. It was suggested somewhere. It just pollutes my folder as it is a misleading name as is a not docstrings > md converter, but a md > html.

Sphinx + Pandoc. After fixing an UTF-8 issue, I gave up on Sphinx as I have a single .py script to convert and the autodoc setting of the quickstart did not parse my script. I tried in Python to import sphinx.ext.autodoc, but TBH the documentation was too long and I gave up.

NB

There is a year-old unanswered Stack Overflow question on the topic, but I hope that by giving a lot more detail I will get an answer.

Answer 1

The other answers are great. But I thought I (the OP) ought to share what I do these days (a year or two after the question).

I use Sphinx and its Markdown extension. Do the following:

TL;DR: See Gist snippet.

Sphinx-markdown-builder

You need sphinx-markdown-builder python module.

 pip install sphinx sphinx-markdown-builder;

Run Sphinx

Not the autodoc, the apidoc!

sphinx-apidoc -o Sphinx-docs . sphinx-apidoc --full -A 'Matteo Ferla'; cd Sphinx-docs;

Configuration

Fix the conf.py file, by following the following or just lazily copy paste the echo command below.

Manual

First uncomment the lines. These are otherwise commented out.

import os
import sys
sys.path.insert(0, os.path.abspath('../'))

Note the change to ../

One weirdness is that the magic methods get ignored. To override this, add this anywhere:

def skip(app, what, name, obj, would_skip, options):
    if name in ( '__init__',):
        return False
    return would_skip
def setup(app):
    app.connect('autodoc-skip-member', skip)

A thing to note: The docstrings ought to be written in restructuredtext (RST). If they are in Markdown, you need to add a mod - see this. The two are similar, but different. For example, a single backquote is required for <code> in Markdown, while two are for RST. If in doubt, several blog posts discuss the merits of RST documentation over Markdown.

Typehinting

RST typehints (:type variable: List) are obsolete as proper typehinting def foo(variable: Optional[List[int]]=None) -> Dict[str,int]: has been introduced since 3.6. To make these work:

 pip install sphinx-autodoc-typehints

And add 'sphinx_autodoc_typehints' at the end of the extensions list. Note the package has hyphens while the module has underscores.

TL;DR

Copy paste this:

echo " import os
import sys
sys.path.insert(0,os.path.abspath('../'))
def skip(app, what, name, obj,would_skip, options):
    if name in ( '__init__',):
        return False
    return would_skip
def setup(app):
    app.connect('autodoc-skip-member', skip)
extensions.append('sphinx_autodoc_typehints')
 " >> conf.py;

Showtime

Then it is showtime.

make markdown;

Copy the files and clean however you fancy.

mv _build/markdown/* ../; rm -r Sphinx-docs;

Repeat Apidoc for new files

It should be noted that when new files are added, the apidoc command needs to be repeated. Nevertheless, I highly recommend generating documentation midway as I often realise I am doing something wrong when I see the docs.

But briefly, apidoc will add for each file a automodule command, so this could be added manually or even expanded:

.. automodule:: my_module
   :members:
   :inherited-members:
   :undoc-members:
   :show-inheritance:

There's also the commands autoclass, autofunction, autoexception, for specific cases. In the case of autoclass if the class inherits many base classes in separate files to rightfully keep filesizes under 250 lines, the property :inherited-members: is a nice addition to this —thus avoiding having to describe the private base classes.

Read the docs: the common way

It should be said that there's a trend to not have documentation in GitHub but in Read the docs. My guess is because:

• avoids this docstrings-to-markdown business
• some users get confused by GitHub
• looks nicer
• other do it

Despite this, it requires some set up due to the module requirements. In another SO post is a long list of pitfalls and tricks —briefly IMO users, such as myself, make three mistakes:

1. missing modules or the target module
2. forget to hard refresh the browser
3. enabling the sphinx.ext.autodoc extension

However, if one has written markdown documentation in GitHub these can be imported too, with a simple workaround in the docs/source/config.py file:

from m2r2 import parse_from_file  # noqa

for markdown_filename, srt_filename in {'../../README.md': 'introduction.rst', '../../otherfile.md': 'side_note.rst'}.items():
    with open(srt_filename, 'w') as fh:
        fh.write(parse_from_file(markdown_filename))

The working path in config.py is its folder, not the base folder of the repo. m2r is a markdown to restructured text converter, but it has been abandoned and is broken due to changes in a different module: r2r2 fixes it. One issue is that links may need to be checked, especially if files moved around or are relative to the base URL (slash prefixed), eg. [Foo](/md_docs/foo).

Answer 2

I have found some other options for doing this:

https://github.com/coldfix/doc2md

Little convenience tool to extract docstrings from a module or class and convert them to GitHub Flavoured Markdown. Its purpose is to quickly generate README.md files for small projects.

https://github.com/freeman-lab/myopts

This module provides a command line tool for parsing a Python file and generating nice looking markdown with your function definitions. It's extremely opinionated and rigid! But also extremely easy to use.

Answer 3

I have a little bit of code that I use to generate an index file from a project. It's not exactly what you're looking for, but with a little wiggle you could add an if statement for py files (where I only have html and png) and grab the doc = "your DocStrings."... https://gist.github.com/Krewn/6e9acdadddb4bf2a56c0

# WARNING RUNNING THIS FILE WILL OVERIDE EXISTING readme.md FILE IN CWD

import os

class indexer:
    path = "~"
    username = "" # !!! Enter your github username in the provided quotes.
    site = "http://"+username+".github.io"
    proj = ""     # !!! Enter your repository name in provided quotes.
    prod = []
    loc=[]

    def __init__(self,p):
        self.path=p
    def fprep(self,name):
        name.replace(".","")
        name.replace("\\","/")
        return(name)
    def refPrep(self):
        ref = self.site+"/"+self.proj
        for qw in self.loc:
            ref+="/"+qw
        return(ref)
    def HtmlFrek(self,adir):
        self.loc.append(adir)
        os.chdir(adir)
        pys = [f for f in os.listdir('.') if os.path.isfile(f) and f.split(".")[len(f.split("."))-1]=="py"]
        for i in pys:
            Open the file i get the __doc__ string and append it to ret
        for k in folders:
            if(k.__contains__(".")):
                continue
            ret+=self.HtmlFrek(k)
        os.chdir("..")
        del self.loc[len(self.loc)-1]
        return(ret)

    def HtmlProd(self):
        ret = ""
        pys = [f for f in os.listdir('.') if os.path.isfile(f) and f.split(".")[len(f.split("."))-1]=="py"]
        for i in pys:
            Open the file i get the __doc__ string and append it to ret
        folders = [f for f in os.listdir(".") if not os.path.isfile(f)]
        for k in folders:
            if(k.__contains__(".")):
                continue
            ret+=self.HtmlFrek(k)
        self.prod = ret
        return(ret)

i = indexer(".")
q=i.HtmlProd()
#print i.prod

w = open("readme.md","w")
w.write(q)
w.close()

Answer 4

I've found pydoc-markdown quite easy to use. The first command will install the library and the second one will create a README from your module named MY_MODULE:

pip install pydoc-markdown
pydoc-markdown -m MY_MODULE -I $(pwd) > README.md