Getting started with Sphinx when your source files aren't in the project base folder [duplicate]

Question

I have a question regarding the Sphinx autodoc generation. I feel that what I am trying to do should be very simple, but for some reason, it won't work.

I have a Python project of which the directory is named slotting_tool. This directory is located at C:\Users\Sam\Desktop\picnic-data-shared-tools\standalone\slotting_tool

I set up Sphinx using sphinx-quickstart. Then my directory structure (simplified) is as follows:

slotting_tool/
|_ build/
|_ source/
|___ conf.py
|___ index.rst
|_ main/
|___ run_me.py

Now, I set the root directory of my project to slotting_tool by adding the following to the conf.py file.

import os
import sys
sys.path.insert(0, os.path.abspath('..'))

Next, I update my index.rst file to look like this:

.. toctree::
   :maxdepth: 2
   :caption: Contents:

.. automodule:: main.run_me
   :members:

When trying to build my html using the sphinx-build -b html source .\build command, I get the following output, with the no module named error:

(base) C:\Users\Sam\Desktop\picnic-data-shared-tools\standalone\slotting_tool>sphinx-build -b html source .\build
Running Sphinx v1.8.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: [] 0 added, 1 changed, 0 removed
reading sources... [100%] index
WARNING: autodoc: failed to import module 'run_me' from module 'main'; the following exception was raised:
No module named 'standalone'
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices... genindex
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 1 warning.

The HTML pages are in build.

There are no HTML pages that refer to run_me.py in build. I have tried setting my root directory to all different kinds of directories and I have tried replacing all dots . with backslashes \ and so forth, but can't seem to find out what I'm doing wrong.

By the way, the statement that standalone is not a module is in fact true, it is just a directory without an __init__.py. Don't know if that might have caused some trouble?

Anyone have an idea?

Answer 1

This is the usual "canonical approach" to "getting started" applied to the case when your source code resides in a src directory like Project/src instead of simply being inside the Project base directory.

Follows these steps:

1. Create a docs directory in your Project directory (it's from this docs directory the commands in the following steps are executed).

2. sphinx-quickstart (choose separate source from build. Places .html and .rst files in different folders).

3. sphinx-apidoc -o ./source ../src

4. make html

This would yield the following structure (provided you .py source files reside in Project/src):

Project
|
├───docs
│   │   make.bat
│   │   Makefile
│   │
│   ├───build
│   └───source
│       │   conf.py
│       │   index.rst
│       │   modules.rst
│       │   stack.rst
│       │
│       ├───_static
│       └───_templates
└───src
        stack.py

In your conf.py you'd add (after step 2):

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join('..', '..', 'src')))

Also include in conf.py:

extensions = ['sphinx.ext.autodoc', 'sphinx.ext.napoleon']

And in index.rst you'd link modules.rst:

Welcome to Project's documentation!
================================

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   modules
      
   
Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

Your stack.rst and modules.rst were auto-generated by sphinx-apidoc, no need to change them (at this point). But just so you know this is what they look like:

stack.rst:

stack module
============

.. automodule:: stack
   :members:
   :undoc-members:
   :show-inheritance:

modules.rst:

src
===

.. toctree::
   :maxdepth: 4

   stack

---

---

After `make html` open `Project/docs/build/index.html` in your browser, the results:

and:

Answer 2

Let's take an example with a project: dl4sci-school-2020 on master branch, commit: 6cbcc2c72d5dc74d2defa56bf63706fd628d9892:

├── dl4sci-school-2020
│   ├── LICENSE
│   ├── README.md
│   ├── src
│   │   └── __init__.py
│   └── utility
│       ├── __init__.py
│       └── utils.py

and utility package has a utils.py module:

Follow this process(FYI, I'm using sphinx-build 3.1.2):

1. create a docs/ directory under you project:

mkdir docs
cd docs

2. start sphinx within docs/, and just pass your project_name, your_name & version of your choice and rest keep defaults.

sphinx-quickstart

you will get below auto-generated in your docs/ folder

├── docs
│   ├── Makefile
│   ├── build
│   ├── make.bat
│   └── source
│       ├── _static
│       ├── _templates
│       ├── conf.py
│       └── index.rst

Since, we created a separate docs directory so we need sphinx find where to find build files and python src module. So, edit the conf.py file, you can use my conf.py file too

import os
import sys
basedir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..'))
sys.path.insert(0, basedir)

Now, to enable access to nested multiple packages & modules if any, you need to edit index.rst file.

.. toctree::
   :maxdepth: 2
   :caption: Description of my CodeBase:

   modules

The modules picks up content from modules.rst file which we will create below: Make sure you're still in doc/ to run the below command

sphinx-apidoc -o ./source ..

The output you get:

├── docs
│   ├── Makefile
│   ├── build
│   ├── make.bat
│   └── source
│       ├── _static
│       ├── _templates
│       ├── conf.py
│       ├── index.rst
│       ├── modules.rst
│       ├── src.rst
│       └── utility.rst

now run:

make html

Now, go and open in browser of your choice,

file:///<absolute_path_to_your_project>/dl4sci-school-2020/docs/build/html/index.html

have you beautiful documentation ready

https://i.sstatic.net/5pvLu.jpg

FYI, You can switch any theme of your choice, I found sphinx_rtd_theme and extension sphinxcontrib.napoleon super dope!. Thanks to their creators, so I used it.

below does the work!

pip install sphinxcontrib-napoleon
pip install sphinx-rtd-theme

You can host your documentation it on readthedocs enjoy documenting your code!