Possible to publish docusaurus cross-repo from multiple-repositories, to aggregate docs together?

Question

My question: Are there any docusaurus features out of the box (beyond https://github.com/facebook/docusaurus/pull/764) that will make the following easier? (I've asked this here because their github question template tells me issues of that type will be shut, and to ask them over here instead).

In my company we have several different repositories containing documentation in markdown and also markdown generated from source-code documentation from a variety of different coding languages.

I would like to explore using docusaurus to define a central site, but pull in documentation from a number of different repositories.

I'd like to do that:

• to get a centralised search index
  • to aid discoverability
• to get a centrally owned consistent theme/UX
• to publish onwards into confluence so that non-technical users can find and browse content if that becomes the company policy to use ( :( )
• to retain all the advantages of docs-close-to-code

This is the structure that docusaurus expects:

docs/            # all documentation should be placed here
website/
  blog/
  build/         # on yarn run build
  core/
    Footer.js
  package.json
  pages/
  sidebars.json
  siteConfig.js
  static/

and this is the structure of published website that I'd like to end up with:

/v1/products/{product}/{version}/{language}/{content as from docs/}
# e.g.
/v1/products/spanner/{version}/en-GB/readme.html

/v1/internal/{gh-org}/{gh-repo}/{language}/{content as from docs/}
#e.g.
/v1/my-org/my-repo/{version}/en-GB/readme.html
/v1/my-org/my-repo/{version}/en-GB/proto-generated.html

(v1 is there because I predict I'll have forgotten something, and it lets me hedge against that and make later breaking-change redirects easier)

and I think therefore this is the intermediate structure I'll need to aggregate things into:

docs/
  product/
    language/
      prose|generated-lang 
  gh-org/
    repo/
      language/
        prose|generated-lang
website/
  blog/
  product/
    language/
      prose|generated-lang 
  gh-org/
    repo/
      language/
        prose|generated-lang
  core/
    Footer.js
  package.json
  pages/
  product/
    language/
      prose|generated-lang 
  gh-org/
    repo/
      language/
        prose|generated-lang
  sidebars.json
  siteConfig.js
  static/
  product/
    language/
      prose|generated-lang 
  gh-org/
    repo/
      language/
        prose|generated-lang

... does that hang together?

I can git clone via bash or submodules quite readily to arrange this; that's not particularly an issue. I want to know if there are things that exist already that will allow me to avoid needing to do that - e.g. native features of docs-site tools, bazel rules, whatever.

Answer 1

If you don't require a single page app and don't need React (docusaurus mentions this here), you can accomplish this using MkDocs as your static site generator and the multirepo plugin. Below are the steps to get it all setup. I assume you have Python installed and you created a Python venv.

1. python -m pip install git+https://github.com/jdoiro3/mkdocs-multirepo-plugin
2. mkdocs new my-project
3. cd my-project
4. Add the below to your newly created mkdocs.yml. This will configure the plugin.

plugins:
  - multirepo:
      repos:
        - section: Repo1
          import_url: {Repo1 url}
        - section: Repo2
          import_url: {Repo2 url}
        - section: Repo3
          import_url: {Repo3 url}

Now, you can run mkdocs serve or mkdocs build, which will build a static site with all the documentation in one site.

This will:

• get a centralised search index to aid discoverability
• get a centrally owned consistent theme/UX (I suggest using Material for MkDocs)
• retain all the advantages of docs-close-to-code

A similar plugin could probably be written for docusaurus.

Answer 2

You can use a script to pull those md files, put them in the right location and then build docusaurus. You can do this with Github's actions upon any change to one of your source repos automatically