This is a survey of the state of the art in developer documentation - how the most popular open source tools manage their documentation.
I'm currently updating the documentation of an open source tool, and I thought I'd start with surveying the prior art of how successful projects are doing documentation today.
I'm particularly interested in how these projects invite contributions, comments, bugs, and handle versioning and scaling down to mobile UIs.
I thought I'd write this up in case it's useful to others. Our projects are typically not this popular, so the tradeoffs the popular projects make sometimes won't be right for us. Their solutions might be too heavyweight. But they've certainly been successful at growing a developer audience, and probably hit a lot of the pitfalls and iterated a few times, so it's useful to see how they solve documentation.
We'll be investigating these projects' docs, somewhat arbitrarily chosen as the top most starred projects on GitHub:
- Visual Studio Code
- Home Assistant
This methodology misses important projects that aren't hosted on GitHub, e.g. Python's Docs, or the Linux Kernel's Docs, but the point of this is to take a quick sample, not be exhaustive/representative.
Finally, remember the technology used to build docs is perhaps the least important part of the docs: the words that are written, and the information hierarchy presented to the user, are far more important than whether the site generator uses Go or Ruby. That said, let's dive in!
Example Page: https://www.tensorflow.org/api_docs/python/tf/concat
generate2.py script generates the TensorFlow docs from Markdown files in the TensorFlow Documentation Git repo, and Python Doc Comments, and Javadoc in the main TensorFlow repo. There is a Contributor Guidelines page to walk people through how to do this.
TensorFlow has no "Edit this page" button to take you to the source of the page, though there is a link to the Issue Tracker.
There is different API documentation per-version.
There is a survey: "Is this page helpful" allowing one-click voting from 1-5 stars.
Visual Studio Code
VS Code's docs include an "Edit this Page" link, which take you to the source Markdown file in GitHub. The Markdown has front-matter, like used by Jekyll and Hugo static site generators.
VS Code builds their docs with Gulp build tool, which invokes a
scripts/build.sh which doesn't appear to be present in the repository – probably a Microsoft-internal file. So I'm not sure if contributors can run the site locally! Microsoft's guidance is, naturally, to edit the Markdown file using Visual Studio Code.
There is a Contributor Guide, and a survey on each page:
Ansible has extremely good documentation, in that I can usually copy/paste from their examples to solve my problems. I wish all sites had examples as good as Ansible's.
Ansible has edit links at the bottom of each doc page. Some of the links 404, I've filed an issue about that.
There isn't a button to file a documentation bug – perhaps the doc bugs are low-quality? And there's no survey either.
Ansible uses the popular Sphinx Doc system to generate documentation, also used by Python and the Linux Kernel. Sphinx has docs written in ReStructured Text
.rst files, which is like Markdown but less popular.
Ansible's docs are part of the main Ansible git repository, not a separate repository. This makes it easier to update the code at the same time as the docs.
The docs are versioned, with a dropdown to select older versions.
Example Page: https://vuejs.org/v2/guide/conditional.html#v-else-if
Props to the Vue team for the #BlackLivesMatter banner!
Vue has a edit button at the bottom of each page, which takes you to Markdown source with YAML front-matter.
The docs are in a separate Git repository from the main vuejs project. The
README.md indicates the site is built with
hexo, a Node.JS-based static site generator I hadn't heard of before, and deployed automatically on merge using Netlify. I suppose it makes sense for a JS project to use a JS-based static site generator, so contributors don't have to learn another language ecosystem.
Translations exist, and are organised as forks of the original docs repository. That would make it difficult to keep them up to date, but does optimise for ease of starting a translation.
There are no surveys or links to file a bug on the documentation page. Versioned docs are accessible:
Example Page: https://reactjs.org/docs/forwarding-refs.html
Each React doc page has an "Edit this page" link at the bottom, which takes you to a Markdown file on GitHub with YAML Front-Matter. The docs are stored in a
reactjs.org repository, separate from the main React code.
Docs are available for some old versions of React, but strangely not all. There's no link to file a bug about the documentation, and no survey.
Example Page: https://angular.io/guide/user-input
Angular has a very subtle 'Pencil' icon at the top of each page, which takes you to a Markdown file in the main
angular GitHub repository. Angular's docs build with
yarn using dgeni, a custom doc generator used by Angular and Protractor projects.
I'm not seeing any survey on the page.
Example Page: https://keras.io/api/layers/initializers/
Keras has no "Edit this page" link. This would be a neat opportunity for someone to add one, for an extremely popular library.
Example Page: https://flutter.dev/docs/cookbook/design/drawer
Flutter has a subtle 'source code' button that takes you to Markdown files with YAML front-matter hosted on a
flutter/website repository on GitHub, separate from the main Flutter code.
Flutter also has a "bug" link to create an issue for their documentation, which pre-fills information about the doc page.
Next/previous links exist, unlike many of the other doc sites.
No survey is presented.
Example Page: https://kubernetes.io/docs/concepts/#kubernetes-objects
Kubernetes docs have a strong statement about not tolerating racism in their community. I appreciate it.
Kubernetes docs have a big edit button at the top. At the bottom they have an extremely comprehensive set of feedback loops: a Yes/No helpfulness survey, Edit This Page, and Create an Issue, and even last-modified-time for the page with a link to the latest commit! I particularly appreciate the last-modified-time, as it can be a signal to whether the page is up to date.
All translations are part of the same repository, but they don't seem to be updated at the same time.
The docs are Markdown with YAML front matter. The
README.md says the site is built using the Hugo Go-based static site generator. Kubernetes is also written in Go: again, we see a project choosing a static site generator written in the same language as the rest of the project.
Interestingly, it looks like the site was converted from Jekyll to Hugo two years ago, in a +123k line diff. That looks like a big job, though I hear Hugo has an automated converter. Looks like the rationale was: Hugo offers better multi-language support and faster build times.
We chose Hugo after months of research and conversations with other open source translation projects. [...] Hugo's multilingual support is built in and easy.
Another advantage of Hugo is that build performance scales well at size. At 250+ pages, the Kubernetes site's build times suffered significantly with Jekyll. We're excited about removing the barrier to contribution created by slow site build times.
It's always interesting to see people migrate from one platform to another, and their reasons for it! It wouldn't have been easy to migrate.
Kubernetes has a Contributors Guide for documentation. Continuous Deployment seems to be handled by Netlify.
Example Page: https://www.home-assistant.io/docs/
README.md indicates the site uses
bundle exec rake preview to see the site, and the
jekyll build. So this is a Jekyll-based website. Ruby-based Jekyll is the most popular static site generator.
Continuous Deploys are handled by Netlify. There's no survey on the docs page, or direct link to file issues about the page, but there is a "Need Help?" page which links to the GitHub Issues page for the docs.
This has been a short tour of how the most popular open source projects manage their documentation. What have we learned?
- Everyone's checking their code into version control as their source of truth. Nobody is using a database-backed Content Management System like Wordpress for their docs.
- Everyone's using a static site generator, and it doesn't really matter if it's custom or a standard one. Jekyll is most popular, but a lot of projects opt for a static site generator written in the same language as their library.
- Markdown remains extremely popular as a doc format, with all projects using Markdown except for Ansible, which uses ReStructured Text as required by Sphinx. None of the projects surveyed are using raw HTML for doc content, nor Wiki Markup, BBCode, AsciiDoc, or LaTeX. Of course there are different flavours of Markdown though...
- Static site generators seem to have mostly converged on "YAML front matter" as being the way to configure pages.
- "Edit This Page" buttons are popular to try to convert users into documentation contributors.
- "File a bug about this page" is not very popular – perhaps the bugs are low-quality and triaging them is a pain?
- Some teams are putting feedback surveys on their docs, but only some of the really big corporate-backed players who can pay people to do the analysis.
- Netlify is a popular way to do Continuous Deployment and hosting.
- Almost all sites are caring about responsive design, scaling down to mobile-phone size. I wonder how many people are reading API docs on their phone. It's probably not a trivial number! I'd love to see some audience numbers on this.
- Some projects keep their docs in the same git repository, and many don't. A best practice hasn't emerged here.
- Many projects allow you to view API docs for old versions, though this is not consistently supported.
- Nobody seems to have comments on their docs any more. Comments used to be very popular. PHP's user-contributed comments on docs were often famous for being more useful than the docs themselves. Perhaps nobody wants to moderate comments any more? Maybe everyone had Disqus comments then Disqus started putting ads on their page?
- Everyone is doing translations differently: some not at all, some as forks of the main repo, some as subdirectories.
Further Work / Questions not answered here
- How do these sites handle search? Static Site Generators don't typically output a search index.
- Do different patterns emerge for smaller, or less popular open source projects?
- What about the projects that aren't on GitHub, for whatever reason? Perhaps they're too big, or older, or just prefer being hosted elsewhere?
- If documentation is in a separate repository from the main code, how are they kept in sync?
- What do professional Tech Writers think are the best practices?
- Is there a way we could standardise these patterns, so every project doesn't have to reinvent best practices? What would that look like?
- What's the next big thing in documentation, that all of these sites are missing?