State of reasonml.org 2020-Q2 / Pt. 2
In our previous blogpost we talked about the rationale and general concepts of the
reasonml.org project. In this post we want to highlight how we make sure that our documentation is easy to manage and easy to refactor.
When we extracted the API docs from the BuckleScript codebase into markdown files, we noticed that a substantial number of examples in the JS / Belt module did not compile when we tried to copy them in a different file (mostly due to missing context, or not fully qualified module references). One can guess how many people have been frustrated with compiler errors. For a project of the depth and scope of BuckleScript, automating away these sorts of mistakes is critical so the core team can continue focusing on delivering faster compile times and more features.
To tackle this issue, we built a simple mechanism to automatically verify every API example. It extracts all the
reason example code fences from the markdown, and then compiles them with BuckleScript.
A markdown example with a code snippet might look like this:
some text with some broken `List.head` example: ```reason example Belt.List.head(); /* None */ List.head([1, 2, 3]); /* This should actually be Belt.List */ ```
Running the test with the broken example would produce:
As you can see, the error points you to the right error location within the
list.mdx file by replacing the relative snippet line number with the absolute line number of the mdx file.
A full run of the example test suite on current master yields following results:
Total Examples: 693 Failed: 0 Success: 693
Leveraging the machines do the hard work of tracking and verifying nearly 700 snippets means that the rest of us can invest in higher-level experiences.
We also created a test to verify hyperlinks that cross-reference to other relative pages, which makes it hard to link to non-existing pages within reasonml.org.
As an example, this is how a hyperlink to a non-existent page will be reported:
node scripts/test-hrefs.js -------Results for './pages/markdown-guide.mdx'---------- ./pages/markdown-guide.mdx:14 => ok ./pages/markdown-guide.mdx:14 => failed / Unknown href './doesnt-exist' in line 14:52 ----------- Summary: Total Links: 1125 Failed: 1 Success: 1125 Tip: You can also run tests just for specific files / globs: `node scripts/test-hrefs.js "pages/belt_docs/*.mdx"` ----------- Error Preview: ./pages/markdown-guide.mdx: Unknown href './doesnt-exist' in line 14:52
Here are the statistics for all internal cross-links within the boundaries of the documentation platform:
Total Links: 1125 Failed: 0 Success: 1125
The summary reports 1125 relative links that point to different sections on reasonml.org. This includes relative hrefs such as
This feature itself already showcases the value we provide with our new platform. Our goal is to make contributions and doc updates almost frictionless. We can even use this mechanism to test e.g. major compiler upgrades.
All the tests run automatically on CI, and since we are using NextJS to build our platform, we also offer live preview deployments for each PR which makes it easier for maintainers and contributors to spot visual regressions as well.
You might ask yourself why we did the markdown extraction process in the first place. As for right now, the tool for generating documentation from code comments, called
odoc, is not capable of generating simple structured data, such as JSON.
odoc data is pretty complex, the only output format right now is plain Html. You probably have seen
odoc generated Html already, such as the Reason API docs on reasonml.github.io.
We want to achieve better UX and integrate the documentation seamlessly in our design system and application. In our next post of the series, we will explain how we'll achieve this with the aid of our doc-tools project.
So far we've mostly been talking about what we did and what technical tools we build. Actually we think it's more important to talk more about you, the community, taking part in all of this.
Right now, the official documentation websites are maintained by the Reason / BuckleScript core team. Some websites are part of the source code repository, such as ReasonReact, some are separate. Issue trackers are flooded with requests, and the maintainers only have a limited amount of time to tackle them. We try to share the burden by slowly but steadily taking over the documentation part so the core team can focus on improving the compiler speed or shipping new features.
Also knowing just one project well is not enough to build a documentation platform for the whole ecosystem. For instance, it doesn't make sense to only write about Reason syntax features when most of the interesting features are coupled to the Reason JS compiler (BuckleScript).
We tried our best to raise enough funding and set our budgets in a way that allows us to allocate time and resources to work full time on the documentation for 2020. That means that we can build up competences with a holistic view on the whole Reason platform, and can spend enough time on reviewing PRs, giving feedback and allocating tasks. We also want to make sure to lower the bus factor by adding more core documentation maintainers in the future. Continuous and deep collaboration, proper guidance and mutual trust will be the foundation for this.
To wrap things up, we think that we'll be able to establish a reliable documentation focused culture to share responsibilities within the community. That said, we'll try to be clear on our processes and goals, since we believe these are key aspects for successful collaboration.
In this article we described how test automation across the whole Reason documentation platform can help us to proactively detect and fix errors. We also explained how we'll improve the contribution process for community curated docs.
In case you have any questions or comments, make sure to let us know in the Reason Discord
Happy docs browsing!
This article is part of the "State of reasonml.org 2020-Q2" series (please stay tuned for the upcoming posts):
Part 4) It's all Opinions: We'll dive into our documentation philosophy, our vision of the Reason Platform and ideas on how to tackle the "Reason Native" documentation.
Part 1) Introduction to reasonml.org