Personal Documentation

Intro

As a developer you probably spend a lot of time dealing with documentation. Reading it online for packages, frameworks, API's etc. You have probably written some documentation (hopefully) for the projects that you work on, even if that documentation is not publicly available. It's vital to our field of work. Great documentation saves us time and makes our lives easier. Poor documentation is frustrating and wastes time. I'm here to advocate for a specific kind of documentation, that will be more useful to you than any other. Personal Documentation.

What is personal documentation?

Personal Documentation is for you to keep track of things that you learn. Even a person with great memory (which I don't have) will forget the specifics of a solution to a problem they solved years ago. Or even months, weeks or yesterday. We are all doomed to look up the same information over and over. To re-write boilerplate for design patterns we have already written. I once looked up a question on StackOverflow and found that the question had been answered by myself two years ago. I had no memory of this.

What format should personal documentation be in?

People do all kinds of things. Here is developer Josh Branchaud who takes notes entirely in Vim Script and posts them in a public git repo called TIL. That repo has over 11k stars...

In this Medium post The importance of Personal Documentation Mark Mo describes how he keeps his personal Infosec notes in a 68 tab 23mb Excel Spreadsheet.

So the short answer is whatever works for you. But I have some opinions on what the best format is.

Requirements

Ideally our personal documentation should be able to do the following.

• Be accessible on any device
• Be publicly available
• Support Categories and Tags
• Table of Contents (with hash links)
• Searchable
• Support Syntax Highlighting
• Support Links
• Support Photos
• Embedded content (iframes, videos, components)
• Low friction to add new content
• Version controlled

Also

• Free (or low cost)
• Low Tech Debt

The ideal format which satisfies all these requirements is Markdown files published with Static Site Generators (SSGs). Let's get into why that is.

Why Markdown

Practically all of the documentation that you are already reading and writing is already written using Markdown, for reasons you already know. It supports code blocks, links and tables natively. But more importantly, it can be parsed readily into websites using converters like remark.

That makes it ideal for satisfying our first two conditions:

• Be accessible on any device
• Be publicly available

But really it's greatest strength is its transferability.

Tech Debt

If our only goal was to publish online there are a lot of other tools that could handle this well. Any decent Headless Content Management System (HCMS) could manage this type of content.

• Wordpress
• Contentful
• Strapi
• Ghost

Strapi especially is a very cool piece of software. But the drawback to all of these is that they lock our content into a proprietary format that cannot be transferred (easily) to another management system. The last thing we want is to have to abandon our notes if we want to change the infrastructure around how we manage them.

Markdown is:

• future proof
• can be transferred readily between SSG's
• easy version control
• low distraction
• wide range of editors
• publication ready
• extensible

Recommended Editors

The best Markdown editor is your IDE of choice. You can instantly preview your content, can use linters to help you format, and can quickly manage your version control. Theres simply no debate on this for me. I use Visual Studio Code.

Notable Alternative Editors

Obsidian is a desktop/mobile application to write and organize Markdown based notes. You can pay to publish (like Ghost), and pay to sync between devices. There are a lot of plugins that extend it abilities like add a calendar or Kanban board. It has some cool features.

FrontMatter is an IDE Extension that turns the editor into a full on HCMS, with a media library and everything right in your IDE. It's designed to integrate with SSG's and has some really cool features.

Recommended Markdown Structure

There is a way to structure both the files and frontmatter for Markdown that gives the most compatibility with the most SSGs. I experimented by attempting to transfer my 250+ markdown posts to various SSGs and made note of the snags. I've broken that content out into another post here:

📘 Writing Static Site Generator Compatible Markdown

I'll just break out the conclusions here.

File Structure

Structure your files so that every folder is a post. Include the images associated with that post in the same folder.

blog
├── post-title-one
│   ├── index.md
│   ├── 1.png
│   └── 2.png
└── post-title-2
    ├── index.md
    ├── 1.jpg
    ├── 2.jpg
    └── 3.jpg

This allows for relative image links and keeps associated images with their text. Additionally this let's you write faster as you don't have to come up with unique image names. Image naming, like variable naming, is one of the sources of slowdown while writing. If you can eliminate that you can write faster, which means you will write more often.

YAML Format

---
title: "Some Title"
description: "All text strings wrapped in parentheses"
date: 2023-03-14
tags:
 - Markdown
 - Gatsby
 - Astro
 ---

Note especially the format of the date and tags array.

How To Publish

Once you have written your Markdown files you will commit them to a repository just like you would any other code. Then you set up a CI/CD pipeline to listen to the changes on that repository, automatically build your site files and publish them. It sounds a lot more complicated than it is and we'll get into how to do that here.

Hosts

There are several free hosting options available for statically generated sites. The best are

• Netlify
• AWS
• Github Pages

Netlify

Let me just jump to the end here and tell you that Netlify is the best. They are by FAR the easiest to setup, they have a very generous free tier, they are very fast. I have only good things to say about them. Just use Netlify.

The process is roughly this. Once you have setup your SSG and you are pushing changes to github, you login to Netlify and link the repo. Netlify will even automatically detect what your SSG is and identify the correct build folder for the files. You click some confirmation buttons and that's it. It will automatically listen for changes on the repo, build the site files and host them.

Once you have done that you just have to point your DNS records to the netlify address and you are live on a custom domain.

AWS

AWS can do everything we need it to do for free, but you have to set it all up manually. I've written several posts about how to do this.

📘 AWS CodePipeline for React App to S3 Bucket

📘 AWS CodePipeline for Dockerized React App to S3 Bucket

However as i've noted in the 2nd post, this is the path to sadness. It takes a lot of time to setup and you need to be familiar with AWS to even get started. If you are a power user who wants to be in control of everything have fun.

Github Pages

I know that this is a popular option, although I haven't used it. It can be used with various static site generators, but it requires manually setting up configuration files, which places makes it more complicated than Netlify, but still significantly easier than AWS.

Static Site Generators

Static Site Generators are the frameworks that will convert your Markdown notes into websites.

Here is the index of all the static site generators for you to choose from: Jamstack.org/generators. There are literally hundreds.

Some of them are better for Markdown files than others however. Also if you are like me and you are into React, then you get some really cool options. If you don't know React then use the filters on that site to help you find one that is more in tune with your skillset.

The options that I have taken the closest look at recently are the following

• Gatsby
• Docusaurus
• Astro
• Eleventy

Let's briefly go over the strong points of each of these.

Gatsby

Gatsby is what I use for this site, it's extremely flexible and powerful, but could be considered a little heavy. It uses GraphQL for data mapping, which you'll have to at least wrap your head around if you want to use it. I'm a big fan of Gatsby,and they were actually just acquired by Netlify so the two should go together nicely in the future.

I've been told it doesn't play nicely with TypeScript though.

Docusaurus

Docusaurus is build by Facebook so naturally it's React based. Its a framework built specifically for documentation and blogging. It has some really cool features like built in integration for Algolia Search. I was impressed by Docusaurus and I think I'm going to use it for the documentation for my bidbear.io project.

Astro

Astro seems to be more geared towards integration with Headless CMSs. It will work with Markdown files, but does not currently support the suggested file structure above. It does have TypeScript support though.

Eleventy

Eleventy is the odd man out here. Gaining popularity very quickly, it is very lightweight, un-opinionated and customizable. If I had more time to play around with this I think this would be a very good and stable option.

The first three are all React based, Eleventy is templated with something called Nunjucks which is a Javascript templating language similar to EJS.

Search

The best way to integrate search is by using Algolia. Don't be overwhelmed, it's actually quite easy to integrate. There is a plugin available for all the major SSGs, or they have a standalone package/component combo that can be integrated into any React application called DocSearch.

Extensions

If you want to ensure maximum compatibility between various static site generators or Markdown file organizers, you'll want to stick as closely as possible to the vanilla CommonMark Markdown standard. However by adding some extensions to our SSGs we can really take Markdown up a notch with some exciting features.

Syntax Extensions

It's extremely useful to be able to highlight rows in your syntax like this:

let someFunction = ()=>{
    if (x){
        submitApplication(x);
    };
}

to do that all I have to do is add an extension and then add comments // highlight-row into my code blocks

Another useful one is adding filenames

index.js

let someFunction = ()=>{
    if (x){
        submitApplication(x);
    };
}

and you'll also notice the little language tag on the upper right of those blocks. Also an extension.

Mermaid

Let's you add diagrams, charts, timelines and more into Markdown files

mermaid.js

MDX

Or if you really want to get crazy and you like React. MDX is a flavor of Markdown where you can directly mix in React components into your Markdown text, which basically opens up the door to anything you can dream of. Just install the MDX plugin in whatever React based SSG you are using and you are off to the races.

Why you should self publish your personal documentation

It's helpful to others. This website gets 11k individual users every month.

There is a lot of useful information out there that is behind paywalls, like medium.com. Don't write great content and then put it up behind a paywall. By self publishing you can ensure that people will actually be able to access your content.

Sites like StackOverflow can be very useful. They can also be hostile, especially to new users. The scope of each post is also extremely limited, you won't find any full walkthroughs there.

Comments

Recent Work

Basalt

basalt.software

Free desktop AI Chat client, designed for developers and businesses. Unlocks advanced model settings only available in the API. Includes quality of life features like custom syntax highlighting.

Learn More

BidBear

bidbear.io

Bidbear is a report automation tool. It downloads Amazon Seller and Advertising reports, daily, to a private database. It then merges and formats the data into beautiful, on demand, exportable performance reports.

Learn More