---
title: "Code and Repo Documentation"
description: READMEs and using Github Pages and Quarto to make websites for documentation
date: September 1, 2023
date-modified: today
author: 
  - name: Frank Aragona
    email: frank.aragona@doh.wa.gov
    affiliations:
      - Washington Department of Health
      - Data Integration/Quality Assurance
knitr: true
editor: 
  markdown: 
    wrap: 72
filters: 
  - line-highlight
categories:
  - data democratization
  - documentation
  - Quarto
---

# Introduction

![](images/nwpage_tree_logo.png){style="float: right;"
fig-alt="Hex logo that says \"Data Science in a Box\"" fig-align="right"
width="250"}

This site was created using [Quarto](https://quarto.org/), Github, and
uses a [Github Action](https://github.com/features/actions) to
automatically render when a commit is pushed to the main branch of this
repository.

Quarto is a framework for creating documentation, slideshows, articles,
blogs, books and websites using markdown. It can execute R, Python and
other programming languages within the document.

Github Actions uses a `.yml` file in the repository to trigger an action
based on a certain event. In this case, when a commit is pushed to the
main branch the `.yml` will trigger this Quarto website to render to the
`gh-pages` branch of the repository and publish the github page. This
section will give details on how to

1.  Create the website
2.  Create, edit, and troubleshoot the Github Action to render the site

# Create a Quarto Website

Go to:
`File > New Project… > New Directory > Quarto Website > Create Project`

There is an option to use `renv` which is a virtual environment for the
R project. This is helpful, but if you're unfamiliar with `renv` you can
un-check it.

![](images/create_quarto.gif)

<br>

The Quarto project will come with a `_quarto.yml` file. This is similar
to an rmarkdown `yml` header where you can specify how you want your
document to be style and what format to output it to.

# How to edit the website (add chapters and change the style)

Most websites have a main file that sources all the htmls, css,
javascript and other files into one. Quarto is the same - it sources all
the markdown files and css files into one `.yml` file that dictates the
output appearance and functionality of the site as a whole. Think of
`.yml` files or headers as instructions for a document's style, output,
and functions. It is the same thing as the `yaml` header in rmarkdown
files.

## Edit/Add Sections and Chapters

To add a section, open up the `_quarto.yml` file and scroll to the
`navbar` section

::: myframe
```{yaml filename="_quarto.yml"}
#| source-line-numbers: "7-40"
project:
  type: website
website:
  title: "COE Github Standards"
  search: true
  
  navbar: 
    background: primary
    left: 
      - text: Home
        href: index.qmd
      - text: Github Organization Standards
        menu: 
          - href: std/security.qmd
            text: "0: Security Standards"
          - href: std/lic.qmd
            text: "1: Choosing a License"
          - href: std/templates.qmd
            text: "2: Org Policy Setting"
```
:::

<br>

This is where all of the `qmd` files are sourced and the instructions on
how to format and style the navigation bar in the website.

Currently, the project is set up to have each section have it's own drop
down menu in the navbar. In a section, use `- href:` to specify a file
and `text:` to give the file a custom name in the website.

Each chapter exists within a sub-folder, so to add a chapter make sure
create the qmd in its sub-folder and then reference the sub-folder and
chapter in the `.yml`. For example, if you make a new chapter called
`new-chapter.qmd` and it exists in the `covid` section/sub-folder, you
need to reference it in the `.yml` file like: `covid/new-chapter.qmd`

## Website Style

You can customize many aspects of the website in the `.yml` file itself
with the `format:` function. There are a ton of themes included in
Quarto [here](https://bootswatch.com/) and you can also add a [custom
`css` and/or
`scss`](https://quarto.org/docs/output-formats/html-themes.html#custom-themes)
file to your project. I *think* you can even go super in depth and
customize the javascript components of the site, but I'm not entirely
sure how to do that yet. [This website](https://nbdev.fast.ai/) has a
ton of custom `css` components with Quarto, and possibly uses custom
javascript, so it could be a place to start if you're interested.
Basically, you need to embed the `css` file into your `_quarto.yml` file

::: myframe
```{yaml filename="_quarto.yml"}
format:
  html:
    theme: 
      - cosmo
      - assets/styles.scss
    scss: assets/styles.scss
    # css: styles.css
    toc: true
    highlight-style: assets/custom.theme
```
:::

# How to edit a chapter

First you must clone our github repository. For more information on
cloning the repo, [follow these
instructions](git.qmd#Clone%20this%20repository)

Once you have a local clone you can start editing these files and push
updates.

Basic steps:

1.  Open the R Project
2.  Open the folder/section you want to edit
3.  Open the specific chapter in that section you want to edit (each
    chapter is a `.qmd` file)
4.  Save the change and push to the main branch (or make a pull request
    to the main branch)

Once a commit is push to the main branch, it will trigger a Github
Action to re-render the website and publish it to the main github
repository

## Open the R project

This is a Quarto website that is contained in a `.rproj` file path. The
R project contains all the documents used to create this website. Begin
by opening the R project when should be in your local clone under
`C:\Users\XXXXXXX\Projects\Sequencing_2.0\sequencing_documentation\sequencing_documentation.Rproj`

![](images/open_doc.gif)

## Open the files

This project has `.qmd` files (Quarto Markdown files) that each
represent a chapter in the website. All of the `.qmd` files are knitted
together (using R `knitr`) which compiles all of the files to be sourced
into htmls.

This website is set up to have each major section contain multiple
chapters. To open a chapter, the bottom right pane in your R Studio
window should contain folders for each section, highlighted below

::: myframe
```{bash}
#| source-line-numbers: "14-21,23-30"
$ tree /f
C:.
│   .gitignore
│   about.qmd
│   index.qmd
│   standards.Rproj
│   _quarto.yml
│
├───assets
│       custom.theme
│       styles.css
│       styles.scss
│
├───std
│   │   creds.qmd
│   │   lic.qmd
│   │   public_code.qmd
│   │   security.qmd
│   │   templates.qmd
│   │
│   └───images
│
├───tools
│   │   how_to.qmd
│   │   iac.qmd
│   │   link_code.qmd
│   │   readme.qmd
│   │   release.qmd
│   │   renv.qmd
│   │   teams.qmd
│   │
│   └───images
```
:::

<br>

The `.qmd` files are inside of these folders. Select one to edit.

## Commit changes

Once you're done editing, push the change to the main branch (or make a
new branch, and then a pull request for the main branch). More one this
in the [git](git.qmd) chapter

# How to link external code to the site

The code in this website is automatically linked to the code in the
repository. This lets us update the code and not need to copy and paste
any new changes to the documentation/github page

To do this, open your *external* R script in your repo. In this case
we're using `external_script.R`. Now wrap the chunks of code you want to
link with comments like this, `## ---- libraries` and `## ---- stop`:

::: smallframe
```{r filename="external_script.R"}
#| echo: true
#| eval: false
## ---- libraries
library(pacman)
p_load(
  reticulate,
  fs,
  lubridate,
  dplyr,
  stringr,
  magrittr,
  readr,
  httr
)
## ---- stop
```
:::

The `## ---- libraries` signals the beginning of a chunk. the
`## ---- stop` signals the end of a chunk.

<br>

Now you can call this chunk in your *github page/quarto document* like
this:

Scan the external R script for code chunks:

::: smallframe
```` markdown
```{{r setup}}
#| echo: false
knitr::read_chunk(file.path(getwd(),"external_script.R"))
```
````
:::

<br>

Call the code chunk you want in the `{r}` header within the chunk. like
this `{r libraries}`:

::: smallframe
```` markdown
```{{r libraries}}

```
````
:::

<br>

And now the document will output any code in that code chunk and can
also execute that code chunk if you want. Here's what the output will
look like in this case:

![](images/code-chunk.PNG)

# Publish the site/Github Actions

Github Actions allow you to automate tasks in your repository. Quarto
has functions for Github Actions that allow you to automatically render
your Quarto document and publish it to Github Pages. [Github
Pages](https://pages.github.com/) is a free service from Github to host
a website. Documentation for R and Python code is usually found in a
Github Page within the package repository

To create this process, I followed the [Quarto dev
documentation](https://github.com/quarto-dev/quarto-actions/blob/main/examples/example-01-basics.md#quarto-actions-basics):

> Add the GitHub Actions workflow to your project
>
> 1.  Copy
>     [quarto-publish-example.yml](https://github.com/quarto-dev/quarto-actions/blob/main/examples/quarto-publish-example.yml)
>     to `.github/workflows/quarto-publish.yml`. Uncomment the "Publish
>     to GitHub Pages (and render)" action. No further changes are
>     needed to the action (in particular, do not edit the line below to
>     add a secret to this file. This file has the same permissions as
>     your repository, and might be publicly readable)
>
> 2.  run `quarto publish gh-pages` locally, once
>
> 3.  Quarto needs to configure the repository for publishing through
>     GitHub Actions. To do this, run `quarto publish gh-pages` locally
>     once.
>
> 4.  Now, add and commit the workflow file you have just created, and
>     push the result to GitHub. This should trigger a new action from
>     GitHub that will automatically render and publish your website
>     through GitHub pages.
>
> Note that GitHub Pages uses a gh-pages branch in your repository,
> which will be automatically created if one doesn't exist.

<br>

## Example `YAML` Workflow

The `.yml` workflow for this project looks something like this:

`on:` is a tag indicating when the action will run. Right now it will
run when any code gets pushed to the main branch in the documentation
folder or `lineages_public_repo.R` script

::: myframe
```{yaml filename="quarto-publish.yml"}
#| eval: false
#| echo: true
on: 
  push:
    branches:
      - main
    paths:
      - documentation/**
      - lineages_public_repo.R
```
:::

`jobs:` is a tag that tells a Github virtual machine what to run and
what operating system to run it on. In this case `ubuntu` with the
`latest` version. This can be `windows`, `linux` or `macOS`.

::: myframe
```{yaml filename="quarto-publish.yml"}
#| eval: false
#| echo: true
name: Render and Publish

jobs:
  build-deploy:
    runs-on: ubuntu-latest
```
:::

<br>

Now we have the steps:

-   `env` will find the renv folder
-   `uses: actions/checkout@v3` will refresh the repo and pull the
    latest changes
-   `uses: quarto-dev/quarto-actions/setup@v2` will install quarto
-   `uses: actions/cache@v1` and the code below it will set up `renv`
    and use the cached packages to install them onto the Github virtual
    machine

::: myframe
```{yaml filename="quarto-publish.yml"}
#| eval: false
#| echo: true
    env:
      RENV_PATHS_ROOT: ~/.cache/R/renv
      
    steps:
      - name: Check out repository
        uses: actions/checkout@v3
        
      - name: Set up Quarto
        uses: quarto-dev/quarto-actions/setup@v2
        
      - name: Prep CURL install
        run: sudo apt-get update

      - name: Install CURL Headers
        run: sudo apt-get install libcurl4-openssl-dev

      # - name: Setup Renv
      #   uses: r-lib/actions/setup-renv@v2
  
      - name: Cache packages
        uses: actions/cache@v1
        with:
          path: ${{ env.RENV_PATHS_ROOT }}
          key: ${{ runner.os }}-renv-${{ hashFiles('**/renv.lock') }}
          restore-keys: |
            ${{ runner.os }}-renv-
      
      - name: Restore packages
        shell: Rscript {0}
        run: |
          if (!requireNamespace("renv", quietly = TRUE)) install.packages("renv")
          renv::restore()
```
:::

<br>

And finally,

-   `uses: quarto-dev/quarto-actions/publish@v2` will render the site by
    running `quarto render`
-   `with: target: gh-pages path: documentation/_site` lets you know
    which branch and path to render the site to

::: myframe
```{yaml filename="quarto-publish.yml"}
#| eval: false
#| echo: true
      - name: Publish to GitHub Pages (and render)
        uses: quarto-dev/quarto-actions/publish@v2
        with:
          target: gh-pages
          path: documentation/_site
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # this secret is always available for github actions

```
:::

## Using `renv` in the GH Action

If you need to constantly update your website with code chunk, this is
the best way to do it. It is also probably safer and better than the
`_freeze` way, but it requires a better understanding of Github Actions
and virtual environments.

`renv` is an R package for creating a project level virtual environment.
In other words, `renv` will create project specific folders that contain
the specific R package versions you use in an project. [More on this
here](../wiki/venv.qmd)

To use `renv` in a Github Action, you can put

::: smallframe
```{yaml filename="quarto-publish.yml"}
#| eval: false
#| echo: true
- name: Setup Renv
  uses: r-lib/actions/setup-renv@v2
```
:::

or use the renv cache code in the `yaml` section above

## Using a `_freeze` file

If you only need to execute the code once or just need to render a
non-executable code chunk once, make sure you have this code in your
`_quarto.yml` file:

::: smallframe
```{yaml filename="_quarto.yml"}
#| eval: false
#| echo: true
execute:
  error: true
  freeze: true
```
:::

and then run this in your terminal window:

`quarto render name-of-specific-document-or-chapter.qmd`

This will render that specific document in the website, execute code
chunks if they are set to execute (`eval: true`) and then it will create
a **`_freeze`** file. The `_freeze` file will save a snapshot of that
specific document and *not* re-render it in the Github Action. This
means you can render other parts of the website, but any files in the
`_freeze` folder will stay the same as they are in the freeze. If you
need to make changes to a freeze document, run the quarto render code
again after making changes.

This is also documented in the [Quarto dev
documentation](https://github.com/quarto-dev/quarto-actions/blob/main/examples/example-01-basics.md#quarto-actions-basics)

## Troubleshooting

So you did these steps:

1.  Create the `quarto-publish.yml`
2.  Run `quarto publish gh-pages` in the **terminal**
3.  Push all the files in your git to the main branch

If this works on your first try then the universe is taking extra
special care of you.

If not, you are like the rest of us poor souls:/

<br>

The first thing I would check is the error in your Github repo's Action
tab.

**If the error is something like `jsonlite not installed` or
`some package not installed`** then it most likely means your are trying
to commit a chunk of code in the documentation. *Even if you are not
executing the code*, Github Actions will punish you. There are a couple
options to fix this, depending on your priorities.

1.  If you don't care about executing your code and/or only need to push
    that part of the script once, consider using the `_freeze` option
2.  If you need to execute code *or* need to programmtically render the
    document with code chunks often, consider using `renv` or a similar
    package installation method
