Thoughts from a Github Workflow Virgin!¶

Clickbaity? Maybe, but it's true... I was, until this morning at least, a virgin in terms of Github workflows. Now I'm not and it's been both a simultaneously enjoyable and somewhat infuriating experience popping my workflow cherry, so I figured I'd share my thoughts.

Background¶

The background for this foray into Github workflows originates from my move from WordPress to MkDocs. One of my requirements was to be able to write anywhere and then when I commit, have the site automagically rebuilt and uploaded to the server. Along the way, the requirements expanded to the following:-

The master branch of the repository for the blog is the work area. Anything committed there should be considered a work in progress
The live branch of the repository for the blog should be what is on the live site. Commit here and the site is rebuilt and published to the live site location (this build should not include drafts)
The preview branch of the repository for the blog should be a preview version of the live site. Commit here and a preview version of the site is built and published at a designated password protected location so I can take a look at how things are looking and make sure anything fancy I'm doing is going to work on the live server (this build should include drafts)

I could in theory dispense with the preview branch and just have the master branch rendered and uploaded, but if I'm honest, I'm trying to avoid paying for a Github account for as long as I can, so I don't want workflow time being burned everytime I commit a work in progress as I only have a limited amount of free time. At the time of writing, a free account gets 2000 minutes of free workflow time, and whilst this probably seems a lot, if by chance your job happens upon a runner that is chugging along and for whatever reason your workflow takes 10 minutes instead of it's usual 1, you've just burned 9 minutes, so limiting the total number of executions seems like a good plan to me.

Initial Joy¶

After some early frustration with networking issues (my own doing), I was able to achieve a fully operational workflow that built and deployed my site when I pushed to the live branch. I'm not going to lie, it was a bit daunting... it would perhaps have been less so had I actually taken the time to read all the available documentation before I got started... but who wants to read manuals??? Am I right???

So, here are the answers to some of the questions I had along the way...

What shall I call the workflow file? Anything you like. It has no bearing on anything and as far as I can tell it will only be used if you don't provide a name for the workflow.
Do all jobs in the workflow run in the same runner? No. It's designed for parallel execution, so whilst you can specify dependencies (i.e. job 2 requires that job 1 has completed), you must assume that each job will start life in a totally blank runner.
Can you use reusable workflows to avoid copy/paste workflow development? No, not that I'm aware of... more on this later.

Sample Workflow¶

This is the workflow for building the preview version of my site as it stands at the time of writing.

Github Workflow For Building My Blog Preview
name: "Build Blog (Preview)"
on:
    push:
        branches:
            - preview
jobs:
    previewbuild:
        runs-on: ubuntu-latest
        steps:
# Common - Setup environment
            - name: Setup Python
              uses: actions/setup-python@v4.7.0
              with:
                python-version: 3.11.4
            - name: Install MkDocs
              run: pip install mkdocs=="1.5.0"
            - name: Install MkDocs Lightbox Plug-in
              run: pip install mkdocs-glightbox=="0.3.4"
            - name: Install MkDocs Categories Plug-in
              run: pip install mkdocs-categories-plugin=="0.4.0"
            - name: Install MkDocs Include Markdown Plug-in
              run: pip install mkdocs-include-markdown-plugin=="4.0.4"
            - name: Install MkDocs Macros Plug-in
              run: pip install mkdocs-macros-plugin=="1.0.2"
            - name: Install MkDocs Minify Plug-in
              run: pip install mkdocs-minify-plugin=="0.7.0"
            - name: Install Material for MkDocs
              run: pip install mkdocs-material=="9.2.0b2"
            - name: Install Material Extensions
              run: pip install mkdocs-material-extensions=="1.1.1"
            - name: Install Markdown Callouts
              run: pip install markdown-callouts=="0.3.0"
            - name: Install SSHPASS
              run: sudo apt install sshpass
# Check out source (Be careful which branch is being checked out)
            - name: Checkout Site Source
              uses: actions/checkout@v3
              with:
                ref: preview
                token: ${{ secrets.GITHUB_TOKEN }}
# Preview Pre-build
            - name: Change Site URL
              run: sed -i 's/com\/site\//com\/somedir\//' mkdocs.yml
# Build the site
            - name: Build Site
              run: mkdocs build
# Preview Post-build
            - name: Make HTACCESS File
              run: echo -e "AuthType Basic\nAuthName \"Athena's Pad Preview\"\nAuthUserFile /etc/somehtaccesspasswordfile\nrequire valid-user" > ./site/.htaccess
# Upload to server (Be careful with destination directory)
            - name: RSync With Server
              run: sshpass -p '${{secrets.SOME_PASSWORD}}' rsync -r -L --delete -e 'ssh -p 65535 -o StrictHostKeyChecking=no' ./site/ ${{secrets.SOME_USERNAME}}@${{secrets.SOME_HOSTNAME}}:somedir

Breakdown Of Workflow

name: "Build Blog (Preview)"

Provide a display name for the workflow.

on:
    push:
        branches:
            - preview

Specify when this workflow will be started. In this case, it's when changes are pushed to the branch preview

jobs:
    previewbuild:
        runs-on: ubuntu-latest
        steps:

Each workflow consists of jobs. In this case we have a single job with the shortname previewbuild. It's going to run on the ubuntu-latest runner and in this case consists of the following steps.

Each step provides a name and either specifies the command to run href=#__codelineno-4-1> - name: Setup Python uses: actions/setup-python@v4.7.0 with: python-version: 3.11.4 - name: Install MkDocs run: pip install mkdocs=="1.5.0" - name: Install MkDocs Lightbox Plug-in run: pip install mkdocs-glightbox=="0.3.4" - name: Install MkDocs Categories Plug-in run: pip install mkdocs-categories-plugin=="0.4.0" - name: Install MkDocs Include Markdown Plug-in run: pip install mkdocs-include-markdown-plugin=="4.0.4" - name: Install MkDocs Macros Plug-in run: pip install mkdocs-macros-plugin=="1.0.2" - name: Install MkDocs Minify Plug-in run: pip install mkdocs-minify-plugin=="0.7.0" - name: Install Material for MkDocs run: pip install mkdocs-material=="9.2.0b2" - name: Install Material Extensions run: pip install mkdocs-material-extensions=="1.1.1" - name: Install Markdown Callouts run: pip install markdown-callouts=="0.3.0" or the Github action it uses. When using an action you provide values for the action inputs using with, so in this example we're using the predefined action actions/setup-python and we're using version 4.7.0 of that action. We're telling it which version of Python we want using the variable python-version. There are lots of options for this action so a trip to the Github marketplace is advisable to get to the reference documents for it.

With Python installed we can use pip to install all the dependencies for MkDocs

            - name: Install SSHPASS
              run: sudo apt install sshpass

Now a little OS adjustment. We're going to need sshpass later on for getting the rsync connection up and running, so have the OS install it.

# Check out source (Be careful which branch is being checked out)
            - name: Checkout Site Source
              uses: actions/checkout@v3
              with:
                ref: preview
                token: ${{ secrets.GITHUB_TOKEN }}

Checkout a copy of the site (as pushed into the preview branch)

# Preview Pre-build
            - name: Change Site URL
              run: sed -i 's/com\/site\//com\/somedir\//' mkdocs.yml

Do an in-place modification of the MkDocs configuration file (change the site_url from the live version to one for the preview). We're using sed (installed as standard with the OS) which uses a regular expression to specify the search string. The file is modified inplace (-i).

# Build the site
            - name: Build Site
              run: mkdocs build

Call the MkDocs build process.

# Preview Post-build
            - name: Make HTACCESS File
              run: echo -e "AuthType Basic\nAuthName \"Athena's Pad Preview\"\nAuthUserFile /etc/somehtaccesspasswordfile\nrequire valid-user" > ./site/.htaccess

Generate the .htaccess file for the directory that will contain the preview on the server.

# Upload to server (Be careful with destination directory)
            - name: RSync With Server
              run: sshpass -p '${{secrets.SOME_PASSWORD}}' rsync -r -L --delete -e 'ssh -p 65535 -o StrictHostKeyChecking=no' ./site/ ${{secrets.SOME_USERNAME}}@ ${{secrets.SOME_HOSTNAME}}:somedir

Deploy the site to the server using rsync. We could potentially use some kind of key file, but if that file (or the repo) was to be compromised, the password would be revealed. Using this approach, the password is stored in the secure secrets area of the repo, it cannot be read, and does not feature in any of the logging. This is pretty secure and really I'm not worried about adopting this approach.

So lets explain what's going on here... sshpass is used to push the password into the ssh session rsync establishes and we're adjusting the configuration of that session using -e 'ssh -p 65535 -o StrictHostKeyChecking=no', which specifies a custom SSH port (I don't run my server on the default) and we turn off the host key checking since this connection will be on a new blank canvas everytime so we don't want any headaches with the server being unknown to the client.

Saving Time¶

My original deployment used lftp to deploy the built site to my server. What a horrendous idea that turned out to be... the deployment took over 5 minutes, so much overhead, so I switched over to rsync and an SSH connection. What a bloody good idea that was! Deployments using rsync currently take around 7 seconds. Even though it's free at 5 minutes, my 2000 free minutes would get me 400 builds... but why wouldn't you save all the time you could?

Over 5 minutes down to under 1 minute is a good saving and it's important to remember that the workflow time will only ever increase as your site grows. The majority of the time spent on the job at the moment is the setup of Material for MkDocs at around 13 seconds (nearly twice as long as the deployment). Could this be optimised? Maybe, but for now... my free time should be ample for my needs.

Execution time is definitely something you should consider when building out workflows, and you should definitely be aware of what happens with multiple jobs.

Conclusion¶

Given this is my first foray into the world of Github runners and workflows, I'm both pleased with how easy it's been to get the desired outcome and disappointed that some aspects of it have proven so infuriating. The key issues I found were the inability to debug failures and perhaps a lack of clarity in the documentation and/or a lack of understanding on my part.

Debugging Failures¶

Consider how you might go about debugging a Linux command you've never used before. Personally, I'm most likely to fire up a terminal, get an SSH session to one of my servers and tinker. Experiment with the command. So this morning, my workflow dev cycle went something like this... read a limited log, edit the workflow script, commit the updated script to Git, push it up to the origin, commit it to one of my build branches and then wait for the workflow to be executed so I could read another limited log. It was annoying to say the least and whilst ultimately the issues I experienced were down to me, figuring that out was a nightmare. If would have been so much easier if you could do something like this:-

Workflow Breakpoint Idea

- name: Make HTACCESS File
  run: echo -e "AuthType Basic\nAuthName \"Athena's Pad Preview\"\nAuthUserFile /etc/somehtaccesspasswordfile\nrequire valid-user" > ./site/.htaccess
- breakpoint
# Upload to server (Be careful with destination directory)
- name: RSync With Server
  run: sshpass -p '${{secrets.SOME_PASSWORD}}' rsync -r -L --delete -e 'ssh -p 65535 -o StrictHostKeyChecking=no' ./site/ ${{secrets.SOME_USERNAME}}@${{secrets.SOME_HOSTNAME}}:somedir

When the runner encounters breakpoint, if some conditions are met (like maybe you've just executed this on-demand using a special Debug Workflow button) you get an SSH session to the runner at that point and you can inject the command as it would be executed, allowing you to tinker with it. Clearly there is some kind of session connection as commands are injected and the output is captured... being able to drop the user into that control loop would be amazing. Maybe don't even have the breakpoint command, simply allow the user to step over or step into.

Documentation Clarity and/or My Lack of Understanding¶

The background at the start doesn't really explain the order of origin for the requirements. The live branch was always a thing, but the preview branch actually became a requirement when the build process for the live branch was done (admittedly using lftp - this was the source of many of my problems, or more specifically the setup of both it and my server to allow FTP connections). Long story short, I needed a means of testing bigger changes without doing a build and pushing a live site, so the preview branch was born along with it's deployment workflow, and here is where my pain started.

For the preview build there are a couple of things I need to do differently (minor configuration tweaks and an additional file to activate the basic authentication for protecting the directory). No problem, engage in some copy/paste coding and job's a good 'un!

NO! NO! NO!

If you find yourself getting taken over by this mentality, take yourself outside, touch grass and remind yourself that if you engage in such practices you will be summarily flogged by your future self (or some other poor soul who has to maintain your code)!

Seriously though, at this point I saw in the Github documentation the term reusable workflow and I dived in!

So here's the thing... if you use reusable workflows, you can't also use steps in your job. I mistakenly thought I could just reuse a workflow like an action... WRONG! Each job in your workflow can reuse one reusable workflow, so instead of a series of steps your workflow must be a series of jobs and whilst you can specify dependencies (i.e. one job requires the completion of some other job or jobs), each job is going to be executed on a different runner... so my setup MkDocs job prepared a runner with Python and all the MkDocs components, the preview build pre-build job then executed and promptly fell over because the target of it's operations didn't exist because the source files hadn't been checked out on that runner.

Now, at this point I'd already answered the survey about my experience with workflows (I gushed about them and suggested the debugging idea above), but this apparent inability to reuse bits and pieces really cheesed me off. Enter this article and my desire to make sure I wouldn't be talking out of my bung hole. I've been reading and you can, although I don't think it's documented particularly well, create custom actions that allow you to reuse steps in multiple workflows, so let's take a look at that.

For starters, actions must exist in their own directory in .github/actions and for composite actions you must have action.yaml. So, I've now created the following custom actions.

.github/
  actions/
    deploy - Action to deploy the built site to my hosting
    prebuild - Pre-build actions (executed after checkout and setup, before MkDocs build)
    postbuild - Post-build actions (executed after MkDocs build, before deploy)
    setuprunner - Action to install all the required software on the runner

And my preview build workflow has been reduced to this...

Workflow - .github/workflows/buildpreview.yml

name: "Build Blog (Preview)"
on:
    push:
        branches:
            - preview
jobs:
    previewbuild:
        runs-on: ubuntu-latest
        steps:
            - name: Checkout Site Source
              uses: actions/checkout@v3
              with:
                  ref: preview
                  token: ${{ secrets.GITHUB_TOKEN }}
            - name: Setup runner
              uses: ./.github/actions/setuprunner
            - name: Prebuild
              uses: ./.github/actions/prebuild
              with:
                  buildmode: preview
            - name: Build Site
              run: mkdocs build
            - name: Postbuild
              uses: ./.github/actions/postbuild
              with:
                  buildmode: preview
            - name: Deploy Site
              uses: ./.github/actions/deploy
              with:
                  targetdirectory: preview
                  ftppassword: '${{secrets.FTP_PASSWORD}}'
                  ftpusername: '${{secrets.FTP_USERNAME}}'
                  ftpserver: '${{secrets.FTP_SERVER}}'

Now, that's much better because the live build workflow is the same with just minor adjustments and from what I've seen, I could probably get rid of the changes by using conditional actions in the custom actions to do different things depending on which branch we're operating on.

And so you can get an idea of the custom actions, here's the prebuild action.

Custom action - .github/actions/prebuild/action.yaml

name: "Prebuild Setup"
author: "AthenaOfDelphi"
description: "Carry out the pre-build actions"
inputs:
    buildmode:
        required: true
        description: "Build Mode (preview or live)"
runs:
    using: "composite"
    steps:
        - name: Change Site URL
          if: ${{inputs.buildmode=='preview'}}
          run: sed -i 's/com\/site\//com\/somedir\//' mkdocs.yml
          shell: bash
        - name: Turn Off Drafts
          if: ${{inputs.buildmode=='live'}}
          run: sed -i 's/draft:\ true/draft:\ false/' mkdocs.yml
          shell: bash

What's not particularly well documented? Using local actions. I found this approach (i.e. referencing local files) in an article by Rene Hernandez and then using local actions works a treat. My original attempt tried to use the action in the repo and it wouldn't work and then when I tried to use the local action copies, I forgot I hadn't actually checked them out (the check out was the last action in the setuprunner action). Moving the check out to the main script before the first custom action solved it, so now I have the code reuse I wanted. I think there is some scope for further improvement (by modifying the prebuild, postbuild and deploy to look at the branch that triggered the workflow rather than an input). Get that done and we'll end up with a single workflow.

Overall¶

I gotta say, for a free service... I'm impressed. I've been able to setup automated build and deploy for my website and whilst the site itself isn't fully finished and ready for live deployment yet, just being able to write about this without the baggage of the shitty WordPress editor is amazing. Being able to build and deploy a preview is a huge bonus. If I'd simply indulged in some copy and paste programming I would have been done a long time ago and who knows I may have even finished this article and the rest of the site migration, but I'm glad I didn't because now I know a lot more about Github workflows, which can only be a good thing.

And on that note... one step closer to ditching WordPress... YAY!!!