Skip to content

Latest commit

 

History

History
120 lines (99 loc) · 4.33 KB

README.org

File metadata and controls

120 lines (99 loc) · 4.33 KB

databaseToToml

This tool can be used to extract the data from the MySQL database and generate the required TOML files + Markdown files & directory structure needed for the Hugo project.

Installation

At this time we don’t have a CI task yet to generate a binary that can just be downloaded. So for now, we have to build from source.

First we need a recent version of Nim. If the Nim compiler and nimble (the Nim package manager) are in PATH, compilation is a simple:

nim c -d:release databaseToToml.nim

The tool is extremely simple. See the help:

./databaseToToml -h
basti at void in ~/src/tinyIap/staticExample/databaseToToml ツ ./databaseToToml -h            
Usage:
  main [optional-params] 
Options:
  -h, --help                                     print this cligen-erated help
  --help-syntax                                  advanced: prepend,plurals,..
  --host=               string  "localhost"      set host
  -u=, --user=          string  "root"           set user
  -p=, --password=      string  ""               set password
  -m=, --mhbTable=      string  "modulhandbuch"  set mhbTable
  -s, --showDataframes  bool    false            set showDataframes

Essentially, the tool only needs information about the host of the database, a user & password (currently it has to be handed as an argument. We can make that e.g. an environment variable).

So, if the database is local and we just have a root user, we’d call the tool as:

./databaseToToml --password <foo>

The tool reads all module & course information and writes the data (currently) to /tmp.

For each “Prüfungsordnung” one directory is generated:

  • /tmp/PO2006
  • /tmp/PO2014
  • /tmp/Other

These contain both the TOML files with all the data as well as the basic Markdown files needed for Hugo.

Static binary building

We build a statically linked binary in the CI using an Alpine Linux docker image in Github Actions. Each time a semver-like tag is pushed (i.e. v0.1.0, v15.0.2, …) the build is automatically triggered and a new release is made including the static binary.

There are a few things to note for this.

We use Alpine Linux as it is tiny and uses musl by default. musl makes static linking of libraries comparatively easy.

Statically linking on the available Ubuntu images in Github Actions does not really work. It effectively tries to shoehorn dlopen calls into the binary somehow. Locally using Void Linux building a static binary from my local libmariadbclient.a works perfectly fine however.

Alpine Linux’s package repository contains everything we need, but choosing the right dependencies (openssl says hello) is a bit of a pain. The static version of MariaDB in Alpine is apparently linked against the openssl package { and requiring openssl-libs-static, not openssl3-libs-static }, nor libressl!). If someone knows where these things are documented, let me know.

Also, don’t forget:

  • -lz and zlib-static
  • -lcrypto and not only -lssl!

To summarize, we install the following packages:

apk add mariadb mariadb-static mariadb-dev \ # MariaDB and the static library (`libmariadbclient.a`)
    zlib-static \ # for `libz.a`
    openssl openssl-libs-static \ # for the correct `libssl.a` and `libcrypto.a`
    musl-dev \ # general musl related build tools
    git gcc make \ # general stuff to build things (maybe not needed/installed already?)
    nim nimble \ # to build Nim stuff. Currently has version 1.4.8, which is fine for us

See here: ../.github/workflows/auto_release.yml It’s possible we only need the static packages or don’t have to install them explicitly (maybe they are installed as part of the main packages, didn’t check).

With these things in mind, the following are the important NimScript switches:

let
  mariadbIncludeDir = "/usr/include/mysql"
switch("passC", "-I" & mariadbIncludeDir)
switch("define", "useMariadbHeader")
switch("passL", "-L" & mariadbLibDir)
switch("passL", mariadbLibFile)
switch("passL", "-lmariadbclient")
switch("dynlibOverride", "mariadb")
switch("dynlibOverrideAll") ## just disable dynlib...
switch("passL", "-lz")
switch("passL", "-lssl")
switch("passL", "-lcrypto")

which can be found here: ./config.nims