Skip to content

Commit

Permalink
Update manual
Browse files Browse the repository at this point in the history
  • Loading branch information
pcinereus committed Jul 11, 2024
1 parent 2422dd0 commit 6f121b4
Show file tree
Hide file tree
Showing 12 changed files with 1,188 additions and 27 deletions.
521 changes: 513 additions & 8 deletions docs/manual.qmd

Large diffs are not rendered by default.

429 changes: 429 additions & 0 deletions docs/resources/diagram2.tikz

Large diffs are not rendered by default.

35 changes: 35 additions & 0 deletions docs/resources/references.bib
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
@Manual{R-2024,
title = {R: A Language and Environment for Statistical Computing},
author = {{R Core Team}},
organization = {R Foundation for Statistical Computing},
address = {Vienna, Austria},
year = {2024},
url = {https://www.R-project.org/},
}
@Article{brms,
title = {{brms}: An {R} Package for {Bayesian} Multilevel Models Using {Stan}},
author = {Paul-Christian Bürkner},
journal = {Journal of Statistical Software},
year = {2017},
volume = {80},
number = {1},
pages = {1--28},
doi = {10.18637/jss.v080.i01},
encoding = {UTF-8},
}

@book{Gelman-2007-2007,
Address = {Cambridge, UK},
Author = {Gelman, A. and Hill, J.},
Publisher = {Cambridge University Press},
Title = {Data Analysis Using Regression and Multilevel/hierarchical Models},
Year = {2007}}

@Manual{DHARMa,
title = {DHARMa: Residual Diagnostics for Hierarchical (Multi-Level / Mixed)
Regression Models},
author = {Florian Hartig},
year = {2022},
note = {R package version 0.4.6},
url = {https://CRAN.R-project.org/package=DHARMa},
}
152 changes: 152 additions & 0 deletions docs/resources/standardisations.tikz
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@

\usetikzlibrary{shapes,arrows,shadows,positioning,mindmap,backgrounds,decorations, calc,fit, decorations.text, decorations.pathreplacing,decorations.pathmorphing, shadings,shapes.geometric, shapes.multipart,patterns, quotes}

\pgfdeclarelayer{level1}
\pgfdeclarelayer{level2}
\pgfsetlayers{background,level1,level2}

\tikzstyle{Labels} = [font={\fontspec[Scale=1]{ArchitectsDaughter-Regular}}]
%% \tikzstyle{Labels} = [font={\fontspec[Scale=1]{ArchitectsDaughter}}]
%\tikzstyle{Description} = [font={\fontspec[Scale=1]{ArchitectsDaughter}}]
\tikzstyle{code} = [font=\ttfamily]
\tikzstyle{Messy} = [decorate,decoration={random steps,segment length=3pt, amplitude=0.3pt},thick]
%\tikzstyle{Messy} = []
%\tikzstyle{Messy} = [decorate,decoration={random steps,segment length=10pt, amplitude=0.1pt},thick]
\definecolor{Bin}{rgb}{0.9,0.9,0.2}%
\definecolor{Docker}{rgb}{0.26,0.65,0.91}%
\definecolor{OS}{rgb}{1,0.8,0.4}%
\definecolor{cloud}{HTML}{89DCF0}%
\definecolor{Push}{HTML}{0078D7}%

\newcommand{\Arrow}[5] {
\coordinate (#5) at (#1);
\draw[thick, Messy, rotate around={#4:(#1)}] (#1) -- ++(0,#3*1) -- ++(#2,#3*-0.5) -- ++(0.25*#2*-0.3,#3*0.5) -- ++(0.25*#2*2,#3*-1) -- ++(0.25*#2*-2,#3*-1) -- ++(0.25*#2*0.3,#3*0.5) -- ++(0.25*#2*-4,#3*-0.5) -- ++(0,#3*1) -- cycle;
}

\begin{tikzpicture}
[
node distance=0mm,
title/.style={font=\fontsize{8}{8}\color{black!90}\bfseries},
boxframe/.style={Messy, draw=black!50, inner sep=0mm, fill=white},
]

%% Most metals labels
\begin{scope}
\begin{pgfonlayer}{level2}
\node [text width = 2.5cm, title, Messy, Labels] (most_metals_labels) {Ag, Co, Cu, Hg, Ni, Pb \& Zn};
\end{pgfonlayer}
\end{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 3cm, boxframe, fit={(most_metals_labels)}]
(most_metals_labels_box) {};
\end{pgfonlayer}

\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 3cm, anchor=west, right = 0.5cm of most_metals_labels_box.east,
text width = 7cm, draw, Messy] (most_metals_wh_box) {};
\end{pgfonlayer}
\end{scope}

\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, below = 0.2cm of most_metals_wh_box.north, text width = 2.5cm, title, Messy, Labels] (most_metals_wh_title) {Whole Harbour};

\node [anchor=east, below = 0.8cm of most_metals_wh_title.south west, Labels, font=\bfseries, Labels] (most_metals_wh_less) {Fe/Al < 1.3};
\node [anchor=east, Labels, text=blue] at ($(most_metals_wh_less.east) +(0,-0.7cm)$) {50,000 mg/kg Al};

\node [anchor=west, below = 0.8cm of most_metals_wh_title.south east, font=\bfseries, Labels] (most_metals_wh_more) {Fe/Al > 1.3};
\node [anchor=west, Labels, text=blue] at ($(most_metals_wh_more.west) +(0,-0.7cm)$) {20,000 mg/kg Fe};
\Arrow{$(most_metals_wh_title.south) +(0.4, -0.2)$}{0.8}{0.15}{-140}{A1};
\Arrow{$(most_metals_wh_title.south) +(0.1, -0.0)$}{0.8}{0.15}{-40}{A2};
\end{pgfonlayer}
\end{scope}

%% Vanadium
\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 2cm, text width = 2.5cm,
anchor=north west, draw, Messy] at ($(most_metals_labels_box.south west) +(0,-0.25cm)$) (v_box) {};
\end{pgfonlayer}
\end{scope}
\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, minimum height = 2cm, align=center, below = 0cm of v_box.north, text width = 2.5cm, title, Messy, Labels] (v_labels_box) {V};
\end{pgfonlayer}
\end{scope}

\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 2cm, anchor=west, right = 0.5cm of v_box.east,
text width = 7cm, draw, Messy] (v_wh_box) {};
\end{pgfonlayer}
\end{scope}
\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, below = 0.2cm of v_wh_box.north, text width = 2.5cm, title, Messy, Labels] (v_wh_title) {Whole Harbour};
\node [anchor=north, Labels, text=blue] at ($(v_wh_title.south) +(0,-0.5cm)$) {20,000 mg/kg Fe};
\end{pgfonlayer}
\end{scope}



%% Arsenic
\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 3cm, text width = 2.5cm,
anchor=north west, draw, Messy] at ($(v_box.south west) +(0,-0.25cm)$) (as_box) {};
\end{pgfonlayer}
\end{scope}
\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, minimum height = 3cm, align=center, below = 0cm of as_box.north, text width = 2.5cm, title, Messy, Labels] (as_labels_box) {As};
\end{pgfonlayer}
\end{scope}

\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 3cm, anchor=west, right = 0.5cm of as_box.east,
text width = 7cm, draw, Messy] (as_wh_box) {};
\end{pgfonlayer}
\end{scope}
\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, below = 0.2cm of as_wh_box.north, text width = 2.5cm, title, Messy, Labels] (as_wh_title) {};

\node [anchor=east, below = 0.8cm of as_wh_title.south west, font=\bfseries, Labels] (as_wh_inner) {Inner Harbour};
\node [anchor=east, Labels, text=blue] at ($(as_wh_inner.east) +(0,-0.7cm)$) {20,000 mg/kg Fe};

\node [anchor=west, below = 0.8cm of as_wh_title.south east, font=\bfseries, Labels] (as_wh_outer) {Outer Harbour};
\node [anchor=west, Labels, text=blue] at ($(as_wh_outer.west) +(0,-0.7cm)$) {Mn (i.e As/Mn ratio)};
\Arrow{$(as_wh_title.south) +(0.4, -0.2)$}{0.8}{0.15}{-140}{A1};
\Arrow{$(as_wh_title.south) +(0.1, -0.0)$}{0.8}{0.15}{-40}{A2};
\end{pgfonlayer}
\end{scope}

%% Hydrocarbons
\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 2cm, text width = 2.5cm,
anchor=north west, draw, Messy] at ($(as_box.south west) +(0,-0.25cm)$) (h_box) {};
\end{pgfonlayer}
\end{scope}
\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, minimum height = 2cm, align=center, below = 0cm of h_box.north, text width = 2.5cm, title, Messy, Labels] (h_labels_box) {Hydrocarbons};
\end{pgfonlayer}
\end{scope}

\begin{scope}
\begin{pgfonlayer}{level1}
\node [minimum height = 2cm, anchor=west, right = 0.5cm of h_box.east,
text width = 7cm, draw, Messy, Labels] (h_wh_box) {};
\end{pgfonlayer}
\end{scope}
\begin{scope}
\begin{pgfonlayer}{level2}
\node [anchor=center, below = 0.2cm of h_wh_box.north, text width = 2.5cm, title, Messy, Labels] (h_wh_title) {Whole Harbour};
\node [anchor=north, Labels, text=blue] at ($(h_wh_title.south) +(0,-0.5cm)$) {1\% TOC};
\end{pgfonlayer}
\end{scope}

\end{tikzpicture}
22 changes: 13 additions & 9 deletions md/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,23 @@ represent an action performed in furtherance of the analysis and of
which there are reportable diagnostics. For example, once the
application loads, the first Stage of the pipeline is to prepare the
environment. The first Task in this Stage is to load the necessary R
packages used by the codebase. Whilst this technically, this action
packages used by the codebase. Whilst technically, this action
consists of numerous R calls (one for each package that needs to be
loaded), the block of actions are evaluated as a set.

Initially, all tasks are reported as "pending" (<span class="fas
fa-clock"></span>). As the pipeline progresses, each Task is evaluated
and a status is returned as either "success" (<span class="fas
fa-circle-check"></span>) or "failure" (<span class="fas
fa-circle-xmark"></span>).
Initially, all upcoming tasks are reported as "pending" (<span
class="fas fa-clock"></span>). As the pipeline progresses, each Task
is evaluated and a status is returned as either "success" (<span
class="fas fa-circle-check"></span>) or "failure" (<span class="fas
fa-circle-xmark"></span>).

The Stage that is currently (or most recently) being run will be
expanded, whereas all other Stages will be collapsed (unless they
contain errors). It is also possible to expand/collapse a Stage by
double clicking on its title (or the small arrow symbol at the left
side of the tree).

As the pipeline progresses, Task logs are written to a log_file and
As the pipeline progresses, Task logs are written to a log file and
echoed to the **Logs** panel. Each row represents the returned status
of a specific Task and are formatted as:

Expand All @@ -37,7 +37,7 @@ of a specific Task and are formatted as:
useful for helping to diagnose the source and cause of issues.

The Logs in the Log panel are presented in chronological order and
will autoscroll such that the most recent log is at the bottom of th e
will autoscroll such that the most recent log is at the bottom of the
display. If the number of Log lines exceeds 10, a scroll bar will
appear on the right side of the panel to help reviewing earlier Logs.

Expand All @@ -46,6 +46,10 @@ The Status and Logs are
completely refreshed each time the application is restarted.
</div>

The Progress panel also has a tab (called **Terminal-like**) also has
The Progress panel also has a tab (called **Terminal-like**) which provides
an alternative representation of the status and progress of the
pipeline.

Under the **Logs** panel, there is a **Model Logs** panel. This panel
provides additional status and progress about the fitting and
processing of individual statistical models.
11 changes: 6 additions & 5 deletions md/data_raw_instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,18 +33,19 @@ the _focal_ Sheet of the **Data** and **Validation** tabs:
places, yet the actual underlying data is full resolution.

- the table in the **Validation** tab displays more details about
which fields or rows of the _focal_ Sheet that failed validation
tests.
which fields or rows of the _focal_ Sheet failed validation tests.

If there were no validation issues, this table will be empty.
Otherwise, the description field will indicate the nature of the
violation and in the case of issues with an individual record, the
offending row will be presented across the remaining cells in the
row. For more information about the validation tests, please refer
to the **Data requirements** box (to the right of this box).
to the **Data requirements** box (to the right of this box in the
app).

Underneath both the Data and Validation tables, there is a **Download
as csv** button. Via this button, you can download a comma separated
text file version of the data in the table for further review in a
spreadsheet. Once you click this button, you will be prompted to
navigate to a suitable location to store the file.
spreadsheet of your choice. Once you click this button, you will be
prompted to navigate to a suitable location to store the file.

2 changes: 1 addition & 1 deletion md/dharma.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ many statistical models, discerning genuinely violation-indicating
patterns in residuals from artifacts due to the model type can be
really difficult - if not impossible.

Within R, the _DHARMa_ (Diagnostics for HierArchical Regression Models)
Within R, the _DHARMa_ (Diagnostics for HierArchical Regression Models: @DHARMa)
package generates standardised residuals via simulation and uses these
as the basis of a range of tools to diagnose common modelling issues
including outliers, heterogeneity, over-dispersion, autocorrelation.
Expand Down
2 changes: 1 addition & 1 deletion md/instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ turn green. Grey buttons are disabled.
Clicking on button will run that stage. Once a stage has run, the
button will change to either green (success), yellow (orange) or red
(failures) indicating whether errors/warnings were encountered or not.
If the stage was completed successfully, the bottom corresponding to
If the stage was completed successfully, the button corresponding to
the next available stage will be activated.

Sidebar menu items that are in orange font are active and clicking on
Expand Down
4 changes: 3 additions & 1 deletion md/processed_data_instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,7 @@ the processing, the following new fields will be created:
| Fe_Al_normalisation | what the sample **would be** normalised against Fe or Al (or not applicable) if it could be normalised |
| Normalised against | what the sample **was** normalised against (if it was normalised) |
| Normalisation flag | whether the normalisation has switched between Fe and Al over time for this site |


: {tbl-colwidths="[20,80]"}

</div>
9 changes: 9 additions & 0 deletions md/raw_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ fields listed in their respective tables):
| Sample_ID | unique sample ID | must contain characters |
| >C*¹ | observed concentration of hydrocarbons within a specific size bin in sediment sample | must contain only numbers or start with a '<' symbol |

: {tbl-colwidths="[10,50,40]"}

1: where the '*' represents a string of characters defining the
size bin (such as '10 _C16'). There should be numerous of these
fields </div>
Expand All @@ -37,6 +39,7 @@ fields listed in their respective tables):
| Sample_ID | unique sample ID | must contain characters |
| TOC (%) | observed total organic carbon (as a percentage of the sample weight) | must contain only numbers |

: {tbl-colwidths="[10,50,40]"}

</div>

Expand All @@ -58,9 +61,15 @@ fields listed in their respective tables):
| Baseline_site | the unique site ID of the corresponding baseline sample | must contain characters (cannot be blank) |
| Baseline_acquire_date_site | the date and time of the corresponding baseline sample | must be in datetime format |

: {tbl-colwidths="[25,35,40]"}

</div>

- notes - this sheet is not processed or validated

::: {.callout-note}
All input data must be placed in the `/input` folder of the Project
prior to starting the app.
:::


18 changes: 18 additions & 0 deletions md/temporal_analysis_effects.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,3 +28,21 @@ polarity and evidence of change via a color.
- light green: weak evidence of a decline
- mid green: evidence of a decline
- dark green: strong evidence of a decline

The full modelled posteriors are depicted in the figure under the
table. The y-axis represents each of the contrasts (comparisons
between sets of years or years and the Baseline). The posterior
distributions are coloured according to the polarity and degree of
evidence as outlined above.

The full and summarised posteriors can be presented on either a
**percentage** scale (default) or **fold** scale and this is
controlled via a dropdown box above the table.

- the percentage scale is interpreted as the percentage change between
the contrasting years. For example, percentage changes of -20, 20
and 0 indicate a 20% decline, 20% increase and no change
respectively.
- the fold scale is interpreted as the fold change between the
contrasting years. For example, fold changes of 0.5, 2 and 1
indicate a halving, a doubling and no change respectively.
10 changes: 8 additions & 2 deletions md/temporal_analysis_overview_instructions.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
The table presents a matrix in which:

- the rows represent the variables
- the columns represent the zones
- the columns represent the variables
- the rows represent the sites/zones/areas
- the colour of the cells represents the polarity and evidence of change:

- red: strong evidence of an increase
Expand All @@ -11,10 +11,16 @@ The table presents a matrix in which:
- light green: weak evidence of a decline
- mid green: evidence of a decline
- dark green: strong evidence of a decline
- gray: model not completed (typically due to a lack of data)

- models built on data that includes values below limit of
reporting/detection are marked with a black flag (<span class="far
fa-flag"></span>)

The selectors can be used to filter what subset of data are presented
in the table:

- scale of aggregation (site/zone/area)
- whether or not the data are standardised
- the type of variable (metals, hydrocarbons or both)
- the focal year to compare to the baseline (2019/2020)

0 comments on commit 6f121b4

Please sign in to comment.