Portage data workflow

[bpbond]

Sample workflow for processing raw data to L0. This includes

• A Quarto file that takes all the *.dat files in the raw folder, reads them in, parses out the Logger and Table info, and writes them to the L0 folder. It then generates a summary table.
• A driver script that renders the Quarto file, putting the output html into the L0 folder as documentation.
• Sample Raw/ data (see below)

Sample html documentation file: https://rpubs.com/bpbond/970986

Raw folder before processing:

Compass_PTR_UP_313_WaterLevel600A.dat
Compass_PTR_UP_313_TerosTableA.dat
Compass_PTR_UP_313_SapflowA.dat

L0 folder after processing:

Compass_PTR_UP_313_SapflowA_L0.csv
Compass_PTR_UP_313_TerosTableA_L0.csv
Compass_PTR_UP_313_WaterLevel600A_L0.csv
raw_to_L0_20221115145825.html   <--- see link above

This is very much a first pass—feedback welcome @stephpenn1

[bpbond]

Added what the SERC folks call the "normalization" step: matching loggernet variables with their design_link entries; the updated data files are moved to L1_normalize/.

Example html output: https://rpubs.com/bpbond/971152

Need to stick the expand_df code somewhere it'll get tested — probably in compasstools?

[bpbond]

@stephpenn1 FYI in e86ca27 I added a skeleton L1a.qmd file; all it does is move/split the data into yyyy_mm_<logger> folders—nothing else, as we said you'd write the L1a code. I just wanted to get this in place so I can proceed to the L1b step.

[bpbond]

Visualization of files moving through the processing pipeline:

=========================================
 20221202.1317 Starting L0 
|
|- Raw/ 
|	|- Compass_PTR_UP_313_SapflowA.dat 
|	|- Compass_PTR_UP_313_TerosTableA.dat 
|	|- Compass_PTR_UP_313_WaterLevel600A.dat 
|	|- README.md 
|	|
|	|- done/ 
|		|- README.md 
|
|- L0/ 
|	|- README.md 
|
|- L1_normalize/ 
|	|- README.md 
|
|- L1a/ 
|	|- README.md 
|
|- L1b/ 
	|- README.md 

=========================================
 20221202.1317 Starting L1_normalize 
|
|- Raw/ 
|	|- Compass_PTR_UP_313_SapflowA.dat 
|	|- Compass_PTR_UP_313_TerosTableA.dat 
|	|- Compass_PTR_UP_313_WaterLevel600A.dat 
|	|- README.md 
|	|
|	|- done/ 
|		|- README.md 
|
|- L0/ 
|	|- Compass_PTR_UP_313_SapflowA_L0.csv 
|	|- Compass_PTR_UP_313_TerosTableA_L0.csv 
|	|- Compass_PTR_UP_313_WaterLevel600A_L0.csv 
|	|- README.md 
|
|- L1_normalize/ 
|	|- README.md 
|
|- L1a/ 
|	|- README.md 
|
|- L1b/ 
	|- README.md 

=========================================
 20221202.1317 Starting L1a 
|
|- Raw/ 
|	|- Compass_PTR_UP_313_SapflowA.dat 
|	|- Compass_PTR_UP_313_TerosTableA.dat 
|	|- Compass_PTR_UP_313_WaterLevel600A.dat 
|	|- README.md 
|	|
|	|- done/ 
|		|- README.md 
|
|- L0/ 
|	|- README.md 
|
|- L1_normalize/ 
|	|- Compass_PTR_UP_313_SapflowA_L1_norm.csv 
|	|- Compass_PTR_UP_313_TerosTableA_L1_norm.csv 
|	|- Compass_PTR_UP_313_WaterLevel600A_L1_norm.csv 
|	|- README.md 
|
|- L1a/ 
|	|- README.md 
|
|- L1b/ 
	|- README.md 

=========================================
 20221202.1317 Starting L1b 
|
|- Raw/ 
|	|- Compass_PTR_UP_313_SapflowA.dat 
|	|- Compass_PTR_UP_313_TerosTableA.dat 
|	|- Compass_PTR_UP_313_WaterLevel600A.dat 
|	|- README.md 
|	|
|	|- done/ 
|		|- README.md 
|
|- L0/ 
|	|- README.md 
|
|- L1_normalize/ 
|	|- README.md 
|
|- L1a/ 
|	|- README.md 
|	|
|	|- 2022_11_Compass_PTR_UP_313/ 
|		|- Compass_PTR_UP_313_SapflowA_2022_11.csv 
|		|- Compass_PTR_UP_313_WaterLevel600A_2022_11.csv 
|		|- README.md 
|
|- L1b/ 
	|- README.md 

[bpbond]

Superseded by #57