CTD SFER / SE US MBON seus-mbon-cruise-ctd-processing

CTD processing & reporting for the SEUS MBON research cruise data.

"Raw" data comes from https://gcoos5.geos.tamu.edu/erddap. The data hosted there has already had some preprocessing done, but this cleans it even further. The preprocessing includes cutting off the upcast using SBEDataProcessing from SeaBird. The settings file (.psa) is not (yet) included here. Other SBEDataProcessing modules may be used. Processing methods are documented in Chrissy_2009_SeaBird CTD Processing Manual.pdf.

After processing the cleaned data will be in ../seus-mbon-cruise-ctd-data/cleaned/. A version of the cleaned data is hosted by USF IMaRS at here.

Usage

Data ingestion is completed through rendering of a research notebooks into a quarto website. The data is downloaded from ERDDAP in a pre-render step, then processed by the .qmd research notebooks.

1. clone this repository
2. build the site: quarto render --no-cache
   • NOTE: you can run with cache, but this may not generate the ../seus-mbon-cruise-ctd-data/cleaned/*.csv files
3. view the site: quarto preview
4. publish to github pages:

export QUARTO_USE_FILE_FOR_PROJECT_OUTPUT_FILES="quarto_file_output_list.txt"
export QUARTO_USE_FILE_FOR_PROJECT_INPUT_FILES="quarto_file_input_list.txt"
quarto publish

adding a cruise

The list of CTD casts is in /data/ctd_datasetid_cruisename_stationname_mapping.csv. To add a cruise, add rows with the ERDDAP dataset_ids for each cruise station CTD cast.

CTD Station ID Alignment

Each cast has a cast_id (or ERDDAP dataset_id), which contains a cruise_id and a station ID. Sometimes the station_id from the upstream data publishers does not match station_id standard names determined by the FRESCA project. To handle this issue a ID mapping is applied. The mapping can be found in the file /data/ctd_datasetid_cruisename_stationname_mapping.csv. If a station ID is not aligned properly: edit the .csv file, clear any cached files, and re-run the processing.

Details of the pre-render & render workflow

A few steps happen when the quarto publish is run.

1. pre-render jobs will happen. These are specified _quarto.yml. These include downloading the data and generating a .qmd document for each CTD.

• Sometimes there are caching issues. To resolve these delete all files within ctd_reports/ctd_reports/*.

2. quarto render will be run on all files listed under render in _quarto.yml to create the .html files.
3. The .html are uploaded to github pages.

In this workflow, you will primarily be editing the ctd_reports/ctd_reports_template.qmd file.

workflow

To work with this repository:

1. edit cruise_report/cruise_report_template.qmd as desired.
   • note that the params in the header can be edited for testing a specific cruise.
2. quarto preview or quarto publish to generate the site.

Running Tests

This project uses testthat for unit testing. To run the tests:

# From R console
testthat::test_dir("tests/testthat")

# Or using devtools
devtools::test()

Or from the command line:

Rscript -e 'testthat::test_dir("tests/testthat")'

Clear Cache (make clean)

mkdir backupFiles
mv data backupFiles/.
mv ctd_reports/ctd_reports/ backupFiles/.
mv cruise_report/cruise_reports/ backupFiles/.