
NEWS update (enhancement): accounting lock
Source:vignettes/enhancement_accountinglock.Rmd
enhancement_accountinglock.Rmd
Add overwrite protection
A response to https://github.com/NOAA-OWP/RRASSLER/issues/6
Because RRASSLER overwrites data and databases can get large, it’s
desirable to have training wheels (an affectionate term I assure you, I
don’t know how to ride this bike) on some of the processing steps in
case a user errantly sets one of the relevant commands off (primarily
those under the post-process
category). As Rob points out, the most straightforward way to accomplish
this is to version the tracking sheets, the leftmost of which is
accounting.csv
which is replaced after merging the many
disparate files into their RRASSLE’d outputs. This PR adds an overwrite
flag to in the refresh_master_files
function.
It’s also been coded to default to not create this duplication
so that the behavior of the toolchain remain consistent with the first
release. Finally, this will place backups in the same directory in order
to avoid creating a more sprawling folder structure. This is a great
feature to have since it gives us a safety check and an informal history
into the evolution of the catalog, but it’s worth keeping in mind that
RRASSLER data is not intended to serve as an archive that keeps versions
of data, but a catalog that facilitates model accounting and access.
Here’s a quick demonstration of this new functionality over the sample data!
ras_dbase <- file.path(fs::path_package("", package = "RRASSLER"),"extdata","sample_output","ras_catalog",fsep = .Platform$file.sep)
message(paste("Running in:",ras_dbase))
What’s currently in the folder?
fs::dir_tree(path = ras_dbase, recurse = FALSE)
## /tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog
## ├── HUC8.fgb
## ├── OWP_ras_model_catalog.csv
## ├── XS.fgb
## ├── accounting.csv
## ├── model_footprints.fgb
## ├── model_map.html
## ├── model_map_files
## ├── models
## ├── point_database.parquet
## └── stac
Existing functionality: nothing new is generated
RRASSLER::refresh_master_files(path_to_ras_dbase = ras_dbase,is_verbose = TRUE)
## Writing layer `XS' to data source
## `/tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog/XS.fgb' using driver `FlatGeobuf'
## Writing 177 features with 1 fields and geometry type Line String.
## Writing layer `model_footprints' to data source
## `/tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog/model_footprints.fgb' using driver `FlatGeobuf'
## Writing 9 features with 8 fields and geometry type Polygon.
## [1] TRUE
fs::dir_tree(path = ras_dbase, recurse = FALSE)
## /tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog
## ├── HUC8.fgb
## ├── OWP_ras_model_catalog.csv
## ├── XS.fgb
## ├── accounting.csv
## ├── model_footprints.fgb
## ├── model_map.html
## ├── model_map_files
## ├── models
## ├── point_database.parquet
## └── stac
Added feature: File backup.
refresh_master_files(path_to_ras_dbase = ras_dbase,is_verbose = TRUE,overwrite = FALSE)
## Writing layer `XS' to data source
## `/tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog/XS.fgb' using driver `FlatGeobuf'
## Writing 177 features with 1 fields and geometry type Line String.
## Writing layer `model_footprints' to data source
## `/tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog/model_footprints.fgb' using driver `FlatGeobuf'
## Writing 9 features with 8 fields and geometry type Polygon.
## [1] TRUE
fs::dir_tree(path = ras_dbase, recurse = FALSE)
## /tmp/RtmpEB7FiH/temp_libpath40d22c77b526/RRASSLER/extdata/sample_output/ras_catalog
## ├── HUC8.fgb
## ├── OWP_ras_model_catalog.csv
## ├── XS.fgb
## ├── accounting.csv
## ├── accounting_20250604011801.csv
## ├── model_footprints.fgb
## ├── model_map.html
## ├── model_map_files
## ├── models
## ├── point_database.parquet
## └── stac
Note that we now have a new accounting_{yyyymmddhhmmss}.csv backup.