Skip to contents

ingest_into_database

Source: R/ingest_into_database.R
ingest_into_database.Rd

ingest files from a source directory on your drive into a formatted input directory

Usage

ingest_into_database(
  path_to_ras_dbase,
  top_of_dir_to_scrape,
  code_to_place_in_source,
  proj_override = NULL,
  apply_vdat_trans = FALSE,
  is_quiet = FALSE,
  is_verbose = FALSE,
  overwrite = FALSE,
  parallel_proc = TRUE,
  free_treads = 2
)

Arguments

path_to_ras_dbase

A path to a directory to write your RRASSLED directory to. See Methods and Structures for more details. is location agnostic so this can be either a local path or an s3 bucket

top_of_dir_to_scrape

The top of the directory to look for models. Will greedy search and find all models as described in Ingest logic

code_to_place_in_source

a string to place into the model source column. Useful to distinguish data authors

proj_override

a string to override projection information should none be found, Default: NULL

apply_vdat_trans

Should VDATUM be applied to the HEC-RAS model geometry. See https://vdatum.noaa.gov/, Default: FALSE

is_quiet

flag to determine whether print statements are suppressed, TRUE to suppress messages and FALSE to show them, Default: FALSE

is_verbose

flag to determine whether internal print statements (i.e. cross section parsing, vdat trans, file info) are suppressed, TRUE to show these messages and FALSE to suppress them, Default: FALSE

overwrite

overwrite files if we find identical models, Default: FALSE

parallel_proc

Flag to determine if this should this parallel process, will check for enough free cores and boot this back if it exceeds available resources. Will suppress all intermediate messages if active, Default: TRUE

free_treads

number of threads to leave free if parallel processing, Default: 2

Value

a RRASSLE'd catalog of models

Details

here 'ingest' means add to our accounting system and database refers to our folder structure

See also

glue str_sub detectCores, makeCluster registerDoParallel foreach

Other ingest: cloud_ingest_record(), disk_ingest_record(), ingest_FEMA6_BLE()

Examples

if (FALSE) { # \dontrun{
if(interactive()){
 #EXAMPLE1
 # ras_dbase <- file.path("~/data/ras_catalog/")
 ras_dbase <- file.path("./inst/extdata/sample_output/ras_catalog/")

 dir_to_scrape <- "./inst/extdata/sample_ras/FEMA-R6-BLE-sample-dataset/"
 RRASSLER::ingest_into_database(path_to_ras_dbase = ras_dbase,top_of_dir_to_scrape = dir_to_scrape,code_to_place_in_source = "test: FEMA6",proj_override = "EPSG:2277",apply_vdat_trans = FALSE,is_quiet = FALSE,is_verbose = TRUE,overwrite = FALSE,parallel_proc = FALSE)

 dir_to_scrape <- "./inst/extdata/sample_ras/ras2fim-sample-dataset/input_iowa/"
 RRASSLER::ingest_into_database(path_to_ras_dbase = ras_dbase,top_of_dir_to_scrape = dir_to_scrape,code_to_place_in_source = "test: Iowa input",proj_override = "EPSG:26915",apply_vdat_trans = FALSE,is_quiet = FALSE,is_verbose = TRUE,overwrite = FALSE,parallel_proc = FALSE)

 dir_to_scrape <- "./inst/extdata/sample_ras/ras2fim-sample-dataset/output_iowa/"
 RRASSLER::ingest_into_database(path_to_ras_dbase = ras_dbase,top_of_dir_to_scrape = dir_to_scrape,code_to_place_in_source = "test: RAS2FIM V1",proj_override = "EPSG:26915",apply_vdat_trans = FALSE,is_quiet = FALSE,is_verbose = TRUE,overwrite = FALSE,parallel_proc = FALSE)

 # Sys.setenv("AWS_ACCESS_KEY_ID" = "AKIASUPERSECRET","AWS_SECRET_ACCESS_KEY" = "evenmoresecret","AWS_DEFAULT_REGION" = "us-also-secret")
 dir_to_scrape <- "./inst/extdata/sample_ras/FEMA-R6-BLE-sample-dataset/"
 RRASSLER::ingest_into_database(path_to_ras_dbase = "s3://ras-models/",top_of_dir_to_scrape = dir_to_scrape,code_to_place_in_source = "test: FEMA6",proj_override = "EPSG:2277",apply_vdat_trans = FALSE,is_quiet = FALSE,is_verbose = TRUE,overwrite = FALSE,parallel_proc = FALSE)
 }
} # }