Skip to contents

Pipeline

End-to-end pipeline entry points.

run_pipeline()
Run the end-to-end hydrofabric pipeline for a single VPU
run_vpus()
Build a set of VPUs, then merge them
presplit_reference()
Pre-split a multi-VPU reference fabric into per-VPU slices
prepare_hydrolocations()
Prepare an externally-supplied hydrolocation set for embedding
check_reference_input()
Validate a reference fabric before building from it
check_grid_coverage()
Check that the supplied grids cover the domain
check_hydrolocation_input()
Validate a hydrolocation set before building from it
check_reference_coincident_geometry()
Find reference reaches drawn on top of one another
check_reference_divide_pairing()
Find reference reaches that end outside their own divide
check_reference_downstream_resolves()
Find reference reaches whose downstream neighbour is missing
check_reference_severed_terminals()
Find reference reaches that terminate on a channel that carries on
hf_compare_products()
Compare a rebuilt product against the one it replaces
hl_reference_index()
Index a reference for hydrolocation checking
hf_input_manifest()
Record the inputs a build read
hf_verify_manifest()
Verify inputs against a recorded manifest
hf_output_manifest()
Record what a build produced
hf_verify_output_manifest()
Verify products against an output manifest

Refactor

Split/collapse flowlines and raster-partition divides.

refactor_fabric()
Refactor a reference hydrofabric (split, collapse, reconcile; refactor phase)
refactor_flowlines()
Refactor reference flowlines into reconciled reaches
reconcile_collapsed_flowlines()
Reconcile collapsed flowlines
split_flowlines()
Split flowlines (sequential, flowpath_id-based)
events_to_exclude_cats()
Build exclude_cats from event/split_event data
reconcile_divides()
Reconcile divides to refactored flowlines (CRS-aware, lr-capable)
partition_reconcile_divides()
Partition and Reconcile Divides with Raster-Based Splitting
partition_reconcile_divides_from_objects()
Partition and Reconcile Divides from In-Memory Objects

Aggregate

Aggregate the network and divides to distribution scale.

aggregate_fabric()
Aggregate a refactored fabric to the distribution network (aggregate phase)
aggregate_network()
Aggregate flowpath network topology without divide geometry
construct_agg_divides()
Construct aggregated divides from Stage-1 rec_div via rec_id_list
prepare_network()
Prepare Hydrologic Network
add_network_type()
Annotate Flowpaths and Divides with Type Info
extend_flowpaths_to_mainstem()
Extend aggregated flowpath geometry along its main-stem sub-flowlines

NextGen

Build the NextGen product and finalize the data model.

nextgen_fabric()
Build the NextGen data model from refactored + aggregated layers (nextgen phase)
nextgen_from_gpkg()
Build a NextGen hydrofabric from a single GeoPackage
add_flowlines()
Add Sub-Flowline Segments to Aggregated Hydrofabric
build_aspatial_network()
Build an aspatial long-form network crosswalk
enforce_flowpath_flowdir()
Enforce upstream->downstream orientation on aggregated flowpath geometry
flag_flowpath_constituents()
Flag whether each flowline is a direct constituent of its flowpath's line
flag_routeable_flowlines()
Flag the routeable flowlines (the routing network)
add_upstream_index()
Add a nested-set upstream index to a hydrofabric GeoPackage
finalize_data_model()
Finalize NextGen output to the Lynker hydrofabric data model spec

Global merge

Merge per-VPU outputs into domain products with global ids.

merge_vpus()
Merge per-VPU fabrics into domain products with globally unique IDs

QA, lineage & invariants

The correctness contract and reference-id traceability.

check_invariants()
Pipeline invariant checks
enforce_integer_ids()
Enforce integer typing and indexes on hydrofabric id columns
lineage_build()
Build the initial lineage table from refactor_flowlines output
lineage_attach_aggregated()
Attach aggregated flowpath IDs to a lineage table
lineage_attach_ngen()
Attach NextGen IDs (fp-N, cat-N) to a lineage table
lineage_member_csv()
Recompute member_comid CSV strings from a lineage table