datHere/ckan_geoconnex_bulk_runner
1
0
Fork 0
mirror of https://github.com/dathere/ckan_geoconnex_bulk_runner.git synced 2026-10-04 23:08:10 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
📦 (WIP) Program that sets up a bulk integration between a CKAN water data hub's datasets and vector geospatial features with Geoconnex. Ran as a Docker container and built with Rust.
48 commits 1 branch 15 tags 230 KiB
  • Rust 95.8%
  • Dockerfile 4.2%
Find a file
Exact
rzmk e5098e7cb4 fix: use validate subcommand for nabu
2026-10-01 23:42:33 -04:00
.github/workflows ci: clarify job name 2026-09-15 15:24:19 -04:00
bulk_loader feat: use nabu SHACL validator and update deps 2026-09-15 11:15:05 -04:00
generate_release fix: label and use intended response values for debugging 2026-09-15 15:27:19 -04:00
geoconnex_utils fix: use validate subcommand for nabu 2026-10-01 23:42:33 -04:00
.gitignore feat: use nabu SHACL validator and update deps 2026-09-15 11:15:05 -04:00
Cargo.lock feat: use nabu and /api/3/action/gztr_geoconnex_dataset_jsonld, remove jsonschema 2026-09-15 15:22:21 -04:00
Cargo.toml build: remove legacy Python lib from root Cargo.toml 2026-09-17 08:28:47 -04:00
README.md docs: update README to reflect recent changes 2026-09-15 19:56:34 -04:00

README.md

ckan_geoconnex_bulk_runner

https://github.com/user-attachments/assets/779fe866-d511-44f3-91a9-a1c2e1cfa189

Status: This codebase is currently a work in progress and more documentation is planned.

The ckan_geoconnex_bulk_runner codebase is part of a multi-software-system infrastructure to sync water data hubs using CKAN to the Geoconnex knowledge graph using the ckanext-gztr CKAN extension.

  • generate_release: Compatible CKAN datasets for all connected water data hubs are uploaded to a JSONL file in the latest GitHub release.
  • bulk_loader: Requests and outputs the latest JSONL file from the latest GitHub release. This is ran as a Docker container for each CKAN instance by Geoconnex on a periodic frequency to upload all water data hub web resources to the Geoconnex knowledge graph following the Geoconnex bulk contribution specification.
  • geoconnex_utils: Helper functions used throughout the ckan_geoconnex_bulk_runner project including JSON-LD construction and JSON schema validation.

This runner is implemented for a water data hub with the relevant fields and/or ckanext-gztr. For questions reach out to datHere, Center for Geospatial Solutions, or add an issue/discussion.

Acknowledgements

The ckan_geoconnex_bulk_runner software system is built in collaboration with several organizations. Please view the bottom of gztr.dathere.com to learn more about each organization.