DAWGS

Database Abstraction Wrapper for Graph Schemas

Purpose

DAWGS is a collection of tools and query language helpers to enable running property graphs on vanilla PostgreSQL without the need for additional plugins.

At the core of the library is an abstraction layer that allows users to swap out existing database backends (currently Neo4j and PostgreSQL) or build their own with no change to query implementation. The query interface is built around openCypher with translation implementations for backends that do not natively support the query language.

Development Setup

For users making changes to dawgs and its packages, the go mod replace directive can be utilized. This allows changes made in the checked out dawgs repo to be immediately visible to consuming projects.

Example

replace github.com/specterops/dawgs => /home/zinic/work/dawgs

Building and Testing

The Makefile drives build and test automation. The default make target should suffice for normal development processes.

make

For validation before handing off a change, run the full test target:

make test_all

make test_all runs unit tests and the integration suites. Integration tests use the CONNECTION_STRING environment variable and run against the backend selected by that connection string's scheme.

The shared integration cases under integration/testdata/cases and integration/testdata/templates are expected to be semantically equivalent across supported backends. Avoid driver-specific skips or expected results in those files; add driver-scoped integration coverage instead when a backend-only capability needs to be tested.

Benign local examples:

export CONNECTION_STRING="postgresql://dawgs:weneedbetterpasswords@localhost:65432/dawgs"
export CONNECTION_STRING="neo4j://neo4j:weneedbetterpasswords@localhost:7687"

Neo4j connection strings may use neo4j://, neo4j+s://, or neo4j+ssc://; a single path segment selects the Neo4j database name.

Use make test for unit tests only and make test_integration for integration tests only.

Test Metrics

make test writes unit test coverage artifacts under .coverage/:

make test

The stable coverage profile is .coverage/unit.out, and the function coverage summary is .coverage/coverage.txt.

Cyclomatic complexity, CRAP, and quality signal reports are available through dedicated metric targets:

make complexity
make crap
make quality
make metrics

make complexity writes .coverage/cyclomatic.txt. make crap reruns unit tests for a fresh coverage profile, then writes .coverage/crap.txt, .coverage/crap.json, .coverage/quality.txt, .coverage/quality.json, and a standalone HTML report at .coverage/metrics.html. The quality section summarizes semantic drift, backend equivalence, integration/template invariants, fuzz health, mutation score, and benchmark drift. Signals that need external captures are reported as pending unless their input files are provided. Generated parser files, tests, vendor code, and testdata are excluded from these reports. The HTML report embeds its CSS and JavaScript directly in the document, so it can be opened without network access.

Optional quality inputs can be supplied through Make variables:

make quality BACKEND_RESULT_ARGS="-backend-result pg=.coverage/integration-pg.json -backend-result neo4j=.coverage/integration-neo4j.json"
make quality BENCHMARK_REPORT=.coverage/benchmark.json BENCHMARK_BASELINE=.coverage/benchmark-baseline.json
make quality FUZZ_REPORT=.coverage/fuzz.json MUTATION_REPORT=.coverage/mutation.json

make quality_backend captures PostgreSQL and Neo4j integration results for backend equivalence comparison. It requires PG_CONNECTION_STRING and NEO4J_CONNECTION_STRING. make quality_bench writes benchmark markdown and JSON captures for later baseline comparison. Benchmark drift comparison can be performed by make quality through tools/metrics when BENCHMARK_REPORT and BENCHMARK_BASELINE are provided.

Run the package benchmark suite with:

make test_bench

Use cmd/benchdiff to compare benchmarks between two committed refs without changing the active worktree:

go run ./cmd/benchdiff -base main -target HEAD -kind unit

For integration benchmark comparisons, provide the same CONNECTION_STRING used by integration tests:

export CONNECTION_STRING="postgresql://dawgs:weneedbetterpasswords@localhost:65432/dawgs"
go run ./cmd/benchdiff -base main -target HEAD -kind all -driver pg -fail-regression 10%

The harness writes raw outputs and a Markdown report under .bench/runs/ by default. The report begins with comparison findings, includes the raw benchstat output for each benchmark suite, and ends with a table of all captured benchmark numbers.

The integration benchmark runner includes committed base, adcs_fanout, and traversal_shapes datasets by default. The traversal shape suite checks expected result counts for chain, fanout, bounded cycle, disconnected, edge-kind-selective, and multi-path shortest-path scenarios before recording timings.

make plan_corpus captures plan diagnostics for the shared Cypher integration corpus. It accepts either CONNECTION_STRING for one backend or PG_CONNECTION_STRING and NEO4J_CONNECTION_STRING for both backends, then writes JSONL captures and markdown/JSON summaries under .coverage/.

go run ./cmd/graphbench captures runtime diagnostics for the scale corpus under benchmark/testdata/scale. The current modes are postgres_sql, local_traversal, and neo4j; AGE is reference-design input only and is not a direct comparison mode yet. The command can emit JSONL records plus Markdown and JSON summaries, and can compare current timings against a previous JSONL baseline.

go run ./cmd/retriever dumps and loads live Dawgs graph databases as manifest-based collections of compact OpenGraph-derived fragments. It supports PostgreSQL and Neo4j, gzip and zstd compression, checksum validation before load, optional deterministic property scrubbing, and a read-throughput benchmark mode. It can also package dumps as single HPKE/ML-KEM encrypted TAR archives. See cmd/retriever/README.md for dump, encrypted archive, load, scrubbed dump, metrics verification, and benchmark examples. The same import/export functionality is available to library consumers from github.com/specterops/dawgs/retriever; callers provide an already-open graph.Database, and archive helpers support both path-based and stream-based APIs. The package exposes CLI-matching default option constructors, structured progress callbacks, manifest/metrics helpers, HPKE key envelope reader/writer helpers, and typed errors for validation, compatibility, checksum, metrics, and count mismatches.

PostgreSQL translates exact string property equality with a JSON string type guard and properties ->> extraction, so indexes created on expressions such as properties ->> 'objectid' and properties ->> 'name' can be used for selective anchors without matching JSON booleans or numbers. Simple relationship count fast paths depend on the schema's kind_id-first edge index for efficient typed counts.

PostgreSQL property index regression coverage is hard-failing under the manual_integration tag. The synthetic plan test translates Cypher to PgSQL, disables sequential scans for the EXPLAIN, and requires explicit node property indexes to appear in the JSON plan:

CONNECTION_STRING="postgresql://dawgs:weneedbetterpasswords@localhost:65432/dawgs" \
  go test -tags manual_integration ./integration -run TestPostgreSQLPropertyIndexPlans

Substring and suffix predicates are intentionally not promoted to blanket schema indexes. PostgreSQL deployments can request explicit TextSearchIndex/trigram property indexes for fields that need CONTAINS, STARTS WITH, or ENDS WITH. The hard regression only asserts current index-compatible literal forms; dynamic parameter/property forms that lower to helper functions are intentionally outside that contract until their lowering changes.

Thresholds are report-only by default. To enforce the configured thresholds, run:

make metrics_check

The defaults can be adjusted with CYCLO_TOP, CYCLO_OVER, CRAP_TOP, CRAP_OVER, and BENCHMARK_REGRESSION.