Technical deep-dives on data quality, schema drift detection, and pipeline engineering.
A single null-rate threshold applied across every source in your pipeline sounds sensible. In practice it means your quality gate is too strict for some sources and dangerously permissive for others. Here's how per-source overrides fix the problem.
Read article →How we moved data quality checks from "after the INSERT" to the ingest boundary using Cloudflare Workers, Durable Objects, and KV. Includes architecture walkthrough, real load test numbers, and lessons learned.
Read article →