# DataScreenIQ

> DataScreenIQ is a real-time data quality screening API. Send any JSON or CSV payload and get PASS / WARN / BLOCK in under 10ms. It detects schema drift, null spikes, type mismatches, outliers, and 18 other quality checks at the ingest boundary — before bad data reaches your database.

Important notes:
- DataScreenIQ is an API-backed screening engine, not a library that runs locally
- All checks run in-memory on Cloudflare Workers — no data is stored, only aggregated statistics
- Free tier: 500K rows/month, no credit card required
- Python SDK available: `pip install datascreeniq`
- Supports JSON and CSV input via API, plus pandas DataFrames via the SDK

## API Reference

- [Screen endpoint](https://datascreeniq.com/api-reference.html): POST /v1/screen — screen a data payload and get a quality report
- [OpenAPI spec (JSON)](https://datascreeniq.com/openapi.json): Full OpenAPI 3.0 specification
- [OpenAPI spec (YAML)](https://datascreeniq.com/openapi.yaml): Full OpenAPI 3.0 specification in YAML

## Documentation

- [Homepage](https://datascreeniq.com): Product overview, pricing, and signup
- [Privacy architecture](https://datascreeniq.com/privacy.html): How data is processed and what is stored
- [Terms of service](https://datascreeniq.com/terms.html): Terms and conditions

## Python SDK

- [PyPI package](https://pypi.org/project/datascreeniq/): Install with `pip install datascreeniq`
- [GitHub repository](https://github.com/AppDevIQ/datascreeniq-python): Source code, examples, and issue tracker
- [SDK README](https://github.com/AppDevIQ/datascreeniq-python/blob/main/README.md): Full SDK documentation with quickstart, response structure, and pipeline integration examples
- [Usage examples](https://github.com/AppDevIQ/datascreeniq-python/blob/main/examples/usage.py): Code examples for screening, pipeline guards, file screening, and DataFrame screening

## Integration Examples

- [GitHub Action](https://github.com/AppDevIQ/datascreeniq-python/tree/main/examples/integrations/github-action): Screen CSV/JSON files on every PR
- [Airflow DAG](https://github.com/AppDevIQ/datascreeniq-python/tree/main/examples/integrations/airflow): Quality gate task between extract and load
- [dbt post-hook](https://github.com/AppDevIQ/datascreeniq-python/tree/main/examples/integrations/dbt): Screen model output after dbt run
- [Prefect flow](https://github.com/AppDevIQ/datascreeniq-python/tree/main/examples/integrations/prefect): Quality gate flow with alerting
- [Google Colab notebook](https://colab.research.google.com/github/AppDevIQ/datascreeniq-python/blob/main/examples/integrations/colab/datascreeniq_demo.ipynb): Interactive demo — try DataScreenIQ in 60 seconds

## Quality Checks

DataScreenIQ runs 18 quality checks in a single pass:

- Null rate detection (WARN ≥30%, BLOCK ≥70%)
- Type mismatch detection (WARN ≥5%, BLOCK ≥20%)
- Empty string rate detection
- Duplicate rate detection
- IQR-based outlier detection (1.5× IQR)
- HyperLogLog approximate distinct counts (±2% error)
- Enum/cardinality tracking for low-cardinality fields
- Timestamp detection and recency checks (WARN ≥24h, BLOCK ≥72h)
- Schema drift detection: field added, field removed, type changed, null spike, empty string spike, new enum value, row count anomaly, timestamp staleness
- Percentiles (p25/p50/p75/p95) for numeric fields
- Health score calculation (0.0–1.0) with configurable thresholds

## Pricing

| Plan | Price | Rows/month |
|------|-------|------------|
| Developer | Free | 500K |
| Starter | $19/mo | 5M |
| Growth | $79/mo | 50M |
| Scale | $199/mo | 500M |

## Quick Start

```python
import datascreeniq as dsiq

client = dsiq.Client("dsiq_live_...")  # get free key at datascreeniq.com

report = client.screen([
    {"order_id": "ORD-001", "amount": 99.50, "email": "alice@corp.com"},
    {"order_id": "ORD-002", "amount": "broken", "email": None},
], source="orders")

print(report.status)      # BLOCK
print(report.health_pct)  # 34.0%
print(report.summary())   # 🚨 BLOCK | Health: 34.0% | Type mismatches: amount | Null rate: email=67%
```

## Optional

- [Postman collection](https://www.postman.com/vignesh-datascreeniq-3090309/datascreeniq-s-workspace/): Run in Postman
- [Hoppscotch](https://hoppscotch.io/?method=POST&url=https%3A%2F%2Fapi.datascreeniq.com%2Fv1%2Fscreen): Try in browser
- [RapidAPI listing](https://rapidapi.com): Available on RapidAPI marketplace
- [Blog](https://deviq.hashnode.dev): Technical articles about data quality engineering