MorningDrop: A Local AI Agent for Data Quality
See how MorningDrop uses local Gemma and data contracts to check nightly exports safely. Explore its workflow, safeguards, trade-offs, and setup.
MorningDrop: A Local AI Agent That Checks Nightly Data Drops
Nightly data exports are easy to ignore when they work, and expensive to miss when they don't. A supplier "helpfully" changes USD to usd, refunds show up with negative quantities, or a new file appears with no one agreeing on what the fields mean. The real problem usually isn't missing data. It's the repetitive, manual work of checking data you didn't produce - so someone else can trust it.
MorningDrop is a small, open-source agent built for that gap. You run it locally on a schedule. It checks new files against data contracts, then uses a local Gemma model to summarize what went wrong and suggest fixes. The model isn't trusted to validate anything - nor is it trusted to write an unchecked report. Its output gets compared to the contract rules and to the facts from the run.
This article walks through how the pipeline works, why the model is intentionally constrained, and what to think about before using a local AI agent for routine data quality checks. The core idea is simple: keep deterministic checks in code, and use AI only where language helps.
What MorningDrop does
MorningDrop watches a folder of incoming exports and checks files that are new since the previous run. It's aimed at the kind of unglamorous data sources that actually break: vendor CSVs, storefront exports, CRM extracts, and shared-folder files - because that's often where integrations end up.
The workflow is deliberately small:
- Find new files in the configured folder.
- Validate rows against data contracts.
- If a file has no contract, draft one for a person to review.
- Send a compact summary of validation failures to a local Gemma model.
- Check the model's response before using it in the report.
- Write a Markdown report, a JSON report, and a short
latest.txtsummary.
The project uses DataContractGate to validate data against YAML contracts. A contract can specify types, required fields, regular-expression patterns, allowed values, and ranges. CSV values are coerced to the expected types first. If a value can't be parsed, it stays a string - so the validator can report a type mismatch instead of quietly hiding the problem.
The model doesn't get whole data rows. It receives a compact account of what failed: counts, field names, validation messages, and the relevant contract rules. Rows that pass don't need a natural-language explanation, so they don't get sent to the model.
That separation matters. The validator answers, "Does this data match the rules?" Gemma helps answer, "How do we explain the result quickly?" Those are different jobs, and mixing them makes it harder to trust what you're reading.
A real run, not a polished dashboard
The sample project includes the kinds of issues that show up in real exports: inconsistent currency casing, refunds represented by a quantity of -1, a country code sent as usa, and a new shipments file with no contract.
In the example run, orders_2026-10-24.csv has 240 rows, and 63 fail validation. The report points to currency and quantity as the main problems. A customer file has four failing rows. The shipments file has no contract yet. Instead of quietly accepting the new dataset, MorningDrop drafts a proposed contract for review.
On a clean night, the output is much shorter: one drop checked and every row passed. In that case, the model isn't called at all. That's a practical choice for cost and reliability: if there's nothing to explain, there's no reason to add another dependency.
The project's demo and source repository show the command-line workflow and report formats. The demo was recorded on a CPU-only machine; the "wait" shown there is trimmed, so it's worth treating runtime as an operational detail, not assuming the model will respond instantly.
How the pipeline keeps the model in bounds
The most important design decision isn't "which model." It's that MorningDrop treats the model's output as untrusted input.
Even small local models can produce text that sounds right while being wrong. During development, the project saw models that repeated the prompt instead of summarizing, invented failure counts for a file without a contract, or suggested a quantity range that didn't match the actual rule. Even valid JSON wouldn't reliably catch that. The model can return the "right shape" while describing a different run.
MorningDrop checks the answer in two steps.
First, validate the output structure
AISchemaGate validates the model's response against a separate YAML contract. For example, the status has to be one of green, yellow, or red; summary lines have to fit defined length limits; and each suggested fix has to include a file, field, kind, and suggestion.
This catches malformed or incomplete responses. It also makes the expected output shape explicit, which helps when you change prompts or swap models. The same contract-based idea shows up at both ends: input data has rules, and generated output has rules too.
Then, check the answer against the run's facts
Structure is only the first layer. Grounding checks verify that the summary matches the validation results. The status has to line up with the worst file status. The summary has to name the worst file. Any numbers mentioned have to already exist in the known facts. Suggested fixes have to point to a real file and a field that actually failed.
These checks stop "pretty writing" from turning into false confidence. If the response fails validation, MorningDrop can send the violations back to Gemma for another try. Incorrect suggested fixes can be quarantined while valid ones are kept. If the model still fails - or Ollama is unavailable - a plain template generates the report from the validation facts.
That fallback is a key reliability feature: the model helps readability, but it isn't the only thing that makes the overnight job work. If the model is down, the data still gets checked and you still get a report.
Proposed contracts are drafts, not policy
When a new dataset appears, MorningDrop can infer a proposed contract from observed types, ranges, small enums, and date patterns. It compiles the draft to confirm the contract is valid, then saves it under contracts/_proposed/.
It doesn't automatically enforce the inferred contract. That's intentional. Inference can speed up the first pass, but a sample of data can't tell you every business rule. A field that looks like it has only three values might get a fourth next month. A range inferred from last night's file might reflect historical data instead of a real limit.
Someone who actually understands the dataset should review the draft and move it into the enforced contracts folder. Automation can reduce contract-writing time, without pretending the model knows the business semantics.
Why run the model locally
MorningDrop uses Gemma through Ollama, but the point is bigger: the language model runs on the same machine as the files. The project can also work with other local OpenAI-compatible servers, including llama.cpp, vLLM, and LM Studio, by changing configuration.
For customer or operational exports, local execution can make the data-handling boundary easier to reason about. The source describes a design where rows don't leave the machine and the model receives only validation facts. That can help when files include personal information, order values, or data governed by internal rules or agreements. Still, local execution doesn't automatically make a deployment compliant. You still need to review access controls, retention, machine security, and your organization's data requirements.
Local execution also removes some dependencies from a scheduled job. There's no hosted API key to rotate, no per-request fee, and no provider outage to interrupt a run. That said, local setup has real costs. You need hardware with enough capacity, you need a model installed and available, and you need a way to monitor failed jobs. A CPU-only run can take tens of seconds for a model summary, and smaller models may be more likely to fail the response checks and trigger a fallback.
Model size is a trade-off, not a quality label. A smaller model might fit constrained hardware, but it may require more retries or fallbacks. A larger model may produce better summaries, but it uses more memory and takes longer. The practical question isn't "Which model is best?" It's "Which model is reliable enough on the hardware and schedule I actually have?"
A hosted model could produce smoother prose. MorningDrop's design favors control and offline operation over that convenience. For a report that's basically a few grounded lines, the operational benefit of keeping the workflow local often matters more than having prettier language.
The project is MIT-licensed, as are the contract tools it identifies. You can inspect the checks instead of trusting an opaque service to decide whether an AI-generated summary is trustworthy. For context on the local runtime, see Ollama's documentation.
Where MorningDrop fits - and where it doesn't
MorningDrop works best when you get recurring files from systems you don't control, and someone is already spending time checking them by hand. It's especially useful when a full data platform would be more than you need: a nightly folder, a handful of contracts, and a report that makes the workflow less fragile.
Scheduling is also pretty conventional. Run morningdrop run from cron, a systemd timer, or launchd, or use the watch mode to poll for files. You don't need hosted orchestration just to inspect a local directory. That said, the normal operational basics still matter: only pick up completed exports, decide what happens to old files, keep reports as long as you need, and alert someone if the scheduled process fails.
What to verify before putting it on a schedule
A sensible rollout is incremental:
- Start with one dataset. Pick a file where the expected fields and values are understood.
- Write or review its contract. A validator is only as useful as the rules it enforces.
- Test representative failures. Include malformed types, missing fields, and boundary values - not just a clean sample.
- Check the reports against known facts. Make sure file counts, failing fields, and suggested fixes match the run.
- Exercise the fallback. Stop the local model or use one that fails the response checks. The validator and report should still work.
- Schedule only after the file handoff is reliable. If a job reads a file while another system is still writing it, you'll treat partial data as a real failure.
That last part is easy to miss. Data validation can't fix an ambiguous file-delivery process. If exports arrive with unstable names, are written in place, or sometimes show up late, make that behavior visible in the workflow. A robust check should distinguish "the data is invalid" from "the expected file hasn't arrived yet."
MorningDrop isn't a replacement for a full data observability platform, schema registry, or incident-management process. It's a focused tool for a repeatable task. If you need cross-system lineage, historical anomaly detection, ownership workflows, or large-scale orchestration, a scheduled local agent may be one component in a larger system - not the whole answer.
The project reports tests using scripted fake models, CI across Python 3.9 and 3.12, and an end-to-end CI job that installs Ollama and runs Gemma on a CPU runner. Those checks are useful signals, but you should still test the exact file formats, contracts, machine, and schedule you plan to use. A passing demo doesn't remove the need to validate your own operating conditions.
The time savings are real in the project's stated use case: swapping a 15-30 minute morning review for an overnight check that covers every row. More importantly, automation can inspect every record consistently instead of relying on a quick visual scan. The trade-off is that you have to maintain contracts and operational safeguards. That's usually worth it when the same checks recur daily - but it's less compelling for a one-off file.
Conclusion
MorningDrop is a good example of putting a small language model into a system that doesn't rely on trusting it blindly. Contracts do the actual checks with testable results. Gemma turns failure facts into a readable summary. A second contract and grounding checks constrain what you see, and a template keeps the job useful even when the model can't respond.
That split of responsibilities is the part worth copying. Use AI where language helps, keep deterministic rules in code, and build a fallback before you rely on the process unattended. If you regularly inspect vendor exports or other nightly drops, review the MorningDrop repository, try it on a representative file, and confirm that its contracts and failure handling match your workflow before letting it run automatically.