Kenmark T.
AI Systems & Automation Builder
Case Study — AI Content Operations

Finding the hidden failure in an AI content pipeline

A system was passing its own compliance checks — and still letting factual problems reach human review. The fix wasn't a better check. It was noticing that two different questions had been collapsed into one.

RoleSystems design, build & operation
DomainRegulated finance / mortgage
StatusLive pilot · 2 brands
The problem

Content that followed the rules but still wasn't safe

A financial-services group needed social content produced at a steady cadence across multiple brands. In a regulated space, that's not a volume problem — it's a trust problem. Every post carries factual and compliance risk, and a single wrong number can create exposure for the business.

I designed and built a content operations system that takes a monthly strategy, drafts platform-specific posts, checks them against brand and compliance rules, and routes everything to a human for approval before anything is scheduled. It runs on an automated daily production cadence with a human approval gate — not unattended publishing.

The system worked. Posts were generating, compliance was catching violations, drafts were landing in the review queue. And yet factual problems were still occasionally surfacing at human review — the last place you want to be relying on a person to catch them.

The discovery

The compliance gate was answering the wrong question

When I traced why, the insight wasn't about a broken node. It was structural. The compliance gate was checking whether a post faithfully followed its brief — the right tone, the approved offer, no banned topics. It was doing that well.

But nothing was checking whether the brief itself was true.

The system verified that posts followed their instructions. Nobody was verifying that the instructions were factually supported in the first place.

A post could be perfectly compliant — on-brand, correctly formatted, obeying every rule — and still rest on a statistic the model had produced without a source. "Faithful to the brief" and "true" are two different questions, and the architecture had quietly merged them into one. Once that was clear, the whole class of failures made sense: they weren't compliance failures at all. They were verification failures wearing a compliance mask.

The decision

Separate the responsibilities, keep the human gate

The architectural response was to stop treating this as one job. I split the pipeline into distinct responsibilities — research, verification, strategy, content creation, compliance, and human approval — so that each answers exactly one question and can't paper over another's gap.

The key addition was a verification step that sits before strategy: claims get checked against their actual source material, not against the model's own memory, so an unsupported fact is caught and excluded before it ever becomes the foundation of a post. This is the architecture I designed to close the truth-checking gap; parts of it are built and parts are still being wired and validated.

Just as important was what I chose not to automate. Final approval stays with a human. The system does the repeatable work and routes every genuine judgment call — brand nuance, factual edge cases, anything ambiguous — to a person. Automating that away would have been easy and wrong.

The proof

Failures the pilot exposed before publication

A live system is only as trustworthy as the risks it exposes and contains before publication. These are real catches from the pilot — shown as what was caught and why it mattered, not how it was wired.

Caught in audit safety review

A revision loop that could have published without a human

The workflow that revises rejected content and sends it back was, in an earlier version, configured to publish that revised content automatically instead of returning it for approval — at the single most sensitive point in the system, on material that had already failed review once.

Why it mattered In a regulated pipeline, auto-publishing machine-revised content that already failed review is exactly the gate you cannot lose. I identified the risk during a safety audit and redesigned the revision path to preserve mandatory human approval. I've verified the live configuration returns revised content as a draft; a final supervised end-to-end run is the last confirmation before this path runs unattended.
Held before review compliance gate

Invented statistics and figures, stopped at the gate

Across the pilot the compliance gate repeatedly held content that had fabricated specifics the brief never supplied — an invented time-saving statistic and dollar figures that appeared from nowhere. Different fabrications, each caught automatically and held rather than shipped.

Why it mattered These are the failures that, in finance, turn into real exposure. Cataloguing them as distinct classes — invented number, invented figure, unsupported claim — is what drove the verification redesign above, instead of patching them one at a time.
Caught at review · traced by me config cascade

A guide that didn't exist, and the setting behind it

A post promoting a resource that didn't exist reached human review and was rejected there — the backstop working as intended. Rather than just fix the post, I traced it to a single configuration value that had silently switched the engine's behaviour for a whole category of posts, then mapped exactly which posts were affected and corrected all of them.

Why it mattered Two things had to work: the human gate caught what the automated checks weren't designed to, and the diagnosis went past the symptom to the upstream cause and its full blast radius. Finding that is the difference between a patch and a fix.
The result — stated honestly

A working pilot, with the hard parts named

Operating as a live pilot across two brands, generating platform-specific content and routing it through human approval.
Core workflows and safeguards are functioning — content generation, compliance holds, and approval routing; the safety fix on the revision path is verified in configuration, with a final supervised run pending.
The factual-verification layer is partly built and partly in progress; longer-term performance measurement is still being instrumented.

I'd rather show you a pilot with its edges named than a "finished product" that hides them. What's proven is proven; what's in progress is labelled in progress. In regulated work, that distinction is the job.

Client details and implementation mechanics have been limited to protect business confidentiality.

What it demonstrates

Not tool-wiring — operating judgment

Have a process that's grown past what one person can hold?

That's the work I do — designing systems that run reliably and know where to stop and ask a human.

Discuss an automation pilot