Live AI systems need care: monitoring, evaluation, model changes and a steady stream of improvements. An ongoing partnership keeps what is live healthy and moving, on a rhythm you can see.
Why an ongoing partnership exists
An AI system is not a machine you switch on and walk away from. Providers deprecate models, behaviour drifts with every release, and the work it does keeps growing.
The partnership is the answer to that: a standing cycle of review, evaluation and small improvements, with the evidence shown to you every month. When there is nothing worth improving in a cycle, we will say so rather than invent work.
What you get
These are the commitments behind every ongoing partnership; the exact scope is agreed with your team in writing.
The evaluation set agreed at launch keeps running against the live system. You see accuracy, error rates and cost each cycle, including the cases the system got wrong.
What to verify: Ask to see a sample evaluation report from a current engagement, and how often it is produced.
Providers deprecate models, and prompt behaviour drifts with every release. Changes are tested against the evaluation set in a separate environment first, and the results are shown to you before anything is swapped in production.
What to verify: Ask what happens, step by step, when a model we depend on is deprecated, and who signs off the replacement.
Each cycle starts with a review of what the live system is doing, then we agree the next increments together. Improvements ship as small, verifiable changes rather than a yearly big reveal.
What to verify: Ask for the cycle calendar: when reviews happen, when increments ship, and what evidence each ships with.
As the system takes on more decisions, the review steps, audit trails and access rules grow with it. The governance you had at launch is revisited, not assumed still valid.
What to verify: Ask how the human review steps and audit trails are updated when the system's scope changes.
Documentation, runbooks and walkthroughs are kept current as the system evolves, so your team can take over at any point if the partnership ends.
What to verify: Check the documentation is current, not a snapshot from launch day, and that your engineers can run the evaluation set themselves.
AI in the engagement
A live AI system is measured, versioned and owned, not left to drift.
Evaluation does not end at launch. Real examples from production are added to the set, and the bar the system must pass is revisited as its role grows.
Prompts, retrieval setups and pipelines are versioned and tested like code. A change goes through review and evaluation before it reaches production, never by live editing.
When the live system misbehaves, a named engineer responds, not a queue. You know who to call and what the response times are.
The monthly cycle
The same four stations every month, so you always know where the partnership stands.
Working principles
Rhythm, small increments and an honest exit decide whether an ongoing partnership actually works. We make those commitments explicit rather than implied.
The cycle is fixed: review the metrics, agree the increments, build them, report the results. You always know where the month stands.
Improvements ship in small pieces with their own evidence, so a bad change is caught early and can be rolled back without drama.
One senior engineer owns the partnership and is your single point of contact across every cycle.
The agreement runs on a rolling basis after the initial term. If you leave, you leave with the code, the documentation, the evaluation sets and the knowledge, intact.
Security & trust
The partnership works on your live systems and your customers' data. Access, ownership and data handling are agreed in writing and revisited every cycle.
Everything runs in your cloud and your repositories. The partnership works on your systems, not around them.
What we can access is agreed in writing at the start and reviewed every cycle. Access shrinks or is revoked when the work no longer needs it.
Production data stays in your environment. Where test data is needed, it is anonymised or synthetic unless you agree otherwise in writing.
Working arrangements
The cycle runs remotely inside your rituals, in your cloud and your repositories. Quarterly reviews and kick-offs can be in person where being in the room genuinely helps.
Reviews, planning and daily work happen remotely inside your tools and rituals.
Evaluation, staging and production all live in your cloud and your accounts, from day one to the last cycle.
Kick-offs, quarterly reviews and handover sessions can be in person where being in the room genuinely helps.
How it starts
Most partnerships start from a handover: a sprint, a fixed-scope project or a pod build we then keep healthy. The first cycle is agreed in writing before it starts.
Most partnerships start from a handover: a sprint, a fixed-scope project or a pod build we then keep healthy. We agree the cycle calendar, the metrics and the response commitments in writing.
Each cycle opens with the numbers: evaluation results, error rates, cost and what changed since last time. Nothing is hidden in a dashboard you never see.
The agreed increments are built and shipped in small pieces, each with its own evaluation evidence before it reaches production.
The cycle closes with a report of what shipped, what it changed in the numbers, and the proposal for the next cycle. Then it starts again.
A pod builds new products and features. An ongoing partnership keeps live systems healthy and evolving: monitoring, evaluation, model changes and a steady stream of improvements. Many clients run both, with the pod feeding the partnership.
After the initial term it runs on a rolling monthly basis. You can end it with 30 days' notice and keep everything we have built.
We test the replacement against the evaluation set in a separate environment, show you the results including regressions, and agree the switch date with you before anything changes in production.
Yes, and it is a common path. The partnership starts by taking ownership of the handover: current documentation, the evaluation set and what is running in production.
Review the metrics, agree the increments, build and ship them with evidence, report the results and propose the next cycle. The rhythm is the same every month, so you always know where things stand.
The standing evaluation runs, production error and drift signals, samples of human review decisions, and the cost of running the system. You see all of it each cycle.
When the system is still being built and the scope is open, a pod or a sprint fits better. A partnership is for systems that are live and need to stay that way.
Let's talk
Tell us what is live today and what worries you about it. We will tell you honestly whether an ongoing partnership fits, and what the first cycle would look like.