Product & workflow fit
Review the intended users, supported tasks, acceptable outcomes and points where a person must take control.
A successful demonstration is not evidence of reliable operation across real workflows.
Services / AI Safety & Reliability
Production readiness review.Know what stands between demo and dependable.
Before you expand a pilot or launch an AI-enabled product, examine how it behaves, fails and recovers. We turn release assumptions into tested evidence, prioritised findings and a practical plan for your team.
A scoped engineering review. Clear limitations. No launch guarantees.
The business case
The demo works. Can the team operate the release?
A pilot can prove that an idea is useful without proving that it is ready for more users, sensitive data or consequential actions. A readiness review examines the gap before exposure grows.
When to bring us in
Before a customer launch, enterprise rollout, pilot expansion, major model change or handover from a development team. Existing production systems can also be reviewed when incidents, costs or unclear ownership expose operational gaps.
What you get
An evidence-backed assessment of the agreed release scope, a prioritised remediation backlog, retest requirements and an operating handover. The review informs a release decision; it does not replace your accountable owners, legal advisers or specialist certification.
Review coverage
Six dimensions of a real release.
We select checks according to the product, its exposure and your existing evidence. An internal read-only assistant and an agent executing financial actions should not have identical launch criteria.
AI quality & safety
Inspect evaluation coverage, representative data, failure handling, tool permissions and model-change risks.
Check the whole application, not just the model's answer quality.
Application & integration reliability
Review state management, retries, idempotency, dependency failures and recovery from interrupted work.
Actions with side effects need stronger controls than read-only suggestions.
Security & information handling
Assess authentication, authorisation, tenant isolation, secrets, retention and exposure through prompts, logs and tools.
A readiness review does not replace a separately scoped penetration test or legal assessment.
Capacity, latency & cost
Examine expected traffic, concurrency, provider quotas, queue behaviour, response times and spend controls.
Results apply to the workload, environment and dependencies actually examined.
Release & operational ownership
Review rollout, rollback, monitoring, incident response, support handover and named decision owners.
A system is not operationally ready if nobody can recognise or respond to its failure.
The release checkpoint
Bring the evidence together. Keep the return path open.
Quality, reliability, security and operating evidence converge at a shared decision point. Open findings return for remediation and retesting; accepted release conditions shape the rollout rather than disappearing after the meeting.
A review is not a launch switch
The recommendation can be to hold, release under defined conditions or proceed within the reviewed scope. Your product, engineering and risk owners decide whether the remaining exposure is acceptable.
Scope changes the conclusion
Approval for a limited cohort is not approval for unrestricted use. Changes to traffic, permissions, data, providers or workflow behaviour can invalidate assumptions and trigger another review.
Evidence before opinion
Turn launch assumptions into verifiable checks.
We agree what ready means for your product and release scope before assessing it. The review links requirements to evidence, rather than assigning a universal maturity score.
Define the release boundary
Identify the version, environment, user groups, workflows, data access and integrations being reviewed. Separate a limited pilot from a broad public launch.
Set acceptance criteria
Agree measurable quality, response-time, availability and cost criteria with your owners. Include critical failure cases and conditions that must block rollout.
Trace the evidence
Connect each check to a reproducible test, configuration inspection, observed workflow or document. Record evidence dates and versions so conclusions remain interpretable.
Make unknowns visible
Missing access, incomplete tests and unrepresentative environments remain explicit evidence gaps. Not examined is never presented as passed.
AI behaviour
Good answers are only part of a working system.
For LLM applications, agents, retrieval systems and AI-enabled products, we review the interaction between model behaviour and application controls.
Representative evaluation
Inspect coverage of real user tasks, ambiguous requests, unsupported queries and important edge cases. Review data provenance, holdout separation and how human judgements are calibrated.
Grounding & output validation
Check retrieval relevance, source access, citations where appropriate and schema validation. Review abstention and escalation when context is missing or outputs are invalid.
Tools & consequential actions
Assess permission boundaries, approval gates, duplicate-action protection and stopping conditions. Verify that rejected or interrupted requests do not leave unintended side effects.
Change & regression
Review how prompts, models, retrieval indexes and tool definitions are versioned. Confirm what is retested after a change and how a degraded candidate is kept out of rollout.
Failure & recovery
Inspect the paths that a demo rarely reaches.
Reliability depends on what happens when providers slow down, integrations reject a request or work stops halfway through. Tests are bounded and authorised, normally in a representative non-production environment.
Dependency degradation
Exercise agreed timeout, rate-limit and unavailable-service scenarios. Inspect bounded retries, backoff, circuit breaking and honest fallback behaviour instead of silent success.
State & side effects
Review transactional boundaries, deduplication keys, queue acknowledgements and resumable jobs. Check how partial completion is surfaced and reconciled.
Data & service recovery
Inspect backup coverage, restoration evidence, migration safety and recovery procedures. A backup setting alone is not evidence that a restore works.
Human fallback
Verify handoff ownership, the context passed to reviewers and the behaviour when nobody is available. Restricted or manual operation may be safer than an unbounded automated fallback.
Security & privacy
Readiness includes the boundaries around the application.
We inspect selected controls and evidence within the agreed scope, working with your security and privacy owners. High-risk findings may require a deeper specialist assessment.
Identity & isolation
Review authentication, server-side permissions, tenant boundaries and service identities. Include unauthorised access paths and privileged tool execution where agreed.
Sensitive information
Map information sent to models, stored in application state and captured in telemetry. Inspect minimisation, redaction, retention and access controls against confirmed requirements.
Secrets & supply chain
Review secret handling, dependency hygiene, deployment permissions and provider processing terms. Track exceptions rather than treating a clean scanner result as a complete security assessment.
Adversarial exposure
Examine existing evidence for prompt injection, data leakage and tool abuse. A dedicated red-team or penetration-test engagement remains a separate scope when needed.
Performance & economics
Understand the workload before committing to the launch.
We review capacity and cost using agreed scenarios, not hypothetical peak numbers. Production behaviour can differ from a staging test and external provider limits can change.
End-to-end latency
Measure user-visible response time, queue delay, first-response time where relevant and completion latency. Inspect distributions and slow paths, not only averages.
Concurrency & saturation
Exercise representative traffic mixes and identify limits in workers, databases, connection pools and model quotas. Observe behaviour near saturation and during recovery.
Spend & resource controls
Review cost per completed workflow, retries, long conversations, tool calls and token growth. Inspect usage limits, alerting and controls for runaway workloads.
Capacity assumptions
Document expected usage, caching assumptions, vendor constraints and what was not load-tested. Your team agrees operational targets and headroom based on business risk.
Launch & operations
A release needs a way forward and a way back.
We look for a practical operating plan that connects release conditions, live signals and people who can act. Documentation is checked against what the system and team can actually do.
Controlled rollout
Review feature flags, user cohorts, staged exposure and release checks. Define signals that pause expansion and make the decision owner explicit.
Rollback & disablement
Check how the team reverts code, configuration and model versions, or disables consequential actions. Database migrations and irreversible external actions may need forward recovery rather than rollback.
Monitoring & incident response
Inspect dashboards, quality signals, actionable alerts, incident routing and runbooks. Test selected alert and response paths instead of assuming a configured monitor reaches the right person.
Support & ownership
Confirm escalation paths, known limitations, access, operational training and ownership of unresolved findings. Support hours and response commitments are agreed separately, not implied by a review.
Findings & recommendation
A report your team can act on.
Each finding records the affected path, supporting evidence, impact, priority, owner, recommended action and verification needed to close it. Severity and release impact are assessed in the context of your agreed criteria.
Release blockers
Issues that breach agreed mandatory criteria or expose unacceptable risk for the proposed release. Record the evidence, affected workflows, remediation owner and required retest.
Conditional release items
Issues that may be acceptable only with documented restrictions, compensating controls or a limited rollout. The accountable owner must explicitly accept those conditions.
Follow-up improvements
Non-blocking work with a clear rationale and priority. Separate improvements from required launch work so the team can make a realistic plan.
Evidence gaps
Checks that could not be completed or conclusions limited by access, data or environment fidelity. Explain the impact on confidence and what would close the gap.
The final report names the reviewed versions, environment, checks performed, limitations and residual risks. An evidence gap is not a pass, and accepting risk is a recorded business decision, not a technical guarantee.
Tools & frameworks
Use the tools that reveal the actual risk.
We work with your existing delivery and observability setup where possible. These are relevant options, selected for the application, authorised test environment and evidence needed—not a requirement to adopt another stack.
AI evaluation
promptfooDeepEvalRagaspytestHuman review
Choose tests and scoring methods to match the task. Automated or model-based scores are calibrated against reviewed examples where they influence release decisions.
Workflow & API testing
PlaywrightpytestVitestOpenAPI
Exercise real user journeys, schema contracts, permissions, integration failures and recovery paths using your existing test tooling where possible.
Load & resilience
k6LocustToxiproxy
Run bounded workload tests and controlled dependency-failure scenarios in an authorised environment. Provider quotas and test costs are agreed before execution.
Tracing & operational evidence
OpenTelemetryLangfusePrometheusGrafanaSentry
Correlate request traces, AI behaviour, exceptions and resource usage with controlled access and sensitive-data handling.
Security checks
OWASP ZAPSemgrepTrivyGitleaks
Support targeted application, code, image and secret checks. Findings require triage; scanners do not establish that the application is secure.
Delivery & infrastructure
GitHub ActionsGitLab CIDockerKubernetesTerraform
Review reproducible releases, configuration, deployment permissions and recovery in the infrastructure you actually use. These tools are options, not prerequisites.
Tool selection depends on supported capabilities, licensing, information-handling requirements and operating constraints. No single test suite or scanner proves production readiness.
Delivery & outputs
From a release question to a bounded recommendation.
The review is collaborative, with evidence and findings discussed as they emerge. Remediation and further specialist testing are explicitly scoped rather than hidden inside an open-ended assessment.
- 01
Scope & acceptance
Agree the release boundary, decision owners, mandatory checks, test permissions and available evidence.
Output: Review plan, acceptance criteria and access requirements.
- 02
Inspect & reproduce
Review architecture, configuration, existing tests, operating material and representative workflows. Establish what the current evidence supports.
Output: Evidence map, baseline observations and initial gaps.
- 03
Exercise priority risks
Run the agreed quality, integration, performance and failure-path checks. Investigate material findings with your engineers.
Output: Reproducible results and prioritised findings with limitations.
- 04
Remediate & retest
Your team or a separately agreed engineering scope addresses priority issues. Recheck changed paths and relevant regressions.
Output: Retest evidence, residual risks and updated release recommendation.
- 05
Decision & handover
Walk through the findings with accountable owners and document release conditions, open work and reassessment triggers.
Output: Readiness report, remediation backlog and operating handover.
A useful first sprint
Start with a paid two-week sprint and a bounded release question, without a required long-term commitment. It can establish the evidence baseline and examine priority risks; comprehensive coverage or remediation may require further work.
What your team brings
An accountable release owner, engineering access, architecture and deployment information, a representative environment, relevant test data and existing evidence. We agree access permissions, test impact limits and sensitive-data handling before execution.
After the review
Keep the checks. Keep the ownership.
A review should leave your team better equipped for the next release, not dependent on a one-off report. We hand over agreed test assets, findings, operating material and the triggers for reassessment.
Reviewable handover
Your code belongs to you from day one. Agreed scripts, configurations and documentation are handed over subject to data rights and third-party licences. Findings remain traceable to evidence, with named owners for open work.
Support is separately defined
Ongoing monitoring, incident response, routine reviews and remediation are not automatically included. We can define an additional engineering or support engagement, with clear responsibilities and commitments.
Related assurance services
Go deeper where the evidence calls for it.
Combine a release review with focused evaluation, adversarial assessment or operational visibility when those areas need more depth.
Questions before review
The practical details.
Is this only for AI products?
The service focuses on AI-enabled applications, but readiness also depends on ordinary software, infrastructure and operations. We review both within the agreed boundary; model evaluation alone is not a production review.
Can you review a product built by another team?
Yes, subject to authorised access and an agreed scope. We work from your architecture, code, environments, test evidence and operating material. Restrictions or missing information are recorded as limitations.
Do you provide a pass score or certify the product?
We provide findings and a scoped release recommendation against agreed criteria, not an independent certification or universal score. Your accountable owners retain the release decision and any residual-risk acceptance.
Will you test against production?
Representative staging or isolated environments are preferred for failure and load testing. Any production checks require explicit authorisation, bounded execution, impact limits and a stop procedure. Destructive tests are not assumed.
Will you fix the issues you find?
Remediation can be handled by your engineers or included in a separately agreed engineering scope. A review does not imply unlimited implementation work; priorities, ownership and retesting are made explicit.
How long does a review take?
Timing depends on system complexity, access, evidence quality and required tests. A paid two-week sprint can address a bounded first scope; full coverage and remediation may need additional work. We agree deliverables before starting.
How long does the recommendation remain valid?
It applies to the reviewed version, environment, workload and release conditions. Material changes to models, prompts, data, integrations, permissions or traffic can require reassessment. It is not a guarantee of future reliability.
Before the next release
Bring the product.
Let's examine what ready means.
Tell us what you plan to launch, who will use it and what would be costly to get wrong. We'll define a practical first review.