How to Run a 90-Day AI Pilot That Actually Ships

Why Most AI Pilots Fail

Let me be direct: most AI pilots do not fail because the technology did not work. They fail because they were never designed to reach production. They picked a flashy demo instead of a costly process, they never set a baseline to measure against, and they had no plan for the boring 20 percent that a real system has to handle. So they produce an impressive demo, a vague good feeling, and nothing that ships.

A pilot is not a science experiment. It is the first phase of a production system, run small and measured, so you can decide whether to scale on evidence instead of hope. Ninety days is enough to prove or kill a well-scoped use case, if you run it with discipline. This guide is that discipline.

A good pilot answers one question with a number: does this process, automated, beat the manual baseline enough to justify scaling it. Everything else is theater.

We run AI engagements this way, and we prove patterns on our own products, Exfinity and OGuardAI, before we bring them to clients. This is the playbook. For the business framing of what to automate, see our real AI isn't a chatbot guide, and this is core AI services and consulting work.

Who Cares About What

The Rule: Pick the Process, Not the Demo

The single most important decision is what to pilot, and it happens before any code. The right target is not the most impressive, it is the most repeated and most expensive.

Score candidate processes on three axes.

High volume?           ──▶ yes ──┐
Rule-heavy with        ──▶ yes ──┼──▶ pilot this
   exceptions?                   │
A person spends hours  ──▶ yes ──┘
   on the boring 80%?

A process that runs thousands of times a year and eats senior time is a strong pilot. A process that runs occasionally is not worth a custom build, no matter how interesting. Our consulting team helps score these honestly, and our use cases show the shape of good candidates.

Set the Baseline Before You Build

You cannot prove a pilot worked if you never measured what it replaced. Before any code, measure the manual process: how long it takes, how much it costs, how often it errs, how the team feels about it. This baseline is the yardstick, and agreeing on it up front prevents the "it feels better" argument that kills serious evaluation.

Without this, the pilot's success is a matter of opinion. With it, the scale decision is a matter of arithmetic. Our AI observability guide covers instrumenting these from day one.

The 90 Days, Phase by Phase

Here is the shape of a pilot that reaches a real decision.

Weeks 1 to 3: Scope and baseline. Pick the process, measure the manual version, define what "better" means as a number, and design the smallest system that could prove it. No building yet beyond a spike.

Weeks 4 to 8: Build the core. Build the retrieval or the pipeline or the agent, on production foundations, not a throwaway. Wire in the human review queue from the start, because the exceptions are part of the system, not an afterthought.

Weeks 9 to 11: Run on real data. Put real work through it alongside the manual process, and measure against the baseline. Watch the confidence and error numbers, not just the happy path.

Week 12: Decide. Compare to the baseline and make an honest ship-or-kill call. If the numbers hold, you have the first phase of a production system, not a demo to rebuild.

Weeks 1-3    Scope + baseline
Weeks 4-8    Build core + human loop
Weeks 9-11   Real data, measured
Week 12      Ship-or-kill decision

The key discipline: build on foundations that can become production, so a successful pilot expands instead of being thrown away and rebuilt. We cover that transition in our prototype to production guide.

A Worked Example: A Support-Triage Pilot

Make the playbook concrete with one pilot, a support-ticket triage system that drafts first responses.

Weeks 1-3   Baseline: measure current triage. A ticket takes 8 minutes
            to first response on average, 400 tickets a week, agents
            spend most of it finding the right policy. Define "better":
            cut time-to-first-response, hold quality, at 80% coverage.
Weeks 4-8   Build: retrieval over the policy base, a draft-reply agent,
            and a review queue where an agent edits and sends. Built on
            production foundations, not a throwaway.
Weeks 9-11  Real data: run it beside the manual process on live tickets.
            Measure time saved, edit rate (how often agents change the
            draft), and any wrong policy cited.
Week 12     Decide: drafts are used with light edits on most tickets,
            time-to-first-response is down meaningfully, wrong-policy
            rate is at or below the manual baseline. Ship it, expand
            to the next queue.

Notice what made it work. There was a real baseline, so "better" was a number, not a feeling. The review queue was there from day one, so quality stayed contained and the agents saw a tool that helped them. And it was built to become production, so shipping meant expanding, not rebuilding. If instead the edit rate had been high and the wrong-policy rate up, that is a clear iterate-or-kill signal, and finding that out in 90 days for a modest cost is exactly the point. Our prototype to production guide covers the expansion step.

Ship With a Human in the Loop

A pilot that fully automates a judgment task is not a pilot, it is a liability test. The right design keeps a person on the exceptions and anything high-stakes from day one. The system handles the volume, a human handles the edges, and you measure how much volume the system can safely take.

This is the human-in-the-loop pattern, and in a pilot it does double duty: it keeps errors contained while you learn, and it shows the team the tool helps them rather than replaces them, which matters enormously for adoption. Our AI governance guide covers the accountability side.

Handle Compliance in the Pilot, Not After

The fastest way to kill a promising pilot is to discover at scale-up time that it was never going to pass a data-protection or EU AI Act review. Handle it during the pilot. If the process touches personal data, protect it at the boundary from the start, as covered in our PII-safe RAG guide. If it touches regulated decisions, keep the human oversight and the audit trail in from day one. A pilot that ignores compliance is not a smaller version of the production system, it is a different system that cannot scale. Our GDPR and AI guide covers what to build in.

Signals You Are Ready to Pilot

A pilot is worth running when several of these hold.

• You have a specific process in mind that is high-volume, repetitive, and eats senior time on the boring part.
• Leadership wants proof before committing to a larger build, and a demo is not enough to convince them.
• A previous AI effort stalled at the demo stage and never reached production, and you want to avoid repeating that.
• You can name a baseline you would measure against, or you are willing to spend the first weeks establishing one.
• You have access to real data and a process owner who will engage, not just an idea in a slide deck.

If most of these are true, you are ready, and the 90-day structure will give you a clear ship-or-kill answer. If you cannot name the process or the baseline, that is the first conversation to have, and our consulting team runs exactly that scoping. A pilot without a target is just a demo with a deadline, so the readiness is in the specifics, not the enthusiasm.

The Decision: Scale, Iterate, or Kill

At the end, you make one of three honest calls, and all three are fine outcomes.

Scale, if the numbers beat the baseline enough to justify it. Iterate, if it is close and you can see the specific fix. Kill, if it does not work, and killing a bad pilot cheaply is a success, not a failure, because you spent 90 days instead of a year finding out. The whole point of the discipline is to make this decision cheap and clear. Then you take the next process. Our methodology is built around this staged, evidence-driven expansion. Start at contact or get a scoped quote.

Common Ways This Goes Wrong

1. Piloting the demo, not the process. Pick the most repeated, most expensive workflow, not the flashiest.
2. No baseline. Without a measured before, success is an opinion. Measure first.
3. A throwaway prototype. Build on production foundations so success expands instead of rebuilds.
4. No human loop. Full automation of judgment ships errors. Keep a person on the exceptions.
5. Compliance as an afterthought. A pilot that ignores it cannot scale. Build it in.
6. No kill criteria. Define what failure looks like up front, so you can end a bad pilot cheaply.

Who Builds This

Oronts is a founder-led software company in Munich. Refaat Al Ktifan, our founder and solution architect, leads a senior team that ships AI pilots designed to reach production, not to impress and stall. We prove patterns on our own products, Exfinity and OGuardAI, before bringing them to clients. We scope the process, set the baseline, build on production foundations, and give you a ship-or-kill decision with a number behind it. See our services and solutions pages.

Takeaways

• A pilot is the first phase of a production system, not a science experiment.
• Pick the most repeated, most expensive process, not the most impressive demo.
• Set a measured baseline before you build, so the scale decision is arithmetic, not opinion.
• Build on production foundations and ship with a human in the loop from day one.
• End with an honest ship-or-kill call. Killing a bad pilot cheaply is a win.

Ninety days is enough to know. The discipline is not speed, it is honesty: a real target, a real baseline, and the courage to kill it if the number does not hold.

Ready to run a pilot that ships or dies cleanly? Tell us the process. Start at contact or get a scoped quote.