Many AI agent projects do not die on technology — they die on not knowing whether a step counts as done. There are seven steps between requirements and pilot, and each should produce one checkable deliverable; a step without a deliverable runs on feeling.

Questions to ask before procurement

  • Where is the pilot's boundary — which process, which request types, how many users? How are out-of-scope requests handled?
  • What is the pilot's success criterion, and who judges it? Is it written as a number, not as “it works well”?
  • Who is the business owner? During the pilot, who can stop it, and who provides data and signs off acceptance?
  • Where does the data come from, and who approves permissions? Are the knowledge base's sources, update owners, and confidentiality levels confirmed?
  • What are the exit conditions if the pilot fails? How are data and systems cleaned up afterwards, and how are costs settled?

How to judge technically

Accept the project stage by stage along the delivery chain. Negency's chain has seven steps — requirements & baseline, architecture, development, integration, testing, controlled launch, maintenance — each with a checkable output, and nothing ships below the threshold. Judge a project plan on three points:

  • Every stage has a deliverable. The requirements stage produces a scenario specification and baseline data, not a verbal agreement; the architecture stage produces a layered diagram and a permission matrix, not a few slides.
  • Every stage has a gate. No integration without a passing evaluation; no pilot without passing integration tests. Gates belong in the contract, not in “we will try”.
  • The pilot has a control. Pilot results must be compared against the pre-launch baseline: for the same batch of requests, what did humans produce, what did the agent produce, and where do they differ? A pilot without a baseline can prove neither success nor failure.

The Measure function of the NIST AI Risk Management Framework follows the same logic: define measurable indicators before claiming a system is usable. The evaluation set should come from real business samples and cover failure cases — a set that only tests success scenarios cannot reveal production risk.

Where this applies

This deliverables list fits enterprises running their first AI agent pilot, especially where real business systems must be connected. It does not fit pure research exploration (no business owner, no acceptance criteria) or mature teams that have already run several pilots — those need a scale-up and operations list, not a from-scratch deliverables list.

Risks

  • Scope creep. Once a pilot “works well”, everyone adds scenarios. The pilot scope must be fixed in writing; new scenarios go to the next round of evaluation.
  • Underestimated data quality. Outdated documents, contradictory versions, and missing permission labels in the knowledge base will all surface during the pilot. Data cleanup belongs in the plan; do not assume the data is already clean.
  • Underestimated integration effort. Connecting a system is not just calling an interface — it includes permission mapping, exception handling, and audit trails. Estimate each integration point separately.
  • Missing acceptance criteria. Without written criteria, everyone disagrees at the end of the pilot about whether it worked. Criteria must be signed before the pilot starts, not written afterwards.
  • Pilot becomes a permanent demo. Without a scale-up or stop decision point, a pilot keeps “piloting”, consuming budget without producing production value. The pilot plan must state the decision date and the decision criteria.

Deliverables checklist

  • Requirements & baseline: scenario specification, baseline data (human-handled samples), success criteria and exit conditions
  • Architecture: layered architecture diagram, reads/writes in separate lanes, permission and approval matrix
  • Development: knowledge base inventory (sources, update owners, confidentiality levels), evaluation set and pass thresholds
  • Integration: tool integration list (each system interface, permission mapping, exception handling), MCP or API integration notes
  • Testing: functional test report, evaluation report (including failure-case analysis), security testing (prompt-injection defence)
  • Controlled launch: pilot run plan (scope, users, duration), monitoring and rollback plan, human fallback process
  • Review: pilot run report (against baseline), scale-up or stop recommendation, next-stage plan

Sources

Next step: if you are planning an agent pilot, start with Negency enterprise AI systems & intelligent solutions — a seven-layer technical architecture and a diagnose, architect, pilot, deploy, operate chain, with each step's output and gate stated plainly; or take the AI scenario screening to confirm which service line fits your scenario.