Use this checklist when an internal-tools team needs to connect an AI run investigation to a human review decision. Read From AI run logs to an internal review app for the architecture and tradeoffs.
This is a proposed prototype built from documented platform capabilities. It is not a ready-made template or a claim that the complete application has been runtime-tested.
Prerequisites and version boundary
- Complete one synthetic request and inspect its matching run using the gateway evaluation checklist
- Record the installed version and exact supported interfaces
- Use synthetic data and an account authorized for the test application
- Choose what success means: a reviewer can inspect a run, save a linked decision and read it back
The v0.4.1 README documents Data Models, CRUD APIs, workflow-backed APIs, React/TSX blocks and MCP. The newer log-structure note describes a development schema snapshot, not a stable API contract. In particular, do not assume its task-level grouping exists in v0.4.1.
1. Establish a supported read path
- Match a synthetic prompt marker and time to a run
- Inspect the available model, tool and final-output evidence
- Identify the authorized API or built-in view that supplies the required reference and fields
- Record missing fields and version differences
The v0.4.1 console log handlers require console authorization. The application key used for model requests is not a grant to inspect administration logs.
Do not write directly into internal log tables or invent an endpoint from a table name. If the installed version lacks the required supported read interface, use manual inspection and record that integration gap.
2. Define your business record
Create a small Data Model for the review process. Suggested fields:
| Field | Purpose |
|---|---|
| Application reference | Identify the owning application |
| Run reference | Link to original execution evidence |
| Review status | New, investigating or resolved |
| Issue category | Group the team’s own investigation categories |
| Reviewer note | Store the human conclusion |
| Timestamps | Track creation and latest update |
These are your proposed business fields, not built-in log columns. Prefer references over copied conversation content. Define required values, who can edit them and what happens when the referenced run is removed by retention cleanup.
3. Build and test the human flow
- Generated CRUD operations can create and read the review record
- A React/TSX list and detail view display the correct record
- A status change persists and survives a fresh read
- The original evidence is accessible only to an authorized reviewer
- A second user without the required access cannot retrieve another application’s evidence
- Unavailable or still-processing evidence is labeled accurately
Keep human review status separate from execution status and answer quality. A successful request can contain a poor answer; a resolved review does not change the original execution outcome.
4. Evaluate MCP access separately
If an agent needs to assist, begin with the documented progressive mcp_list, mcp_get, mcp_call discovery pattern. Verify what the installed version actually exposes, authenticated identity and permissions.
- Expose only capabilities needed by the agent
- Start with read-only inspection
- Review write operations independently before enabling them
- Keep the project context available for complex application work
MCP tooling and ready-to-use application templates are still being refined. Do not treat this checklist as a no-code, no-context application generator.
5. Check data lifecycle and failure boundaries
Choose retention, backups and access policies for both the source evidence and review records. Keep private prompts, credentials and full logs out of public issues; report only sanitized reproduction details through the project issue tracker.
Persistent run records support inspection. They do not alone prove durable execution, automatic crash recovery, safe retries or exactly-once external effects. Test such requirements separately against the installed version.
For deeper manual investigation, use Token BOM. Keep a short prototype receipt with passed, failed and untested checks before expanding the application.