Application request execution

This page explains how the application layer starts, coordinates, executes, and finishes a model request. It is the canonical maintainer guide for request ownership shared by HTTP, terminal, application plugins, and background work.

The key distinction is:

SessionRequestCoordinator owns the application-facing request operation;
AgentApplication.send_request(...) is the lower-level event generator;
ToolLoopRunner performs provider and tool-loop phases;
FastAPI and terminal are adapters that delegate to the application layer.

Main implementation files

core/python/agent_app/request_execution.py
request admission, preparation, background execution, public event publication, and cleanup;
core/python/agent_app/application_future.py
AgentApplication, request lifecycle actions, persistence reconciliation, cancellation state, and the low-level request generator;
core/python/agent_app/application_owner.py
application-generation admission and reload draining;
core/python/agent_app/tool_loop.py
provider execution, streaming, tool calls, checkpoints, and cancellation;
application/python/agent_terminal_app/server.py
HTTP validation, idempotency, response mapping, and route definitions;
application/python/agent_terminal_app/terminal_app.py
terminal-facing delegation and current-session state;
application/python/agent_terminal_app/terminal.py
prompt-toolkit input and event rendering.

Ownership map

Component	Responsibility
`SessionRequestCoordinator`	Single-flight admission, request preparation, worker ownership, public event publication, and cleanup
`AgentApplication`	Sessions, effective config, lifecycle actions, cancellation, persistence, notifications, and the low-level request generator
`AgentApplicationOwner`	Admit work against one application generation and wait for admitted work before reload
`ToolLoopRunner`	Provider phases, streaming, tool execution, request checkpoints, and cooperative cancellation
Event store	Public request events, checkpoint markers, session cursors, and frontend polling
FastAPI	HTTP parsing, idempotency keys, status/error translation, and serialization
Terminal	User input, current-session presentation state, event polling, and rendering
Application plugin or background process	Choose when to start a request and react to the returned start/result object

The normal background flow is:

HTTP / terminal / plugin
        |
        v
AgentApplication.start_session_request(...)
        |
        v
SessionRequestCoordinator
  1. acquire application generation
  2. validate and reserve the session
  3. optionally append a message
  4. capture the request baseline
  5. start the request worker
        |
        v
AgentApplication.send_request(...)
        |
        v
ToolLoopRunner -> provider -> tools -> provider ...
        |
        v
session persistence + public events + cleanup

Application request APIs

AgentApplication exposes four coordinated request operations.

Start from existing history

started = app.start_session_request(
    session_id,
    overrides={"temperature": 0.2},
    stream=True,
)

This starts a background request without appending a message. It is the application-layer equivalent of:

POST /sessions/{session_id}/send

Atomically append and start

started = app.start_session_request_with_message(
    session_id,
    role="user",
    content="Explain the latest result.",
    metadata=None,
    overrides={},
    stream=True,
)

Admission occurs before append. If another request is already active, the message is not appended.

This is the application-layer equivalent of:

POST /sessions/{session_id}/send_message

Run synchronously

result = app.run_session_request(
    session_id,
    overrides={},
    stream=True,
    on_checkpoint=handle_live_session,
)

This uses the same admission, preparation, event publication, and cleanup logic but runs in the calling thread. Terminal compatibility APIs use this form when they need a final result tuple.

Inspect the active request

request_id = app.get_active_session_request_id(session_id)

This is the authoritative application-process single-flight state. Adapters must not maintain a separate active-request map.

Query derived status for explicit sessions

snapshot = app.get_session_request_statuses(["session-one", "session-two"])

SessionRequestStatusService checks coordinated active state first. When no request is active, it classifies the persisted conversation tail and caches that classification by the session store's opaque revision. Empty history is idle, a user tail is not_started, a final assistant tail without tool calls is succeeded, and every other incomplete conversational tail is interrupted.

FastAPI exposes this through POST /sessions/request-statuses. The request must contain explicit session_ids; omitted IDs never mean all sessions. Missing sessions are partial results. The response authority is derived_transient, so it does not claim durable failure/cancellation history and can later delegate to persisted operation state.

Cancel

app.cancel_request(request_id)

Cancellation remains owned by AgentApplication. It is cooperative: providers and tool loops observe a RequestCancellation, but a blocking provider operation may not stop immediately.

Start and result objects

A background start returns SessionRequestStart:

SessionRequestStart(
    status="started",  # or "already_running"
    session_id=session_id,
    request_id=request_id,
)

When admission finds an existing request, it returns status="already_running" with that existing request ID.

A synchronous run returns SessionRequestResult:

SessionRequestResult(
    session_id=session_id,
    request_id=request_id,
    final_session=serialized_session_or_none,
    final_messages=[...],
)

Admission and preparation

Application-generation admission

Before preparing a request, the coordinator asks ApplicationControl to acquire an owner operation.

The operation must refer to the same AgentApplication instance as the coordinator. This prevents background work from starting through an application generation that has already been replaced.

While reload is pending, owner admission raises ApplicationReloadPendingError. A request is rejected before an optional message append.

The worker retains the owner operation after an HTTP route returns its immediate acknowledgement. Reload waits for the worker to release it.

Applications constructed without an AgentApplicationOwner remain supported; in that case no owner-operation handle is required.

Session lock and writable view

Preparation runs under the shared per-session lock:

reject a selected readonly snapshot;
load the current session and its configured agent runtime;
check whether the coordinator already has an active request;
reserve the generated request ID;
optionally append and persist a message;
capture the current session file state as the request baseline.

The active request is reserved before append. This provides the important invariant:

An append-and-start operation that loses admission does not append.

Configuration overrides

Request overrides are merged in this order:

session-owned Session.metadata["overrides"];
explicit overrides supplied to the coordinated request operation.

Explicit request overrides win on conflicts.

The merged mapping is passed to AgentApplication.send_request(...). AgentApplication.resolve_effective_config(...) combines it with the agent base configuration and uses the result for message conversion, lifecycle actions, provider setup, and tool preparation.

Atomic message append

Append-and-start resolves effective config, calls AgentCore.add_message(...), persists the resulting immutable session, publishes message_appended, and records a checkpoint before the request worker begins.

Multipart attachment content follows the same path. HTTP validates and normalizes the multipart payload before calling the application API.

Background and synchronous execution

Both forms call the same internal execution path.

Background start

The coordinator creates one daemon worker thread for the prepared request and returns SessionRequestStart immediately.

The worker owns:

iteration of the low-level request generator;
internal checkpoint filtering;
public event publication;
final result collection;
active-request cleanup;
application-generation operation release.

Unexpected worker exceptions are logged. Application/provider failures that are represented as normal request events still flow through the event store.

Synchronous run

run_session_request(...) performs the same work in the caller's thread. It is useful for terminal compatibility, tests, and other callers that need the final result directly.

If the session already has an active request, the synchronous form raises SessionRequestAlreadyRunningError.

Low-level request lifecycle

AgentApplication.send_request(...) remains the low-level generator. It receives a prepared core, session, base config, merged overrides, request ID, and file-state baseline.

Its lifecycle is:

reconcile the prepared session against persisted state;
run request_prepare lifecycle actions;
resolve effective config and streaming mode;
emit request_started;
prepare tools;
install the live RequestCancellation;
run provider/tool-loop phases;
persist request checkpoints;
run request_complete or request_error lifecycle actions;
expire request-retained notifications;
emit request_completed;
clear application cancellation state.

Tool-loop phases

ToolLoopRunner performs one or more phases:

provider response
    |
    +-- no tool calls --> final response
    |
    +-- tool calls --> execute tools --> checkpoint --> next provider phase

The runner receives callbacks for:

cooperative cancellation;
checkpoint persistence;
reconciling session state before another phase;
refreshing config and prepared tools when the session changes.

Streaming fallback

The application may initially request streaming. If the provider reports that streaming is not implemented before yielding tool-loop events, the application marks that provider class as non-streaming and retries the phase without streaming.

This fallback belongs to the low-level generator, not the coordinator or HTTP adapter.

Persistence and reconciliation

The request starts from a session/file-state baseline captured during preparation.

Before request execution and at subsequent checkpoints, application reconciliation checks that persisted history is still compatible. It rejects conflicting message or protected-metadata changes instead of silently overwriting them.

Compatible non-protected metadata updates can be merged. This is important for application-owned state such as notifications that may change while other work is running.

On a conflict, the application emits an error event, preserves a conflict backup where configured, and completes the request without replacing the conflicting stored history.

Internal checkpoints and public events

ToolLoopRunner and AgentApplication.send_request(...) may yield an internal event containing a live Session object:

{
  "type": "session_checkpoint",
  "session_id": "...",
  "request_id": "...",
  "session": "<live Session object>"
}

SessionRequestCoordinator does not publish this object to the public event store.

Background execution consumes it internally.
Synchronous execution may pass its Session to on_checkpoint.

All other request events are published through AgentApplication.publish_event(...).

Checkpoint persistence records a serializable checkpoint marker in the event store. The per-session cursor points to the latest persistence boundary.

GET /sessions/{session_id}/messages derives frontend active_request_id from the checkpoint at that cursor. This keeps the message snapshot, cursor, and request state aligned.

For event shapes and polling semantics, see the Event API.

Cancellation timing

Cancellation can arrive before or after the live RequestCancellation object is installed.

AgentApplication.cancel_request(request_id) always records the request ID in its pre-cancelled set. If the live object already exists, it is cancelled immediately. Otherwise, send_request(...) creates the live cancellation object in the cancelled state when request preparation reaches that point.

The tool loop checks cancellation:

before provider phases;
while streaming;
before and during tool execution;
while waiting for async or blocking components that register callbacks.

A non-cooperative provider may finish its current operation before cancellation is observed.

Reload and application close

Managed application reload

The request worker owns an ApplicationOperation for its exact application generation. AgentApplicationOwner does not replace the application until all admitted operations release.

During a pending reload:

HTTP middleware rejects new application operations;
direct coordinator starts also fail owner admission;
active requests are allowed to finish.

Direct application close

AgentApplication.close() closes its coordinator before other runtime resources.

Coordinator close:

rejects new starts;
requests cooperative cancellation for active request workers;
prevents later public event publication through the closed coordinator.

Normal owner-managed replacement should already have drained request workers before close.

Adapter responsibilities

FastAPI

FastAPI owns transport concerns:

request-body validation;
multipart normalization;
HTTP idempotency keys;
HTTP status and exception mapping;
JSON serialization;
route declarations.

FastAPI does not own:

active request state;
request IDs;
worker threads;
request event publication;
message append transaction semantics;
application-generation retention.

/send returns the coordinator's started or already_running result. /send_message maps already_running to HTTP 409 and does not append a second message.

HTTP idempotency remains separate from application single-flight admission. Replaying an idempotency key returns the original HTTP response; starting a different request for an already-active session returns the active request result.

Terminal

Ordinary terminal input calls atomic append-and-start. The explicit /append operation remains append-only.

The terminal polls and renders public events. It does not create its own request worker or hold a separate owner operation.

Assistant lifecycle start/end events carry message_index, equal to the LLM phase's existing start_index. During streaming this is the reserved index the assistant message will occupy if the phase commits. Cancellation after partial output can leave that displayed index without a stored message; the terminal does not persist an incomplete assistant solely to make the index expandable.

TerminalApplication.poll_new_events() applies the shared application event compactor before rendering. Redundant assistant/tool stream chunks may be dropped when their aggregate terminal event is present, but tool_partial(phase="end") is preserved because it carries the final display-aware tool result and stored message index. Terminal renderers do not reconstruct a lost end event from tool_results.

TerminalApplication.send_request(...) remains as a synchronous compatibility API and delegates to run_session_request(...).

Application plugins and background work

Application plugins should use the coordinated APIs rather than invoking the low-level generator directly.

For example, background compaction can start a follow-up request without appending a message:

started = app.start_session_request(session_id)

This preserves ordinary admission, reload ownership, event publication, cancellation, and frontend request visibility.

Failure semantics and invariants

The application-layer request path maintains these invariants:

at most one coordinated request is active for a session;
a failed duplicate append-and-start does not append;
every admitted request clears admission in a finally path;
every owner-operation handle is released;
internal live Session objects are not published as public events;
HTTP-specific exceptions and idempotency do not enter agent_app;
request IDs are stable across start responses, events, cancellation, and cleanup;
stale application generations cannot start new work;
session conflicts are reported instead of silently overwriting history.

Source and test map

Primary implementation:

core/python/agent_app/request_execution.py
core/python/agent_app/application_future.py
core/python/agent_app/application_owner.py
core/python/agent_app/tool_loop.py
application/python/agent_terminal_app/server.py
application/python/agent_terminal_app/terminal_app.py
application/python/agent_terminal_app/terminal.py

Primary tests:

application/python/tests/test_session_request_coordinator.py
application/python/tests/test_agent_terminal_app_server.py
application/python/tests/test_terminal_application.py
application/python/tests/test_application_reload_owner.py
core/python/tests/test_application_layer_streaming.py
core/python/tests/test_application_tool_loop_session_concurrency.py

When changing request ownership, run at minimum:

pytest core/python/tests -q
pytest application/python/tests -q
npm --workspace @crystal-lattice/frontend-sdk test

For provider, mobile, and desktop follow-up expectations, see the repository development workflow in the root AGENTS.md.

Requests also participate in the shared application-operation admission and lookup model. See Application operations for cross-producer session exclusivity, minimal active registration, managed-process integration, plugin-owned status, and future operation-menu extensions.