Agents and sessions¶

An agent is a component that carries out a task by talking to a model. Like every other ankka component, its handler returns a description rather than doing the work: which instructions to send, which message, which tools the model may call, which guardrails apply, and what shape the answer takes. The platform runs the conversation that description asks for. It calls the model, dispatches tool calls, records the conversation and counts tokens.

This page explains how that works and why it is built this way. Agents shows how to write one.

ankka has a second kind of agent. An autonomous agent is handed a task rather than a request, works it over as many iterations as it needs with nobody waiting, and ends it with a typed result; this page is about the request agent, which answers while its caller waits.

An agent is addressed by session¶

An agent instance is identified by a session id, not by an entity id. A session is one conversation. Every request to an agent names the session it belongs to, and the platform shards agents by that id across the service's instances, exactly as it shards entities by theirs.

Requests to one session are handled strictly one at a time. A second request that arrives while the first is still talking to the model waits in a queue until the first has finished. Without that rule, two overlapping requests would read the same history, both append to it, and neither turn would acknowledge the other. With it, a conversation is a sequence, however many callers it has.

Different sessions run concurrently, on any instance. Fanning out to several agents at once is safe for that reason: the concurrency is across sessions, and each session's own history is only ever touched by one request.

Session memory is an event sourced entity¶

A session's conversation is stored by an event sourced entity that the platform registers for you, with the component id ankka-session-memory. (AgentRuntime.descriptors also registers the three components autonomous agents use — ankka-task, ankka-agent-instance and the consumer ankka-task-cascade — which cost nothing in a service that has none.) Each user message, model reply and tool result is an event in its journal, and the conversation is the fold of those events. Several properties follow from that, rather than being features built separately:

Durability. A conversation survives a restart, a rolling deployment and a crash, because it is a journal like any other.
Collaboration. Several agents share a conversation by addressing the same session id. Every stored message records which agent wrote it, so an agent can read the whole conversation or only what particular agents said.
Observation. Memory events can be consumed like any entity's events. Compaction is built that way.

What an agent reads from memory and whether it writes to it is part of its effect. It can read the whole history, only the last few messages, only messages from named agents, or nothing at all. A routing agent that decides which specialists to consult typically reads and writes nothing, so its routing chatter never reaches the specialists.

The agent loop¶

The loop is the platform's, not the agent's. For each request it:

runs the input guardrails against the user's message, before any model is called;
assembles the request: instructions, the conversation as memory allows, any extra context the handler attached, and the declared tools;
calls the model;
if the model asked for tools, runs them, sends their results back, and calls the model again;
runs the output guardrails against the final answer, before anything is written to memory;
writes the turn to session memory, with the tokens it used, and replies.

A request may take at most a fixed number of tool round trips, 100 by default, and fails with an internal error if the model is still asking for tools after that. The limit is set per agent.

Tools are declared, not discovered¶

A tool is a name, a description the model reads, a set of typed parameters, and a function. In Scala a tool is built with FunctionTool, and each parameter's type supplies both the JSON Schema the model is shown and the decoder that reads the model's arguments back. They come from one instance on purpose: a parameter cannot be described to the model as an integer and then decoded as something else. In Python a tool's parameters are the fields of an input dataclass, which the SDK turns into the schema.

A tool that fails does not fail the request. Its error goes back to the model as an error result, and the model usually recovers by correcting its arguments.

Guardrails¶

A guardrail inspects text and either passes it or rejects the interaction with a reason. Input guardrails run before the model is called, so a rejection costs nothing. Output guardrails run before memory is written, so a rejected answer leaves no trace in the conversation. Guardrails run in the order they are declared and stop at the first rejection, so put the cheap ones first.

On a streaming handler, output guardrails see the answer only after its tokens have been delivered. They can still keep it out of memory, but they cannot take back what the reader saw. Anything that must never be shown belongs in an input guardrail.

Structured answers¶

A handler can ask for the answer as text, as a stream of tokens, or decoded into a type. A typed answer lets a workflow act on what a model decided, for example a list of specialists to consult, and treat a reply that does not decode as a failure rather than as prose.

Model providers¶

Everything above sits on a ModelProvider, which completes a request and optionally streams one; a provider that cannot stream falls back to one completion. ankka speaks two wire protocols, Anthropic's and OpenAI's, and every provider is one of them pointed at an address: AnthropicProvider and OpenAiProvider ship with ankka and use each vendor's official Java SDK for transport only, and the same two speak to any server that serves either protocol — Mistral, Groq, Gemini, a router, or a model on the developer's own machine — under a name the service chooses. A preset names an endpoint ankka has tested. A session is ankka's, so an agent on one provider continues what an agent on another began; see Model providers. TestModelProvider answers from a script and fails loudly when the script runs out, so a test whose model quietly returned a default cannot pass by accident.

Every model call reports the tokens it used: input, output, and cache reads and writes where the provider reports them. The platform adds them to the session and shows them in the local console. It reports tokens, not money, because the provider reports tokens; there is no price table, and cost is shown as unknown rather than as zero.

Judgments¶

A text model is not the only thing a service can ask. A judgment is the answer to a set of typed questions about a state — a choice among described options, a score on a scale of described levels, a yes or no — from a System One model, which writes no text: every question is answered at once, each restricted to the shape it allows and carrying the probabilities behind it. It is the tool for a decision with a bounded set of answers that code then acts on, and it answers in a fraction of a second at a small fraction of a text model's cost.

It sits on its own seam, JudgmentProvider, beside ModelProvider, because it fits none of the shapes around it. It is not a component: a judgment has no identity, state or lifecycle. It is not another model behind the text seam, which takes a conversation and tools and returns prose and tool calls; a System One model takes and returns neither. And it is not a kind of agent: it keeps no conversation and calls no tools. An agent's handler asks for one as an effect and replies with it, and a judged guardrail asks questions about the text going into or out of a model instead of matching a pattern. A judgment reads no session history and writes no message, but the tokens it spends are recorded against the session it was asked in, as a figure of their own: a sum over two models priced a hundred times apart would mean nothing. See Judgments.

Compaction¶

A long conversation eventually outgrows any model's context window. Compaction replaces the oldest part of a session with a summary, keeping the most recent messages verbatim, because those are what the model needs in full.

It runs as a consumer of session memory's own events, not inside the agent's request. The request that pushes a session over the limit therefore never waits for a summarisation call. The limit is a character count, not a token count, and should be set well below the model's context window. Compacting a session twice folds the earlier summary into the new one rather than stacking summaries.

The summariser calls the model provider directly rather than through an agent. An agent would need a session, and the only sensible session is the one being summarised, so it would append its own turns to the history it is trying to shorten. Compaction is best effort: a summarisation that fails is skipped and the consumer moves on, because a consumer that kept retrying one session would stall compaction for every session.

Agents in Python¶

A Python agent declares the same things: instructions, tools, guardrails and the answer's shape. Its handler returns the plan as data. The loop does not run in the Python process. It runs in the sidecar, which calls the model, keeps and compacts the session and counts tokens. The Python process is asked for three things only: to plan a request, to run a tool with the model's arguments, and to check a guardrail.

Two consequences matter when deploying. The model's API key is configured on the sidecar, and the Python process never holds it or calls a model. A session is the sidecar's entity, so it survives the Python process restarting.