Runtime configuration¶
An ankka service needs no configuration to run on a laptop: with no environment variables it connects to
Postgres on localhost:5432 as ankka/ankka, serves HTTP on port 9000, and forms a one-node cluster by
joining itself. Deployed on the platform, it needs no configuration either, because the platform sets
every variable that says where it is running. This page lists what can be changed and what the platform
controls.
How configuration is layered¶
The runtime reads HOCON configuration, stacked from four layers. A key is taken from the highest layer that sets it:
- JVM system properties, such as
-Dankka.ask-timeout=20s. - The service's own
application.conf. - The cluster overlay for where the process runs, chosen by
ANKKA_CLUSTER_MODE. - The runtime's
reference.conf, which holds every default.
The cluster overlay decides only how a node finds its peers. ANKKA_CLUSTER_MODE selects it:
| Mode | Set by | How nodes form a cluster |
|---|---|---|
local, the default |
nobody | Join the nodes named in ANKKA_CLUSTER_SEED_NODES, or join yourself; bind loopback on a random port. |
kubernetes |
the platform | Discover peers through the Kubernetes API; bind the pod's address on fixed ports. |
Any other value stops the service at startup with a message naming the known modes. Because the service's
application.conf sits above the overlay, it can override any choice the platform's overlay made.
Environment variables reach the configuration through ${?VARIABLE} substitutions, so an environment
variable overrides a default only where the table lists one. In the kubernetes overlay the substitutions
are required rather than optional: a missing POD_IP is a startup failure naming it, not a node that
quietly binds loopback.
Settings¶
The table is generated from the runtime's configuration files.
| Variable | Configuration key | Default | Applies in |
|---|---|---|---|
ANKKA_SERVICE_NAME |
ankka.service.name |
"" |
every service |
ANKKA_INSTALLATION |
ankka.replication.installation |
"" |
every service |
ANKKA_INSTALLATIONS |
ankka.replication.named |
"" |
every service |
ANKKA_VIEW_WATCH_BOUND |
ankka.view.watch-bound |
1000 |
every service |
ANKKA_VIEW_UNREAD_BOUND |
ankka.view.unread-bound |
256 |
every service |
ANKKA_VIEW_FETCH_SIZE |
ankka.view.fetch-size |
256 |
every service |
ANKKA_SECRET_KEY |
ankka.secrets.key |
"" |
every service |
ANKKA_SECRET_BACKEND |
ankka.secrets.backend |
"" |
every service |
ANKKA_SECRET_RECORDS_URL |
ankka.secrets.records-url |
"" |
every service |
ANKKA_SECRET_MOVE |
ankka.secrets.move |
"" |
every service |
ANKKA_SECRET_VERSIONS_KEPT |
ankka.secrets.versions-kept |
2 |
every service |
ANKKA_CLOUD_ACCOUNT |
ankka.cloud.account |
"" |
every service |
ANKKA_CLOUD_LOCATION |
ankka.cloud.location |
"" |
every service |
ANKKA_DATABASE |
ankka.database |
"" |
every service |
ANKKA_DB_LINE |
ankka.history-line |
"" |
every service |
ANKKA_SERVICE_CLIENT_TIMEOUT |
ankka.service-client.timeout |
30s |
every service |
ANKKA_KEYRING_ADDRESS |
ankka.keyring.address |
"" |
every service |
ANKKA_STORAGE_BACKEND |
ankka.storage.backend |
"" |
every service |
ANKKA_STORAGE_DIR |
ankka.storage.directory |
".ankka/storage" |
every service |
ANKKA_OBSERVABILITY_ADDRESS |
ankka.observability.local-address |
"" |
every service |
ANKKA_DB_HOST |
pekko.persistence.r2dbc.connection-factory.host |
"localhost" |
every service |
ANKKA_DB_PORT |
pekko.persistence.r2dbc.connection-factory.port |
5432 |
every service |
ANKKA_DB_NAME |
pekko.persistence.r2dbc.connection-factory.database |
"ankka" |
every service |
ANKKA_DB_USER |
pekko.persistence.r2dbc.connection-factory.user |
"ankka" |
every service |
ANKKA_DB_PASSWORD |
pekko.persistence.r2dbc.connection-factory.password |
"ankka" |
every service |
ANKKA_DB_SSL_MODE |
pekko.persistence.r2dbc.connection-factory.ssl.mode |
"" |
every service |
ANKKA_DB_SSL_ROOT_CERT |
pekko.persistence.r2dbc.connection-factory.ssl.root-cert |
"" |
every service |
ANKKA_DB_SSL_CERT |
pekko.persistence.r2dbc.connection-factory.ssl.cert |
"" |
every service |
ANKKA_DB_SSL_KEY |
pekko.persistence.r2dbc.connection-factory.ssl.key |
"" |
every service |
ANKKA_HTTP_INTERFACE |
ankka.http.interface |
"0.0.0.0" |
every service |
ANKKA_HTTP_PORT |
ankka.http.port |
9000 |
every service |
ANKKA_HTTP_MAX_STREAM_BYTES |
ankka.http.max-stream-bytes |
5g |
every service |
ANKKA_SOCKET_MAX_FRAME_SIZE |
ankka.http.socket.max-frame-size |
64KiB |
every service |
ANKKA_SOCKET_UNREAD_FRAMES |
ankka.http.socket.unread-frames |
64 |
every service |
ANKKA_SOCKET_KEEP_ALIVE |
ankka.http.socket.keep-alive |
20s |
every service |
ANKKA_SSE_HEARTBEAT |
ankka.http.sse.heartbeat |
15s |
every service |
ANKKA_PROJECT_GRANTS |
ankka.grants.file |
"" |
every service |
ANKKA_PROJECT_DECLARATIONS |
ankka.grants.declarations |
"" |
every service |
ANKKA_MACHINE_ISSUER |
ankka.machines.issuer |
"" |
every service |
ANKKA_MACHINE_JWKS_URL |
ankka.machines.jwks-url |
"" |
every service |
ANKKA_MACHINE_JWKS_CA |
ankka.machines.jwks-ca |
"" |
every service |
ANKKA_GRPC_INTERFACE |
ankka.grpc.interface |
"0.0.0.0" |
a service that serves gRPC |
ANKKA_GRPC_PORT |
ankka.grpc.port |
9090 |
a service that serves gRPC |
ANKKA_OTLP_ENDPOINT |
ankka.telemetry.endpoint |
"" |
a service that exports telemetry |
ANKKA_OTLP_HEADERS |
ankka.telemetry.headers |
"" |
a service that exports telemetry |
ANKKA_MCP_CONNECT_TIMEOUT |
ankka.agent.mcp.connect-timeout |
10s |
a service with agents |
ANKKA_MCP_CALL_TIMEOUT |
ankka.agent.mcp.call-timeout |
60s |
a service with agents |
ANKKA_CLUSTER_SEED_NODES |
ankka.cluster.seed-nodes |
"" |
local mode |
ANKKA_CLUSTER_PORT |
pekko.remote.artery.canonical.port |
0 |
local mode |
POD_IP |
pekko.remote.artery.canonical.hostname |
required, set by the platform | kubernetes mode |
ANKKA_SECRET_RECORDS_URL |
ankka.secrets.records-url |
"https://ankka-controlplane.ankka-controlplane.svc:9000" |
kubernetes mode |
POD_IP |
pekko.management.http.hostname |
required, set by the platform | kubernetes mode |
ANKKA_CLUSTER_SERVICE |
pekko.management.cluster.bootstrap.contact-point-discovery.service-name |
required, set by the platform | kubernetes mode |
ANKKA_CLUSTER_CONTACT_POINTS |
pekko.management.cluster.bootstrap.contact-point-discovery.required-contact-point-nr |
required, set by the platform | kubernetes mode |
ANKKA_CLUSTER_POD_SELECTOR |
pekko.discovery.kubernetes-api.pod-label-selector |
required, set by the platform | kubernetes mode |
Settings with no environment variable, overridable in the service's own application.conf:
| Configuration key | Default | Applies in |
|---|---|---|
ankka.ask-timeout |
10s |
every service |
ankka.query-resend-after |
2s |
every service |
ankka.replication.poll |
1s |
every service |
ankka.replication.handover.hold-writes |
off |
every service |
ankka.replication.handover.timeout |
30s |
every service |
ankka.view.listener-backoff.min |
1s |
every service |
ankka.view.listener-backoff.max |
30s |
every service |
ankka.secrets.timeout |
10s |
every service |
ankka.secrets.record-timeout |
5s |
every service |
ankka.secrets.secret-manager.endpoint |
"https://secretmanager.googleapis.com" |
every service |
ankka.secrets.secret-manager.token |
"" |
every service |
ankka.secrets.secret-manager.identity |
"" |
every service |
ankka.keyring.cache.max-keys |
10000 |
every service |
ankka.keyring.cache.ttl |
5m |
every service |
ankka.keyring.cache.outage-bound |
60s |
every service |
ankka.keyring.cache.foreign-ttl |
30s |
every service |
ankka.keyring.notice-bound |
60s |
every service |
ankka.keyring.fetch-timeout |
2s |
every service |
ankka.storage.descriptor-directory |
"" |
every service |
ankka.storage.timeout |
30s |
every service |
ankka.storage.max-signed-url |
7d |
every service |
ankka.storage.default-signed-url |
15m |
every service |
ankka.tls.cluster-directory |
"" |
every service |
ankka.tls.service-directory |
"" |
every service |
ankka.tls.reload-interval |
1m |
every service |
ankka.http.tls.enabled |
off |
every service |
ankka.probe.enabled |
off |
every service |
ankka.probe.port |
7627 |
every service |
ankka.observability.ring-capacity |
4096 |
every service |
ankka.observability.max-counted-handlers |
1024 |
every service |
ankka.observability.call-window |
10m |
every service |
ankka.observability.call-buckets |
60 |
every service |
ankka.observability.observe.enabled |
off |
every service |
ankka.observability.observe.port |
7628 |
every service |
ankka.observability.observe.peer |
"ankka://platform/controlplane" |
every service |
ankka.observability.max-external-services |
32 |
every service |
ankka.observability.max-external-methods |
256 |
every service |
ankka.http.body-timeout |
10s |
every service |
ankka.grants.reload-interval |
10s |
every service |
ankka.grpc.max-message-size |
4MiB |
a service that serves gRPC |
ankka.grpc.max-connection-age |
2m |
a service that serves gRPC |
ankka.grpc.shutdown-grace |
5s |
a service that serves gRPC |
ankka.grpc.keepalive-time |
30s |
a service that serves gRPC |
ankka.grpc.keepalive-timeout |
10s |
a service that serves gRPC |
ankka.telemetry.interval |
1s |
a service that exports telemetry |
ankka.telemetry.batch-size |
512 |
a service that exports telemetry |
ankka.telemetry.export-timeout |
5s |
a service that exports telemetry |
ankka.telemetry.max-backoff |
30s |
a service that exports telemetry |
ankka.telemetry.metric-interval |
10s |
a service that exports telemetry |
ankka.telemetry.shutdown-timeout |
3s |
a service that exports telemetry |
ankka.telemetry.service-name |
"" |
a service that exports telemetry |
ankka.telemetry.project |
"" |
a service that exports telemetry |
ankka.mcp.client-keys.last-use-interval |
1 minute |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.client-keys.query |
${pekko.persistence.r2dbc.query} |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.client-keys.query.refresh-interval |
500ms |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.sign-in.token-lifetime |
1 hour |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.sign-in.refresh-lifetime |
30 days |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.sign-in.grant-lifetime |
10 minutes |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.sign-in.client-lifetime |
30 days |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.sign-in.rotate-every |
30 days |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.mcp.sign-in.keep-for |
31 days |
a service with an MCP endpoint that admits client keys or signs people in |
ankka.cluster.formation |
join-self-or-seeds |
local mode |
ankka.join-self-if-no-seed-nodes |
on |
local mode |
ankka.cluster.formation |
bootstrap |
kubernetes mode |
ankka.join-self-if-no-seed-nodes |
off |
kubernetes mode |
ankka.cluster.join-within |
120s |
kubernetes mode |
ankka.tls.cluster-directory |
"/var/run/secrets/ankka/cluster" |
kubernetes mode |
ankka.tls.service-directory |
"/var/run/secrets/ankka/service" |
kubernetes mode |
ankka.http.tls.enabled |
on |
kubernetes mode |
ankka.probe.enabled |
on |
kubernetes mode |
ankka.observability.observe.enabled |
on |
kubernetes mode |
What each variable means¶
HTTP¶
ANKKA_HTTP_INTERFACEis the address the HTTP server binds,0.0.0.0by default.ANKKA_HTTP_PORTis the port the HTTP server binds,9000by default. On the platform it is set from the descriptor'sport, and a descriptor may not set it directly. Set it locally to run a second service beside the first.ANKKA_SOCKET_MAX_FRAME_SIZEis the largest frame a socket's client may send, as the UTF-8 bytes of the whole message,64KiBby default. A larger one closes the socket1009, too large, without being read. Behind a sidecar it may be at most3MiB, since each frame crosses to the process as one gRPC message.ANKKA_SOCKET_UNREAD_FRAMESis how many frames may wait for a socket's handler that has not received them,64by default. One more closes the socket1008, unread: a handler that only sends must still read.ANKKA_SOCKET_KEEP_ALIVEis how long a socket may be quiet before the platform pings it,20sby default. It must be shorter than the server's idle timeout,pekko.http.server.idle-timeout(60s), which would otherwise cut a quiet socket off; a service whose keep-alive is not shorter does not start. A process-hosted service's sidecar holds its sockets, so all three are given to the sidecar and never to the process.ANKKA_SSE_HEARTBEATis how long a stream of server-sent events may be quiet before the service sends a heartbeat, an event with no data that a browser does not dispatch,15sby default. When it is not shorter thanpekko.http.server.idle-timeout, the heartbeat is sent at half the idle timeout instead, so a quiet stream — a watch of a view that writes nothing for a minute, say — is never ended by it. A wait for a workflow's end served as a stream sends itsheartbeatevents at the same interval, from the service and from the sidecar beside a process alike.ANKKA_HTTP_MAX_STREAM_BYTES(ankka.http.max-stream-bytes) is the largest body a stream route (postStream,putStream) reads,5gby default. A request declaring a larger one is answered413and the handler never runs. A route that reads its body whole keepspekko.http.server.parsing.max-content-length. See HTTP endpoints.
gRPC¶
These apply to a service that registers a GrpcServer; see gRPC endpoints.
ANKKA_GRPC_INTERFACEis the address the gRPC server binds,0.0.0.0by default.ANKKA_GRPC_PORTis the port the gRPC server binds,9090by default. On the platform it is set from the descriptor'sgrpcPortwhen the descriptor says"grpc": true, and a descriptor may not set it directly. A service started with it set and noGrpcServerregistered refuses to start, saying why.
Database¶
The runtime keeps its journal, snapshots, durable state, view rows, projection offsets and timers in one
Postgres database. On the platform these five are set from the database provisioned for the service, and a
descriptor that sets any ANKKA_DB_* variable brings its own database instead.
ANKKA_DB_HOSTis the Postgres host,localhostby default.ANKKA_DB_PORTis the Postgres port,5432by default.ANKKA_DB_NAMEis the database,ankkaby default.ANKKA_DB_USERis the user,ankkaby default.ANKKA_DB_PASSWORDis the password,ankkaby default. A provisioned database has none: its role logs in by certificate.ANKKA_DB_SSL_MODEturns on TLS to the database:require,verify-caorverify-full. Empty, the default, the connection is plain. The platform setsverify-fullfor a provisioned database.ANKKA_DB_SSL_ROOT_CERTis the file of the authority the server's certificate is verified against.-
ANKKA_DB_SSL_CERTandANKKA_DB_SSL_KEYare a client certificate and its PKCS#8 key to log in with instead of a password. The key is read when a connection is opened, so a renewed certificate reaches the next connection without a restart. -
ANKKA_DATABASEisnonefor a service that has no database at all: one made of consumers, endpoints and agents. The platform sets it for a descriptor that says"database": "none". With it the runtime opens no connection — the secret store is unavailable and the timer scheduler refuses every timer — and an entity, a view, a workflow or a timed action registered in the service refuses the start, naming itself. Empty, the default, the service has a database, the platform's or its own. -
ANKKA_DB_LINE(ankka.history-line) names the line of history the service's database is on: the database cluster it was first started on. The platform sets it on a service switched to a restore; on the project database it is the clusterANKKA_DB_HOSTnames. At start the service records when it first ran on that line, and every message published from a journal event carriesce-id<line>/<persistence id>/<sequence>, so an event published again after a restore keeps its id. Empty in a local run, nothing is recorded and messages keep random ids.
A view's query can be answered as a stream of rows, and a declared query watched (Views):
ANKKA_VIEW_WATCH_BOUNDis how many watches one instance holds open, every view together,1000by default. One more is refused, naming the bound.ANKKA_VIEW_UNREAD_BOUNDis how many rows a watch holds for a watcher that has not read them, one per row key,256by default, unless the watcher gives its own. Past it, the watcher's overflow strategy decides.ANKKA_VIEW_FETCH_SIZEis how many rows the database yields per fetch of a stream,256by default: what a stream holds in hand however many rows it gives.
Each reaches the runtime that holds the watches — beside a Python or TypeScript process, the sidecar, never the process.
Never point two services at one database. Timers, view tables and projection offsets are not separated by service, so two services sharing a database delete each other's timers and overwrite each other's views.
Calling other services¶
ANKKA_SERVICE_CLIENT_TIMEOUT(ankka.service-client.timeout) is how long a call to another service waits for its answer,30sunless set; connecting has five seconds of its own and there is no per-call timeout. A descriptor may give it, and it goes to the runtime that makes the call — beside a Python or TypeScript process, the sidecar, never the process. See Calling other services.
Secret store¶
-
ANKKA_SECRET_KEY(ankka.secrets.key) is the key the service's secret store encrypts the values it keeps with: the standard base64 of exactly 32 bytes, asopenssl rand -base64 32writes it. On the platform the operator makes one per service unless the descriptor sets it, and gives it only to the platform's own program, never to a process or a module. Empty, the default, the service starts and keeping or reading a secret fails naming the variable. Set to anything that is not 32 bytes of base64, the service does not start. See Secrets a service keeps. -
ANKKA_SECRET_BACKEND(ankka.secrets.backend) is where service secrets are kept:postgres, the default, for the table in the service's own database;secret-managerfor Google Secret Manager. The installation's setting, given by the operator; a descriptor cannot set it. ankka.secrets.timeoutis how long one keep, read or removal waits for the backend before it fails as unavailable,10sunless set.ankka.secrets.record-timeoutis how long the record of a read waits to be acknowledged before the read fails as unavailable,5sunless set: no value is used without its record.ANKKA_SECRET_RECORDS_URL(ankka.secrets.records-url) is where the record of each read is written. A deployed service is given the control plane's address; empty, on your machine, the record is a line in the log.ANKKA_SECRET_MOVE(ankka.secrets.move) is the phase of a move to Secret Manager the instance performs when it starts:copy,checkorremove. Empty, the default, no move.ANKKA_SECRET_VERSIONS_KEPT(ankka.secrets.versions-kept) is how many versions of a service secret Secret Manager keeps,2unless set.ankka.secrets.secret-manager.endpointis Secret Manager's address, Google's unless a test runs a stand-in;ankka.secrets.secret-manager.tokenis a fixed access token for that stand-in, and empty, as deployed, means the pod's own token from the metadata server;ankka.secrets.secret-manager.identitysays whose secrets these are,<project>/<service>, when the service's certificate cannot.ANKKA_CLOUD_PROVIDER(ankka.cloud.provider),ANKKA_CLOUD_ACCOUNT(ankka.cloud.account) andANKKA_CLOUD_LOCATION(ankka.cloud.location) are the installation's cloud:noneorgcp, the Google Cloud project the secrets are kept in, and the region they are kept in, automatic when empty.
Storage client¶
ANKKA_STORAGE_BACKEND(ankka.storage.backend) is what stands behind every bucket the storage client reaches:s3,directoryormemory. Empty, the default, choosess3when the service is given a store's variables (ANKKA_S3_ENDPOINT, or a supplied bucket's),directorywhen it runs on a developer's machine, and otherwise no store, so every call is refused namingprovisionObjectStorage.memoryis what the test kits use.ANKKA_STORAGE_DIR(ankka.storage.directory) is where the directory store keeps its objects, one directory per bucket,.ankka/storageunder the working directory by default.ankka.storage.descriptor-directoryis where the service's ownservice.jsonis read from for the buckets it declares, on a developer's machine and in a test; empty, the default, is the working directory.ankka.storage.timeoutis how long one request to the store waits for its answer before failing asUnavailable,30sby default.ankka.storage.max-signed-urlis the longest span a signed URL may be made for,7d, andankka.storage.default-signed-urlthe span one is made for when the caller gives none,15m.
The buckets themselves come from the ANKKA_S3_* variables the platform gives a service, described in
Object storage on the platform. See
Objects a service keeps.
Keyring client¶
How a service holds the keys of the data subjects whose personal fields it reads and writes.
ANKKA_KEYRING_ADDRESS(ankka.keyring.address) is where the installation's keyring answers. Only the platform sets it, and a descriptor that gives it is refused. Unset, a service run on a developer's machine with a database keeps its keys in a local keyring in its own database, and a service in a cluster reaches no keyring: every read and write of a personal field is refused as unavailable.ankka.keyring.cache.max-keysis how many subject keys one instance holds at once,10000; the least recently used goes first.ankka.keyring.cache.ttlis how long a key is held before it is asked for again,5m.ankka.keyring.cache.outage-boundis how long a held key still serves while the keyring cannot be reached, counted from when the keyring was last heard,60s; past it, a read or write that needs a key the instance must ask for fails asUnavailable. It may not exceedankka.keyring.notice-bound; a service whose settings say otherwise does not start.ankka.keyring.notice-boundis how long the instance's channel to the keyring may go unconfirmed before the instance drops every key it holds,60s: the bound on how long an erased data subject can still be read by a new fetch.ankka.keyring.fetch-timeoutis how long one key fetch waits before it fails asUnavailable,2s.
Replication¶
These configure replicated entities. A service that names no installations replicates nothing, and its replicated entities behave as any event sourced entity.
ANKKA_INSTALLATION(ankka.replication.installation) names the installation this service runs at, a DNS label of at most 63 characters. It is the platform's to name, so a descriptor that sets it is refused; locally, set it in the environment of each process.ankka.replication.installationslists the other installations the service may replicate with, by name. For the runtime's own replication transport each names its journal's database asdb { host, port, name, user, password }, which reads that installation's records directly. The variablesANKKA_INSTALLATION_<NAME>_DB_HOST,_DB_PORT,_DB_NAME,_DB_USERand_DB_PASSWORDadd an installation and its database from the environment, the name lower-cased with_read as-. They reach the runtime alone, never a process beside it. A replication transport supplied on the classpath, such asankka-replication, needs only the names. An installation set tonullin a configuration file above another is not named, which is how a later file takes one out.ANKKA_INSTALLATIONS(ankka.replication.named) names installations by name alone, comma-separated (leipzig,bratislava), for a transport supplied on the classpath, which needs no database. Each name is lower-cased and must be a DNS label of at most 63 characters, or the service refuses to start naming it. Each is added toankka.replication.installationswith no database; a name that also has a database, in the configuration or fromANKKA_INSTALLATION_<NAME>_DB_*, is one installation with that database. It is the platform's to set, so a descriptor that sets it is refused. With names and neither a database for each nor a transport on the classpath, the service refuses to start, naming the installation and both ways of reaching it.ankka.replication.poll,1s, is how often the runtime's own transport looks for records a follower has not yet copied. A read at a follower may be this far behind its home.ankka.replication.handover.hold-writes,offby default, decides what the old home does while a handover waits for the new home to hold everything. Off, it goes on handling writes and completes the handover at the first moment the new home has caught up. On, it holds writes while it waits, and abandons the handover afterankka.replication.handover.timeout,30s.
ANKKA_INSTALLATION=leipzig \
ANKKA_INSTALLATION_VIENNA_DB_HOST=localhost ANKKA_INSTALLATION_VIENNA_DB_NAME=vienna \
ANKKA_INSTALLATION_VIENNA_DB_USER=ankka ANKKA_INSTALLATION_VIENNA_DB_PASSWORD=ankka sbt run
Local clusters¶
ANKKA_CLUSTER_PORTfixes the cluster's remoting port inlocalmode, which is random by default so that several services can share a machine. Fix it on the node that others will join.ANKKA_CLUSTER_SEED_NODESis a comma-separated list of node addresses to join inlocalmode, such aspekko://ankka@127.0.0.1:17355. Empty, the node joins itself.
ANKKA_CLUSTER_PORT=17355 sbt run
ANKKA_CLUSTER_SEED_NODES=pekko://ankka@127.0.0.1:17355 ANKKA_HTTP_PORT=9001 sbt run
Set by the platform¶
These are set by the platform on every deployed instance, and a descriptor that sets one is refused.
ANKKA_CLUSTER_MODEselects the cluster overlay; the platform setskubernetes.POD_IPis the pod's address, which the node binds and advertises to its peers.ANKKA_CLUSTER_SERVICEis the Kubernetes Service through which peers are discovered.ANKKA_CLUSTER_POD_SELECTORis the label selector that identifies this service's pods, so a node never mistakes another service's pods for its own.ANKKA_CLUSTER_CONTACT_POINTSis how many peers must be found before a new cluster forms: the smaller of the instance count and two.ANKKA_NAMESPACE_PREFIXis how a project id becomes a namespace,ankka-<project>, which the service client uses to address another service by name.
The installation's cloud is named for the platform's own processes and set on no instance:
ANKKA_CLOUD_PROVIDER, ANKKA_CLOUD_ACCOUNT, ANKKA_CLOUD_LOCATION, ANKKA_CLOUD_KMS_KEY,
ANKKA_CLOUD_ACKNOWLEDGEMENT_BOUND and ANKKA_CLOUD_ROTATION_GRACE are read by the operator, the
control plane and the cloud provider, and a descriptor that sets one is refused. See
The cloud provider.
In kubernetes mode the remoting port is fixed at 17355, the management port at 7626 and the readiness
port at 7627, and every one of them but readiness is mutual TLS with certificates the platform mounts under
/var/run/secrets/ankka/. See Runtime endpoints and
Networking and TLS.
Agents and models¶
ANTHROPIC_API_KEYis the key for Anthropic's API. A Scala service reads it when it constructs its model provider. In a process-hosted service it belongs to the sidecar, which runs the agent loop; the platform routes it there, and the sidecar configures the provideranthropicfrom it.OPENAI_API_KEYis the key for OpenAI's API, read the same way; the sidecar configuresopenaifrom it.ANKKA_MODEL_NAMEnames the Anthropic modelanthropicuses. It is read by the sidecar.ANKKA_MODEL_<NAME>_PROTOCOL,_URL,_MODEL,_KEY,_SHAPEand_PRESETconfigure a model provider named<name>on the sidecar: either protocol at any address, by hand or from a preset.ANKKA_MODEL_DEFAULTnames the provider that answers an agent naming none, when the sidecar has more than one. Model providers lists every setting, preset and refusal.ANKKA_MODEL_SCRIPTgives the sidecar a scripted model instead of a real one, for tests: a JSON array of turns, or the path of a file holding one. Each turn is{"text": "..."}, a tool call{"tool": "name", "arguments": {...}}, several tool calls{"tools": [...]}, or{"refusal": "..."}, consumed in order;{"when": "<part of the user's message>", "text": "..."}is a standing rule used once the turns run out. With both a key and a script set, the key wins.
Every variable beginning ANTHROPIC_, OPENAI_ or ANKKA_MODEL_ in a process-hosted service's
descriptor goes to the sidecar, never to the process.
ANKKA_MODELS_ADDRESSis where the installation's model server answers its services, set by the platform on the platform's container of every instance when the installation has a model server, and on nothing when it has none. A descriptor that sets it is refused. The operator and the control plane read the same variable from the model server's component, and the operator also readsANKKA_MODELS_STATUS_URL, the model server's status document, andANKKA_MODELS_NAMESPACE, where it runs. The three are set together or not at all: the operator refuses to start with some and not others. See Models served by the installation.
The model server's keeper¶
The keeper beside Ollama in the models component reads its own variables, set by the component:
ANKKA_KEEPER_ADDRESSis the address services call, required; it is what the status document reports.ANKKA_KEEPER_OLLAMAis where Ollama answers,http://127.0.0.1:11434unless set. Ollama is bound to the pod's loopback, so the keeper is the only way in.ANKKA_KEEPER_DECLARATIONis the declaration file,/etc/ankka/models/models.jsonunless set: themodels-declarationConfigMap'smodels.json, mounted.ANKKA_KEEPER_PORTis the mutual TLS port services call, 8443 unless set;ANKKA_KEEPER_STATUS_PORTserves the status document without TLS to the operator, 8080 unless set; andANKKA_KEEPER_PROBE_PORTanswers the readiness probe, 7627 unless set.ANKKA_KEEPER_PASS_INTERVALis how often the keeper compares what is declared with what is held,30sunless set. An edit to the declaration is also noticed when the mounted file changes.ANKKA_KEEPER_PARALLELis how many calls Ollama answers at once, 1 unless set, which the keeper reports; set it to the same number as Ollama's ownOLLAMA_NUM_PARALLEL.-
ANKKA_OTLP_ENDPOINT,ANKKA_OTLP_HEADERSandANKKA_INSTALLATIONexport the keeper's traces and metrics as a service's are; with no endpoint nothing is exported. -
ANKKA_MCP_CONNECT_TIMEOUT(ankka.agent.mcp.connect-timeout) is how long the runtime waits for each MCP server an agent lists to answer when the service starts,10sunless set; a server that does not answer in time fails the start, naming it. ANKKA_MCP_CALL_TIMEOUT(ankka.agent.mcp.call-timeout) is how long a call to an MCP server's tool waits for its answer,60sunless set; a call that takes longer reaches the model as the tool's error.-
ANKKA_MCP_<SERVER>_URLis where the MCP server of that name is, required when its declaration gives no address and replacing one when it does;<SERVER>is the name in upper case with-as_. Any other variable beginningANKKA_MCP_holds the value of a header a declaration names, such as a server's token. Like the model's variables, everyANKKA_MCP_variable goes only to the platform's program — beside a Python or TypeScript process, the sidecar, which connects to the servers — and a WebAssembly module'sconfiganswers it absent. See MCP servers. -
TYPESAFE_API_KEYis the key for TypeSafe AI's API, which answers judgments. A Scala service reads it when it constructsJevProvider.fromEnv(), which fails at startup when it is not set. TYPESAFE_BASE_URLis optional: where that provider's requests go instead ofhttps://api.typesafe.ai, such as a gateway that passes them through unchanged.
Neither is routed to a sidecar: judgments are available to Scala services only.
Broker topics¶
ANKKA_KAFKA_BOOTSTRAP_SERVERSis the Kafka bootstrap address.ProjectionRuntime.fromEnv()connects a Scala service to it, and the sidecar connects a process-hosted service to it. The sidecar needs it only for a view sourced from a topic or a consumer that produces to one, and refuses to start without it when the service has either, naming the variable. A Scala service may pass its broker toProjectionRuntime.withKafkain code instead.ANKKA_KAFKA_TLS_DIRECTORYis where the certificate a service presents to the broker is, astls.key,tls.crtandca.crt. With it the service connects over TLS, presenting that certificate and picking up its renewal on the next connection; without it, in plain text.ANKKA_KAFKA_TOPIC_PREFIXis what the broker's names for the project's topics start with. A component names a topic as its project declared it, and the prefix is added where the topic is handed to the broker.
On an installation with a broker the platform sets all three on every service with components, naming
the installation's broker, the service's own certificate and <project>.; for a process-hosted service it
sets them on the sidecar and on the process. A descriptor that sets any variable beginning
ANKKA_KAFKA_ names a broker of its own instead: the platform sets none of them, and the service connects
as the descriptor says. See The installation's broker.
ANKKA_PROJECT_DECLARATIONSnames the file of the project's declarations — its topics with their partitions, whether they are compacted and their contract, and its declared brokers — which the platform mounts at/var/run/ankka/project/topics.json. The runtime reads it once when the service starts and refuses a component whose stated contract is not the declared one, or that names a broker the project does not declare. Unset, nothing is checked. See Contracts.ANKKA_PROJECT_GRANTSnames the file of the grants the service's project has made, whichCallers.grantedadmits by. Unset, the runtime readsgrants.jsonbeside the declarations file, which is where the platform mounts it, so the platform sets nothing for it. The runtime re-reads the file when it changes, looking every ten seconds: a grant takes effect, and a revoked grant refuses its caller and closes what it admitted, with no restart. With neither variable set no caller holds a grant. See Cross-project access.ANKKA_MACHINE_ISSUERandANKKA_MACHINE_JWKS_URLsay where a registered machine's token comes from and where the keys it is signed with are read;ANKKA_MACHINE_JWKS_CAis the authority that address is trusted by, the service certificate's ownca.crtwhen unset. A request through the gateway carrying a valid token from that issuer is that machine. Unset, the runtime readsmachines.jsonbeside the declarations file, which is where the platform writes them, so the platform sets none of the three; with neither, no token is verified and a bearer leaves the caller as it was.ANKKA_TOPIC_BROKER_<NAME>_BOOTSTRAP_SERVERS,ANKKA_TOPIC_BROKER_<NAME>_SHAPE,ANKKA_TOPIC_BROKER_<NAME>_SECRET_DIRECTORYandANKKA_TOPIC_BROKER_<NAME>_NAMEare a broker the project declares, one set per broker with its name upper-cased and-as_: its address, the shape of its credential (certificateorsasl), the directory the project secret holding the credential is mounted at, and its declared name. The platform sets them on the platform's container of every service in the project, and never on a process. A component names the broker for one topic; see A topic on another broker.ANKKA_TOPIC_DEFAULT_RETENTION,ANKKA_TOPIC_DEFAULT_RETENTION_SIZE,ANKKA_TOPIC_DEFAULT_CLEANUP,ANKKA_TOPIC_DEFAULT_TOMBSTONE_WINDOW,ANKKA_TOPIC_DEFAULT_MIN_COMPACTION_LAG,ANKKA_TOPIC_DEFAULT_MAX_COMPACTION_LAG,ANKKA_TOPIC_DEFAULT_COPIES,ANKKA_TOPIC_DEFAULT_MIN_IN_SYNC,ANKKA_TOPIC_LONGEST_RETENTION,ANKKA_TOPIC_LARGEST_RETENTION_SIZE,ANKKA_TOPIC_MOST_COPIESandANKKA_TOPIC_WARNING_THRESHOLDare the control plane's, not a service's: what a topic's first declaration is given for a setting it leaves out, the bounds every declaration is held to, and the retention below which a view reading a topic is warned. They are read once when the control plane starts and are set on its Deployment; a value it cannot read stops it. Their shipped values and meanings are on The installation's broker.ANKKA_BROKER_NODESis the control plane's too: how many nodes the installation's broker has, whichGET /installationreports asbroker.nodes. The control plane's Deployment sets1, for the one-node broker it ships with, and the three-node broker's configuration sets3. Unset, the installation reports the node count as not stated rather than guessing it.ANKKA_SERVICE_NAME, orankka.service.name, is a service's name when it runs on a developer's machine. Each view or consumer that reads a topic reads under a consumer group named for it, so two services on one broker never share one; with no name stated, a group is named for its component alone. A deployed service's name is read from the certificate the platform issued it, so the key is not read there, and a descriptor that sets the variable is refused. See Broker topics.
Token verification¶
A service that verifies its users' tokens lists the issuers it accepts as a named set, read by
Oidc.authenticate() in a Scala service and by the sidecar for every other language. The set is not a
setting of reference.conf, so it has no row in the table above. For a process-hosted service the
variables go to the sidecar container only; the process never sees them, and a WebAssembly module's
config call answers each as absent.
| Variable | Required | Meaning |
|---|---|---|
ANKKA_AUTH_ISSUERS |
to verify anything | The issuers' names, comma separated. Each is a word of letters, digits and -, starting with a letter, and listed once. |
ANKKA_AUTH_<NAME>_ISSUER |
for each name | The issuer a token must name, exactly. A trailing / is ignored. |
ANKKA_AUTH_<NAME>_JWKS_URL |
for each name | Where the issuer's keys are fetched. |
ANKKA_AUTH_<NAME>_AUDIENCE |
for each name | The audience a token must be for. |
ANKKA_AUTH_<NAME>_CA |
no | A file of PEM certificates the keys fetch trusts, and nothing else. Without it, the JVM's own trust store. |
ANKKA_AUTH_<NAME>_TYP |
no | A typ claim a token must carry, such as Bearer for Keycloak. Without it, the claim is not checked. |
ANKKA_AUTH_<NAME>_CLOCK_SKEW |
no | The tolerance on a token's expiry and start, 60s by default. It accepts ms, s and m suffixes; a bare number is seconds. |
ANKKA_AUTH_REALM |
no | The realm a challenge names, ankka by default. |
<NAME> is the issuer's name upper-cased, with - as _. Variables of names that are not listed are not
read, so the control plane's own ANKKA_AUTH_ISSUER, ANKKA_AUTH_JWKS_URL and ANKKA_AUTH_JWKS_CA,
described on Identity and machine accounts, can sit beside a service's set. A
name listed twice, a name that is not a word, a required variable missing, a CA that is not a file or a
skew that is not a duration stops the service starting, with every problem named at once.
MCP endpoints¶
A service whose MCP endpoint admits client keys or opts into brokered sign-in registers the MCP runtime,
and these settings are read only then. They have no variables; a service overrides them in its
application.conf.
ankka.mcp.client-keys.last-use-intervalis how often a client key's last use is written down,1 minuteby default. A key used on every request is noted in memory and written at most once an interval, so a busy assistant does not write on every call; a key's listing is that far behind at most.ankka.mcp.client-keys.queryis the read journal each instance follows the keys with, the runtime's own with its connection pool, polling every500ms(query.refresh-interval). It is what bounds how soon a key issued or revoked on one instance is known on the others.ankka.mcp.sign-in.token-lifetimeis how long a token the sign-in broker mints is good for,1 hour;refresh-lifetime, its refresh token's,30 days, replaced each time it is used.ankka.mcp.sign-in.grant-lifetimeis how long a pending sign-in and the code it becomes wait to be used,10 minutes;client-lifetime, how long a registered client that is never used is kept,30 days.ankka.mcp.sign-in.rotate-everyandankka.mcp.sign-in.keep-forare how often the broker makes a new signing key,30 days, and how long each is kept to verify what it signed,31 days. One key is always kept.
A brokered endpoint signs people in at the first issuer its ACL lists, as a client of that issuer. Two more variables of the issuer's set say who the broker is there:
| Variable | Required | Meaning |
|---|---|---|
ANKKA_AUTH_<NAME>_CLIENT_ID |
for each issuer of a brokered endpoint's ACL | The client id the broker is registered as at the issuer. Without it, a service with a brokered endpoint does not start. |
ANKKA_AUTH_<NAME>_CLIENT_SECRET |
no | The client's secret, for an issuer that registered the broker as a confidential client. Without it, the broker is a public client and proves itself with PKCE alone. |
Process-hosted services¶
A service in another language runs as a process beside the sidecar, and the two find each other on loopback. The platform sets these variables on the two containers, and a descriptor may not.
ANKKA_PROCESS_PORTis the port the process serves the protocol on,9010by default. The Python SDK reads it when it starts listening.ANKKA_PROCESS_ADDRESSis where the sidecar finds the process,127.0.0.1:9010by default.ANKKA_SIDECAR_PORTis the port the sidecar serves its client API on for the process,9011by default.ANKKA_SIDECAR_ADDRESSis where the process finds the sidecar,127.0.0.1:9011by default. The Python SDK's component client reads it.ANKKA_SIDECAR_BINDis the address the sidecar's client API binds,127.0.0.1by default. It differs only when the sidecar runs in a container and the process on the host, as in local development with Docker Compose.ANKKA_OBSERVABILITY_ADDRESS(ankka.observability.local-address) is thehost:porta service run outside Kubernetes binds its local observability endpoint to, loopback on an ephemeral port by default. The Python, TypeScript and Go integration kits set it on their sidecar container so the test can reach the endpoint's keyring routes through a published port. A descriptor may not set it.ANKKA_SIDECAR_DISCOVERY_TIMEOUTis how long the sidecar waits for the process to answer the discovery handshake before giving up,60sby default. It acceptsms,sandmsuffixes; a bare number is seconds.
ANKKA_LOCAL_CALLER_TOKEN is the secret a test presents to name a caller outside a cluster. The Python
integration testkit generates one, passes it to the sidecar it starts, and puts it on a request through
kit.caller_headers(caller). Nothing sets it in a cluster, where the caller comes from a certificate and
the header is not read.
The Python and TypeScript integration testkits read ANKKA_SIDECAR_IMAGE to choose the sidecar image they
start. Without it, a released SDK starts ghcr.io/thinkmorestupidless/ankka-sidecar at its own version, and an
unreleased one (version 0.0.0, from a checkout of the repository) starts ankka-sidecar:latest.
Web-hosted services¶
A web-hosted service's process is given two variables, and a descriptor may not declare either:
PORTis the port the process listens on: the descriptor'sprocessPort,8080by default.ANKKA_SERVICES_URLis where the process calls other services by name,http://127.0.0.1:7630in a cluster.
The platform's proxy beside the process is configured by the operator through variables starting
ANKKA_PROXY_; they are the operator's to write. The operator itself reads two more on its own
Deployment: ANKKA_PROXY_IMAGE, the proxy image it runs beside every web-hosted service, without which
such a service fails with operator has no proxy image; and ANKKA_HTTPS_PORT, the port the gateway is
reached on, from which the proxy states the address a request from the internet was sent to. The
installation's overlays set both. The web hosting reference is what the process is told.
Settings without a variable¶
These are overridden in the service's application.conf or with a system property.
ankka.ask-timeoutis how long a component client call waits before failing with theTimeouterror code,10sby default. A process-hosted service's sidecar uses it as the time it waits for the process to answer a command.ankka.http.body-timeoutis how long the HTTP server waits for a request body to arrive in full,10sby default.ankka.grpc.max-message-sizeis the largest request a gRPC call may send,4MiBby default. A larger one is refusedRESOURCE_EXHAUSTEDbefore any handler runs.ankka.grpc.max-connection-ageis how long a caller's connection to the gRPC server lasts before it is asked to reconnect,2mby default; it is what brings an instance added to a service into the rotation of callers already connected.ankka.grpc.shutdown-graceis how long calls in progress, streams included, are given to finish when a connection reaches its age or the service stops,5sby default. What is left then endsUNAVAILABLE.ankka.grpc.keepalive-timeandankka.grpc.keepalive-timeoutare how often an idle gRPC connection is asked whether its caller is still there,30s, and how long the answer is waited for,10s. A caller that went away without saying so is found within the two, and its calls are cancelled.ankka.local-grpc-services."<name>", set tohost:port, is where a service on this machine calls the gRPC endpoint of the service called<name>, before looking for it among the services running here.ankka.observability.ring-capacityis how many spans each instance keeps in memory for the local console and the metrics endpoint,4096by default. The oldest are overwritten; nothing is persisted.ankka.observability.call-windowis how far back a service's topology counts the calls between its components,10mby default. A call made before the window is forgotten, and two handlers that have not called each other inside it are not shown as calling each other.ankka.observability.call-bucketsis how many slices the window is kept as,60by default. A call leaves the window when its slice does, so more slices forget in finer steps and use more memory for each pair of handlers.ankka.observability.max-external-servicesis how many other services a topology shows by name,32by default. Calls to any service beyond that are counted together as other services.ankka.observability.max-external-methodsis how many gRPC methods of other services a call is recorded under by name,256by default; calls to any method beyond that are recorded together as other methods.ankka.observability.max-counted-handlersis how many pairs of component and handler the invocation counts name,1024by default; any beyond are counted together as(other). These are the counts a service exports as its metrics, since it started.ankka.telemetry.endpoint, fromANKKA_OTLP_ENDPOINT, is the OpenTelemetry collector a service exports its spans and metrics to,http://host:4318orhttps://host[:port]; empty, the default, exports nothing and starts nothing.ankka.telemetry.headers, fromANKKA_OTLP_HEADERS, is what is sent with every export, asname=value,name=value, and is never logged. On the platform the operator sets both from the installation's settings and a descriptor may not; read only by a service with theankka-telemetry-otlpmodule. See Telemetry.ankka.telemetry.interval,1s, is how often recorded spans are read and sent, sooner when half the trace window is unread;ankka.telemetry.batch-size,512, is the most spans in one export; andankka.telemetry.export-timeout,5s, is how long one export may take.ankka.telemetry.max-backoff,30s, is the longest wait between tries while the collector cannot be reached;ankka.telemetry.metric-interval,10s, is how often the metrics are sent; andankka.telemetry.shutdown-timeout,3s, is how long a stopping instance spends sending what it holds.ankka.telemetry.service-nameandankka.telemetry.projectsay who an instance is where it has no certificate to say so, such as a developer's machine; on the platform the certificate is used instead.ankka.observability.observe.enabled,ankka.observability.observe.portandankka.observability.observe.peerstart the listener the installation's control plane reads a deployed instance's topology from: off by default, on inkubernetesmode, on port7628, admitting only the peerankka://platform/controlplane. Leave them to the overlay.ankka.cluster.formationandankka.cluster.seed-nodesare set by the cluster overlays. Leave them to the overlay.ankka.join-self-if-no-seed-nodesisoninlocalmode andoffinkubernetesmode, where joining itself would split the service into several clusters.ankka.tls.cluster-directoryandankka.tls.service-directoryare where the cluster and service certificates are read from, empty inlocalmode and set by thekubernetesoverlay.ankka.tls.reload-interval,1m, is how often a changed certificate file is noticed.ankka.http.tls.enabledmakes the HTTP server mutual TLS;offlocally,oninkubernetesmode.ankka.probe.enabledandankka.probe.port,7627, are the plain readiness listener, on only inkubernetesmode.ankka.local-services.<name>is the address of another service on this machine for the service client, for every component that calls one and for the runtime beside a Python or TypeScript process, which is given it as a JVM option (JAVA_OPTS=-Dankka.local-services.<name>=…), such as"http://127.0.0.1:9001". Without it the client asks the local console's registry.
The runtime also sets Pekko's own settings. Two of them shape how a service behaves:
pekko.cluster.split-brain-resolver.active-strategy = keep-majority: after a network partition the side with the majority of instances survives. This is why instance counts should be odd.pekko.cluster.sharding.passivation.default-idle-strategy.idle-entity.timeout = 120s: an entity idle for two minutes is unloaded from memory, and rebuilt from its journal on its next command.
The CLI¶
The ankka CLI reads ANKKA_URL, ANKKA_TOKEN, ANKKA_PROJECT, ANKKA_CA and ANKKA_CONFIG. They are
described with the CLI's other settings in CLI.