Self-hosting Sophie

Sophie is designed to run as a set of microservices to ensure scalability and high availability:

All components are containerized and typically run using Podman.

Before starting, ensure you have the following installed on your host system:

The recommended way to deploy Sophie is using the provided Ansible playbooks in the directory.

Copy to and fill in the required values. Key variables include:

Set to a Sentry DSN to enable error reporting and performance traces. With a DSN, Sophie traces every transaction by default (); lower this rate to reduce volume, or set it to to disable traces while keeping error reporting. Sampling is at the parent transaction level, so the same rate also applies to non-AI requests. The Pydantic AI integration records agent/model/tool spans, token usage, prompts, replies, and tool inputs/outputs in captured traces. These can contain personal data and produce high Sentry volume; set an appropriate retention policy. For Ansible deployments, set in the operator environment to override the default in all bot, scheduler, and REST processes.

Sophie also adds spans for its own AI caches and state transitions. AI request metrics still use the existing and settings.

To deploy the stable environment:

To deploy the beta environment (includes scheduler and REST API):

Mini App login at validates Telegram init data with the bot token using v1. Init data has an explicit one-day lifetime; older credentials are rejected, so clients must reopen the Mini App to obtain fresh init data rather than reuse it indefinitely. Malformed init data (including missing required fields) or validated data without a user returns HTTP 400. Invalid hashes and expired credentials return HTTP 401. A valid Telegram user not registered in Sophie's database still receives HTTP 403. The successful token response is unchanged.

Welcome messages go only to the members they greet, one message per member with their own name. Nothing is posted to the chat, so clean-welcome cleanup has nothing to delete.

Captcha prompts also go to each new member as ephemeral messages, rather than into the chat. Only new members see them, one prompt each; nothing is left to delete after the captcha is passed.

If a member leaves before Telegram accepts their ephemeral welcome or captcha prompt, returns and logs , , the chat ID and the recipient ID. This applies to Rich messages, traditional text and single-media saveables. A missing reply target is retried without the reply, but always with the same ephemeral recipient; a departed recipient is never retried publicly. Only the exact failure on an ephemeral send is treated this way: unrelated failures and this error on public sends are still surfaced.

A prompt whose security note is an album is still posted to the chat: cannot address one member, and splitting the album into separate ephemeral messages is no way around it — Telegram accepts at most five ephemeral messages per user.

Sophie's AI models, the endpoints they are served from and the API keys used to reach them live in the database, not in the configuration file. They are managed at runtime with operator commands, so adding a model or rotating a key never needs a redeploy.

The migration creates catalog models and the provider; the migration creates and providers for moderation and transcription. All three start without keys. Configure credentials in the database before enabling AI, using in a private chat with the bot or updating the provider through the operator API.

The initial catalog also includes a model. To use it, create an OpenAI-compatible provider named with its endpoint and key via or the operator API. No AI provider credentials are read from environment variables.

is , , or (a key for a vendor SDK rather than a chat-completions endpoint — do not point models at one). A role is — for example — or just for the purposes that are not per-chat (, ). Purposes are , , , and , ; modes are , , , and .

A mode with no model for a purpose falls back to the tier, so you only need to define the roles you want to differ. Changes take effect on every process within a few seconds without a restart.

is the model's authoritative positive integer context capacity. The migration fetches the public OpenRouter models API once, matching exact first and exact catalog second. Existing configured capacities are preserved; unmatched models stay undefined and are reported. Set those manually with or the operator API. Network or invalid-response errors stop the migration before it changes any capacities. Rollback removes only unchanged migration-added values and preserves later operator edits.

Modern context requires a configured capacity for every possible chatbot candidate, including the last-resort model. Missing capacities stop the request with an AI configuration failure reply and are reported for diagnosis; no provider-profile or model-name estimate is used. Legacy context can still use models with undefined capacities.

Quota accounting and provider-debug output use Pydantic AI's native token counts. Zero input or output tokens remain zero; pricing uses the actual input/output split rather than estimating a missing side.

Research reads with the other workflow settings once per workflow. That tier is the default for every research model attempt; a tier configured on the model's catalog role can override it.

The existing flag and chat mode control moderation. Mistral and OpenAI classifier requests retain their normal retry budget. If a supported transient provider failure (including overload/rate limits, service outages or transport timeouts) exhausts that budget, Sophie logs the failure without sending a chat message and stops processing the current message across bot handlers. An unclassified message is not a clean moderation result: it stays in the chat, and its author is not banned or restricted because of the failed check. The message is not added to AI history. There is no deferred classification or extra retry.

Failures are logged with operation, model and error type. When Sentry is configured, the log includes a reference to a structured provider-failure event. Moderation retry breadcrumbs and terminal events omit provider response text and SDK tracebacks, which can contain the private classifier input. Persistent authentication/configuration errors and unexpected programming errors still follow the normal error-reporting path; they are not presented as temporary classifier unavailability.

Chatbot replies show an in-progress message while they stream. Manual translation (, ) and also show progress. All three use STFU Rich rendering with a fixed animated AI emoji and a three-emoji footer; translation and research edit the same Rich message for the result. Automatic translation stays silent until its result is ready. The chatbot and manual translation initially show a random working message after the animated AI emoji. When reasoning or an activity begins, that space stays empty until answer text streams in; progress appears below. Streamed reasoning appears once as an italic “Reasoning...” activity below the header, without revealing reasoning text. Later reasoning passes, including those after tool calls, do not add another activity. Every tool invocation and retry stays in the bottom activity list, including repeated calls and tools hidden from the final header. New streamed answer text clears the activity list. Later tool calls appear below the answer text already shown; the next text update clears those entries in turn.

Manual and retain their current activity list until the final translation replaces the progress message. A replied voice starts with “Transcribing voice message...” before transcription, followed by “Translating...”. Replied images and videos show processing stages (including video audio transcription) before translation. Chatbot replies stack the same media stages during context preparation.

Video transcription reads the first audio stream even when the container lists a video stream before it.

Completed replies replace the animation with a static AI emoji, eligible used-tool titles without tool icons (Search first when used), and a battery footer in its own paragraph. Each tool has its own setting in ; set it to to hide that tool from the final header without hiding its in-progress activity. Custom emoji IDs remain in the tool metadata but are not rendered in the progress activity or completed header. The low, middle, and high battery icons correspond to 0–32%, 33–65%, and 66–100%. adds the model beside the battery reading. Only the assistant's answer is stored in conversation history; the displayed header, tool titles, and battery footer are not. Legacy context replays recent tool calls and results, with each stored result capped by . Proactive replies use the same inline custom-emoji header and bottom battery footer. Header styles are no longer configurable.

Modern history is opt-in. Run the model-context-size migration and configure every unmatched model, including last-resort candidates, before enabling it:

keeps one serialized session per chat, topic and AI mode. It appends new chat messages and native assistant/tool exchanges once, rather than rebuilding the complete recent-message block on every turn. When the budget is reached, it deletes old background batches first, then whole completed turns. Tool calls and returns stay paired. There is no generated summary or compaction message; an oversized current request fails instead of being truncated.

Admission counts model-facing instructions, text and tool arguments/results with each candidate's local tokenizer, not the serialized session JSON. Known OpenAI model names use (including / , using the GPT-5 mapping). GLM-5.3 and GLM-5.3-Flash use Hugging Face with only 's , pinned to revision ; provider prefixes and routing suffixes on both model names are accepted. Unrecognized model names use the bundled OpenAI tokenizer as an estimate. and in define an application policy: limit exposure to estimation error by capping the registered capacity at 8192 and keeping 80% of the input left after output and tool-schema reserves. These chosen safety limits are not a measured tokenizer ratio or a provider context-window claim; models with known tokenizers use their registered capacities. A smaller registered window still wins, and exhausted reserves leave zero input capacity, so admission rejects that candidate. This policy cannot guarantee a fit for an unknown tokenizer; register a supported tokenizer to use a model's larger window confidently. No model weights or remote token-counting API are used.

Persisted history is pruned with the primary model's tokenizer and capacity. Each attempted candidate, including image-eligible fallbacks and the last-resort model, gets an independently budgeted request projection; smaller fallback windows never delete primary session history. Runtime snapshots are restored in the projection if pruning removes the turn carrying them. An oversized current request fails before it reaches that candidate's provider.

The default uses the registered model capacity minus reserves. Positive values remain explicit input caps, including existing chat and global overrides; clear these overrides with to use the model-aware default. This setting does not replace the registered capacities. Admission also reserves the configured response limit (2048 tokens when unlimited), 4096 tokens of safety/tool-schema headroom, and per-message/part framing allowances. Each image reserves 4096 estimated tokens, independently of its transport size. These are conservative allowances, not a guarantee of exact provider usage: chat templates, image handling and hidden provider additions can differ. Stored event costs are recounted on each build, so changing models or tokenizers does not reuse stale token estimates.

Cache retention is separate from these budgets: the shared message cache expires after 48 hours, but retained native session events are pruned by tokens, not age. The modern builder reads the eligible cache without a latest-10 seed cap; legacy/count-limited seed policies do not control native session growth. Changes to cold-start seed selection or 48-hour cache retention are deferred.

Admission budgets the history before a run; newly generated tool results can grow beyond the reserved headroom. It does not impose a new per-tool-step truncation policy.

The runtime Docker image bundles the pinned GLM tokenizer and every OpenAI encoding provided by its installed version during the dependency build stage. Loading a tokenizer never downloads assets at runtime and does not require writable caches. Missing or invalid assets for a recognized model fail the request rather than silently changing its token estimate.

For source-checkout development, prepare the same assets once with network access:

Generated files live in and are excluded from Git and the Docker build context. CI test setup runs the same generator and smoke-checks the required tokenizers before running pytest. The image always generates its own assets from the locked dependencies and pinned GLM revision. No model weights or provider credentials are needed. Tokenizer instances load off the event loop and are cached per process.

Stable instructions precede conversation history. Each request carries one current runtime snapshot with the date, escaped XML chat notes and 1-based XML memory items; earlier runtime blocks are removed structurally from replayed history. Previous-chat summary context uses only the newest stored daily summary for that chat, with its rendered in both runtime XML and chatbot instructions. An empty or missing latest summary clears that context without generating a replacement. Current/replied speakers and memory references can appear in the entertainment roster; old private alias maps do not grow the roster forever. Group and channel titles are included as escaped XML in runtime snapshots, never in stable instructions. Private-chat names are omitted from the conversation-title field.

Speakers and message references use session-local and labels. Typed builders select these labels instead of real handles or Telegram IDs for attribution fields. Entertainment alone includes literal first-name attribution. Its outgoing first-name mentions resolve to usernames only when the session identity lookup is unambiguous; known ambiguous names remain plain. Other modern modes keep alias-only attribution. Authored messages, notes, summaries, configuration prompts and arbitrary native tool data are not scanned or redacted. Only XML syntax is escaped. Native provider message parts and signatures replay unchanged; there is no generic sanitization processor. Memory facts use identity anchors, stored as internal links and re-aliased for a new session, so first-name jokes do not attach to a different person after reset.

Supported providers receive caching options: Anthropic-compatible OpenRouter models use explicit cache markers, native Anthropic uses its cache controls, and native OpenAI uses an opaque session cache key. Generic compatible endpoints receive no unsupported OpenAI cache fields. Actual cache reads/writes are recorded from provider usage; cache hits remain provider-dependent.

clears the shared message cache and native sessions in one epoch-fenced transaction. Pre-reset incoming messages and modern deliveries cannot repopulate it after their handler or generation finishes. Tool history and memory are also cleared. Switching the flag off uses ; it does not reinterpret the modern session. After a deployment that retires old display headers, reset affected histories if previously decorated cache entries remain.

The AI moderator classifies messages against nine categories and deletes anything that crosses a threshold. It does not go through the AI catalog above — it calls a dedicated moderation classifier, chosen with the feature flag:

Switching backend is two steps: put the key in the catalog, then flip the flag. Neither needs a restart — the client is rebuilt as soon as the catalog version changes.

The two providers report different categories, and Sophie normalises them onto its own nine. , , and have no OpenAI equivalent and never trigger on that backend.

Chat admins run to get a table of the nine categories and a button for each one. Pressing a button walks that category through Off → Low → Medium → High.

The level multiplies the classifier's score before it is compared to the threshold, so a chat can be made more or less sensitive without an operator retuning anything. The factors are feature flags:

Off skips the category entirely rather than scaling it to zero.

Every threshold is a feature flag, so it can be changed per chat with no redeploy:

Flags are named , because categories that Sophie groups together do not score alike: needs a much lower cut-off than . These are the operator-level tuning; per-chat sensitivity is the detection level above.

The "message deleted" notice removes itself after (30 by default); set it to to keep the notices in the chat.

The Sophie-help assistant can start a sub-agent that reads Sophie's own source code when the documentation cannot answer a question. It costs several model requests per question, so usage is limited per chat.

Groups do not get it from their AI mode. is a space or comma separated list of group IDs allowed to use it anyway, for the chats where people ask how Sophie works.

Every run is bounded: , and cap one run, caps how many runs one chat may start per day, and the tokens are charged to that chat's AI quota like any other feature.

The model it uses is the catalog model holding the role, so it is swapped like any other: . Prefer a cheap one — the daily cap is what bounds the damage, not the price per run.

The sub-agent can only read files inside the package, and only through search and bounded reads — it never executes anything and never sees configuration or data.

If you prefer to run containers manually, you can use the following logic (based on the Quadlet templates):

The scheduler is the same image but runs with .

The REST API is the same image but runs with .

Sophie implements a unique proxying system where the Stable instance can redirect traffic to the Beta instance. This allows seamless transitions and testing of new features.

When enabled, the bot can route requests between instances based on configuration, allowing for "canary" style deployments or easy beta testing for specific users/chats.