Logs and troubleshooting

The node logs to standard output and names most problems there; the table below lists the common ones with their fixes.

Logs

docker compose logs -f tavrn shows the node’s log on stdout/stderr. TAVRN_LOG_LEVEL (default info) takes a level or per-module directives (info,sqlx=warn, debug for a short investigation); TAVRN_LOG_FORMAT=json writes one JSON object per line for log collectors. Logs contain addresses, usernames and request metadata, so treat them as personal data and rotate them (Docker’s json-file driver with max-size/max-file, or your collector’s retention). The output of the supervised LiveKit process is forwarded into the node’s log. Useful lines: LiveKit starting media=… (announced addresses and ports), new node signing key generated (first start), the background storage encryption progress, Auto-moderation: message flood.

Performance

A node answers most API requests in a few milliseconds; these settings and behaviours keep it that way and help to find the exceptions.

• Database connections stay warm. Each instance keeps at least 2 PostgreSQL connections open while idle and closes further idle ones only after 30 minutes (every connection is replaced after 2 hours). A connection is checked with a ping only when it was idle for more than 30 seconds, not on every request. So the first clicks after a quiet period do not pay a new connection (TCP, TLS, authentication) and the re-preparation of the statements.
• Slow requests. TAVRN_LOG_SLOW_REQUEST_MS=250 logs every request that took 250 ms or more: slow request method=GET route=/api/v1/rooms/{room_id} status=200 duration_ms=312.4 queries=9. The route is the template, so the lines group well and carry no ids; queries counts the database statements of the request (a high count points at a missing batch, a long duration with few statements at the database or a slow external call). The line is at level info, so a TAVRN_LOG_LEVEL of warn hides it unless it names the target (warn,tavrn_node::slow_request=info). Counting costs a little per statement, so leave it off when you are not looking.
• Slow statements. TAVRN_DB_SLOW_STATEMENT_MS (default 1000) logs a warning with the statement text for every statement at or above the threshold; lower it (e.g. 100) to find the statements behind slow requests.
• The web client is served from memory without a copy per request, compressed (about 70 % smaller for scripts and WebAssembly) and with an ETag, so a revalidation of index.html answers 304.
• Stalled clients. A WebSocket client that stops reading (a frozen tab, a broken mobile link) is disconnected once a send has waited 15 seconds; the user then counts as offline (and gets web pushes again) instead of staying “online” until the kernel gives up on the connection. The heartbeat re-checks the sessions of all connected devices of an instance together, a few statements per 30 seconds instead of two per connection.
• Background workers (federation outbox, housekeeping, event subscriptions, voice sweeps, directory, whiteboard sweeper) start again after a panic; the log shows background worker panicked, starting it again with the worker’s name, and the panic itself as a panic line with its location.

Common problems

These are the problems operators run into most often, each with its cause and fix.

Symptom	Cause and fix
The container exits at once with Configuration error:	One line per problem with the variable name; fix them all, docker compose up -d again (exit code 2).
TAVRN_CACHE_URL: Valkey/Redis at host:port is not reachable (…), exit code 3	Valkey down, wrong host, password or TLS scheme. See Failure behaviour.
The node refuses to start because of storage.key	The file is missing or does not belong to this database. Restore keys/ from the same backup as the database; never let the node create a new one (Keys and backups).
database migration failed (schema newer than the binary? …)	An older image against a newer database: start the newer image again, or restore the backup taken before the upgrade (Upgrade and rollback).
The node refuses to start because of TAVRN_WEBAUTHN_RP_ID	The public URL’s host is not covered by the relying party id; see Passkeys.
The Android app says “This node does not accept the app’s passkeys yet”	TAVRN_WEBAUTHN_APP_ORIGINS lacks the app’s origin (the hash of the key the installed APK is signed with), or assetlinks.json of the relying party id does not list that key; see Passkeys in the apps.
Rate limits or bans hit everybody at once	The proxy’s address is not in TAVRN_TRUSTED_PROXIES, so every request seems to come from the proxy. Add it.
Uploads fail with 413	The proxy’s body limit is below TAVRN_UPLOAD_MAX_BYTES (nginx client_max_body_size, Caddy request_body max_size).
The web client loads but never connects or keeps reconnecting	WebSockets are not passed through (Upgrade/Connection headers, “Websockets Support” in NPM) or the proxy’s read timeout is short.
/rtc/validate does not answer 401	/rtc is not routed to tavrn.
Voice connects but nobody hears anyone, or only some users	The UDP range or TCP 7881 is not published 1:1, not open in the firewall or not forwarded; or the wrong address is announced. Compare LiveKit starting media=… with the router and run the connection test. LAN clients without NAT loopback: add the LAN address to TAVRN_RTC_ADVERTISE_IPS.
Docker cannot publish the UDP range	The range lies in the ephemeral range 32768–60999 and a socket on the host occupies a port; move the range above 61000.
Users on corporate or hotel networks cannot join voice	Enable TURN, ideally over TLS on 443 (TURN).
/healthz shows voice_problem	Usually none of the hostnames in TAVRN_RTC_EXTERNAL_IP/TAVRN_RTC_ADVERTISE_IPS resolves.
/readyz answers 503 with "livekit": false	LiveKit is not running: look for livekit-server exited, restarting in the log; outside the image check TAVRN_LIVEKIT_BINARY.
Other nodes cannot reach yours, or the directory does not list it	https://<TAVRN_DOMAIN>/.well-known/tavrn must be reachable with a valid certificate; with delegation check the proxy rule or SRV record; a static copy may be stale after a key change. TAVRN_DIRECTORY_LISTING must be on.
A peer answers 503 federation.conformance_pending or your node is blocked by a peer	Conformance checks; run tavrn-node doctor --peer <domain> and see Conformance checks.
No e-mails arrive	Run tavrn-node send-test-mail you@example.org; check TAVRN_SMTP_URL and TAVRN_SMTP_FROM.
“Forgot password” answers 503 mail.unavailable	The node has no TAVRN_SMTP_URL, so it sends no reset links (they would only end up in the log). Configure SMTP, or reset the user’s second factor or account as a node admin.
Users are signed out with 401 auth.session_revoked (WebSocket close 4001)	The device’s sessions ended: logout, a password change or reset, the device was removed, or a node admin disabled, banned or reset the account. Expected; the client signs in again.
WebSockets close with 4002 too_many_connections	The user holds more than TAVRN_WS_MAX_SOCKETS_PER_USER connections on this instance (many tabs or a client reconnecting in a loop).
SSO login fails	Run tavrn-node check-config: it checks the issuer and prints the redirect URI to register; the issuer must match exactly (Authentik with the trailing slash).
Avatars or animated images are refused, 503 media.busy	ffmpeg missing (TAVRN_MEDIA_FFMPEG) or too many conversions at once (TAVRN_MEDIA_WORKERS).
Webhook posts fail with 409 webhook.pending_key or 400 webhook.sealed_required	See Webhooks in end-to-end encrypted channels.
Push notifications stop for everybody	Egress HTTPS blocked, or keys/vapid.key was replaced (users must re-enable push). Watch tavrn_push_failed_total.
Push does not work on iPhone or iPad	The web app must be added to the home screen first.
503 node.leader_unavailable or node.cache_unavailable with several instances	See Failure behaviour.

---

This page is generated from tavrn-node/docs/HANDBOOK.md (AGPL-3.0-only) when the website is built; its links were adapted for this site.