Skip to content

Troubleshooting

uvx hailer doctor runs the startup checks (config, notebook, credentials, kernel; in docker mode also docker, image and data). It starts no kernel and talks to no running one. uvx hailer status lists this workspace's running Docker kernels.

Message Meaning and fix
Marimo is not running at <url> (...) with the hint End the chat (/exit) and run uvx hailer again The kernel this session started has stopped (killed, removed with uvx hailer kernel stop, or out of memory). Hailer never looks for another server: exit the chat and start it again, which starts a new kernel.
Could not finish stopping the Docker kernel or Could not stop the Docker kernel The command exits 1. The containers keep their labels and no longer have a live owner: restore access to Docker and run uvx hailer kernel stop, or let the next start remove them.
Marimo exited early (code N) or Marimo did not answer on http://127.0.0.1:2718 within 60 s, followed by Last lines of its log: uvx hailer notebook could not start marimo. The log tail usually names the cause (port in use by something else, a syntax error in the notebook, marimo not installed in the environment). A server another session runs on the same port never counts: Hailer waits for the one that holds its own token.
An unsafe-local marimo keeps running after Hailer was killed Nothing records an unsafe-local kernel, so no later Hailer ends it. It still needs its token, so other programs cannot use it: end the python -m marimo process with Task Manager or kill.
the notebook is not open in a browser followed by Open http://... in your browser. The server is up but has no kernel session. Open the URL; Hailer opens it for you once at startup. The URL ends in &view-as=present (app view) and, for a server Hailer started, &access_token=...; Ctrl+. in the notebook shows the code.
The browser shows marimo's sign-in page instead of the notebook The link has no access_token: links in the agent's replies and tool results leave the token out, because they go to the model endpoint. Run /notebook in the chat for the signed-in link.
Docker is not installed. The docker kernel, the default, needs the docker command. Install Docker Desktop (Windows, macOS) or Docker Engine (Linux), start it, and run the command again. Only if you accept that notebook code then runs as you, with your files and network, set [kernel] runtime = "unsafe-local" (or HAILER_KERNEL=unsafe-local; uvx hailer notebook --kernel unsafe-local for one run).
Docker is not running. The docker command is there but the engine does not answer (Docker Desktop is closed or still starting). Start Docker Desktop (or the Docker service), wait until it is running, and run the command again (the hint also names runtime = "unsafe-local", for those who accept that notebook code then runs as them). uvx hailer kernel stop says that it could not check for containers. When the hint says The docker command cannot reach an engine through its current context, DOCKER_CONTEXT or docker context use names a context that does not work: docker context use default switches back.
Docker runs windows containers; Hailer's kernel image needs Linux containers. Docker Desktop is set to Windows containers. Choose Switch to Linux containers from its tray icon menu (or, accepting what it means, set [kernel] runtime = "unsafe-local").
The kernel image marimo0.24.2-64a6b78f25cf is not published (or not visible to you): ghcr.io/openafterhours/hailer-kernel:marimo0.24.2-64a6b78f25cf. The registry refused the download: this kernel contract has no published image (a development checkout that changed the image, or a release whose image job has not finished), or the image is not public. uvx hailer kernel build builds it on this machine; or set [kernel].image (or HAILER_KERNEL_IMAGE) to a copy you can reach, such as a company mirror. Could not download the kernel image ..., with docker's own error above it, is a network, proxy or registry problem with the same fixes.
The kernel image <image> has kernel contract marimo0.23.0-0123456789ab; this Hailer needs marimo0.24.2-64a6b78f25cf. (or (no contract label), an image from before kernel contracts) [kernel].image names an image of another kernel contract, and a different marimo would break code mode. uvx hailer kernel pull downloads the matching one, uvx hailer kernel build builds it; or point image at the matching tag.
The data folder (<path>) is the workspace folder. With [kernel] runtime = "docker" it is mounted into the container, so notebook code could read Hailer's own files. (also is a whole drive, contains your home folder, is Hailer's .hailer folder, contains the context folder, is inside C:\Users\<you>\.aws, a folder that holds credentials) Docker mode refuses, on every start (--foreground too), a data folder that would hand notebook code Hailer's own files or your credentials (see Which data folder can be mounted). Keep the data in a folder of its own, such as data/, and point [hailer].data_dir at it. The notebooks folder is never mounted, so no rule applies to it.
The data folder \\server\share\sales is on a network share (UNC path), which Docker cannot mount. (an error in doctor's config row and at every start), or in doctor: the container may not see all of the data folder with ... is on a mapped network drive (Z:); Docker Desktop usually cannot mount it. (a warning) Docker Desktop cannot mount network locations. Copy the data to a folder on a local disk and point [hailer].data_dir at it. N link(s) in the data folder point outside it means symlinks or junctions the container cannot follow: copy those files into the folder.
Not copied into the kernel (only marimo notebooks with plain names are): helpers.py, test_sales.py. Docker mode: the kernel works on copies, and only marimo notebooks with plain names are copied (no conftest.py, test_*.py, *_test.py, setup.py, conf.py, tasks.py and other names tools run, no name of a Python module such as json.py or pandas.py (the warning says which module), no dot or __ folders, no files over 5 MB or deeper than four folders; see Notebook copies). A module such as helpers.py is not a notebook: put shared code in a notebook cell, or rename a notebook that has a test-runner name.
Not copied back from the kernel (only marimo notebooks with plain names are; everything else stays in the kernel and is gone when it stops): .git/, conftest.py, out.csv. Notebook code wrote files into the kernel's notebooks folder that are not notebooks. They never reach your machine; nothing to clean up. If you wanted one (an export), show the result in a notebook instead, or write it with the unsafe-local runtime.
Warning: q3/review.py changed both here and in the kernel. The kernel's version replaced it; the copy that was here is saved as .hailer/notebook-backups/q3/review.<date-time>.py. You edited the notebook on your machine while a docker session ran, and the kernel changed it too. Your version is in the backup; merge what you need and delete the backup.
Warning: could not copy the notebooks back from the kernel (...). The files in <folder> are unchanged. or Warning: could not copy back <name> (...) The kernel stopped answering, or a notebook's path here goes through a link or junction, or is not a file. Nothing on your machine was changed. Ask the agent to retry after fixing the cause; the last copy runs again when the session ends.
Warning: copying the notebooks back from the kernel stopped after 10 s or 50 MiB; the files in <folder> not copied yet are unchanged. (30 s when the session ends) A copy has a deadline and, in the background, a byte budget, so a kernel that answers slowly or with huge notebooks cannot hold Hailer or rewrite your files without end. In the background the rest follows on the next pass; at the end of a session it is not copied.
Warning: stopped before the notebooks were copied back; the kernel's changes since the last copy are lost. A second Ctrl+C interrupted the copy at the end of a docker session. The previous copies (after each turn and every 15 seconds) are on your machine.
marimo stopped (exit code N): it was ended from outside this terminal (uvx hailer kernel stop, Task Manager or kill), or it failed; its own output is above. Printed by an unsafe-local --foreground when its marimo ended without Ctrl+C.
The kernel container exited early (code N)., The forwarder container exited early (code N). or Marimo did not answer on http://127.0.0.1:<port> within 60 s., followed by Last lines of docker logs hailer-kernel-<id>: The docker kernel did not come up; the log lines usually name the cause. Nothing is left running.
The kernel ran out of memory and Docker stopped it; [kernel].memory in hailer.toml raises the limit. Printed by --foreground. Raise [kernel].memory (for example "8g") and start again. In a chat the same event shows as marimo no longer answering.
Note: [kernel].cpus = 8 is more than the 4 CPUs Docker has; the kernel gets 4. Information only. Docker Desktop's resource settings decide how many CPUs its engine has.
A notebook that used to work fails with KeyError: 'DB_PASSWORD' (or a database refuses a login) A docker kernel gets none of your environment variables, and Hailer withholds secret-looking ones from an unsafe-local kernel (doctor shows how many); no setting passes them through. Keep credentials out of notebook code: export the data the analysis needs into the data folder instead.
Warning: unknown key [kernel].pass_env in <path> is ignored. [kernel].pass_env was removed: no setting lets a secret-looking variable through to notebook code. Delete the line.
Warning: [model].runtime in <path> is ignored: runtime belongs under [kernel]. The [kernel] line is missing or commented out, so runtime landed in the table above it and the kernel stays docker. Add or uncomment [kernel] above it.
Invalid [kernel].runtime "local" (or HAILER_KERNEL): the runtime that runs notebook code on this machine is now called "unsafe-local" ... (or Invalid --kernel "local") local was renamed so that nobody runs notebook code unisolated without choosing it. Remove the setting to use Docker (the default), or write "unsafe-local" if you accept that notebook code runs as you, with your files and network.
Invalid [kernel].runtime "..." (or HAILER_KERNEL) or Invalid --kernel "..."; also Invalid [kernel].memory / cpus Fix the value: runtime is "docker" or "unsafe-local", memory a size such as "4g" or "512m", cpus a number above 0.
ModuleNotFoundError: No module named 'hailer' in the starter notebook The kernel's Python lacks Hailer's notebook helpers (hailer.periods), which the starter notebook imports. In docker mode [kernel].image names an image without them: uvx hailer kernel pull or uvx hailer kernel build gets the right one. /exec import hailer.periods in the chat checks it.
Could not remove hailer-kernel-<id>-<suffix>: docker said: ... after uvx hailer kernel stop (exit code 1) Docker refused a removal and the container (or network) is still there. Run uvx hailer kernel stop again; if it keeps failing, remove the one named in the message with docker rm -f (or docker network rm).
not found: <path> for the notebook The configured notebook ([hailer].notebook) does not exist. uvx hailer init creates it from the starter template (your hailer.toml is kept), or fix the path; a deleted active notebook is not the cause, because Hailer already falls back to the configured one when the remembered notebook is gone. New notebooks are created from the chat with /notebook new <name>.
No notebook named '...' in the notebooks folder. or ... is outside the notebooks folder. /notebook open (or the agent's notebook_open) only opens marimo notebooks inside [hailer].notebooks_dir, by their name there (sales.py, q3/review.py); the hint lists the available names. Move the file into the folder or point notebooks_dir at it.
'...' is not a notebook name inside the notebooks folder. A notebook name is a .py path inside the notebooks folder: no .., drive or leading /, no folder starting with . or __, no :, and no Windows device name such as con or nul. Pick another name.
test_sales.py would stay in the docker kernel and be lost when it stops (a file name other tools run). (or (the name of the Python module json)) In docker mode only notebooks with plain names are copied back (see Notebook copies), so Hailer refuses to create one that would be lost. Pick another name, such as sales_checks.py.
INTERNAL_MODEL_API_KEY is not set (required by provider 'internal'), or the same for OPENAI_API_KEY and provider 'openai' Every provider needs a key. Run uvx hailer login <provider> or set the variable in this terminal. Releases up to 0.2.2 could use a ChatGPT login for the openai provider; that is gone, so create an API key.
The model endpoint rejected the API key for provider '...' The endpoint returned 401. Re-run uvx hailer login <provider>.
Unknown model '...' for provider '...' The endpoint does not know [model].name (or the name given to /model). Its reply is quoted after The endpoint said:.
Could not reach the model endpoint at <base_url> Check base_url, VPN or proxy (HTTP_PROXY, HTTPS_PROXY, NO_PROXY), and that the endpoint is running.
The endpoint at <base_url> did not accept the request. The endpoint answered 404/405. The hint names the request Hailer sent (POST <base_url>/responses or POST <base_url>/chat/completions): either base_url is wrong, or the endpoint implements the other API, in which case switch wire_api. The endpoint's own reply is quoted on the last line, after The endpoint said:.
The endpoint at <base_url> rejected the request (HTTP 400) (or 422) The endpoint understood the request but refused a field or a value. The line The endpoint said: quotes its reply: fix what it names (stream = false or stream_options = false on the provider, reasoning_effort = "" under [model] to stop sending reasoning effort). uvx hailer --verbose logs the HTTP traffic.
The endpoint at <base_url> refused access (HTTP 403). The key was accepted but is not allowed for this model, route or organisation. Check the endpoint's access policy and the provider's http_headers / env_http_headers.
The endpoint at <base_url> is unavailable (HTTP 429) (or 5xx) Rate limit or an outage on the endpoint's side. Retry in a moment.
The conversation no longer fits the model's context window. Start again with /new, and lower [model].summarize_after_tokens so older turns are summarised before the model's limit is reached.
Warning: unknown key [model_providers.x].merge_messages ... is ignored (also parallel_tool_calls, requires_openai_auth, [hailer].codex_home, [web].allow_shell_network) Settings from releases up to 0.2.2 that no longer mean anything. The file still loads; delete the keys to silence the warning.
Previous conversation could not be resumed; started a new one. after upgrading from 0.2.2 or earlier Conversations from before 0.2.3 (the Codex releases) were stored elsewhere and do not carry over.
A tool result starting with ERROR: inside the conversation The agent hit a marimo or allowlist problem; the text contains the fix (for example the URL to open).
Marimo at <url> rejected the request (HTTP 401). (or 403) Hailer sends the token of the server it started, so something else now answers on that port (the kernel was stopped and the port reused). End the chat (/exit) and run uvx hailer again.

Logging

Normal runs print only warnings. uvx hailer --verbose (or HAILER_LOG_LEVEL=DEBUG) logs provider, session, marimo and tool activity and shows full tracebacks. Log output passes through a redaction filter that masks bearer tokens and the values of environment variables whose names end in _KEY, _TOKEN, _SECRET or _PASSWORD.