Langfuse Details How Its In-Product Assistant 'Halo' Was Built on Existing Infrastructure
Langfuse has pulled back the curtain on the development of Halo, its in-product AI assistant, revealing how the tool was built by integrating existing infrastructure rather than creating entirely new systems. Launched in June 2026, Halo allows users to interpret traces, build dashboards, and set up evaluations directly within the Langfuse UI.
Architecture and Integration
The development of Halo was largely centered around reusing and extending Langfuse's existing components, including its MCP server and CLI. The company aimed to avoid creating a separate maintenance burden for Halo, ensuring that improvements to core tools would benefit both the in-product agent and external users. A significant step was the expansion of the MCP server in May 2026, which added tools for accessing observations, metrics, scores, comments, datasets, and annotation queues, making project data accessible to agents.
Halo's journey involved several key milestones throughout 2026: a public beta launch in June, the addition of write actions with human-in-the-loop approvals in July, background execution in August, and sandbox-enabled runs later that month. The high-level architecture separates run management, with a browser initiating runs that are then processed by a Langfuse worker. This worker interacts with models and tools via the Langfuse MCP server, with optional Lambda MicroVM sandboxes for processing larger results.
Security and Permissions
Security was a foundational aspect of Halo's design. The agent accesses project data through the public MCP server, with tools and permissions carefully managed. For each run, a temporary API key scoped to the project is created. By default, only read-only MCP tools are available. The system maintains its own mapping of tool permissions, filtering tools based on the signed-in user's role before exposing them to the model. Actions that modify data require explicit approval, with options to save approvals for a conversation, though permissions are rechecked against the user's current role on each run.
Sandbox and Background Execution
While the agent loop itself runs within Langfuse's backend workers, a separate sandbox environment is utilized for tasks requiring file operations or script execution. This sandbox, typically a Lambda MicroVM for Langfuse Cloud, has no default network access and runs code as an unprivileged user. It is suspended and terminated after periods of inactivity. This design ensures that the agent cannot make arbitrary network calls from within the sandbox, with data access strictly controlled through authorized MCP calls. For tasks that involve processing large datasets exceeding a model's context window, the sandbox allows Halo to use tools like grep, head, jq, or short Python scripts. An optional silent parameter for MCP tools allows the agent to fetch data and receive a file path to the result within the sandbox, rather than the full output being loaded into the model's context.
To enable users to close their browser tabs and return to ongoing tasks, the agent loop was moved from an HTTP handler to the worker process. This separation allows runs to continue independently. The web server queues new runs via BullMQ, and the worker executes them, saving progress to Postgres. Server-Sent Events (SSE) are used to stream updates back to the browser. Langfuse has deliberately chosen not to implement automatic retries for failed runs to maintain simplicity, though this decision may be revisited for longer-running or critical tasks.
Future Directions
Langfuse is exploring ways to enhance sandbox script capabilities, potentially granting access to the Langfuse SDK and project LLM connections for tasks like evaluator calibration. This expansion would require careful security measures to ensure scripts only make approved requests. Halo is currently available on Langfuse Cloud and in public beta for self-hosted deployments.
