bench get-app https://github.com/vyogotech/frappe-ai-agent

[![Listed on Frappe Gems](https://frappegems.com/api/method/frappe_gems.seo.badge?app=vyogotech%2Ffrappe-ai-agent)](https://frappegems.com/gems/apps/vyogotech/frappe-ai-agent)

Frappe AI Agent

AI agent service for Frappe/ERPNext — natural-language questions in, structured visual answers out, streamed over Server-Sent Events.

Overview

frappe-ai-agent is the backend that powers the AI sidebar in a Frappe/ERPNext deployment. The browser POSTs a user message; the agent runs a custom envelope-based tool-use loop, calls ERPNext tools through frappe-mcp-server, and streams back a mix of prose and rich content blocks (charts, tables, KPI cards, status lists) for the frontend to render.

Three design points are load-bearing:

1. Permissions stay in Frappe. The browser forwards the user's sid cookie on every chat request. The agent authenticates the request from that cookie and forwards the same sid to MCP for every tool call, so each tool runs under the caller's Frappe user — no shadow admin account, no permission re-implementation.
2. No fabrication. The system prompt forbids inventing data; every value must come from a tool call in the same turn. Tool errors are folded back into the conversation as observations so the LLM can explain what failed instead of aborting.
3. Envelope-protocol streaming. The LLM emits a single JSON envelope per turn whose blocks are one of tool_call | text | table | chart | kpi | status_list. The agent loop runs tool-call blocks itself and re-prompts; non-tool blocks become content / content_block SSE events. The browser sees discrete typed events, not partial markup.

Architecture

Browser (Vue sidebar)
   │  POST /api/v1/chat  (SSE, Cookie: sid=...)
   ▼
frappe-ai-agent  (FastAPI + envelope tool-use loop)
   │
   ├──▶ LLM provider                (Ollama / OpenAI / Anthropic / Google)
   ├──▶ frappe-mcp-server           (MCP Streamable HTTP, sid forwarded)
   │       ▼
   │    ERPNext REST API            (runs as the caller)
   └──▶ Frappe REST                 (AI Chat Session / AI Chat Message)

Per chat request, ChatService.handle_message does the following:

1. Resolve or create an AI Chat Session in Frappe (best-effort; falls back to a temporary in-memory id if Frappe is down).
2. Persist the user message to AI Chat Message.
3. Build a fresh MCP client carrying the caller's sid and load its tools (timeout configurable via mcp_tools_load_timeout_s, default 20 s).
4. Register each tool in ToolRegistry, which surfaces exceptions to the LLM as observations rather than aborting the loop.
5. Build a per-request system prompt with page context and currency.
6. Run run_agent_loop: structured-output envelope → execute any tool_call blocks → re-prompt with results → repeat until the envelope contains terminal blocks. Translate each block into the matching SSE event.
7. Persist the final assistant message (success or error).

Quick start

Prerequisites

• Python 3.12+
• UV for dependency management
• Ollama running locally (http://localhost:11434) with a tool-capable model pulled, or API credentials for OpenAI / Anthropic / Google
• frappe-mcp-server reachable on http://localhost:8080/mcp
• A Frappe/ERPNext instance reachable on http://localhost:8000 (for chat history persistence)

uv sync --all-extras
cp .env.example .env
make serve

The agent listens on http://localhost:8484.

API

POST /api/v1/chat

Streaming chat endpoint. Returns text/event-stream.

Request body (JSON):

{
  "message": "show me unpaid invoices",
  "session_id": "AI-CHAT-0001",
  "context": { "doctype": "Customer", "docname": "ACME", "currency": "USD" }
}

• message — required, 1–32 000 chars, non-whitespace.
• session_id — optional. Omit on the first turn; the agent creates a session and announces its id in the first SSE frame. Pass that id back on subsequent turns to continue the conversation.
• context — optional page context. Recognised keys: doctype, docname, route, currency (ISO 4217 code; default INR). Everything else is ignored.

Authentication — must include a Frappe sid cookie. Missing or empty cookie → 401.

Rate limit — 30/minute per sid by default, enforced by slowapi. Exceeding the limit returns 429. Tune via AI_AGENT_AGENT_RATE_LIMIT. The limit applies per request, not per concurrent connection — a single sid may hold multiple open SSE streams within its quota.

Response stream — newline-delimited data: \n\n frames. Event types:

type	Payload
session	{ id: str } — sent once, before any other event
status	{ message: str } — informational (reserved for future use)
tool_call	{ name: str, arguments: dict } — emitted when the agent invokes a tool
content	{ text: str } — prose token chunks (streams as the LLM generates)
content_block	{ block: dict } — a complete parsed content block (chart, table, …)
error	{ message: str } — fatal error; followed by done
done	{ tools_called: list[str], data_quality, timestamp: str }

data_quality is "high" on success, "low" if the turn ended in error.

The wire contract is encoded as TypedDicts in src/ai_agent/transport/sse_events.py (SessionEvent, StatusEvent, ToolCallEvent, ContentEvent, ContentBlockEvent, ErrorEvent, DoneEvent, plus the SSEEvent union). A validate_event(event: dict) helper in the same module raises ValueError on any drift from the contract — tests/unit/test_chat_service.py::test_every_emitted_event_matches_sse_contract runs it over every event the service emits in a typical turn, so adding a new field server-side without updating the TypedDict is caught at CI time, not in a frontend bug report.

GET /health

Lightweight liveness check: {"status": "ok"}.

GET /health?detail=true probes MCP (/health) and Ollama (/api/tags). Non-Ollama LLM providers are reported as skipped: true — the agent does not call hosted-provider model-list endpoints from a public health route.

GET /config

Returns the resolved LLM provider, model, base URL, and MCP server URL. Useful for the frontend to render a "connected to: …" indicator.

Content blocks

The LLM wraps structured data in { JSON } tags. The block JSON is validated against a Pydantic model, capped at sane size limits, and streamed to the frontend as a single content_block event.

Block	Purpose	Cap
chart	Bar / line / pie / funnel / heatmap / calendar via ECharts	500 datapoints per dataset
table	Sortable rows + columns with optional row→doc link	100 rows
kpi	Horizontal row of metric cards	8 metrics
status_list	Colored status entries	50 items

Tolerances:

• `(orbar,line, …) is accepted as a chart alias —chart_type` is filled in from the tag if the inner JSON omitted it. Some smaller local models reach for this shape first.
• Malformed JSON, unknown types, or oversized payloads fall back to a TextBlock so the user still sees something instead of a dropped message.
• Chart datasets may contain null to mean "no data here" — ECharts renders these as gaps.

The full schemas are in src/ai_agent/blocks/models.py.

Configuration

All settings are loaded from environment or .env with the AI_AGENT_ prefix. Unknown keys with the prefix cause startup to fail rather than silently being ignored.

Variable	Default	Description
AI_AGENT_HOST	0.0.0.0	Bind host
AI_AGENT_PORT	8484	Bind port
AI_AGENT_WORKERS	1	Uvicorn workers. Wired into the Dockerfile CMD as uvicorn --workers ${AI_AGENT_WORKERS:-1}. The agent is stateless per-request (history is fetched from Frappe each turn), so raising workers is safe — bottleneck is Ollama / hosted-LLM throughput.
AI_AGENT_CORS_ORIGINS	["http://localhost:8000"]	JSON list of credentialed-CORS origins. "*" is not allowed because cookies are forwarded
AI_AGENT_LLM_PROVIDER	ollama	ollama, openai, anthropic, google
AI_AGENT_LLM_BASE_URL	http://localhost:11434	Provider base URL
AI_AGENT_LLM_API_KEY	empty	API key for hosted providers
AI_AGENT_LLM_MODEL	qwen3.5:9b	Model identifier
AI_AGENT_LLM_TEMPERATURE	0.2	Low default tightens tool-call argument formatting on small local models
AI_AGENT_LLM_MAX_TOKENS	8192	Max output tokens (Ollama num_predict)
AI_AGENT_LLM_NUM_CTX	16384	Ollama context window. Ignored for hosted providers. The Ollama default of 2048 is too small for system prompt + tool results + answer
AI_AGENT_AGENT_RECURSION_LIMIT	50	Envelope-loop recursion ceiling — small models need headroom while exploring doctype schemas before converging
AI_AGENT_AGENT_RATE_LIMIT	30/minute	slowapi-format per-sid rate limit on POST /api/v1/chat (e.g. 100/hour, 10/second)
AI_AGENT_MCP_SERVER_URL	http://localhost:8080/mcp	MCP Streamable HTTP endpoint
AI_AGENT_MCP_TOOLS_LOAD_TIMEOUT_S	20.0	Per-request bound on tools/list. A timeout becomes a single SSE error event, not a hung stream
AI_AGENT_HEALTH_PROBE_TIMEOUT_S	5.0	Timeout on the agent's own /health reachability pings against MCP and Ollama
AI_AGENT_FRAPPE_URL	http://localhost:8000	Frappe URL for chat history writes
AI_AGENT_OTEL_ENDPOINT	empty	OTLP gRPC endpoint. Empty = tracing disabled
AI_AGENT_OTEL_SERVICE_NAME	frappe-ai-agent	Resource attribute on emitted spans
AI_AGENT_LOG_LEVEL	info	debug / info / warning / error
AI_AGENT_LOG_FORMAT	json	json or console

See .env.example for a working starter file.

LLM providers

The factory in src/ai_agent/integrations/llm.py instantiates a chat model from settings:

• Ollama — direct ChatOllama (so num_ctx and num_predict are wired correctly).
• OpenAI / Anthropic / Google — via LangChain's universal init_chat_model. Install the optional extras (anthropic, google) if you need those.

The default config targets a local Ollama running qwen3.5:9b. The system prompt and the parser are designed to tolerate the kinds of mistakes small models make (chart-type aliases, occasional tool-call formatting drift, etc.), but any tool-capable model will work.

MCP integration

Tools are loaded per-request from frappe-mcp-server via the Streamable HTTP transport (langchain-mcp-adapters). A new MCP client is built for every chat turn so the caller's sid cookie can be attached as a request header — sharing clients across users would leak sessions.

tools/list is bounded by AI_AGENT_MCP_TOOLS_LOAD_TIMEOUT_S (default 20 s). If MCP is unreachable, the user sees a single SSE error event and a done frame; the stream does not hang.

ToolRegistry.ainvoke catches every tool exception (MCP errors, Frappe permission denials, httpx timeouts) and surfaces it to the LLM as a string observation rather than aborting the loop. Permission errors get a clearer prefix (Access denied: permission error — …).

Multi-turn state

The agent does NOT yet replay prior turns into the LLM context — run_agent_loop is called with history=None. Frappe still persists every turn into `AI Chat Mes