bench get-app https://github.com/sanjay-kumar001/idp

[![Listed on Frappe Gems](https://frappegems.com/api/method/frappe_gems.seo.badge?app=sanjay-kumar001%2Fidp)](https://frappegems.com/gems/apps/sanjay-kumar001/idp)

---

Intelligent Document Processing (IDP) for Frappe / ERPNext

> A modern, conversational document‑processing app for Frappe / ERPNext. > Upload an invoice, PO, delivery note, bank statement, or quotation — and the IDP agent extracts structured data, reconciles it against existing ERPNext records, asks for confirmation, and creates / updates the document for you. Built with Vue 3, Frappe UI, PaddleOCR, and a pluggable LLM stack (OpenAI · Anthropic · Ollama).

---

Features

Conversational Document Processing

• Modern single‑page chat UI built with Vue 3, Vite, Tailwind CSS, and Frappe UI
• Sidebar with conversation history, search, filter chips, archive / delete
• Upload PDFs, images, scans, Excel, CSV, or Word — the agent extracts in‑context
• Live ConfirmationCard rendered inline: editable header fields, line items, taxes
• Token‑by‑token assistant streaming via Frappe Realtime (Socket.IO / WebSocket)
• Suggested prompt chips on empty conversations, bulk actions on multi‑card turns
• Persistent cost / tokens / duration footer chip per conversation
• Per‑conversation Output Language and OCR Language overrides
• Optional pre‑flight cost warning before high‑cost turns

Two‑Tier LLM Stack (provider‑agnostic)

• OpenAI (gpt‑4o, gpt‑4o‑mini, …), Anthropic (Claude Opus / Sonnet / Haiku 4.5), Ollama (local LLMs)
• Custom providers can be registered as JSON without code changes
• Cheap tier (Haiku / gpt‑4o‑mini) handles classification, gating, summarisation
• Expensive tier handles extraction and final confirmation
• Routing is configured per purpose (classification, extraction, vision, summarisation, confirmation)
• Anthropic extended‑thinking blocks supported and optionally stripped on replay

Multi‑Format OCR

• PaddleOCR with 12+ languages (English, Hindi, Chinese, Arabic, French, German, Japanese, Korean, Spanish, Tamil, Telugu, Portuguese)
• Vision fallback via Ollama (minicpm-v, llava) when PaddleOCR fails or pypdf text is too thin
• Auto language detection per upload (auto mode)
• Layout analysis and table extraction (toggleable)
• Per‑page pre‑pass: pages without recognised signals are replaced with placeholders in the LLM context

Hybrid Field Mapper

• Rule‑based FieldMapper with built‑in keyword sets (Indian GST, US tax, generic invoice)
• Per‑DocType IDP Extraction Template overrides keywords, validation rules, confidence bands, and LLM fallback thresholds
• LLM hybrid mapping for low‑confidence extractions
• Mapper output cache keyed by (file_hash, target_doctype, mapper_version) — re‑uploading the same bytes within TTL skips the LLM entirely
• Few‑shot exemplars from IDP Extraction Correction ledger injected into the prompt

Schema & Business‑Rule Validation

• Pre‑validation against the target DocType's schema before the LLM extraction loop runs
• Required‑field, range, regex, and in [...] rules definable per template
• Auto‑resolution of missing Supplier / Customer / Item masters (optional)
• Document comparison against existing PO / SO / DN / Invoice records before creation
• Approver routing via JSON rule set (e.g. grand_total > 100000 → Finance Manager)

Document Comparison & Reconciliation

• Side‑by‑side comparison of the extracted invoice with a matching Purchase Order / Sales Order
• Header‑field and line‑item diff with Match / Mismatch / Missing badges
• Bank‑statement reconciliation against Payment Entry / Journal Entry
• Comparison summary used as the agent's narrative reply

Document Search

• Natural‑language search across any ERPNext DocType (Purchase Invoice, Sales Order, Customer, Item, …)
• Filters resolved from the chat — supplier, customer, date range, status, amount thresholds, company, currency
• Powered by the search_documents tool — supports paging, sort order, and a configurable result cap
• Returns a compact result table with the document name, primary fields, and submit / draft status
• Inline drill‑down: click any row to open the ERPNext record in a new tab
• Used by the agent as a building block for "find the matching PO", "is there an unpaid bill from ACME?", and "which Sales Orders are pending delivery?" turns

Document Update

• Edit any field on an existing ERPNext document directly from chat
• Powered by the update_document tool with the same role gating and confirmation flow as create_document
• Schema pre‑validation runs before the update is dispatched — wrong fieldname, value out of range, or invalid link target is reported as a friendly error
• Inline diff in the ConfirmationCard shows old value → new value for every change
• Submittable documents are amended (cancel + new version) rather than mutated in place
• Failed updates surface the underlying Frappe ValidationError / LinkValidationError with the offending field highlighted

Document Delete

• Remove draft documents (or cancel submitted ones) via the delete_document tool
• Requires explicit user confirmation by default — the dispatcher never auto‑deletes
• Role‑gated via IDP Tool Configuration → Allowed / Denied Roles (typically restricted to Finance Manager / System Manager)
• Cascading link checks: deletion is blocked when other ERPNext records reference the target, with the blocking docs listed in the agent reply
• Falls within the global Undo Window — accidental deletes can be reversed by re‑creating from the audit trail in IDP Document Log

Confirmation Card

• Frappe‑UI inspired form rendered inline in the chat
• Edits flow back to the agent → re‑validated → confirmed
• Inline green / amber / red confidence dots on child rows
• Auto‑match, Accept all suggestions, and bulk Apply to all actions
• Per‑row EXISTING / NEW status for items and taxes
• Submit, Save as Draft, Edit, Cancel actions per card

Reversibility & Audit

• Undo Window (default 5 minutes) — created drafts are deleted, submitted submittable docs are cancelled by the originating user (or System Manager)
• Full audit trail in IDP Tool Call Log and IDP Document Log
• Conversation retention with auto‑archive after N days

Plugin System

• Bundle tools (and optionally prompt templates / skills) into a single plugin namespace
• Third‑party Frappe apps register via hooks.py:

  idp_plugins = "acme_logistics.idp:AcmeLogisticsPlugin"
  

• Per‑plugin toggle, priority, and free‑form JSON config — all editable from the desk
• Per‑tool admin overrides: enable/disable, role gates, confirmation gate, token cap

Multi‑Language Output

• Extracted values translated to a target output language while preserving numbers, dates, currency codes, and proper nouns
• Output language and OCR language are independent settings
• Hindi, English, and 10+ output languages supported out of the box
• Per‑skill / per‑prompt‑template language scoping

Cost Controls

• Per‑tool Max Output Tokens cap
• Global LLM Max Output Tokens in IDP Settings
• Inline Text Budget to truncate OCR text shoved into tool results
• Vision Image Max Dim for image resize before vision calls
• Page Pre‑Pass to drop noise pages from the LLM context
• Sliding‑window summariser for long conversations
• Weekly tokens_per_extracted_field regression alert

Supported Target DocTypes (out of the box)

• Purchase Invoice
• Sales Invoice
• Quotation
• Sales Order
• Purchase Order
• Delivery Note
• Purchase Receipt
• Payment Entry
• Journal Entry
• Bank Statement (reconciliation)

Custom DocTypes can be supported by adding an IDP Extraction Template.

---

Demo

▶ Watch the demo on YouTube:

---

Prerequisites

• Frappe Framework (v16+)
• ERPNext (v16+) — required for the supported DocTypes
• Python 3.14+
• Node.js 18+
• PaddleOCR system dependencies (auto‑installed via the app's Python deps)
• An API key from at least one supported LLM provider (OpenAI, Anthropic) or a local Ollama server

---

Installation

# Get the app
bench get-app https://github.com/sanjay-kumar001/idp

# Install on your site
bench --site  install-app idp

# Run migrations
bench --site  migrate

# Build frontend assets
bench build --app idp

# Restart
bench restart

All Python dependencies (PaddleOCR, pypdf, python‑docx, openai, anthropic, …) are installed automatically with the app.

---

Configuration

1. Navigate to IDP Settings at /app/idp-settings
2. Tick Enabled.
3. Pick an LLM Provider (OpenAI / Anthropic / Ollama), paste the API key, set the default model.
4. Add at least one row per purpose in LLM Model Routes (classification, extraction, vision, summarisation, confirmation).
5. Configure OCR:
   • Default OCR language (en, hi, auto, …)
   • Default Output Language (English, हिन्दी, …)
   • Confidence Threshold, Max File Size, Max Pages per PDF
6. Choose a rollout mode:
   • Dry‑run — leave Enable Write Operations off; users see ConfirmationCards but no ERPNext records are created.
   • Live — enable write operations and (optionally) auto‑create missing masters.
7. Turn on the cost‑aware features:
   • Page Pre‑Pass, Mapper Output Cache, Summariser, Strip Thinking Blocks.
8. Set the Undo Window (default 5 minutes) and Active Retention (days).

For full field‑by‑field documentation, see docs/usermanual/idpsettings.md.

---

Usage

After setup, open the IDP chat at:

https:///idp

Drag‑and‑drop a document onto the conversation surface, or use the file picker in the composer. Then ask a question or pick a suggested prompt.

Prompt	What the agent does
"Extract this invoice into a Purchase Invoice for Tara Technologies."	OCR → mapping → ConfirmationCard for a Purchase Invoice.
"Compare this bill against PO PUR-ORD-2026-00430."	Diff header + line items vs. the PO and explain mismatches.
"Create a Sales Invoice from this PDF and translate item descriptions to Hindi."	Extract, translate text fields, render the card in Hindi.
"Resolve the missing suppliers and item codes, then create the document."	Calls resolve_masters / create_master, then create_document.
"Reconcile this bank statement against ERPNext payments for May."	Bank reconciliation engine returns matched / unmatched rows.
"Use the ACME Purchase Invoice template."	Forces a specific IDP Extraction Template for this turn.

The agent streams a natural‑language reply, renders a ConfirmationCard with editable fields, and waits for the user to Submit, Save as Draft, Edit, or Cancel.

---

Architecture Overview

``` idp/ # Python package (Frappe backend) | |-- api/ # @frappe.whitelist() endpoints | |-- conversation.py # Chat turn dispatch | |-- extract.py # OCR + mapper pipeline | |-- upload.py, llm.py, settings.py | |-- advanced.py