Ai

bench get-app https://github.com/theapprenticeproject/Ai

[![Listed on Frappe Gems](https://frappegems.com/api/method/frappe_gems.seo.badge?app=theapprenticeproject%2FAi)](https://frappegems.com/gems/apps/theapprenticeproject/Ai)

# TAP AI - Conversational AI Engine This project extends the TAP AI Frappe application with a powerful, conversational AI layer. It provides a single, robust API endpoint that can understand user questions and intelligently route them to the best tool - a curated knowledge bank, a direct database query, a semantic vector search, or a direct LLM fallback - to provide accurate, context-aware answers. The system is designed for multi-turn conversations, automatically managing chat history to understand follow-up questions. It features **asynchronous processing via RabbitMQ workers**, **voice input/output support**, and **dynamic configuration management** for seamless integration with TAP LMS. Current deployment topology: - **AI application server:** `ai.evalix.xyz` (hosts TAP AI code and workers) - **Remote database server:** `data.evalix.xyz` (PostgreSQL) ## 📋 Table of Contents - [Project Overview](#-project-overview) - [Core Architecture](#-core-architecture) - [System Workflow](#-system-workflow) - [Complete Codebase Structure](#-complete-codebase-structure) - [Dependencies](#-dependencies) - [Installation](#-installation) - [Configuration](#-configuration) - [One-Time Setup](#-one-time-setup) - [Testing](#-testing) - [API Documentation](#-api-documentation) - [Worker System](#-worker-system) - [Core File Descriptions](#-core-file-descriptions) - [Telegram Bot Demo](#-telegram-bot-demo-local-setup) - [Deployment Guide](#-deployment-guide) - [Troubleshooting](#-troubleshooting) --- ## 🎯 Project Overview **TAP AI** is a conversational AI engine built on top of the Frappe framework. It intelligently routes user queries to specialized execution engines: - **Knowledge Bank Tool**: For curated TAP responses, greetings, short support phrases, and high-confidence conversational matches - **Text-to-SQL Engine**: For factual, database-specific queries - **Vector RAG Engine**: For conceptual, semantic, and summarization queries - **Direct LLM Tool**: For open-ended conversation when no knowledge-bank entry fits - **RabbitMQ Worker Architecture**: Asynchronous processing for scalability - **Voice Processing**: STT → LLM → TTS pipeline for voice queries **Key Features:** - Intelligent routing using LLMs - Multi-turn conversation support with history management - Hybrid query execution (Knowledge Bank + SQL + Vector Search + Direct LLM) - Automatic fallback mechanisms with confidence thresholds - Telegram bot integration - Rate limiting and authentication built-in - Voice input/output support via Telegram - Asynchronous processing with RabbitMQ - Dynamic configuration for TAP LMS integration - Admin-controlled DocType exclusion system ## 🛠 Recent Updates - Added a hybrid Knowledge Bank verifier: the system now probes the best KB candidate and asks the LLM to verify whether the candidate appropriately answers the user's query; the LLM either returns the KB response (optionally lightly personalized) or generates a fresh answer. This reduces false positives (e.g., distinguishing "who are you" vs "how are you"). - Router now uses the hybrid verifier for `knowledge_bank` routed queries. - DocType event hooks invalidate the KB cache on insert/update/delete to keep the KB context fresh. - A verifier LLM cache is used to reduce latency for repeated verification queries (TTL configurable). **Technology Stack:** - **Backend**: Python 3.10+ - **Framework**: Frappe (ERPNext) - **LLM**: OpenAI GPT models - **Vector DB**: Pinecone - **Database**: Remote PostgreSQL (`data.evalix.xyz`) - **Message Queue**: RabbitMQ (Pika) - **Caching**: Redis - **Web Framework**: Flask (for Telegram webhooks) - **ORM**: SQLAlchemy **Language Composition:** - **Python**: 107,850 bytes (99%) - **JavaScript**: 564 bytes (1%) --- ## 🚀 Core Architecture The system's intelligence lies in its central router, which acts as a decision-making brain. When a query is received, it follows this flow: 1. **Intelligent Routing:** An LLM analyzes the user's query to determine its intent. 2. **Tool Selection:** - For short, curated conversational intents that match the TAP response bank, it selects the **Knowledge Bank Tool**. - For factual, specific questions (e.g., "list all...", "how many..."), it selects the **Text-to-SQL Engine**. - For conceptual, open-ended, or summarization questions (e.g., "summarize...", "explain..."), it selects the **Vector RAG Engine**. - For open-ended supportive conversation that does not fit the knowledge bank, it selects the **Direct LLM Tool**. 3. **Execution & Fallback:** The chosen tool executes the query. If the knowledge bank misses or returns a low-confidence match, the system falls back to the Direct LLM tool. If SQL fails to produce a satisfactory answer, the system automatically falls back to the Vector RAG engine as a safety net. 4. **Answer Synthesis:** The retrieved data or direct response is returned as a final, human-readable answer. ### System Flow Diagram ```mermaid graph TD subgraph "User Input" User[User Query] end subgraph "API Layer" QueryAPI["api/query.py
Unified Query API (Text + Voice)"] end subgraph "Message Queue" RabbitMQ["RabbitMQ
Message Broker"] end subgraph "Worker Processes" STTWorker["workers/stt_worker.py
Speech-to-Text"] LLMWorker["workers/llm_worker.py
LLM Router"] TTSWorker["workers/tts_worker.py
Text-to-Speech"] end subgraph "Services" Router["services/router.py
Intelligent Router"] KB["services/direct_response_bank.py
Knowledge Bank"] SQL["services/sql_answerer.py
SQL Engine"] RAG["services/rag_answerer.py
RAG Engine"] Direct["services/direct_answerer.py
Direct LLM"] end subgraph "Data Layer" PostgresDB[(Remote PostgreSQL
data.evalix.xyz)] PineconeDB[(Pinecone
Vector DB)] end User -->|Text or Voice| QueryAPI QueryAPI -->|Request| RabbitMQ RabbitMQ -->|audio_stt_queue| STTWorker RabbitMQ -->|text_query_queue| LLMWorker RabbitMQ -->|audio_tts_queue| TTSWorker STTWorker -->|Transcribed Text| RabbitMQ LLMWorker -->|Route Query| Router Router -->|Curated Match| KB KB -->|High confidence| Router KB -->|Miss / low confidence| Direct Router -->|Factual| SQL Router -->|Conceptual| RAG Router -->|Open-ended support| Direct SQL -->|SQL Query| PostgresDB RAG -->|Vector Search| PineconeDB LLMWorker -->|Answer| TTSWorker TTSWorker -->|Audio File| PostgresDB ``` ### ⚙️ Engine Robustness The robustness of the system comes from the specialized design of each engine. #### Text-to-SQL Engine: From Query to Structured Data This engine excels at factual queries because it builds an "intelligent schema" before prompting the LLM. ```mermaid graph TD A[User Query] --> B["1. Inspect Live Frappe Metadata"] B --> C["2. Create Rich Schema Prompt"] C --> D{LLM: Generate SQL} D --> E[Remote PostgreSQL data.evalix.xyz] E --> F[Structured Data Rows] ``` #### Vector RAG Engine: From Query to Rich Context This engine excels at conceptual queries by retrieving semantically relevant documents. #### Knowledge Bank Tool: From Curated Phrase to Direct Answer This tool handles short, high-confidence conversational intents like greetings, acknowledgements, simple help requests, identity questions, and other curated TAP response patterns. ```mermaid graph TD A[User Query] --> B[Normalize and score against curated entries] B --> C{Confidence threshold met?} C -->|Yes| D[Return stored TAP response] C -->|No| E[Fallback to Direct LLM] ``` ```mermaid graph TD A[User Query + Chat History] --> B{LLM: Refine Query} B --> C["1. Select DocTypes"] C --> D["2. Semantic Search"] D --> E["3. Fetch Full Text"] E --> F[Rich Context Chunks] ``` --- ## 📁 Complete Codebase Structure ``` tap_ai/ ├── __init__.py # Package initialization ├── hooks.py # Frappe hooks for app lifecycle ├── modules.txt # Module declaration ├── patches.txt # Database migration patches │ ├── api/ # REST API Endpoints │ ├── __init__.py │ ├── query.py # Unified query endpoint (text + voice, async via RabbitMQ) │ ├── result.py # Unified result polling endpoint (with optional server-side wait) │ ├── voice_query.py # Backward-compatible wrapper alias for unified query │ └── voice_result.py # Backward-compatible wrapper alias for unified result │ ├── services/ # Core execution engines │ ├── __init__.py │ ├── router.py # Intelligent router (brain of system) │ ├── sql_answerer.py # Text-to-SQL engine │ ├── rag_answerer.py # Vector RAG engine │ ├── doctype_selector.py # DocType selection for RAG │ ├── pinecone_store.py # Pinecone vector database integration │ ├── pinecone_index.py # Pinecone index lifecycle │ └── ratelimit.py # API rate limiting utility │ ├── workers/ # RabbitMQ Background Workers │ ├── llm_worker.py # Main LLM routing worker │ ├── stt_worker.py # Speech-to-Text worker (Whisper) │ └── tts_worker.py # Text-to-Speech worker (OpenAI TTS) │ ├── schema/ # Database schema generation │ ├── __init__.py │ ├── generate_schema.py # Schema generator script │ └── tap_ai_schema.json # Generated schema file │ ├── infra/ # Infrastructure utilities │ ├── __init__.py │ ├── config.py # Centralized config loader │ └── sql_catalog.py # Schema catalog loader │ ├── utils/ # Utility functions │ ├── __init__.py │ ├── dynamic_config.py # Dynamic config for TAP LMS integration │ ├── remote_db.py # Remote PostgreSQL connection helpers │ └── mq.py # RabbitMQ publisher utility │ ├── config/ # Frappe app configuration │ └── __init__.py │ ├── public/ # Static assets │ └── .gitkeep │ ├── templates/ # Frappe templates │ ├── __init__.py │ └── pages/ │ └── tap_ai/ # Additional modules (if any) # Root-level files ├── README.md # This file ├── requirements.txt # Python dependencies ├── pyproject.toml # Project metadata & build config ├── license.txt # License information ├── .env # Local environment variables (do not commit secrets) ├── .gitignore # Git ignore rules ├── .vscode/ # VS Code workspace settings ├── .eslintrc # ESLint configuration ├── .editorconfig # Editor configuration ├── .pre-commit-config.yaml # Pre-commit hooks ├── __init__.py # Root package init └── telegram_webhook.py # Telegram bot bridge script ``` --- ## 📦 Dependencies ### Core Dependencies #### Database & ORM - `psycopg2-binary` (or equivalent) - PostgreSQL database driver for remote DB access - `sqlalchemy>=2.0.32` - SQL toolkit and ORM - `sqlalchemy-utils>=0.41.2` - SQLAlchemy utility functions #### LLM & RAG - `openai>=1.40.0` - OpenAI API client (GPT,