Apple
|
382e661f1f
|
feat: complete RAG pipeline integration (ingest + query + Memory)
Parser Service:
- Add /ocr/ingest endpoint (PARSER → RAG in one call)
- Add RAG_BASE_URL and RAG_TIMEOUT to config
- Add OcrIngestResponse schema
- Create file_converter utility for PDF/image → PNG bytes
- Endpoint accepts file, dao_id, doc_id, user_id
- Automatically parses with dots.ocr and sends to RAG Service
Router Integration:
- Add _handle_rag_query() method in RouterApp
- Combines Memory + RAG → LLM pipeline
- Get Memory context (facts, events, summaries)
- Query RAG Service for documents
- Build prompt with Memory + RAG documents
- Call LLM provider with combined context
- Return answer with citations
Clients:
- Create rag_client.py for Router (query RAG Service)
- Create memory_client.py for Router (get Memory context)
E2E Tests:
- Create e2e_rag_pipeline.sh script for full pipeline test
- Test ingest → query → router query flow
- Add E2E_RAG_README.md with usage examples
Docker:
- Add RAG_SERVICE_URL and MEMORY_SERVICE_URL to router environment
|
2025-11-16 05:02:14 -08:00 |
|
Apple
|
49272b66e6
|
feat: add RAG converter utilities and update integration guide
RAG Converter:
- Create app/utils/rag_converter.py with conversion functions
- parsed_doc_to_haystack_docs() - convert ParsedDocument to Haystack format
- parsed_chunks_to_haystack_docs() - convert ParsedChunk list to Haystack
- validate_parsed_doc_for_rag() - validate required fields before conversion
- Automatic metadata extraction (dao_id, doc_id, page, block_type)
- Preserve optional fields (bbox, section, reading_order)
Integration Guide:
- Update with ready-to-use converter functions
- Add validation examples
- Complete workflow examples
|
2025-11-16 03:03:20 -08:00 |
|