daarion-admin/microdao-daarion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
97 Commits 8 Branches 0 Tags
29febee4649dda3615bc148eae6efe583adad128
Commit Graph

97 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Apple
29febee464 fix: Fix matrix_provisioning imports 2025-11-26 12:07:39 -08:00
Apple
5fdb839f5a feat: Add Matrix user provisioning on registration 
- matrix_provisioning.py: Create Matrix users via Synapse admin API
- Auto-create Matrix account when user registers in DAARION
- Return matrix_user_id in registration response
 2025-11-26 12:06:55 -08:00
Apple
064ac5af64 fix: Use direct values in Synapse config (env vars not interpolated) 2025-11-26 12:01:57 -08:00
Apple
385a9382fa fix: Add URL preview blacklist and fix metrics config 2025-11-26 12:00:06 -08:00
Apple
a1b400bf69 fix: Add report_stats to Synapse config 2025-11-26 11:58:45 -08:00
Apple
88d89249f4 fix: Change Synapse port to 8018 (8008 occupied) 2025-11-26 11:58:07 -08:00
Apple
557bfb5ee1 feat: Add Matrix infrastructure (Synapse + Element Web) 
- Synapse homeserver.yaml configuration
- Element Web config.json
- PostgreSQL init script for synapse DB
- Nginx gateway configuration
- Docker Compose for Matrix stack
- README_MATRIX.md documentation
 2025-11-26 11:56:39 -08:00
Apple
5aaf6cbf21 feat: Add Auth Service with JWT authentication 2025-11-26 11:47:00 -08:00
Apple
2c4eb7d432 feat: Add Chat, Agent Console, and Governance UI 
- WebSocket chat client for city rooms
- Chat UI components (ChatMessage, ChatInput, ChatRoom)
- Agent Console with agent list and invoke functionality
- Governance/DAO page with proposals and voting
- Updated navigation with new routes
- Extended API client for agents and microdao
 2025-11-26 09:57:50 -08:00
Apple
25b638ae3d fix: Add SSR support for API client 2025-11-26 09:26:17 -08:00
Apple
2f30f40c0b feat: Add Next.js 15 glassmorphism frontend for DAARION MVP 2025-11-26 09:20:33 -08:00
Apple
5ce1bf1035 fix: enable multimodal processing by removing early message check 
- Commented out early 'if not update.message' validation at line 723
- This was preventing photo and voice messages from being processed
- process_photo() and process_voice() functions are already implemented
- Now photo/voice updates can reach multimodal handlers before text validation
- Fixes issue where bots showed placeholder messages instead of processing media
 2025-11-24 02:51:39 -08:00
Apple
7168aaab1e fix: remove Markdown parse_mode from Telegram messages 
- Telegram API returns 400 Bad Request with invalid Markdown characters
- Switched to plain text mode (no parse_mode) to prevent errors
- Photo and voice processing now work correctly
- Fixes issue where vision-8b responses couldn't be sent to users
 2025-11-24 02:26:07 -08:00
Apple
ccebc2607d feat: gateway complex routing override for DeepSeek 
- Add auto_complex logic: if needs_complex_reasoning, set metadata["provider"] = "cloud_deepseek"
- Add context caching in MemoryClient with 5s TTL
- Add GREENFOOD agent configuration
- This enables automatic routing of complex queries to DeepSeek via explicit_provider_override rule
 2025-11-24 02:14:30 -08:00
Apple
830d13b5a2 feat: стандартизація обробки повідомлень для всіх агентів 
- Створено AgentConfig для уніфікації конфігурації агентів
- Додано універсальні функції: process_photo(), process_document(), process_voice()
- Створено handle_telegram_webhook() для будь-якого агента
- Рефакторинг telegram_webhook() та helion_telegram_webhook()
- Оновлено process_voice() для використання STT Service
- Додано реєстр агентів AGENT_REGISTRY
- Оновлено health endpoint для показу всіх агентів
- Додано інструкції для додавання нових агентів
 2025-11-21 01:50:18 -08:00
Apple
28e0103301 fix: прибрано повідомлення 'Обробляю фото' та оновлено обробку фото через Swapper vision-8b 2025-11-21 01:25:14 -08:00
Apple
9ed17ba907 fix: виправлено get_telegram_file_path для підтримки Helion токена 2025-11-21 01:17:02 -08:00
Apple
7572e45d2a feat: додано обробку фото для Helion бота через Swapper vision-8b 2025-11-21 01:16:33 -08:00
Apple
0ca44dd799 docs: оновлено infrastructure_quick_ref.ipynb - Swapper Service замість Vision Encoder 2025-11-21 00:52:35 -08:00
Apple
3fa206503e docs: оновлено документацію - Swapper Service замість Vision Encoder на Node #1 2025-11-21 00:52:00 -08:00
Apple
6d58532d68 fix: змінено обробку фото для використання Swapper vision-8b замість Vision Encoder 2025-11-21 00:50:57 -08:00
Apple
7d65aeff21 feat: додано обробку фото через Vision Encoder у gateway-bot 2025-11-21 00:37:10 -08:00
Apple
e018b9ab68 feat: додано Node Registry, GreenFood, Monitoring та Utils 2025-11-21 00:35:41 -08:00
Apple
31f3602047 feat: оновлення інфраструктури з Node #2 та нові сервіси 2025-11-21 00:35:03 -08:00
Apple
36c35cab57 fix: Виправити структуру запиту Helion до Router (context на верхньому рівні) 2025-11-17 13:23:41 -08:00
Apple
8c3c0726b4 fix: move helion config to agents section, remove duplicate from policies 2025-11-17 11:46:10 -08:00
Apple
df1cd614cd docs: update GPU specs (RTX 4000 Ada) and Vision Encoder performance with GPU acceleration 2025-11-17 11:39:31 -08:00
Apple
a945b42ba0 feat: re-enable GPU support for Vision Encoder after nvidia-container-toolkit installation 2025-11-17 06:45:54 -08:00
Apple
1fc2f7cfee fix: correct provider field name in smoke test 2025-11-17 06:32:08 -08:00
Apple
b521c9cfc5 fix: register vision_encoder from orchestrator_providers config 2025-11-17 06:14:08 -08:00
Apple
212eb0a5a0 feat: add vision_encoder to orchestrator_providers 2025-11-17 06:11:51 -08:00
Apple
106358417a fix: remove GPU requirement from Vision Encoder for CPU fallback 2025-11-17 05:51:11 -08:00
Apple
6f28171842 docs: add quick deployment guide for Vision Encoder 
- One-command deploy via automated script
- Manual step-by-step deployment
- Verification checklist
- Troubleshooting guide
- Expected results and GPU usage
- Next steps after deployment
 2025-11-17 05:26:52 -08:00
Apple
12ce1fa9b9 feat: add automated deployment script for Vision Encoder 
- 17-step deployment automation
- GPU and Docker GPU runtime checks
- Automatic backup before deployment
- Vision Encoder + Qdrant deployment
- Health checks and smoke tests
- GPU monitoring
- Colorized output with status indicators

Usage: ssh root@144.76.224.179 'cd /opt/microdao-daarion && ./deploy-vision-encoder.sh'
 2025-11-17 05:25:49 -08:00
Apple
4601c6fca8 feat: add Vision Encoder service + Vision RAG implementation 
- Vision Encoder Service (OpenCLIP ViT-L/14, GPU-accelerated)
  - FastAPI app with text/image embedding endpoints (768-dim)
  - Docker support with NVIDIA GPU runtime
  - Port 8001, health checks, model info API

- Qdrant Vector Database integration
  - Port 6333/6334 (HTTP/gRPC)
  - Image embeddings storage (768-dim, Cosine distance)
  - Auto collection creation

- Vision RAG implementation
  - VisionEncoderClient (Python client for API)
  - Image Search module (text-to-image, image-to-image)
  - Vision RAG routing in DAGI Router (mode: image_search)
  - VisionEncoderProvider integration

- Documentation (5000+ lines)
  - SYSTEM-INVENTORY.md - Complete system inventory
  - VISION-ENCODER-STATUS.md - Service status
  - VISION-RAG-IMPLEMENTATION.md - Implementation details
  - vision_encoder_deployment_task.md - Deployment checklist
  - services/vision-encoder/README.md - Deployment guide
  - Updated WARP.md, INFRASTRUCTURE.md, Jupyter Notebook

- Testing
  - test-vision-encoder.sh - Smoke tests (6 tests)
  - Unit tests for client, image search, routing

- Services: 17 total (added Vision Encoder + Qdrant)
- AI Models: 3 (qwen3:8b, OpenCLIP ViT-L/14, BAAI/bge-m3)
- GPU Services: 2 (Vision Encoder, Ollama)
- VRAM Usage: ~10 GB (concurrent)

Status: Production Ready ✅
 2025-11-17 05:24:36 -08:00
Apple
b2b51f08fb Add comprehensive Cursor task for fixing Telegram bots routing and system prompts 2025-11-16 16:45:18 -08:00
Ivan Tytar
765956ab09 Add debug logging to Router llm_provider 2025-11-17 00:40:50 +01:00
Ivan Tytar
6dc35500e5 Fix Helion system_prompt: correct payload.context structure 
Changed from nested 'payload': {'context': {
to proper 'payload': { 'context': { structure
so Router can extract system_prompt correctly
 2025-11-17 00:30:18 +01:00
Ivan Tytar
90168f1bbe Add DAARWIZZ_TELEGRAM_BOT_TOKEN to gateway environment 2025-11-17 00:27:50 +01:00
Ivan Tytar
d110d42195 Fix: separate Telegram bot tokens for DAARWIZZ and Helion 
- Modified send_telegram_message() to accept bot_token parameter
- DAARWIZZ webhook now uses DAARWIZZ_TELEGRAM_BOT_TOKEN
- Helion webhook now uses HELION_TELEGRAM_BOT_TOKEN
- Fixes issue where DAARWIZZ messages were sent via Helion bot
 2025-11-17 00:22:20 +01:00
Ivan Tytar
e90409e2e9 Simplify Cursor task: only fix indent in router_request 2025-11-17 00:07:49 +01:00
Ivan Tytar
116016777c Add Cursor task: fix Helion system_prompt integration 2025-11-17 00:02:31 +01:00
Apple
16630acf0e feat: add memory-service and postgres for Helion agent 
- Add PostgreSQL database in docker-compose.yml
- Add memory-service with health checks
- Create database schema (init.sql) with user_facts, dialog_summaries, agent_memory_events
- Add deployment documentation (DEPLOY-NOW.md)
- Add status reports and troubleshooting guides
 2025-11-16 12:52:23 -08:00
Apple
730b46a2b4 feat: Add automated HTTPS gateway setup for agent webhooks 
- Add setup-nginx-gateway.sh for Let's Encrypt + nginx
- Add register-agent-webhook.sh for easy webhook registration
- Update HELION-QUICKSTART.md with production deployment guide
- Supports unlimited agents without nginx reconfiguration
 2025-11-16 11:13:40 -08:00
Apple
66f19cf499 feat: Add Helion AI agent for Energy Union 
- Add Helion system prompt with 9 user modes
- Add /helion/telegram/webhook endpoint
- Add Helion configuration to docker-compose.yml
- Add HELION-QUICKSTART.md documentation
- Update .env.example with Helion variables

Helion provides technical support for EcoMiner/BioMiner, tokenomics (ENERGY, 1T, kWt), and DAO governance with RAG verification and 4-level compliance.
 2025-11-16 09:30:09 -08:00
Apple
df6000f9d7 fix: export build_memory_section function for fallback prompt 2025-11-16 05:14:40 -08:00
Apple
1ed1181105 feat: add RAG quality metrics, optimized prompts, and evaluation tools 
Optimized Prompts:
- Create utils/rag_prompt_builder.py with citation-optimized prompts
- Specialized for DAO tokenomics and technical documentation
- Proper citation format [1], [2] with doc_id, page, section
- Memory context integration (facts, events, summaries)
- Token count estimation

RAG Service Metrics:
- Add comprehensive logging in query_pipeline.py
- Log: question, doc_ids, scores, retrieval method, timing
- Track: retrieval_time, total_query_time, documents_found, citations_count
- Add metrics in ingest_pipeline.py: pages_processed, blocks_processed, pipeline_time

Router Improvements:
- Use optimized prompt builder in _handle_rag_query()
- Add graceful fallback: if RAG unavailable, use Memory only
- Log prompt token count, RAG usage, Memory usage
- Return detailed metadata (rag_used, memory_used, citations_count, metrics)

Evaluation Tools:
- Create tests/rag_eval.py for systematic quality testing
- Test fixed questions with expected doc_ids
- Save results to JSON and CSV
- Compare RAG Service vs Router results
- Track: citations, expected docs found, query times

Documentation:
- Create docs/RAG_METRICS_PLAN.md
- Plan for Prometheus metrics collection
- Grafana dashboard panels and alerts
- Implementation guide for metrics
 2025-11-16 05:12:19 -08:00
Apple
382e661f1f feat: complete RAG pipeline integration (ingest + query + Memory) 
Parser Service:
- Add /ocr/ingest endpoint (PARSER → RAG in one call)
- Add RAG_BASE_URL and RAG_TIMEOUT to config
- Add OcrIngestResponse schema
- Create file_converter utility for PDF/image → PNG bytes
- Endpoint accepts file, dao_id, doc_id, user_id
- Automatically parses with dots.ocr and sends to RAG Service

Router Integration:
- Add _handle_rag_query() method in RouterApp
- Combines Memory + RAG → LLM pipeline
- Get Memory context (facts, events, summaries)
- Query RAG Service for documents
- Build prompt with Memory + RAG documents
- Call LLM provider with combined context
- Return answer with citations

Clients:
- Create rag_client.py for Router (query RAG Service)
- Create memory_client.py for Router (get Memory context)

E2E Tests:
- Create e2e_rag_pipeline.sh script for full pipeline test
- Test ingest → query → router query flow
- Add E2E_RAG_README.md with usage examples

Docker:
- Add RAG_SERVICE_URL and MEMORY_SERVICE_URL to router environment
 2025-11-16 05:02:14 -08:00
Apple
6d69f901f7 fix: add volumes section for rag-model-cache in docker-compose 2025-11-16 04:42:07 -08:00
Apple
9b86f9a694 feat: implement RAG Service MVP with PARSER + Memory integration 
RAG Service Implementation:
- Create rag-service/ with full structure (config, document_store, embedding, pipelines)
- Document Store: PostgreSQL + pgvector via Haystack
- Embedding: BAAI/bge-m3 (multilingual, 1024 dim)
- Ingest Pipeline: Convert ParsedDocument to Haystack Documents, embed, index
- Query Pipeline: Retrieve documents, generate answers via DAGI Router
- FastAPI endpoints: /ingest, /query, /health

Tests:
- Unit tests for ingest and query pipelines
- E2E test with example parsed JSON
- Test fixtures with real PARSER output example

Router Integration:
- Add mode='rag_query' routing rule in router-config.yml
- Priority 7, uses local_qwen3_8b for RAG queries

Docker:
- Add rag-service to docker-compose.yml
- Configure dependencies (router, city-db)
- Add model cache volume

Documentation:
- Complete README with API examples
- Integration guides for PARSER and Router
 2025-11-16 04:41:53 -08:00