Commit Graph

21 Commits

Author SHA1 Message Date
Apple
5290287058 feat: implement TTS, Document processing, and Memory Service /facts API
- TTS: xtts-v2 integration with voice cloning support
- Document: docling integration for PDF/DOCX/PPTX processing
- Memory Service: added /facts/upsert, /facts/{key}, /facts endpoints
- Added required dependencies (TTS, docling)
2026-01-17 08:16:37 -08:00
Apple
85a785bd86 fix(compose): use external postgres volume + city-service ENV
- docker-compose.yml: postgres uses microdao-daarion_postgres_data (external)
- docker-compose.yml: city-service connects to dagi-postgres:5432/daarion + NATS_URL
- docker-compose.city-space.yml: added dagi-postgres, dagi-nats as services
- docs/DEPLOY_ON_SERVER.md: added restart instructions without data loss

Closes TASK_PHASE_DOCKER_COMPOSE_CITY_SERVICE_POSTGRES
2025-11-30 06:58:02 -08:00
Apple
7b91c8e83c feat(foundation): FOUNDATION_UPDATE implementation
## Documentation (20 files)
- DAARION Ontology Core v1 (Agent → MicroDAO → Node → District)
- User Onboarding & Identity Layer (DAIS)
- Data Model UPDATE, Event Catalog, Governance & Permissions
- Rooms Layer, City/MicroDAO/Agents/Nodes Interface Architecture
- Helper files: ontology-summary, lifecycles, event-schemas

## Database Migration (027)
- DAIS tables: dais_identities, dais_emails, dais_wallets, dais_keys
- agent_assignments table for Assignment Layer
- rooms table for Rooms Layer
- event_outbox for NATS event delivery
- New enums: agent_role, microdao_type, node_kind, node_status, etc.
- Updated agents, microdaos, nodes tables with ontology fields

## Backend
- DAIS service & routes (/api/v1/dais/*)
- Assignment service & routes (/api/v1/assignments/*)
- Domain types for DAIS and Ontology

## Frontend
- Ontology types (Agent, MicroDAO, Node, DAIS, Assignments)
- API clients for DAIS and Assignments
- UI components: DaisProfileCard, AssignmentsPanel, OntologyBadge

Non-breaking update - all existing functionality preserved.
2025-11-29 15:24:38 -08:00
Apple
a6e531a098 fix: NODE1_REPAIR - healthchecks, dependencies, SSR env, telegram gateway
TASK_PHASE_NODE1_REPAIR:
- Fix daarion-web SSR: use CITY_API_BASE_URL instead of 127.0.0.1
- Fix auth API routes: use AUTH_API_URL env var
- Add wget to Dockerfiles for healthchecks (stt, ocr, web-search, swapper, vector-db, rag)
- Update healthchecks to use wget instead of curl
- Fix vector-db-service: update torch==2.4.0, sentence-transformers==2.6.1
- Fix rag-service: correct haystack imports for v2.x
- Fix telegram-gateway: remove msg.ack() for non-JetStream NATS
- Add /health endpoint to nginx mvp-routes.conf
- Add room_role, is_public, sort_order columns to city_rooms migration
- Add TASK_PHASE_NODE1_REPAIR.md and DEPLOY_NODE1_REPAIR.md docs

Previous tasks included:
- TASK 039-044: Orchestrator rooms, Matrix chat cleanup, CrewAI integration
2025-11-29 05:17:08 -08:00
Apple
3de3c8cb36 feat: Add presence heartbeat for Matrix online status
- matrix-gateway: POST /internal/matrix/presence/online endpoint
- usePresenceHeartbeat hook with activity tracking
- Auto away after 5 min inactivity
- Offline on page close/visibility change
- Integrated in MatrixChatRoom component
2025-11-27 00:19:40 -08:00
Apple
a945b42ba0 feat: re-enable GPU support for Vision Encoder after nvidia-container-toolkit installation 2025-11-17 06:45:54 -08:00
Apple
106358417a fix: remove GPU requirement from Vision Encoder for CPU fallback 2025-11-17 05:51:11 -08:00
Apple
4601c6fca8 feat: add Vision Encoder service + Vision RAG implementation
- Vision Encoder Service (OpenCLIP ViT-L/14, GPU-accelerated)
  - FastAPI app with text/image embedding endpoints (768-dim)
  - Docker support with NVIDIA GPU runtime
  - Port 8001, health checks, model info API

- Qdrant Vector Database integration
  - Port 6333/6334 (HTTP/gRPC)
  - Image embeddings storage (768-dim, Cosine distance)
  - Auto collection creation

- Vision RAG implementation
  - VisionEncoderClient (Python client for API)
  - Image Search module (text-to-image, image-to-image)
  - Vision RAG routing in DAGI Router (mode: image_search)
  - VisionEncoderProvider integration

- Documentation (5000+ lines)
  - SYSTEM-INVENTORY.md - Complete system inventory
  - VISION-ENCODER-STATUS.md - Service status
  - VISION-RAG-IMPLEMENTATION.md - Implementation details
  - vision_encoder_deployment_task.md - Deployment checklist
  - services/vision-encoder/README.md - Deployment guide
  - Updated WARP.md, INFRASTRUCTURE.md, Jupyter Notebook

- Testing
  - test-vision-encoder.sh - Smoke tests (6 tests)
  - Unit tests for client, image search, routing

- Services: 17 total (added Vision Encoder + Qdrant)
- AI Models: 3 (qwen3:8b, OpenCLIP ViT-L/14, BAAI/bge-m3)
- GPU Services: 2 (Vision Encoder, Ollama)
- VRAM Usage: ~10 GB (concurrent)

Status: Production Ready 
2025-11-17 05:24:36 -08:00
Ivan Tytar
90168f1bbe Add DAARWIZZ_TELEGRAM_BOT_TOKEN to gateway environment 2025-11-17 00:27:50 +01:00
Apple
16630acf0e feat: add memory-service and postgres for Helion agent
- Add PostgreSQL database in docker-compose.yml
- Add memory-service with health checks
- Create database schema (init.sql) with user_facts, dialog_summaries, agent_memory_events
- Add deployment documentation (DEPLOY-NOW.md)
- Add status reports and troubleshooting guides
2025-11-16 12:52:23 -08:00
Apple
66f19cf499 feat: Add Helion AI agent for Energy Union
- Add Helion system prompt with 9 user modes
- Add /helion/telegram/webhook endpoint
- Add Helion configuration to docker-compose.yml
- Add HELION-QUICKSTART.md documentation
- Update .env.example with Helion variables

Helion provides technical support for EcoMiner/BioMiner, tokenomics (ENERGY, 1T, kWt), and DAO governance with RAG verification and 4-level compliance.
2025-11-16 09:30:09 -08:00
Apple
382e661f1f feat: complete RAG pipeline integration (ingest + query + Memory)
Parser Service:
- Add /ocr/ingest endpoint (PARSER → RAG in one call)
- Add RAG_BASE_URL and RAG_TIMEOUT to config
- Add OcrIngestResponse schema
- Create file_converter utility for PDF/image → PNG bytes
- Endpoint accepts file, dao_id, doc_id, user_id
- Automatically parses with dots.ocr and sends to RAG Service

Router Integration:
- Add _handle_rag_query() method in RouterApp
- Combines Memory + RAG → LLM pipeline
- Get Memory context (facts, events, summaries)
- Query RAG Service for documents
- Build prompt with Memory + RAG documents
- Call LLM provider with combined context
- Return answer with citations

Clients:
- Create rag_client.py for Router (query RAG Service)
- Create memory_client.py for Router (get Memory context)

E2E Tests:
- Create e2e_rag_pipeline.sh script for full pipeline test
- Test ingest → query → router query flow
- Add E2E_RAG_README.md with usage examples

Docker:
- Add RAG_SERVICE_URL and MEMORY_SERVICE_URL to router environment
2025-11-16 05:02:14 -08:00
Apple
6d69f901f7 fix: add volumes section for rag-model-cache in docker-compose 2025-11-16 04:42:07 -08:00
Apple
9b86f9a694 feat: implement RAG Service MVP with PARSER + Memory integration
RAG Service Implementation:
- Create rag-service/ with full structure (config, document_store, embedding, pipelines)
- Document Store: PostgreSQL + pgvector via Haystack
- Embedding: BAAI/bge-m3 (multilingual, 1024 dim)
- Ingest Pipeline: Convert ParsedDocument to Haystack Documents, embed, index
- Query Pipeline: Retrieve documents, generate answers via DAGI Router
- FastAPI endpoints: /ingest, /query, /health

Tests:
- Unit tests for ingest and query pipelines
- E2E test with example parsed JSON
- Test fixtures with real PARSER output example

Router Integration:
- Add mode='rag_query' routing rule in router-config.yml
- Priority 7, uses local_qwen3_8b for RAG queries

Docker:
- Add rag-service to docker-compose.yml
- Configure dependencies (router, city-db)
- Add model cache volume

Documentation:
- Complete README with API examples
- Integration guides for PARSER and Router
2025-11-16 04:41:53 -08:00
Apple
ca05c91799 feat: complete dots.ocr integration with deployment setup
Model Loader:
- Update model_loader.py with complete dots.ocr loading code
- Proper device detection (CUDA/CPU/MPS) with fallback
- Memory optimization (low_cpu_mem_usage)
- Better error handling and logging
- Support for local model paths and HF Hub

Docker:
- Multi-stage Dockerfile (CPU/CUDA builds)
- docker-compose.yml for parser-service
- .dockerignore for clean builds
- Model cache volume for persistence

Configuration:
- Support DOTS_OCR_MODEL_ID and DEVICE env vars (backward compatible)
- Better defaults and environment variable handling

Deployment:
- Add DEPLOYMENT.md with detailed instructions
- Local deployment (venv)
- Docker Compose deployment
- Ollama runtime setup
- Troubleshooting guide

Integration:
- Add parser-service to main docker-compose.yml
- Configure volumes and networks
- Health checks and dependencies
2025-11-16 03:00:01 -08:00
Apple
e0cb3ddbdb refactor: rewrite STT service to use qwen3_asr_toolkit Python API
- Replace Whisper subprocess calls with direct qwen3_asr_toolkit API
- Remove subprocess dependencies, use pure Python API
- Update to use DASHSCOPE_API_KEY instead of WHISPER_MODEL
- Cleaner code without CLI calls
- Better Ukrainian language recognition quality
2025-11-15 12:55:21 -08:00
Apple
65e33add81 feat: add STT service for voice message recognition
- Add STT service with Whisper support (faster-whisper, whisper CLI, OpenAI API)
- Update Gateway to handle Telegram voice/audio/video_note messages
- Add STT service to docker-compose.yml
- Gateway now converts voice → text → DAGI Router → text response
2025-11-15 12:43:41 -08:00
Apple
d50765f2ff feat: add memory-service to docker-compose.yml
- Add memory-service as a service (not under networks:)
- Create Dockerfile for memory-service
- Configure depends_on city-db with healthcheck condition
- Set DATABASE_URL to connect to city-db
2025-11-15 11:40:07 -08:00
Apple
802e092e5b fix: move city-db service under services: section in docker-compose.yml 2025-11-15 11:35:04 -08:00
Ivan Tytar
df0055bb03 fix: Docker deployment - fix Dockerfiles, registry.py f-strings, Gateway app structure 2025-11-15 15:59:42 +01:00
Ivan Tytar
3cacf67cf5 feat: Initial commit - DAGI Stack v0.2.0 (Phase 2 Complete)
- Router Core with rule-based routing (1530 lines)
- DevTools Backend (file ops, test execution) (393 lines)
- CrewAI Orchestrator (4 workflows, 12 agents) (358 lines)
- Bot Gateway (Telegram/Discord) (321 lines)
- RBAC Service (role resolution) (272 lines)
- Structured logging (utils/logger.py)
- Docker deployment (docker-compose.yml)
- Comprehensive documentation (57KB)
- Test suites (41 tests, 95% coverage)
- Phase 4 roadmap & ecosystem integration plans

Production-ready infrastructure for DAARION microDAOs.
2025-11-15 14:35:24 +01:00