microdao-daarion/docs/tasks/PHASE3_ROADMAP.md

PHASE 3 ROADMAP — Core Agent Services

After Phase 2 Agent Integration

Status: 📋 Planning → ✅ SPEC READY
Master Task: PHASE3_MASTER_TASK.md ⭐
Summary: PHASE3_READY.md
Priority: High
Estimated Time: 6-8 weeks
Dependencies: Phase 2 complete

---

🎯 Goal

Replace Phase 2 stubs with production-ready services:

• Real LLM Proxy (multi-provider routing)
• Real Agent Memory (RAG + vector DB)
• Tool Registry (agent actions)
• Agent Blueprint Management (CRUD + versioning)

---

📦 Phase 3 Components

1. LLM Proxy Service (2 weeks)

Purpose: Centralized LLM gateway with routing, rate limiting, cost tracking

Features:

• Multi-provider support (OpenAI, Anthropic, DeepSeek, Local)
• Model selection & routing
• Rate limiting per agent/microDAO
• Cost tracking & billing
• Streaming support
• Error handling & retries
• Prompt sanitization

API:

POST /internal/llm/proxy
{
  "model": "gpt-4",
  "messages": [...],
  "stream": false,
  "max_tokens": 1000,
  "agent_id": "agent:sofia",
  "microdao_id": "microdao:daarion"
}

GET /internal/llm/models
→ List available models

GET /internal/llm/usage?agent_id=agent:sofia&period=30d
→ Usage statistics

Tech Stack:

• FastAPI
• httpx for provider calls
• Redis for rate limiting
• PostgreSQL for usage tracking

Files:

services/llm-proxy/
├── main.py
├── providers/
│   ├── openai.py
│   ├── anthropic.py
│   ├── deepseek.py
│   └── local.py
├── routing.py
├── rate_limiter.py
├── cost_tracker.py
├── models.py
└── config.yaml

---

2. Agent Memory Service (2 weeks)

Purpose: Persistent memory + RAG for agents

Features:

• Short-term memory (recent context)
• Mid-term memory (session/task memory)
• Long-term memory (knowledge base)
• Vector search (RAG)
• Memory indexing (from channel history)
• Memory pruning (for cost/performance)
• Per-agent & per-microDAO isolation

API:

POST /internal/agent-memory/query
{
  "agent_id": "agent:sofia",
  "microdao_id": "microdao:daarion",
  "query": "What did we discuss about Phase 2?",
  "k": 5,
  "memory_types": ["mid_term", "long_term"]
}
→ Top-k relevant memories

POST /internal/agent-memory/store
{
  "agent_id": "agent:sofia",
  "microdao_id": "microdao:daarion",
  "memory_type": "mid_term",
  "content": {
    "user_message": "...",
    "agent_reply": "...",
    "context": {...}
  }
}
→ Store new memory

GET /internal/agent-memory/agents/{agent_id}/stats
→ Memory usage stats

Tech Stack:

• FastAPI
• PostgreSQL (structured memory)
• Qdrant/Weaviate/ChromaDB (vector DB for RAG)
• LangChain/LlamaIndex (RAG helpers)

Files:

services/agent-memory/
├── main.py
├── vector_store.py
├── memory_manager.py
├── rag_engine.py
├── indexer.py
├── models.py
└── config.yaml

---

3. Tool Registry Service (1.5 weeks)

Purpose: Centralized tool definitions & execution for agents

Features:

• Tool catalog (list all available tools)
• Tool execution (secure sandbox)
• Tool permissions (agent → tool mapping)
• Tool versioning
• Execution logs & auditing

Tools (initial set):

• create_task(channel_id, title, description)
• create_followup(user_id, message_id, reminder_text, due_date)
• search_docs(query)
• create_project(microdao_id, name, description)
• summarize_channel(channel_id, period)
• send_notification(user_id, text)

API:

GET /internal/tools/catalog
→ List all tools

POST /internal/tools/execute
{
  "tool_name": "create_task",
  "agent_id": "agent:sofia",
  "microdao_id": "microdao:daarion",
  "parameters": {
    "channel_id": "...",
    "title": "Review Phase 2",
    "description": "..."
  }
}
→ Execute tool, return result

GET /internal/tools/agents/{agent_id}/permissions
→ List tools agent can use

Tech Stack:

• FastAPI
• Dynamic tool loading (plugins)
• Sandboxed execution (Docker/gVisor)
• PostgreSQL (tool definitions, permissions, logs)

Files:

services/tool-registry/
├── main.py
├── catalog.py
├── executor.py
├── sandbox.py
├── permissions.py
├── tools/
│   ├── task_tools.py
│   ├── project_tools.py
│   ├── notification_tools.py
│   └── ...
└── config.yaml

---

4. Agent Blueprint Service (1 week)

Purpose: CRUD + versioning for agent definitions

Features:

• Create/Read/Update/Delete agent blueprints
• Blueprint versioning
• Blueprint templates (archetypes)
• Blueprint validation
• Blueprint inheritance

API:

GET /internal/agents/blueprints
→ List all blueprints

POST /internal/agents/blueprints
{
  "code": "sofia_prime_v2",
  "name": "Sofia Prime v2",
  "model": "gpt-4.1",
  "instructions": "...",
  "capabilities": {...},
  "tools": ["create_task", "summarize_channel"]
}
→ Create blueprint

GET /internal/agents/blueprints/{blueprint_id}
→ Get blueprint

GET /internal/agents/{agent_id}/blueprint
→ Get blueprint for specific agent instance

PUT /internal/agents/blueprints/{blueprint_id}
→ Update blueprint (creates new version)

Tech Stack:

• FastAPI
• PostgreSQL (blueprints, versions)
• YAML/JSON schema validation

Files:

services/agents-service/
├── main.py
├── blueprints/
│   ├── crud.py
│   ├── versioning.py
│   ├── validation.py
│   └── templates.py
├── models.py
└── config.yaml

---

5. Integration Updates (1 week)

Update agent-runtime to use real services:

# Before (Phase 2):
blueprint = await load_agent_blueprint(agent_id)  # Mock
memory = await query_memory(...)  # Stub
llm_response = await generate_response(...)  # Stub

# After (Phase 3):
blueprint = await agents_service.get_blueprint(agent_id)  # Real
memory = await memory_service.query(...)  # Real RAG
llm_response = await llm_proxy.generate(...)  # Real multi-provider

# NEW: Tool usage
if llm_suggests_tool_use:
    tool_result = await tool_registry.execute(tool_name, parameters)
    # Add tool result to context, call LLM again

---

📅 Timeline

Week 1-2: LLM Proxy

• Week 1: Core routing + OpenAI provider
• Week 2: Multi-provider + rate limiting + cost tracking

Week 3-4: Agent Memory

• Week 3: Vector store setup + basic RAG
• Week 4: Memory management + indexing

Week 5-6: Tool Registry

• Week 5: Catalog + basic tools (task, followup)
• Week 6: Executor + permissions + sandboxing

Week 7: Agent Blueprint Service

• CRUD + versioning + validation

Week 8: Integration & Testing

• Update agent-runtime
• E2E testing
• Performance optimization
• Documentation

---

🧪 Testing Strategy

LLM Proxy Testing:

• Unit: Each provider (OpenAI, Anthropic, etc.)
• Integration: Rate limiting, cost tracking
• Load: 100 concurrent requests
• Failover: Provider unavailable scenarios

Agent Memory Testing:

• RAG accuracy: Retrieve relevant memories
• Memory indexing: Auto-index from channels
• Vector search performance: < 500ms
• Memory pruning: Clean old memories

Tool Registry Testing:

• Tool execution: All tools work
• Permissions: Agent cannot use unauthorized tools
• Sandboxing: Tools cannot escape sandbox
• Audit logs: All executions logged

E2E Testing:

• User asks agent to create task → Task created
• User asks agent to summarize → Summary posted
• Agent uses memory correctly in replies
• Multiple providers work (switch between OpenAI/DeepSeek)

---

🎯 Acceptance Criteria

Phase 3 Complete When:

• ✅ LLM Proxy supports 3+ providers
• ✅ Agent Memory RAG works (< 500ms queries)
• ✅ Tool Registry has 5+ working tools
• ✅ Agent Blueprint CRUD works
• ✅ agent-runtime integrated with all services
• ✅ E2E: User → Agent (with tool use) → Result
• ✅ Cost tracking shows LLM usage per agent
• ✅ Memory usage shows per agent/microDAO
• ✅ All services pass health checks
• ✅ Documentation complete

---

📊 Success Metrics

Metric	Target
LLM response time	< 2s (non-streaming)
Memory query time	< 500ms
Tool execution time	< 3s
E2E agent reply	< 5s (with tool use)
LLM cost per request	< $0.05
System uptime	> 99.5%

---

🔗 Dependencies

External Services:

• OpenAI API (for GPT-4)
• Anthropic API (for Claude, optional)
• DeepSeek API (optional)
• Qdrant/Weaviate (for vector DB)

Internal Services:

• PostgreSQL (for all structured data)
• Redis (for rate limiting, caching)
• NATS (for events)

---

💡 Optional Enhancements (Phase 3.5)

LLM Proxy:

• Streaming SSE support
• Local model support (Ollama, vLLM)
• Prompt caching
• A/B testing for prompts

Agent Memory:

• Hierarchical memory (microDAO → team → agent)
• Memory sharing between agents
• Memory snapshots (save/restore agent state)
• Memory analytics dashboard

Tool Registry:

• Tool marketplace (community tools)
• Tool composition (chain tools)
• Visual tool builder
• Tool usage analytics

---

🚀 Quick Start (After Phase 2)

To prepare for Phase 3:

# 1. Review Phase 3 roadmap
cat docs/tasks/PHASE3_ROADMAP.md

# 2. Set up external services
# - Get OpenAI API key
# - Set up Qdrant (Docker or cloud)
# - Set up Redis

# 3. Start with LLM Proxy
mkdir -p services/llm-proxy
cd services/llm-proxy
# Follow PHASE3_LLM_PROXY_TASK.md (to be created)

---

📝 Task Files (To Be Created)

After Phase 2 complete, create detailed tasks:

1. TASK_PHASE3_LLM_PROXY.md (2 weeks)
2. TASK_PHASE3_AGENT_MEMORY.md (2 weeks)
3. TASK_PHASE3_TOOL_REGISTRY.md (1.5 weeks)
4. TASK_PHASE3_BLUEPRINT_SERVICE.md (1 week)
5. TASK_PHASE3_INTEGRATION.md (1 week)

---

🎓 Architecture Evolution

Phase 1 (Complete):

User → Frontend → messaging-service → Matrix → Frontend

Phase 2 (Current):

User → Messenger → agent_filter → Router → agent-runtime (stub) → Reply

Phase 3 (Target):

User → Messenger
    ↓
agent_filter → Router → agent-runtime
    ↓
├─ LLM Proxy → [OpenAI | Anthropic | DeepSeek]
├─ Agent Memory → [Vector DB | PostgreSQL]
├─ Tool Registry → [Task | Project | Notification tools]
└─ Agent Blueprint → [Definitions | Versions]
    ↓
Reply with tool results

---

✅ Current Status

• ✅ Phase 1: Messenger Core (Complete)
• 📋 Phase 2: Agent Integration (In Progress)
• 📋 Phase 3: Core Services (This Roadmap)
• 🔜 Phase 4: Advanced Features (TBD)

---

Ready for Phase 3?

First complete Phase 2, then return to this roadmap for detailed implementation tasks.

---

Version: 1.0.0
Date: 2025-11-24
Status: Planning