microdao-daarion/SCENARIOS.md

DAGI Stack Golden Scenarios

Production test scenarios for validating end-to-end functionality.

---

🎯 Scenario 1: Basic Chat (Telegram → Router → LLM)

Objective: Verify basic LLM routing with RBAC context injection.

Setup

• User: tg:12345 (member role in greenfood-dao)
• Mode: chat
• Expected: LLM response with DAO context

Steps

1. Send message in Telegram

   Привіт! Що це за DAO?
   

2. Expected flow

   Telegram → Gateway (:9300)
              ↓ (enrich with dao_id, user_id)
           Router (:9102)
              ↓ (fetch RBAC context)
           RBAC (:9200)
              ↓ (inject roles: member, entitlements: 4)
           LLM Provider (Ollama :11434)
              ↓ (generate response with context)
           Response to user
   

3. Verify in logs

   docker-compose logs gateway | grep "tg:12345"
   docker-compose logs router | grep "mode=chat"
   docker-compose logs rbac | grep "greenfood-dao"
   

4. Expected response

   • Contains DAO name or context
   • Response time < 5s (if local LLM)
   • No errors in logs

Success Criteria

• ✅ Message received by Gateway
• ✅ Request routed to correct LLM provider
• ✅ RBAC context injected (role: member, entitlements: 4)
• ✅ Response delivered to user
• ✅ Structured logs show full trace (request_id)

---

🚀 Scenario 2: microDAO Onboarding (CrewAI Workflow)

Objective: Validate multi-agent workflow orchestration.

Setup

• User: tg:newcomer001 (guest role)
• Mode: crew
• Workflow: microdao_onboarding

Steps

1. Trigger onboarding

   curl -X POST http://localhost:9102/route \
     -H "Content-Type: application/json" \
     -d '{
       "prompt": "Onboard new member",
       "mode": "crew",
       "metadata": {
         "workflow": "microdao_onboarding",
         "dao_id": "greenfood-dao",
         "user_id": "tg:newcomer001"
       }
     }'
   

2. Expected flow

   Request → Router
              ↓ (match rule: crew_mode, priority 5)
           CrewAI Orchestrator (:9010)
              ↓ (run microdao_onboarding)
           3 Agents:
              - welcomer (greet new member)
              - role_assigner (suggest role)
              - guide (provide next steps)
              ↓
           Response (workflow result + metadata)
   

3. Verify workflow execution

   docker-compose logs crewai | grep "microdao_onboarding"
   docker-compose logs router | grep "use_provider: microdao_orchestrator"
   

4. Expected response

   {
     "status": "completed",
     "workflow": "microdao_onboarding",
     "agents": ["welcomer", "role_assigner", "guide"],
     "output": {
       "welcome_message": "...",
       "suggested_role": "contributor",
       "next_steps": [...]
     }
   }
   

Success Criteria

• ✅ Routing rule matched (priority 5, mode=crew)
• ✅ CrewAI workflow executed
• ✅ All 3 agents completed tasks
• ✅ Workflow metadata returned
• ✅ Execution time < 60s

---

🛠️ Scenario 3: DevTools File Operation

Objective: Validate tool execution through Router.

Setup

• User: tg:admin001 (admin role)
• Mode: devtools
• Tool: fs_read

Steps

1. Request file read

   curl -X POST http://localhost:9102/route \
     -H "Content-Type: application/json" \
     -d '{
       "prompt": "Read README.md",
       "mode": "devtools",
       "metadata": {
         "tool": "fs_read",
         "params": {"path": "README.md"},
         "dao_id": "greenfood-dao",
         "user_id": "tg:admin001"
       }
     }'
   

2. Expected flow

   Request → Router
              ↓ (match rule: devtools_tool_execution, priority 3)
           DevTools Backend (:8008)
              ↓ (POST /fs/read)
           File system (workspace)
              ↓ (read README.md)
           Response (file content)
   

3. Verify tool execution

   docker-compose logs devtools | grep "fs_read"
   docker-compose logs router | grep "use_provider: devtools_local"
   

4. Expected response

   {
     "status": "success",
     "tool": "fs_read",
     "content": "# DAGI Stack\n\n...",
     "size": 11120
   }
   

Success Criteria

• ✅ RBAC verified (admin entitlement: devtools_read)
• ✅ DevTools provider called
• ✅ File content returned
• ✅ Security validated (path not outside workspace)
• ✅ Execution time < 1s

---

🔍 Scenario 4: Code Review Workflow

Objective: Multi-agent analysis with DevTools + CrewAI.

Setup

• User: tg:contributor001 (contributor role)
• Mode: crew
• Workflow: code_review

Steps

1. Submit code for review

   curl -X POST http://localhost:9102/route \
     -H "Content-Type: application/json" \
     -d '{
       "prompt": "Review my code changes",
       "mode": "crew",
       "metadata": {
         "workflow": "code_review",
         "dao_id": "greenfood-dao",
         "user_id": "tg:contributor001",
         "code_path": "src/router.py"
       }
     }'
   

2. Expected flow

   Request → Router → CrewAI (:9010)
              ↓
           3 Agents:
              - reviewer (code quality)
              - security_checker (vulnerabilities)
              - performance_analyzer (bottlenecks)
              ↓
           Aggregated report
   

3. Expected response

   {
     "status": "completed",
     "workflow": "code_review",
     "findings": {
       "quality_score": 8.5,
       "security_issues": 0,
       "performance_warnings": 2
     },
     "recommendations": [...]
   }
   

Success Criteria

• ✅ All 3 review agents executed
• ✅ Aggregated report generated
• ✅ RBAC verified (contributor: code_review entitlement)
• ✅ Execution time < 90s

---

📊 Scenario 5: RBAC Permission Check

Objective: Validate role-based access control.

Setup

• User: tg:guest123 (guest role)
• Attempted action: devtools_write (requires contributor+)

Steps

1. Attempt unauthorized action

   curl -X POST http://localhost:9102/route \
     -H "Content-Type: application/json" \
     -d '{
       "prompt": "Write to file",
       "mode": "devtools",
       "metadata": {
         "tool": "fs_write",
         "dao_id": "greenfood-dao",
         "user_id": "tg:guest123"
       }
     }'
   

2. Expected flow

   Request → Router
              ↓ (fetch RBAC)
           RBAC (:9200)
              ↓ (role: guest, entitlements: [chat_access])
           Router (check entitlement: devtools_write)
              ↓ (DENIED - missing entitlement)
           Error response
   

3. Expected response

   {
     "status": "error",
     "code": "RBAC_PERMISSION_DENIED",
     "message": "User lacks entitlement: devtools_write",
     "user_role": "guest",
     "required_entitlement": "devtools_write"
   }
   

Success Criteria

• ✅ RBAC context fetched
• ✅ Permission check executed
• ✅ Request rejected (403 or error response)
• ✅ Structured error message
• ✅ Audit log entry created

---

🧪 Running Scenarios

Automated Test

# Run all golden scenarios
./test-scenarios.sh

# Run specific scenario
./test-scenarios.sh --scenario chat
./test-scenarios.sh --scenario onboarding

Manual Test

# 1. Start services
docker-compose up -d

# 2. Wait for health checks
sleep 10

# 3. Run smoke tests
./smoke.sh

# 4. Execute scenarios manually (curl commands above)

# 5. Monitor logs
docker-compose logs -f

---

📈 Success Metrics

Metric	Target	Current
Chat response time	< 5s	TBD
Workflow execution	< 60s	TBD
DevTools latency	< 1s	TBD
RBAC resolution	< 100ms	TBD
Error rate	< 1%	TBD

---

🔧 Troubleshooting

Scenario fails: LLM timeout

• Check Ollama: curl http://localhost:11434/api/tags
• Increase timeout in router-config.yml
• Consider GPU acceleration

Scenario fails: RBAC error

• Verify RBAC service: curl http://localhost:9200/health
• Check user exists: curl -X POST http://localhost:9200/rbac/resolve -d '{"dao_id":"greenfood-dao","user_id":"tg:12345"}'

Scenario fails: Gateway not responding

• Check bot token in .env
• Verify Gateway health: curl http://localhost:9300/health
• Check Gateway logs: docker-compose logs gateway

---

Version: 0.2.0
Last updated: 2024-11-15