feat: Add Alateya, Clan, Eonarch agents + fix gateway-router connection
## Agents Added - Alateya: R&D, biotech, innovations - Clan (Spirit): Community spirit agent - Eonarch: Consciousness evolution agent ## Changes - docker-compose.node1.yml: Added tokens for all 3 new agents - gateway-bot/http_api.py: Added configs and webhook endpoints - gateway-bot/clan_prompt.txt: New prompt file - gateway-bot/eonarch_prompt.txt: New prompt file ## Fixes - Fixed ROUTER_URL from :9102 to :8000 (internal container port) - All 9 Telegram agents now working ## Documentation - Created PROJECT-MASTER-INDEX.md - single entry point - Added various status documents and scripts Tokens configured: - Helion, NUTRA, Agromatrix (existing) - Alateya, Clan, Eonarch (new) - Druid, GreenFood, DAARWIZZ (configured)
This commit is contained in:
Apple
183
docs/memory/CUTOVER_CHECKLIST.md
Normal file
183
docs/memory/CUTOVER_CHECKLIST.md
Normal file
@@ -0,0 +1,183 @@
|
||||
# Qdrant Canonical Migration - Cutover Checklist
|
||||
|
||||
**Status:** GO
|
||||
**Date:** 2026-01-26
|
||||
**Risk Level:** Operational (security invariants verified)
|
||||
|
||||
---
|
||||
|
||||
## Pre-Cutover Verification
|
||||
|
||||
### Security Invariants (VERIFIED ✅)
|
||||
|
||||
| Invariant | Status |
|
||||
|-----------|--------|
|
||||
| `tenant_id` always required | ✅ |
|
||||
| `agent_ids ⊆ allowed_agent_ids` (or admin) | ✅ |
|
||||
| Admin default: no private | ✅ |
|
||||
| Empty `should` → error | ✅ |
|
||||
| Private only via owner | ✅ |
|
||||
| Qdrant `match.any` format | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## Cutover Steps
|
||||
|
||||
### 1. Deploy
|
||||
|
||||
```bash
|
||||
# Copy code to NODE1
|
||||
scp -r services/memory/qdrant/ root@NODE1:/opt/microdao-daarion/services/memory/
|
||||
scp docs/memory/canonical_collections.yaml root@NODE1:/opt/microdao-daarion/docs/memory/
|
||||
scp scripts/qdrant_*.py root@NODE1:/opt/microdao-daarion/scripts/
|
||||
|
||||
# IMPORTANT: Verify dim/metric in canonical_collections.yaml matches live embedding
|
||||
# Current: dim=1024, metric=cosine
|
||||
|
||||
# Restart service that owns Qdrant reads/writes
|
||||
docker compose restart memory-service
|
||||
# OR
|
||||
systemctl restart memory-service
|
||||
```
|
||||
|
||||
### 2. Migration
|
||||
|
||||
```bash
|
||||
# Dry run - MUST pass before real migration
|
||||
python3 scripts/qdrant_migrate_to_canonical.py --all --dry-run 2>&1 | tee migration_dry_run.log
|
||||
|
||||
# Verify dry run output:
|
||||
# - Target collection name(s) shown
|
||||
# - Per-collection counts listed
|
||||
# - Zero dim/metric mismatches (unless --skip-dim-check used)
|
||||
|
||||
# Real migration
|
||||
python3 scripts/qdrant_migrate_to_canonical.py --all --continue-on-error 2>&1 | tee migration_$(date +%Y%m%d_%H%M%S).log
|
||||
|
||||
# Review summary:
|
||||
# - Collections processed: X/Y
|
||||
# - Points migrated: N
|
||||
# - Errors: should be 0 or minimal
|
||||
```
|
||||
|
||||
### 3. Parity Check
|
||||
|
||||
```bash
|
||||
python3 scripts/qdrant_parity_check.py --agents helion,nutra,druid 2>&1 | tee parity_check.log
|
||||
|
||||
# Requirements:
|
||||
# - Count parity within tolerance
|
||||
# - topK overlap threshold passes
|
||||
# - Schema validation passes
|
||||
```
|
||||
|
||||
### 4. Dual-Read Window
|
||||
|
||||
```bash
|
||||
# Enable dual-read for validation
|
||||
export DUAL_READ_OLD=true
|
||||
|
||||
# Restart service to pick up env change
|
||||
docker compose restart memory-service
|
||||
```
|
||||
|
||||
**Validation queries (must pass):**
|
||||
|
||||
| Query Type | Expected Result |
|
||||
|------------|-----------------|
|
||||
| Agent-only (Helion) | Returns own docs, no other agents |
|
||||
| Multi-agent (DAARWIZZ) | Returns from allowed agents only |
|
||||
| Private visibility | Only owner sees private |
|
||||
|
||||
```bash
|
||||
# Run smoke test
|
||||
python3 scripts/qdrant_smoke_test.py --host localhost
|
||||
```
|
||||
|
||||
### 5. Cutover
|
||||
|
||||
```bash
|
||||
# Disable dual-read
|
||||
export DUAL_READ_OLD=false
|
||||
|
||||
# Ensure no legacy writes
|
||||
export DUAL_WRITE_OLD=false
|
||||
# OR remove these env vars entirely
|
||||
|
||||
# Restart service
|
||||
docker compose restart memory-service
|
||||
|
||||
# Verify service is healthy
|
||||
curl -s http://localhost:8000/health
|
||||
```
|
||||
|
||||
### 6. Post-Cutover Guard
|
||||
|
||||
```bash
|
||||
# Keep legacy collections for rollback window (recommended: 7 days)
|
||||
# DO NOT delete legacy collections yet
|
||||
|
||||
# After rollback window (7 days):
|
||||
# 1. Run one more parity check
|
||||
python3 scripts/qdrant_parity_check.py --all
|
||||
|
||||
# 2. If parity passes, delete legacy collections
|
||||
# WARNING: This is irreversible
|
||||
# python3 -c "
|
||||
# from qdrant_client import QdrantClient
|
||||
# client = QdrantClient(host='localhost', port=6333)
|
||||
# legacy = ['helion_docs', 'nutra_messages', ...] # list all legacy
|
||||
# for col in legacy:
|
||||
# client.delete_collection(col)
|
||||
# print(f'Deleted: {col}')
|
||||
# "
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Rollback Procedure
|
||||
|
||||
If issues arise after cutover:
|
||||
|
||||
```bash
|
||||
# 1. Re-enable dual-read from legacy
|
||||
export DUAL_READ_OLD=true
|
||||
export DUAL_WRITE_OLD=true # if needed
|
||||
|
||||
# 2. Restart service
|
||||
docker compose restart memory-service
|
||||
|
||||
# 3. Investigate issues
|
||||
# - Check migration logs
|
||||
# - Check parity results
|
||||
# - Review error messages
|
||||
|
||||
# 4. If canonical data is corrupted, switch to legacy-only mode:
|
||||
# (requires code change to bypass canonical reads)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Files Reference
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `services/memory/qdrant/` | Canonical Qdrant module |
|
||||
| `docs/memory/canonical_collections.yaml` | Collection config |
|
||||
| `docs/memory/cm_payload_v1.md` | Payload schema docs |
|
||||
| `scripts/qdrant_migrate_to_canonical.py` | Migration tool |
|
||||
| `scripts/qdrant_parity_check.py` | Parity verification |
|
||||
| `scripts/qdrant_smoke_test.py` | Security smoke test |
|
||||
|
||||
---
|
||||
|
||||
## Sign-off
|
||||
|
||||
- [ ] Dry run passed
|
||||
- [ ] Migration completed
|
||||
- [ ] Parity check passed
|
||||
- [ ] Dual-read validation passed
|
||||
- [ ] Cutover completed
|
||||
- [ ] Post-cutover health verified
|
||||
- [ ] Rollback window started (7 days)
|
||||
- [ ] Legacy collections deleted (after rollback window)
|
||||
Reference in New Issue
Block a user