diff --git a/docs/MEMORY-MODULE-STATUS.md b/docs/MEMORY-MODULE-STATUS.md new file mode 100644 index 00000000..91298f87 --- /dev/null +++ b/docs/MEMORY-MODULE-STATUS.md @@ -0,0 +1,281 @@ +# ๐Ÿง  Agent Memory Module โ€” Status Report + +**ะ”ะฐั‚ะฐ:** 2026-01-10 +**ะ’ะตั€ัั–ั ะดะพะบัƒะผะตะฝั‚ะฐ:** 1.0.0 + +--- + +## ๐Ÿ“Š ะ—ะฐะณะฐะปัŒะฝะธะน ัั‚ะฐั‚ัƒั + +| ะšะพะผะฟะพะฝะตะฝั‚ | NODE1 | NODE2 | NODE3 | +|-----------|-------|-------|-------| +| **PostgreSQL** | โœ… K8s (CloudNativePG) | โŒ | โœ… Docker | +| **Qdrant** | โœ… K8s | โœ… Docker (unhealthy) | โŒ | +| **Neo4j** | โŒ | โš ๏ธ Docker (restarting) | โŒ | +| **Ollama** | โœ… v0.13.5 (fresh) | โœ… v0.12.11 | โœ… v0.13.5 | +| **NVIDIA GPU** | โŒ (CPU only) | โŒ (Apple M4) | โœ… RTX 3090 | +| **LTS-2 (CUDA)** | โš ๏ธ Driver 590.48.01 | N/A | โœ… CUDA 13.0 | + +--- + +## ๐Ÿ–ฅ๏ธ NODE1 โ€” Production Server (Hetzner) + +**IP:** `144.76.224.179` +**OS:** Ubuntu 24.04.3 LTS +**Role:** K8s Master (K3s), Primary Database Host + +### Memory System Components + +#### โœ… PostgreSQL (CloudNativePG) +``` +Status: Running (1/1) +Pod: daarion-postgres-1 +Namespace: daarion +Storage: PVC-backed +``` + +**ะขะฐะฑะปะธั†ั– (Agent Memory Schema):** +| ะขะฐะฑะปะธั†ั | ะŸั€ะธะทะฝะฐั‡ะตะฝะฝั | +|---------|-------------| +| `organizations` | ะœัƒะปัŒั‚ะธั‚ะตะฝะฐะฝั‚ะฝั–ัั‚ัŒ | +| `workspaces` | ะŸั€ะพั”ะบั‚ะธ ะฒ ะพั€ะณะฐะฝั–ะทะฐั†ั–ั— | +| `users` | ะšะพั€ะธัั‚ัƒะฒะฐั‡ั– ัะธัั‚ะตะผะธ | +| `agents` | AI ะฐะณะตะฝั‚ะธ | +| `conversation_threads` | ะšะพะฝั‚ะตะบัั‚ ั€ะพะทะผะพะฒ (Short-term) | +| `conversation_events` | ะกะธั€ะธะน ะปะพะณ ะฟะพะฒั–ะดะพะผะปะตะฝัŒ (Short-term) | +| `thread_summaries` | ะŸั–ะดััƒะผะบะธ ั‚ั€ะตะดั–ะฒ (Mid-term) | +| `long_term_memory_items` | ะคะฐะบั‚ะธ, ะฟั€ะตั„ะตั€ะตะฝั†ั–ั— (Long-term) | +| `memory_feedback` | ะšะพั€ะตะบั†ั–ั— ะบะพั€ะธัั‚ัƒะฒะฐั‡ั–ะฒ | + +#### โœ… Qdrant (Vector Database) +``` +Status: Running (1/1) +Pod: qdrant-5d45444569-hfbzg +Namespace: qdrant +Port: 6333 (internal), 30333 (NodePort) +``` + +**ะšะพะปะตะบั†ั–ั—:** +- `memories` โ€” 1024-dimensional vectors (Cohere embed-multilingual-v3.0) + +#### โœ… External Secrets (Vault Integration) +``` +postgres-credentials: SecretSynced โœ… +memory-service-secrets: SecretSynced โœ… +``` + +#### โœ… Ollama +``` +Version: 0.13.5 +Service: systemd (enabled, running) +API: http://127.0.0.1:11434 +Models: (none downloaded yet) +GPU: NVIDIA Driver 590.48.01 (DKMS installed) +``` + +#### โŒ Neo4j +``` +Status: NOT INSTALLED +``` + +--- + +## ๐Ÿ’ป NODE2 โ€” MacBook M4 (Development) + +**Type:** Apple MacBook Pro M4 +**OS:** macOS Darwin 25.2.0 +**Role:** Development, Local Testing + +### Memory System Components + +#### โŒ PostgreSQL +``` +Status: NOT INSTALLED +Note: Can use NODE1's PostgreSQL via SSH tunnel +``` + +#### โš ๏ธ Qdrant (Docker) +``` +Container: qdrant-vector-db +Status: Up 5 hours (unhealthy) +Ports: 6333-6335 +``` + +#### โš ๏ธ Neo4j (Docker) +``` +Containers: + - dagi-neo4j: Restarting (exit code 1) + - dagi-neo4j-exporter: Restarting (exit code 0) +Image: neo4j:5.15-community +Status: NEEDS ATTENTION +``` + +#### โœ… Ollama +``` +Version: 0.12.11 +Location: /opt/homebrew/bin/ollama +``` + +**ะ—ะฐะฒะฐะฝั‚ะฐะถะตะฝั– ะผะพะดะตะปั–:** +| Model | Size | Modified | +|-------|------|----------| +| deepseek-r1:70b | 42 GB | 7 weeks ago | +| qwen2.5-coder:32b | 19 GB | 7 weeks ago | +| deepseek-coder:33b | 18 GB | 7 weeks ago | +| gemma2:27b | 15 GB | 7 weeks ago | +| gpt-oss:latest | 13 GB | 7 weeks ago | +| llava:13b | 8.0 GB | 6 weeks ago | +| mistral-nemo:12b | 7.1 GB | 7 weeks ago | +| phi3:latest | 2.2 GB | 7 weeks ago | +| starcoder2:3b | 1.7 GB | 7 weeks ago | + +--- + +## ๐ŸŽฎ NODE3 โ€” AI/ML Workstation + +**IP:** `80.77.35.151:33147` +**OS:** Linux (Ubuntu) +**Role:** GPU Compute, ML Inference, GitLab Host + +### Hardware +``` +GPU: NVIDIA GeForce RTX 3090 (24GB VRAM) +Driver: 580.95.05 +CUDA: 13.0 +``` + +### Memory System Components + +#### โœ… PostgreSQL (Docker) +``` +Container: dagi-postgres +Image: postgres (23e88eb049fd) +Status: Up 3 hours +Port: 5432 +``` + +#### โŒ Qdrant +``` +Status: NOT INSTALLED +Note: Can deploy via Docker or K3s +``` + +#### โŒ Neo4j +``` +Status: NOT INSTALLED +``` + +#### โœ… Ollama +``` +Version: 0.13.5 +Location: /usr/local/bin/ollama +GPU: RTX 3090 (CUDA enabled) +``` + +**ะ—ะฐะฒะฐะฝั‚ะฐะถะตะฝั– ะผะพะดะตะปั–:** +| Model | Size | Modified | +|-------|------|----------| +| qwen3:32b | 20 GB | 25 hours ago | +| llama3:latest | 4.7 GB | 2 days ago | + +#### โœ… LTS-2 / CUDA +``` +NVIDIA-SMI: 580.95.05 +Driver Version: 580.95.05 +CUDA Version: 13.0 +GPU Memory: 24576 MiB +Temperature: 56ยฐC +Power: 126W / 390W +``` + +--- + +## ๐Ÿ”— ะั€ั…ั–ั‚ะตะบั‚ัƒั€ะฐ Memory Module + +``` +โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” +โ”‚ AGENT MEMORY SYSTEM โ”‚ +โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ SHORT-TERM โ”‚ โ”‚ MID-TERM โ”‚ โ”‚ LONG-TERM โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ”‚ conversation โ”‚ โ”‚ thread โ”‚ โ”‚ memory โ”‚ โ”‚ +โ”‚ โ”‚ _events โ”‚โ”€โ”€โ”€โ–ถโ”‚ _summaries โ”‚โ”€โ”€โ”€โ–ถโ”‚ _items โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ”‚ (raw log) โ”‚ โ”‚ (rolling โ”‚ โ”‚ (distilled โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ summaries) โ”‚ โ”‚ facts) โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ โ”‚ +โ”‚ โ–ผ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ PostgreSQL (NODE1) โ”‚ โ”‚ +โ”‚ โ”‚ CloudNativePG + pgvector โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ โ”‚ +โ”‚ โ–ผ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ Qdrant (NODE1) โ”‚ โ”‚ +โ”‚ โ”‚ Semantic Search (1024-dim) โ”‚ โ”‚ +โ”‚ โ”‚ Cohere embed-multilingual-v3.0 โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ +โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค +โ”‚ INFERENCE LAYER โ”‚ +โ”‚ โ”‚ +โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚ +โ”‚ โ”‚ NODE1 โ”‚ โ”‚ NODE2 โ”‚ โ”‚ NODE3 โ”‚ โ”‚ +โ”‚ โ”‚ Ollama โ”‚ โ”‚ Ollama โ”‚ โ”‚ Ollama โ”‚ โ”‚ +โ”‚ โ”‚ (CPU) โ”‚ โ”‚ (M4) โ”‚ โ”‚ (RTX 3090) โ”‚ โ”‚ +โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ โ”‚ +โ”‚ โ”‚ API-only โ”‚ โ”‚ 9 models โ”‚ โ”‚ 2 models โ”‚ โ”‚ +โ”‚ โ”‚ inference โ”‚ โ”‚ (42GB max) โ”‚ โ”‚ (20GB max) โ”‚ โ”‚ +โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚ +โ”‚ โ”‚ +โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ +``` + +--- + +## ๐Ÿ“‹ TODO / Recommendations + +### ะšั€ะธั‚ะธั‡ะฝั– +- [ ] **NODE2 Neo4j** โ€” ะ’ะธะฟั€ะฐะฒะธั‚ะธ ะบะพะฝั‚ะตะนะฝะตั€ (ะฟะพัั‚ั–ะนะฝะธะน restart) +- [ ] **NODE2 Qdrant** โ€” ะŸะตั€ะตะฒั–ั€ะธั‚ะธ healthcheck + +### ะ ะตะบะพะผะตะฝะดะพะฒะฐะฝั– +- [ ] **NODE1** โ€” ะ—ะฐะฒะฐะฝั‚ะฐะถะธั‚ะธ ะฑะฐะทะพะฒัƒ ะผะพะดะตะปัŒ ะฒ Ollama (e.g., `llama3:8b`) +- [ ] **NODE3** โ€” ะ’ัั‚ะฐะฝะพะฒะธั‚ะธ Qdrant ะดะปั ะปะพะบะฐะปัŒะฝะพะณะพ ัะตะผะฐะฝั‚ะธั‡ะฝะพะณะพ ะฟะพัˆัƒะบัƒ +- [ ] **NODE1** โ€” ะ’ัั‚ะฐะฝะพะฒะธั‚ะธ Neo4j ะดะปั ะณั€ะฐั„ะพะฒะพะณะพ ะทะฑะตั€ั–ะณะฐะฝะฝั ะทะฒ'ัะทะบั–ะฒ + +### ะžะฟั†ั–ะพะฝะฐะปัŒะฝั– +- [ ] ะะฐะปะฐัˆั‚ัƒะฒะฐั‚ะธ ั€ะตะฟะปั–ะบะฐั†ั–ัŽ PostgreSQL NODE1 โ†” NODE3 +- [ ] ะ ะพะทะณะพั€ะฝัƒั‚ะธ Memory Service (FastAPI) ะฒ K8s +- [ ] ะ†ะฝั‚ะตะณั€ัƒะฒะฐั‚ะธ Cohere API ะดะปั ะฐะฒั‚ะพะผะฐั‚ะธั‡ะฝะพะณะพ ะตะผะฑะตะดะธะฝะณัƒ + +--- + +## ๐Ÿ” Credentials Reference + +| Service | Location | Access | +|---------|----------|--------| +| PostgreSQL | NODE1 K8s | Via Vault โ†’ External Secrets | +| Qdrant | NODE1 K8s | Internal ClusterIP | +| Ollama NODE1 | localhost:11434 | UFW port 11434 | +| Ollama NODE3 | localhost:11434 | SSH tunnel required | +| Cohere API | Memory Service | `nOdOXnuepLku2ipJWpe6acWgAsJCsDhMO0RnaEJB` | + +--- + +## ๐Ÿ“ Related Files + +- `infrastructure/database/agent-memory-schema.sql` โ€” PostgreSQL schema +- `infrastructure/kubernetes/apps/qdrant/deployment.yaml` โ€” Qdrant K8s config +- `services/memory-service/` โ€” FastAPI Memory Service (not deployed yet) +- `INFRASTRUCTURE.md` โ€” Main infrastructure documentation + +--- + +*ะ”ะพะบัƒะผะตะฝั‚ ะทะณะตะฝะตั€ะพะฒะฐะฝะพ ะฐะฒั‚ะพะผะฐั‚ะธั‡ะฝะพ: 2026-01-10 17:10 CET*