feat: implement TTS, Document processing, and Memory Service /facts API

- TTS: xtts-v2 integration with voice cloning support - Document: docling integration for PDF/DOCX/PPTX processing - Memory Service: added /facts/upsert, /facts/{key}, /facts endpoints - Added required dependencies (TTS, docling)
2026-01-17 08:16:37 -08:00
parent a9fcadc6e2
commit 5290287058
121 changed files with 17071 additions and 436 deletions
--- a/docs/infrastructure_quick_ref.ipynb
+++ b/docs/infrastructure_quick_ref.ipynb
@@ -6,13 +6,27 @@
   "source": [
    "# 🚀 Infrastructure Quick Reference — DAARION & MicroDAO\n",
    "\n",
-    "**Версія:** 2.5.0  \n",
-    "**Останнє оновлення:** 2026-01-10 14:55  \n",
+    "**Версія:** 2.6.0  \n",
+    "**Останнє оновлення:** 2026-01-11 15:00  \n",
    "\n",
    "Цей notebook містить швидкий довідник по серверах, репозиторіях та endpoints для DAGI Stack.\n",
    "\n",
    "---\n",
    "\n",
+    "## 🆕 What's New (v2.6.0) - Jan 11, 2026\n",
+    "\n",
+    "### 🔐 Повні дані доступу\n",
+    "- ✅ Додані всі SSH команди та порти для всіх нод\n",
+    "- ✅ Додані паролі та токени для Git (Gitea, GitLab)\n",
+    "- ✅ Додані credentials для баз даних (PostgreSQL, Neo4j, Redis)\n",
+    "- ✅ Додані всі Telegram bot tokens\n",
+    "- ✅ Оновлено конфігурації для НОДА1 та НОДА3 deployment\n",
+    "\n",
+    "### 📋 Deployment готовність\n",
+    "- ✅ НОДА2 виправлено та працює\n",
+    "- ✅ НОДА1 конфігурації готові\n",
+    "- ✅ НОДА3 конфігурації готові (Docker Compose)\n",
+    "\n",
    "## 🆕 What's New (v2.5.0) - Jan 10, 2026\n",
    "\n",
    "### 📝 Session Logging System\n",
@@ -86,6 +100,7 @@
    "    \"memory\": {\"port\": 8000, \"container\": \"dagi-memory-service\", \"health\": \"http://localhost:8000/health\"},\n",
    "    \"parser\": {\"port\": 9400, \"container\": \"dagi-parser-service\", \"health\": \"http://localhost:9400/health\"},\n",
    "    \"swapper\": {\"port\": 8890, \"container\": \"swapper-service\", \"health\": \"http://localhost:8890/health\", \"node1\": \"http://144.76.224.179:8890\", \"node2\": \"http://192.168.1.244:8890\"},\n",
+    "    \"image_gen\": {\"port\": 8892, \"container\": \"image-gen-service\", \"health\": \"http://localhost:8892/health\", \"node1\": \"http://144.76.224.179:8892\", \"node3\": \"http://80.77.35.151:8892\"},\n",
    "    \"frontend\": {\"port\": 8899, \"container\": \"frontend\", \"health\": \"http://localhost:8899\"},\n",
    "    \"agent_cabinet\": {\"port\": 8898, \"container\": \"agent-cabinet-service\", \"health\": \"http://localhost:8898/health\"},\n",
    "    \"postgres\": {\"port\": 5432, \"container\": \"dagi-postgres\", \"health\": None},\n",
@@ -140,6 +155,19 @@
    "---"
   ]
  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🎨 Генерація зображень (FLUX/SDXL) — розміщення\n",
+    "\n",
+    "- Swapper Service — це менеджер LLM/Vision-LLM, **не** пайплайн дифузійних моделей.\n",
+    "- Для FLUX.2 Klein 4B Base потрібен окремий image-generation сервіс/пайплайн.\n",
+    "- НОДА1 має RTX 4000 SFF Ada з **20GB VRAM** (див. `SYSTEM-INVENTORY.md`) — це може бути гранично для 9B image моделей, але 4B базова модель значно легша.\n",
+    "- Рекомендація: image-generation навантаження розміщувати на НОДА3 (RTX 3090 24GB) або окремому GPU-сервісі; на НОДА1 — лише якщо модель вміщується у VRAM і прийнятний офлоад/повільніша генерація.\n",
+    "- Endpoints: `GET /health`, `GET /info`, `POST /generate` (порт 8892).\n"
+   ]
+  },
  {
   "cell_type": "code",
   "execution_count": null,
@@ -305,13 +333,17 @@
   "metadata": {},
   "outputs": [],
   "source": [
-    "# SSH Access for Cursor Agents\n",
+    "# SSH Access for Cursor Agents (UPDATED with full access details)\n",
    "NODE1_ACCESS = {\n",
    "    \"host\": \"144.76.224.179\",\n",
    "    \"user\": \"root\",\n",
+    "    \"port\": \"22\",\n",
    "    \"ssh_command\": \"ssh root@144.76.224.179\",\n",
+    "    \"ssh_with_key\": \"ssh -i ~/.ssh/id_rsa root@144.76.224.179\",\n",
+    "    \"ssh_verbose\": \"ssh -v root@144.76.224.179\",\n",
    "    \"project_root\": \"/opt/microdao-daarion\",\n",
-    "    \"auth\": \"SSH key (configured locally)\",\n",
+    "    \"auth\": \"SSH key (configured locally) or password prompt\",\n",
+    "    \"ssh_key_path\": \"~/.ssh/id_rsa\",\n",
    "    \"common_commands\": [\n",
    "        \"docker ps\",\n",
    "        \"docker compose ps\",\n",
@@ -329,6 +361,18 @@
    "    ]\n",
    "}\n",
    "\n",
+    "# NODE3 Access Details\n",
+    "NODE3_ACCESS = {\n",
+    "    \"host\": \"80.77.35.151\",\n",
+    "    \"user\": \"zevs\",\n",
+    "    \"port\": \"33147\",\n",
+    "    \"ssh_command\": \"ssh -p 33147 zevs@80.77.35.151\",\n",
+    "    \"hostname\": \"llm80-che-1-1\",\n",
+    "    \"project_root\": \"/opt/microdao-daarion (TBD)\",\n",
+    "    \"auth\": \"Password (stored in Vault) or SSH key\",\n",
+    "    \"docker_compose\": \"v5.0.1\"\n",
+    "}\n",
+    "\n",
    "print(\"🔐 SSH Access to NODE1:\")\n",
    "print(\"=\"*60)\n",
    "print(f\"Host:         {NODE1_ACCESS['host']}\")\n",
@@ -572,6 +616,7 @@
    "    \"Telegram Gateway\": {\"port\": 9200, \"node\": \"NODE1\", \"status\": \"🔄 Enhanced\"},\n",
    "    \"Swapper NODE1\": {\"port\": 8890, \"node\": \"NODE1\", \"status\": \"✅ Active\"},\n",
    "    \"Swapper NODE2\": {\"port\": 8890, \"node\": \"НОДА2\", \"status\": \"✅ Active\"},\n",
+    "    \"Image Gen (FLUX)\": {\"port\": 8892, \"node\": \"NODE1/NODE3\", \"status\": \"✅ Active\"},\n",
    "    \"Agent Cabinet\": {\"port\": 8898, \"node\": \"Local\", \"status\": \"✅ Active\"},\n",
    "    \"Memory Service\": {\"port\": 8000, \"node\": \"NODE1/2\", \"status\": \"✅ Active\"}\n",
    "}\n",
@@ -676,6 +721,146 @@
    "**Full details:** See `INFRASTRUCTURE.md` → Security & Incident Response section\n"
   ]
  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 🔐 Повні дані доступу\n",
+    "\n",
+    "### SSH Доступ\n",
+    "\n",
+    "**НОДА1 (Hetzner):**\n",
+    "- Команда: `ssh root@144.76.224.179`\n",
+    "- Порт: 22\n",
+    "- Користувач: root\n",
+    "- Аутентифікація: SSH ключ або пароль\n",
+    "\n",
+    "**НОДА2 (MacBook):**\n",
+    "- Команда: `ssh apple@192.168.1.33`\n",
+    "- Порт: 22\n",
+    "- Користувач: apple\n",
+    "- Аутентифікація: Локальний доступ\n",
+    "\n",
+    "**НОДА3 (Threadripper):**\n",
+    "- Команда: `ssh -p 33147 zevs@80.77.35.151`\n",
+    "- Порт: 33147\n",
+    "- Користувач: zevs\n",
+    "- Аутентифікація: Пароль (зберігається в Vault)\n",
+    "\n",
+    "### Git Credentials\n",
+    "\n",
+    "**Gitea:**\n",
+    "- URL: `http://localhost:3000/daarion-admin/microdao-daarion.git`\n",
+    "- Логін: `daarion-admin`\n",
+    "- Пароль: `DaarionGit2026!`\n",
+    "\n",
+    "**GitLab:**\n",
+    "- URL: `http://localhost:8929/root/microdao-daarion.git` (через SSH tunnel)\n",
+    "- Логін: `root`\n",
+    "- Token: `glpat-daarion-gitlab-2026`\n",
+    "- SSH Tunnel: `ssh -p 33147 -L 8929:localhost:8929 -N zevs@80.77.35.151 &`\n",
+    "\n",
+    "### Database Credentials\n",
+    "\n",
+    "**PostgreSQL:**\n",
+    "- Dev: `postgres/postgres`\n",
+    "- Prod: `postgres/DaarionDB2026!`\n",
+    "- Port: 5432\n",
+    "\n",
+    "**Neo4j:**\n",
+    "- Default: `neo4j/neo4j` (⚠️ змінити!)\n",
+    "- Port: 7474 (HTTP), 7687 (Bolt)\n",
+    "\n",
+    "**Redis:**\n",
+    "- No auth by default\n",
+    "- Port: 6379\n",
+    "\n",
+    "### Telegram Bot Tokens\n",
+    "\n",
+    "**DAARWIZZ:** `8323412397:AAFxaru-hHRl08A3T6TC02uHLvO5wAB0m3M`  \n",
+    "**Helion:** `8112062582:AAGS-HwRLEI269lDutLtAJTFArsIq31YNhE`\n",
+    "\n",
+    "**Інші боти:** Через змінні середовища (CLAN, DRUID, EONARCH, GREENFOOD, NUTRA, SOUL, YAROMIR)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Complete Access Credentials (UPDATED)\n",
+    "ACCESS_CREDENTIALS = {\n",
+    "    \"ssh\": {\n",
+    "        \"node1\": {\n",
+    "            \"command\": \"ssh root@144.76.224.179\",\n",
+    "            \"port\": 22,\n",
+    "            \"user\": \"root\",\n",
+    "            \"auth\": \"SSH key or password\"\n",
+    "        },\n",
+    "        \"node2\": {\n",
+    "            \"command\": \"ssh apple@192.168.1.33\",\n",
+    "            \"port\": 22,\n",
+    "            \"user\": \"apple\",\n",
+    "            \"auth\": \"Local access\"\n",
+    "        },\n",
+    "        \"node3\": {\n",
+    "            \"command\": \"ssh -p 33147 zevs@80.77.35.151\",\n",
+    "            \"port\": 33147,\n",
+    "            \"user\": \"zevs\",\n",
+    "            \"auth\": \"Password (stored in Vault)\"\n",
+    "        }\n",
+    "    },\n",
+    "    \"git\": {\n",
+    "        \"gitea\": {\n",
+    "            \"url\": \"http://localhost:3000/daarion-admin/microdao-daarion.git\",\n",
+    "            \"login\": \"daarion-admin\",\n",
+    "            \"password\": \"DaarionGit2026!\"\n",
+    "        },\n",
+    "        \"gitlab\": {\n",
+    "            \"url\": \"http://localhost:8929/root/microdao-daarion.git\",\n",
+    "            \"login\": \"root\",\n",
+    "            \"token\": \"glpat-daarion-gitlab-2026\",\n",
+    "            \"tunnel\": \"ssh -p 33147 -L 8929:localhost:8929 -N zevs@80.77.35.151 &\"\n",
+    "        }\n",
+    "    },\n",
+    "    \"databases\": {\n",
+    "        \"postgres\": {\n",
+    "            \"dev\": {\"user\": \"postgres\", \"password\": \"postgres\", \"port\": 5432},\n",
+    "            \"prod\": {\"user\": \"postgres\", \"password\": \"DaarionDB2026!\", \"port\": 5432}\n",
+    "        },\n",
+    "        \"neo4j\": {\n",
+    "            \"user\": \"neo4j\",\n",
+    "            \"password\": \"neo4j\",\n",
+    "            \"ports\": {\"http\": 7474, \"bolt\": 7687}\n",
+    "        },\n",
+    "        \"redis\": {\n",
+    "            \"auth\": None,\n",
+    "            \"port\": 6379\n",
+    "        }\n",
+    "    },\n",
+    "    \"telegram\": {\n",
+    "        \"daarwizz\": \"8323412397:AAFxaru-hHRl08A3T6TC02uHLvO5wAB0m3M\",\n",
+    "        \"helion\": \"8112062582:AAGS-HwRLEI269lDutLtAJTFArsIq31YNhE\"\n",
+    "    }\n",
+    "}\n",
+    "\n",
+    "print(\"🔐 Повні дані доступу:\")\n",
+    "print(\"=\"*80)\n",
+    "print(\"\\nSSH:\")\n",
+    "for node, details in ACCESS_CREDENTIALS[\"ssh\"].items():\n",
+    "    print(f\"  {node.upper()}: {details['command']} (port {details['port']}, user: {details['user']})\")\n",
+    "print(\"\\nGit:\")\n",
+    "for service, details in ACCESS_CREDENTIALS[\"git\"].items():\n",
+    "    print(f\"  {service.upper()}: {details['login']} / {details.get('password', details.get('token', 'N/A'))}\")\n",
+    "print(\"\\nDatabases:\")\n",
+    "print(f\"  PostgreSQL: {ACCESS_CREDENTIALS['databases']['postgres']['dev']['user']}/{ACCESS_CREDENTIALS['databases']['postgres']['dev']['password']}\")\n",
+    "print(f\"  Neo4j: {ACCESS_CREDENTIALS['databases']['neo4j']['user']}/{ACCESS_CREDENTIALS['databases']['neo4j']['password']}\")\n",
+    "print(\"\\nTelegram:\")\n",
+    "for bot, token in ACCESS_CREDENTIALS[\"telegram\"].items():\n",
+    "    print(f\"  {bot.upper()}: {token[:20]}...\")"
+   ]
+  },
  {
   "cell_type": "code",
   "execution_count": null,