21691aa042
Security Incident #2 Emergency Response (Jan 9, 2026): - Documented second compromise with NEW crypto miners (softirq, vrarhpb) - Root cause: Docker image auto-restarted after server reboot - Emergency mitigation completed (processes killed, container/images removed, load normalized) - Created comprehensive rebuild task document: TASK_REBUILD_DAARION_WEB.md - Updated INFRASTRUCTURE.md v2.3.0 with Incident #2 timeline and lessons learned - Updated infrastructure_quick_ref.ipynb v2.2.0 with security status Critical Changes: - daarion-web container permanently disabled until secure rebuild - Docker images DELETED (not just container stopped) - Enhanced firewall rules (SSH rate limiting, port scan blocking) - Retry test registered with Hetzner - System load normalized: 30+ → 4.19 - Zombie processes cleaned: 1499 → 5 Files Created/Updated: 1. TASK_REBUILD_DAARION_WEB.md - Detailed rebuild instructions for Cursor agent 2. INFRASTRUCTURE.md - Added Incident #2 to Security section 3. docs/infrastructure_quick_ref.ipynb - Updated security status and version Lessons Learned: - ALWAYS delete Docker images, not just containers - Auto-restart policies are dangerous for compromised containers - Complete removal = container + image + restart policy change Status: Emergency mitigation complete, statement submission pending (deadline: 2026-01-09 12:54 UTC) Hetzner Incident ID: 10F3971:2A (AbuseID) Co-Authored-By: Warp <agent@warp.dev>
727 lines
31 KiB
Plaintext
727 lines
31 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"# 🚀 Infrastructure Quick Reference — DAARION & MicroDAO\n",
|
||
"\n",
|
||
"Версія:** 2.2.0 \n",
|
||
"Останнє оновлення:** 2026-01-09 \n",
|
||
"\n",
|
||
"Цей notebook містить швидкий довідник по серверах, репозиторіях та endpoints для DAGI Stack.\n",
|
||
"\n",
|
||
"**NEW (v2.2.0):** \n",
|
||
"- 🔒 **Security Incident #2** (Jan 9, 2026) - Emergency mitigation completed\n",
|
||
"- ⚠️ **daarion-web permanently disabled** until secure rebuild\n",
|
||
"- ✅ Enhanced firewall rules + retry test registered with Hetzner\n",
|
||
"\n",
|
||
"**v2.1.0:** \n",
|
||
"- 🔒 **Security Incident #1 Resolved** (Dec 2025 - Jan 2026)\n",
|
||
"- ✅ Firewall rules + monitoring deployed\n",
|
||
"\n",
|
||
"**v2.0.0:** \n",
|
||
"- ✅ Мультимодальні сервіси (STT, OCR, Web Search, Vector DB) на НОДА2\n",
|
||
"- ✅ Router Multimodal Support (інтеграція в процесі)\n",
|
||
"- ✅ Telegram Gateway Enhanced (STT + Vision)\n",
|
||
"- ✅ Swapper Service інтеграція в кабінети НОД\n",
|
||
"- ✅ Кабінети мікроДАО з оркестраторами\n",
|
||
"- ✅ Оновлення в реальному часі (кожні 30 секунд)\n",
|
||
"- ✅ Управління мікроДАО в кабінеті DAARION"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Service Configuration (UPDATED with Swapper Service + Frontend + Agent Cabinet)\n",
|
||
"SERVICES = {\n",
|
||
" \"router\": {\"port\": 9102, \"container\": \"dagi-router\", \"health\": \"http://localhost:9102/health\"},\n",
|
||
" \"gateway\": {\"port\": 9300, \"container\": \"dagi-gateway\", \"health\": \"http://localhost:9300/health\"},\n",
|
||
" \"devtools\": {\"port\": 8008, \"container\": \"dagi-devtools\", \"health\": \"http://localhost:8008/health\"},\n",
|
||
" \"crewai\": {\"port\": 9010, \"container\": \"dagi-crewai\", \"health\": \"http://localhost:9010/health\"},\n",
|
||
" \"rbac\": {\"port\": 9200, \"container\": \"dagi-rbac\", \"health\": \"http://localhost:9200/health\"},\n",
|
||
" \"rag\": {\"port\": 9500, \"container\": \"dagi-rag-service\", \"health\": \"http://localhost:9500/health\"},\n",
|
||
" \"memory\": {\"port\": 8000, \"container\": \"dagi-memory-service\", \"health\": \"http://localhost:8000/health\"},\n",
|
||
" \"parser\": {\"port\": 9400, \"container\": \"dagi-parser-service\", \"health\": \"http://localhost:9400/health\"},\n",
|
||
" \"swapper\": {\"port\": 8890, \"container\": \"swapper-service\", \"health\": \"http://localhost:8890/health\", \"node1\": \"http://144.76.224.179:8890\", \"node2\": \"http://192.168.1.244:8890\"},\n",
|
||
" \"frontend\": {\"port\": 8899, \"container\": \"frontend\", \"health\": \"http://localhost:8899\"},\n",
|
||
" \"agent_cabinet\": {\"port\": 8898, \"container\": \"agent-cabinet-service\", \"health\": \"http://localhost:8898/health\"},\n",
|
||
" \"postgres\": {\"port\": 5432, \"container\": \"dagi-postgres\", \"health\": None},\n",
|
||
" \"redis\": {\"port\": 6379, \"container\": \"redis\", \"health\": \"redis-cli PING\"},\n",
|
||
" \"neo4j\": {\"port\": 7474, \"container\": \"neo4j\", \"health\": \"http://localhost:7474\"},\n",
|
||
" \"qdrant\": {\"port\": 6333, \"container\": \"dagi-qdrant\", \"health\": \"http://localhost:6333/healthz\"},\n",
|
||
" \"grafana\": {\"port\": 3000, \"container\": \"grafana\", \"health\": \"http://localhost:3000\"},\n",
|
||
" \"prometheus\": {\"port\": 9090, \"container\": \"prometheus\", \"health\": \"http://localhost:9090\"},\n",
|
||
" \"ollama\": {\"port\": 11434, \"container\": \"ollama\", \"health\": \"http://localhost:11434/api/tags\"}\n",
|
||
"}\n",
|
||
"\n",
|
||
"print(\"Service\\t\\t\\tPort\\tContainer\\t\\t\\tHealth Endpoint\")\n",
|
||
"print(\"=\"*100)\n",
|
||
"for name, service in SERVICES.items():\n",
|
||
" health = service['health'] or \"N/A\"\n",
|
||
" gpu = \" [GPU]\" if service.get('gpu') else \"\"\n",
|
||
" print(f\"{name.upper():<20} {service['port']:<7} {service['container']:<30} {health}{gpu}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 🖥️ Network Nodes\n",
|
||
"\n",
|
||
"### Node #1: Production Server (Hetzner)\n",
|
||
"- **Node ID:** node-1-hetzner-gex44\n",
|
||
"- **IP:** 144.76.224.179\n",
|
||
"- **Role:** Production Router + Gateway + All Services (24/7)\n",
|
||
"- **Location:** Hetzner Cloud (Germany)\n",
|
||
"\n",
|
||
"### Node #2: Development Node (MacBook Pro M4 Max)\n",
|
||
"- **Node ID:** node-2-macbook-m4max\n",
|
||
"- **Local IP:** 192.168.1.244\n",
|
||
"- **Role:** Development + Testing + Backup Router\n",
|
||
"- **Specs:** M4 Max (16 cores), 64GB RAM, 2TB SSD, 40-core GPU\n",
|
||
"- **Location:** Local Network (Ivan's Office)\n",
|
||
"- **Docs:** [NODE-2-MACBOOK-SPECS.md](../NODE-2-MACBOOK-SPECS.md)\n",
|
||
"\n",
|
||
"---"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Network Nodes Configuration\n",
|
||
"NODES = {\n",
|
||
" \"node-1\": {\n",
|
||
" \"name\": \"Hetzner GEX44\",\n",
|
||
" \"ip\": \"144.76.224.179\",\n",
|
||
" \"local_ip\": None,\n",
|
||
" \"role\": \"production\",\n",
|
||
" \"uptime\": \"24/7\",\n",
|
||
" \"ssh\": \"root@144.76.224.179\",\n",
|
||
" \"domain\": \"gateway.daarion.city\",\n",
|
||
" \"services\": \"All (17 services)\",\n",
|
||
" \"specs\": \"See SYSTEM-INVENTORY.md\"\n",
|
||
" },\n",
|
||
" \"node-2\": {\n",
|
||
" \"name\": \"MacBook Pro M4 Max\",\n",
|
||
" \"ip\": None,\n",
|
||
" \"local_ip\": \"192.168.1.244\",\n",
|
||
" \"role\": \"development\",\n",
|
||
" \"uptime\": \"on-demand\",\n",
|
||
" \"ssh\": \"apple@192.168.1.244\",\n",
|
||
" \"domain\": None,\n",
|
||
" \"services\": \"Core only (Router, DevTools, Memory, Ollama)\",\n",
|
||
" \"specs\": \"M4 Max, 16 cores, 64GB RAM, 2TB SSD, 40-core GPU\"\n",
|
||
" }\n",
|
||
"}\n",
|
||
"\n",
|
||
"print(\"DAGI Stack Network Nodes:\")\n",
|
||
"print(\"=\"*80)\n",
|
||
"for node_id, node in NODES.items():\n",
|
||
" print(f\"\\n{node_id.upper()}: {node['name']}\")\n",
|
||
" print(f\" Role: {node['role']}\")\n",
|
||
" print(f\" IP: {node['ip'] or node['local_ip']}\")\n",
|
||
" print(f\" SSH: {node['ssh']}\")\n",
|
||
" print(f\" Uptime: {node['uptime']}\")\n",
|
||
" print(f\" Services: {node['services']}\")\n",
|
||
" if node['domain']:\n",
|
||
" print(f\" Domain: https://{node['domain']}\")\n",
|
||
" print(f\" Specs: {node['specs']}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 🐙 GitHub Repositories",
|
||
"",
|
||
"### 1. MicroDAO (Current Project)",
|
||
"- **Repository:** `git@github.com:IvanTytar/microdao-daarion`",
|
||
"- **HTTPS:** `https://github.com/IvanTytar/microdao-daarion`",
|
||
"- **Remote Name:** `origin`",
|
||
"- **Main Branch:** `main`",
|
||
"- **Purpose:** MicroDAO core code, DAGI Stack, documentation",
|
||
"",
|
||
"### 2. DAARION.city",
|
||
"- **Repository:** `git@github.com:DAARION-DAO/daarion-ai-city.git`",
|
||
"- **HTTPS:** `https://github.com/DAARION-DAO/daarion-ai-city.git`",
|
||
"- **Remote Name:** `daarion-city`",
|
||
"- **Main Branch:** `main`",
|
||
"- **Purpose:** Official DAARION.city website and integrations",
|
||
"",
|
||
"---",
|
||
""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# GitHub Repositories Configuration",
|
||
"REPOSITORIES = {",
|
||
" \"microdao-daarion\": {",
|
||
" \"name\": \"MicroDAO\",",
|
||
" \"ssh_url\": \"git@github.com:IvanTytar/microdao-daarion\",",
|
||
" \"https_url\": \"https://github.com/IvanTytar/microdao-daarion\",",
|
||
" \"remote_name\": \"origin\",",
|
||
" \"main_branch\": \"main\",",
|
||
" \"purpose\": \"MicroDAO core code, DAGI Stack, documentation\",",
|
||
" \"clone_cmd\": \"git clone git@github.com:IvanTytar/microdao-daarion\"",
|
||
" },",
|
||
" \"daarion-ai-city\": {",
|
||
" \"name\": \"DAARION.city\",",
|
||
" \"ssh_url\": \"git@github.com:DAARION-DAO/daarion-ai-city.git\",",
|
||
" \"https_url\": \"https://github.com/DAARION-DAO/daarion-ai-city.git\",",
|
||
" \"remote_name\": \"daarion-city\",",
|
||
" \"main_branch\": \"main\",",
|
||
" \"purpose\": \"Official DAARION.city website and integrations\",",
|
||
" \"clone_cmd\": \"git clone git@github.com:DAARION-DAO/daarion-ai-city.git\"",
|
||
" }",
|
||
"}",
|
||
"",
|
||
"print(\"GitHub Repositories:\")",
|
||
"print(\"=\"*80)",
|
||
"for repo_id, repo in REPOSITORIES.items():",
|
||
" print(f\"\\n{repo['name']} ({repo_id})\")",
|
||
" print(f\" SSH URL: {repo['ssh_url']}\")",
|
||
" print(f\" HTTPS URL: {repo['https_url']}\")",
|
||
" print(f\" Remote: {repo['remote_name']}\")",
|
||
" print(f\" Branch: {repo['main_branch']}\")",
|
||
" print(f\" Purpose: {repo['purpose']}\")",
|
||
" print(f\" Clone: {repo['clone_cmd']}\")",
|
||
"",
|
||
"print(\"\\n\" + \"=\"*80)",
|
||
"print(\"\\nQuick Commands:\")",
|
||
"print(\"\\n# Clone MicroDAO:\")",
|
||
"print(\"git clone git@github.com:IvanTytar/microdao-daarion.git\")",
|
||
"print(\"\\n# Clone DAARION.city:\")",
|
||
"print(\"git clone git@github.com:DAARION-DAO/daarion-ai-city.git\")",
|
||
"print(\"\\n# Add DAARION.city as remote to MicroDAO:\")",
|
||
"print(\"cd microdao-daarion\")",
|
||
"print(\"git remote add daarion-city git@github.com:DAARION-DAO/daarion-ai-city.git\")",
|
||
"print(\"git fetch daarion-city\")",
|
||
""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"🤖 Для агентів Cursor: SSH доступ до НОДА1\n",
|
||
"\n",
|
||
"### Підключення до Production Server\n",
|
||
"\n",
|
||
"**SSH команда:**\n",
|
||
"```bash\n",
|
||
"ssh root@144.76.224.179\n",
|
||
"```\n",
|
||
"\n",
|
||
"**Робоча директорія:** `/opt/microdao-daarion`\n",
|
||
"\n",
|
||
"**Важливо:**\n",
|
||
"- SSH ключ має бути налаштований локально\n",
|
||
"- Працюєте від імені `root`\n",
|
||
"- Завжди перевіряйте `hostname` і `pwd` перед виконанням команд\n",
|
||
"- Не виконуйте деструктивні команди без підтвердження\n",
|
||
"\n",
|
||
"**Повна інструкція:** див. `INFRASTRUCTURE.md` → Для агентів Cursor"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# SSH Access for Cursor Agents\n",
|
||
"NODE1_ACCESS = {\n",
|
||
" \"host\": \"144.76.224.179\",\n",
|
||
" \"user\": \"root\",\n",
|
||
" \"ssh_command\": \"ssh root@144.76.224.179\",\n",
|
||
" \"project_root\": \"/opt/microdao-daarion\",\n",
|
||
" \"auth\": \"SSH key (configured locally)\",\n",
|
||
" \"common_commands\": [\n",
|
||
" \"docker ps\",\n",
|
||
" \"docker compose ps\",\n",
|
||
" \"docker logs <container_name> --tail 50\",\n",
|
||
" \"git status\",\n",
|
||
" \"git pull origin main\",\n",
|
||
" \"systemctl status docker\"\n",
|
||
" ],\n",
|
||
" \"safety_checks\": [\n",
|
||
" \"Always verify hostname before executing commands\",\n",
|
||
" \"Never use 'rm -rf' without confirmation\",\n",
|
||
" \"Never use 'docker rm -f' on production containers\",\n",
|
||
" \"Always check current directory with 'pwd'\",\n",
|
||
" \"Document all changes in git commits\"\n",
|
||
" ]\n",
|
||
"}\n",
|
||
"\n",
|
||
"print(\"🔐 SSH Access to NODE1:\")\n",
|
||
"print(\"=\"*60)\n",
|
||
"print(f\"Host: {NODE1_ACCESS['host']}\")\n",
|
||
"print(f\"User: {NODE1_ACCESS['user']}\")\n",
|
||
"print(f\"Command: {NODE1_ACCESS['ssh_command']}\")\n",
|
||
"print(f\"Project: {NODE1_ACCESS['project_root']}\")\n",
|
||
"print(f\"Auth: {NODE1_ACCESS['auth']}\")\n",
|
||
"print(\"\\nCommon Commands:\")\n",
|
||
"for cmd in NODE1_ACCESS['common_commands']:\n",
|
||
" print(f\" - {cmd}\")\n",
|
||
"print(\"\\n⚠️ Safety Checks:\")\n",
|
||
"for check in NODE1_ACCESS['safety_checks']:\n",
|
||
" print(f\" • {check}\")\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"⌘ Vision Encoder Service (NEW)\n",
|
||
"\n",
|
||
"### Overview\n",
|
||
"- **Service:** Vision Encoder (OpenCLIP ViT-L/14)\n",
|
||
"- **Port:** 8001\n",
|
||
"- **GPU:** Required (NVIDIA CUDA)\n",
|
||
"- **Embedding Dimension:** 768\n",
|
||
"- **Vector DB:** Qdrant (port 6333/6334)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Vision Encoder Configuration\n",
|
||
"VISION_ENCODER = {\n",
|
||
" \"service\": \"vision-encoder\",\n",
|
||
" \"port\": 8001,\n",
|
||
" \"container\": \"dagi-vision-encoder\",\n",
|
||
" \"gpu_required\": True,\n",
|
||
" \"model\": \"ViT-L-14\",\n",
|
||
" \"pretrained\": \"openai\",\n",
|
||
" \"embedding_dim\": 768,\n",
|
||
" \"endpoints\": {\n",
|
||
" \"health\": \"http://localhost:8001/health\",\n",
|
||
" \"info\": \"http://localhost:8001/info\",\n",
|
||
" \"embed_text\": \"http://localhost:8001/embed/text\",\n",
|
||
" \"embed_image\": \"http://localhost:8001/embed/image\",\n",
|
||
" \"docs\": \"http://localhost:8001/docs\"\n",
|
||
" },\n",
|
||
" \"qdrant\": {\n",
|
||
" \"host\": \"qdrant\",\n",
|
||
" \"port\": 6333,\n",
|
||
" \"grpc_port\": 6334,\n",
|
||
" \"health\": \"http://localhost:6333/healthz\"\n",
|
||
" }\n",
|
||
"}\n",
|
||
"\n",
|
||
"print(\"Vision Encoder Service Configuration:\")\n",
|
||
"print(\"=\"*80)\n",
|
||
"print(f\"Model: {VISION_ENCODER['model']} ({VISION_ENCODER['pretrained']})\")\n",
|
||
"print(f\"Embedding Dimension: {VISION_ENCODER['embedding_dim']}\")\n",
|
||
"print(f\"GPU Required: {VISION_ENCODER['gpu_required']}\")\n",
|
||
"print(f\"\\nEndpoints:\")\n",
|
||
"for name, url in VISION_ENCODER['endpoints'].items():\n",
|
||
" print(f\" {name:15} {url}\")\n",
|
||
"print(f\"\\nQdrant Vector DB:\")\n",
|
||
"print(f\" HTTP: http://localhost:{VISION_ENCODER['qdrant']['port']}\")\n",
|
||
"print(f\" gRPC: localhost:{VISION_ENCODER['qdrant']['grpc_port']}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Vision Encoder Testing Commands\n",
|
||
"VISION_ENCODER_TESTS = {\n",
|
||
" \"Health Check\": \"curl http://localhost:8001/health\",\n",
|
||
" \"Model Info\": \"curl http://localhost:8001/info\",\n",
|
||
" \"Text Embedding\": '''curl -X POST http://localhost:8001/embed/text -H \"Content-Type: application/json\" -d '{\"text\": \"DAARION governance\", \"normalize\": true}' ''',\n",
|
||
" \"Image Embedding\": '''curl -X POST http://localhost:8001/embed/image -H \"Content-Type: application/json\" -d '{\"image_url\": \"https://example.com/image.jpg\", \"normalize\": true}' ''',\n",
|
||
" \"Via Router (Text)\": '''curl -X POST http://localhost:9102/route -H \"Content-Type: application/json\" -d '{\"mode\": \"vision_embed\", \"message\": \"embed text\", \"payload\": {\"operation\": \"embed_text\", \"text\": \"test\", \"normalize\": true}}' ''',\n",
|
||
" \"Qdrant Health\": \"curl http://localhost:6333/healthz\",\n",
|
||
" \"Run Smoke Tests\": \"./test-vision-encoder.sh\"\n",
|
||
"}\n",
|
||
"\n",
|
||
"print(\"Vision Encoder Testing Commands:\")\n",
|
||
"print(\"=\"*80)\n",
|
||
"for name, cmd in VISION_ENCODER_TESTS.items():\n",
|
||
" print(f\"\\n{name}:\")\n",
|
||
" print(f\" {cmd}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 📖 Documentation Links (UPDATED)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Documentation References (UPDATED)\n",
|
||
"DOCS = {\n",
|
||
" \"Main Guide\": \"../WARP.md\",\n",
|
||
" \"Infrastructure\": \"../INFRASTRUCTURE.md\",\n",
|
||
" \"Agents Map\": \"../docs/agents.md\",\n",
|
||
" \"RAG Ingestion Status\": \"../RAG-INGESTION-STATUS.md\",\n",
|
||
" \"HMM Memory Status\": \"../HMM-MEMORY-STATUS.md\",\n",
|
||
" \"Crawl4AI Status\": \"../CRAWL4AI-STATUS.md\",\n",
|
||
" \"Vision Encoder Status\": \"../VISION-ENCODER-STATUS.md\",\n",
|
||
" \"Vision Encoder Deployment\": \"../services/vision-encoder/README.md\",\n",
|
||
" \"Repository Management\": \"../DAARION_CITY_REPO.md\",\n",
|
||
" \"Server Setup\": \"../SERVER_SETUP_INSTRUCTIONS.md\",\n",
|
||
" \"Deployment\": \"../DEPLOY-NOW.md\",\n",
|
||
" \"Helion Status\": \"../STATUS-HELION.md\",\n",
|
||
" \"Architecture Index\": \"../docs/cursor/README.md\",\n",
|
||
" \"API Reference\": \"../docs/api.md\",\n",
|
||
" \"Node #2 Specs\": \"../NODE-2-MACBOOK-SPECS.md\"\n",
|
||
"}\n",
|
||
"\n",
|
||
"print(\"Documentation Quick Links:\")\n",
|
||
"print(\"=\"*80)\n",
|
||
"for name, path in DOCS.items():\n",
|
||
" print(f\"{name:<30} {path}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 🎤 Мультимодальні Сервіси (НОДА2)\n",
|
||
"\n",
|
||
"Нові сервіси для розширення можливостей агентів:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import pandas as pd\n",
|
||
"\n",
|
||
"multimodal_services = {\n",
|
||
" \"STT Service\": {\n",
|
||
" \"url\": \"http://192.168.1.244:8895\",\n",
|
||
" \"technology\": \"OpenAI Whisper AI\",\n",
|
||
" \"features\": [\"Voice→Text\", \"Ukrainian/English/Russian\", \"Telegram integration\"],\n",
|
||
" \"endpoints\": [\"/api/stt\", \"/api/stt/upload\", \"/health\"],\n",
|
||
" \"status\": \"✅ Ready\"\n",
|
||
" },\n",
|
||
" \"OCR Service\": {\n",
|
||
" \"url\": \"http://192.168.1.244:8896\",\n",
|
||
" \"technology\": \"Tesseract + EasyOCR\",\n",
|
||
" \"features\": [\"Image→Text\", \"Bounding boxes\", \"6 languages\", \"Confidence scores\"],\n",
|
||
" \"endpoints\": [\"/api/ocr\", \"/api/ocr/upload\", \"/health\"],\n",
|
||
" \"status\": \"✅ Ready\"\n",
|
||
" },\n",
|
||
" \"Web Search\": {\n",
|
||
" \"url\": \"http://192.168.1.244:8897\",\n",
|
||
" \"technology\": \"DuckDuckGo + Google\",\n",
|
||
" \"features\": [\"Real-time search\", \"Region-specific\", \"10+ results\"],\n",
|
||
" \"endpoints\": [\"/api/search\", \"/health\"],\n",
|
||
" \"status\": \"✅ Ready\"\n",
|
||
" },\n",
|
||
" \"Vector DB\": {\n",
|
||
" \"url\": \"http://192.168.1.244:8898\",\n",
|
||
" \"technology\": \"ChromaDB + Sentence Transformers\",\n",
|
||
" \"features\": [\"Vector database\", \"Semantic search\", \"RAG support\"],\n",
|
||
" \"endpoints\": [\"/api/collections\", \"/api/documents\", \"/api/search\", \"/health\"],\n",
|
||
" \"status\": \"✅ Ready\"\n",
|
||
" }\n",
|
||
"}\n",
|
||
"\n",
|
||
"pd.DataFrame(multimodal_services).T\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"ейсу"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"vision_agents = {\n",
|
||
" \"Sofia\": {\n",
|
||
" \"model\": \"grok-4.1\",\n",
|
||
" \"provider\": \"xAI\",\n",
|
||
" \"supports_vision\": True,\n",
|
||
" \"supports_files\": True,\n",
|
||
" \"description\": \"Vision + Code analysis\"\n",
|
||
" },\n",
|
||
" \"Spectra\": {\n",
|
||
" \"model\": \"qwen3-vl:latest\",\n",
|
||
" \"provider\": \"Ollama\",\n",
|
||
" \"supports_vision\": True,\n",
|
||
" \"supports_files\": False,\n",
|
||
" \"description\": \"Vision + Language\"\n",
|
||
" }\n",
|
||
"}\n",
|
||
"\n",
|
||
"pd.DataFrame(vision_agents).T\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 📊 Всі порти сервісів (оновлено)\n",
|
||
"\n",
|
||
"Повний список всіх сервісів з портами:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"all_ports = {\n",
|
||
" \"Frontend\": {\"port\": 8899, \"node\": \"Local\", \"status\": \"✅ Active\"},\n",
|
||
" \"STT Service\": {\"port\": 8895, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
|
||
" \"OCR Service\": {\"port\": 8896, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
|
||
" \"Web Search\": {\"port\": 8897, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
|
||
" \"Vector DB\": {\"port\": 8898, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
|
||
" \"Router\": {\"port\": 9102, \"node\": \"NODE1\", \"status\": \"🔄 Multimodal\"},\n",
|
||
" \"Telegram Gateway\": {\"port\": 9200, \"node\": \"NODE1\", \"status\": \"🔄 Enhanced\"},\n",
|
||
" \"Swapper NODE1\": {\"port\": 8890, \"node\": \"NODE1\", \"status\": \"✅ Active\"},\n",
|
||
" \"Swapper NODE2\": {\"port\": 8890, \"node\": \"НОДА2\", \"status\": \"✅ Active\"},\n",
|
||
" \"Agent Cabinet\": {\"port\": 8898, \"node\": \"Local\", \"status\": \"✅ Active\"},\n",
|
||
" \"Memory Service\": {\"port\": 8000, \"node\": \"NODE1/2\", \"status\": \"✅ Active\"}\n",
|
||
"}\n",
|
||
"\n",
|
||
"pd.DataFrame(all_ports).T\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 🔄 Мультимодальні можливості\n",
|
||
"\n",
|
||
"Статус інтеграції різних типів контенту:\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"multimodal_capabilities = {\n",
|
||
" \"Текст\": {\"frontend\": \"✅\", \"telegram\": \"✅\", \"status\": \"ПРАЦЮЄ\"},\n",
|
||
" \"Голос→Текст\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
|
||
" \"Зображення→Vision\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
|
||
" \"Зображення→OCR\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
|
||
" \"Документи\": {\"frontend\": \"✅\", \"telegram\": \"⚠️\", \"status\": \"ЧАСТКОВА\"},\n",
|
||
" \"Веб-пошук\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
|
||
" \"Knowledge Base\": {\"frontend\": \"✅\", \"telegram\": \"❌\", \"status\": \"ГОТОВИЙ\"}\n",
|
||
"}\n",
|
||
"\n",
|
||
"pd.DataFrame(multimodal_capabilities).T\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 🔒 Security & Incident Response\n",
|
||
"\n",
|
||
"### Incident #1: Network Scanning & Lockdown (Dec 6, 2025 - Jan 8, 2026)\n",
|
||
"\n",
|
||
"**Root Cause:** Compromised `daarion-web` container with cryptocurrency miner (`catcal`, `G4NQXBp`)\n",
|
||
"**Impact:** Server locked by Hetzner for 33 days due to internal network scanning\n",
|
||
"**Resolution:** Container removed, firewall rules implemented, monitoring deployed\n",
|
||
"\n",
|
||
"### Incident #2: Recurring Compromise (Jan 9, 2026) 🔴 ACTIVE\n",
|
||
"\n",
|
||
"**Root Cause:** Compromised Docker image auto-restarted after server reboot \n",
|
||
"**Malware:** NEW crypto miners (`softirq`, `vrarhpb`) - different from Incident #1 \n",
|
||
"**Impact:** \n",
|
||
"- ❌ Second abuse report (AbuseID: 10F3971:2A)\n",
|
||
"- ❌ Critical CPU load: 25-35 (normal: 1-5)\n",
|
||
"- ❌ 1499 zombie processes\n",
|
||
"- ⚠️ Deadline: 2026-01-09 12:54 UTC (~3.5 hours remaining)\n",
|
||
"\n",
|
||
"**Resolution (COMPLETED):** \n",
|
||
"1. ✅ Killed all malicious processes (softirq, vrarhpb)\n",
|
||
"2. ✅ Stopped and removed `daarion-web` container\n",
|
||
"3. ✅ **DELETED Docker images** (78e22c0ee972, 608e203fb5ac) - critical step\n",
|
||
"4. ✅ Cleaned 1499 zombie processes → 5 (normal)\n",
|
||
"5. ✅ System load normalized: 30+ → 4.19\n",
|
||
"6. ✅ Enhanced firewall (SSH rate limiting, port scan blocking)\n",
|
||
"7. ✅ Registered retry test with Hetzner\n",
|
||
"8. ⏳ **PENDING:** User statement submission (URGENT)\n",
|
||
"\n",
|
||
"**Why Incident #2 Occurred:** \n",
|
||
"- Incident #1 removed container but LEFT Docker image intact\n",
|
||
"- Container had `restart: unless-stopped` in docker-compose.yml\n",
|
||
"- Server rebooted → docker-compose auto-restarted from compromised image\n",
|
||
"- NEW malware variant installed (different miners than Incident #1)\n",
|
||
"\n",
|
||
"**What is daarion-web?** \n",
|
||
"- Next.js frontend (port 3000) - NOT critical for core functionality\n",
|
||
"- ✅ Router, Gateway, Telegram bots, API - ALL WORKING\n",
|
||
"- Status: DISABLED until secure rebuild completed\n",
|
||
"\n",
|
||
"**Lessons Learned (Critical):** \n",
|
||
"1. 🔴 **ALWAYS delete Docker images, not just containers**\n",
|
||
"2. 🟡 **Auto-restart policies are dangerous for compromised containers**\n",
|
||
"3. 🟢 **Compromised images can survive container removal**\n",
|
||
"4. 🔵 **Complete removal = container + image + restart policy change**\n",
|
||
"\n",
|
||
"**Next Steps:** \n",
|
||
"1. 🔴 **URGENT:** Submit statement to Hetzner before deadline\n",
|
||
"2. 🟡 Monitor server for 24 hours post-statement\n",
|
||
"3. 🟢 Secure rebuild of daarion-web (see `TASK_REBUILD_DAARION_WEB.md`)\n",
|
||
"4. 🔵 Security audit all remaining containers\n",
|
||
"\n",
|
||
"### Security Measures\n",
|
||
"\n",
|
||
"1. **Egress Firewall Rules** (блокування внутрішніх мереж Hetzner)\n",
|
||
"2. **Monitoring Script** (`/root/monitor_scanning.sh`, runs every 15 min)\n",
|
||
"3. **Security Checklist:**\n",
|
||
" - [ ] Container vulnerability scanning\n",
|
||
" - [ ] Docker Content Trust\n",
|
||
" - [ ] Resource limits (CPU/memory)\n",
|
||
" - [ ] Network segmentation\n",
|
||
" - [ ] Regular security audits\n",
|
||
"\n",
|
||
"**Full details:** See `INFRASTRUCTURE.md` → Security & Incident Response section\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": null,
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Security Configuration (UPDATED with Incident #2)\n",
|
||
"security_config = {\n",
|
||
" \"Firewall Rules\": {\n",
|
||
" \"scripts\": [\"/root/prevent_scanning.sh\", \"/root/block_ssh_scanning.sh\"],\n",
|
||
" \"status\": \"✅ Enhanced\",\n",
|
||
" \"blocks\": [\"10.0.0.0/8\", \"172.16.0.0/12\"],\n",
|
||
" \"allows\": [\"80/tcp\", \"443/tcp\"],\n",
|
||
" \"features\": [\"SSH rate limiting\", \"Port scan blocking\", \"Enhanced logging\"]\n",
|
||
" },\n",
|
||
" \"Monitoring\": {\n",
|
||
" \"script\": \"/root/monitor_scanning.sh\",\n",
|
||
" \"status\": \"✅ Active\",\n",
|
||
" \"interval\": \"15 minutes\",\n",
|
||
" \"log\": \"/var/log/scan_attempts.log\"\n",
|
||
" },\n",
|
||
" \"Incident #1\": {\n",
|
||
" \"date\": \"2025-12-06\",\n",
|
||
" \"malware\": \"catcal, G4NQXBp\",\n",
|
||
" \"recovery_time\": \"33 days\",\n",
|
||
" \"status\": \"✅ Resolved\"\n",
|
||
" },\n",
|
||
" \"Incident #2\": {\n",
|
||
" \"date\": \"2026-01-09\",\n",
|
||
" \"malware\": \"softirq, vrarhpb\",\n",
|
||
" \"mitigation_time\": \"30 minutes\",\n",
|
||
" \"status\": \"⏳ Statement Pending\",\n",
|
||
" \"deadline\": \"2026-01-09 12:54 UTC\",\n",
|
||
" \"actions\": [\"Container removed\", \"Images DELETED\", \"Load normalized\", \"Retry test registered\"]\n",
|
||
" }\n",
|
||
"}\n",
|
||
"\n",
|
||
"import pandas as pd\n",
|
||
"print(\"🔒 Security Configuration:\")\n",
|
||
"print(\"=\" * 80)\n",
|
||
"pd.DataFrame(security_config).T\n"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"metadata": {},
|
||
"source": [
|
||
"## 📝 Notes & Updates\n",
|
||
"\n",
|
||
"### Recent Changes (2025-11-23)\n",
|
||
"- ✅ **Swapper Service інтеграція** в кабінети НОД (тільки в `/nodes/node-1`, `/nodes/node-2`)\n",
|
||
"- ✅ **Оновлення в реальному часі** (кожні 30 секунд) для Swapper Service\n",
|
||
"- ✅ **Кабінети мікроДАО** з оркестраторами (DAARION, GREENFOOD, ENERGY UNION)\n",
|
||
"- ✅ **Управління мікроДАО** в кабінеті DAARION (панель управління всіма мікроДАО)\n",
|
||
"- ✅ **Детальні метрики Swapper Service** (моделі, спеціалісти, конфігурація)\n",
|
||
"- ✅ **Frontend** (port 8899) з кабінетами НОД та мікроДАО\n",
|
||
"- ✅ **Agent Cabinet Service** (port 8898) для метрик агентів\n",
|
||
"\n",
|
||
"### Network Architecture\n",
|
||
"- **Nodes:** 2 (1 production + 1 development)\n",
|
||
"- **Total Services:** 19 (додано Frontend + Agent Cabinet)\n",
|
||
"- **Swapper Service:** Тільки в кабінетах НОД, оновлення в реальному часі\n",
|
||
"- **MicroDAO Cabinets:** 3 (DAARION, GREENFOOD, ENERGY UNION)\n",
|
||
"- **Node Cabinets:** 2 (НОДА1, НОДА2)\n",
|
||
"\n",
|
||
"### Кабінети НОД\n",
|
||
"- **НОДА1:** `http://localhost:8899/nodes/node-1`\n",
|
||
"- **НОДА2:** `http://localhost:8899/nodes/node-2`\n",
|
||
"- **Swapper Service:** Відображається тільки тут, оновлення кожні 30 секунд\n",
|
||
"\n",
|
||
"### Кабінети МікроДАО\n",
|
||
"- **DAARION:** `http://localhost:8899/microdao/daarion` (оркестратор: DAARWIZZ)\n",
|
||
"- **GREENFOOD:** `http://localhost:8899/microdao/greenfood` (оркестратор: GREENFOOD)\n",
|
||
"- **ENERGY UNION:** `http://localhost:8899/microdao/energy-union` (оркестратор: Helion)\n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"**Last Updated:** 2026-01-09 (Security Incident #2 - Emergency mitigation completed) \n",
|
||
"**Maintained by:** Ivan Tytar & DAARION Team \n",
|
||
"\n",
|
||
"---\n",
|
||
"\n",
|
||
"### 🚨 CRITICAL: Active Security Incident\n",
|
||
"- **Incident ID:** 10F3971:2A (Hetzner AbuseID)\n",
|
||
"- **Status:** Mitigation completed, statement submission pending\n",
|
||
"- **Deadline:** 2026-01-09 12:54:00 UTC (~3.5 hours remaining)\n",
|
||
"- **Action Required:** User MUST submit statement at https://statement-abuse.hetzner.com/statements/?token=28b2c7e67a409659f6c823e863887\n",
|
||
"- **Task Document:** `/Users/apple/github-projects/microdao-daarion/TASK_REBUILD_DAARION_WEB.md`"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.11.0"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 4
|
||
} |