daarion-admin/microdao-daarion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
744c1493006f7fab4103edb347d3311008f9daef
microdao-daarion/docs/infrastructure_quick_ref.ipynb
Apple 744c149300
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details
✨ Add automated session logging system 
- Created logs/ structure (sessions, operations, incidents)
- Added session-start/log/end scripts
- Installed Git hooks for auto-logging commits/pushes
- Added shell integration for zsh
- Created CHANGELOG.md
- Documented today's session (2026-01-10)
2026-01-10 04:53:17 -08:00

823 lines
35 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 🚀 Infrastructure Quick Reference — DAARION & MicroDAO\n",
"\n",
"Версія:** 2.4.0 \n",
"Останнє оновлення:** 2026-01-10 XX:XX \n",
"\n",
"Цей notebook містить швидкий довідник по серверах, репозиторіях та endpoints для DAGI Stack.\n",
"\n",
"**🔴 CRITICAL (v2.4.0) - Jan 10, 2026:**\n",
"- 🔴 **Incident #4: NODE1 Host Compromise Suspected**\n",
"- ❌ ALL PostgreSQL images show malware on NODE1 (15-alpine, 16-alpine, 14, 16)\n",
"- ⚠️ **NODE1 UNSAFE** - Do not deploy any containers until verified\n",
"- 📋 **Triage script added**: `scripts/security/triage-postgres-compromise.sh`\n",
"- 🔬 **Verification required**: Test same image digest on clean host\n",
"\n",
"**v2.3.0:** \n",
"- 🖥️ **NODE3 added** - Threadripper PRO 5975WX + RTX 3090 24GB\n",
"- 🚀 Most powerful node for AI/ML workloads (32c/64t, 128GB RAM, 4TB NVMe)\n",
"- ✅ Security verified - clean system\n",
"\n",
"**v2.2.0:** \n",
"- 🔒 **Security Incident #2** (Jan 9, 2026) - Emergency mitigation completed\n",
"- ⚠️ **daarion-web permanently disabled** until secure rebuild\n",
"- ✅ Enhanced firewall rules + retry test registered with Hetzner\n",
"\n",
"**v2.1.0:** \n",
"- 🔒 **Security Incident #1 Resolved** (Dec 2025 - Jan 2026)\n",
"- ✅ Firewall rules + monitoring deployed\n",
"\n",
"**v2.0.0:** \n",
"- ✅ Мультимодальні сервіси (STT, OCR, Web Search, Vector DB) на НОДА2\n",
"- ✅ Router Multimodal Support (інтеграція в процесі)\n",
"- ✅ Telegram Gateway Enhanced (STT + Vision)\n",
"- ✅ Swapper Service інтеграція в кабінети НОД\n",
"- ✅ Кабінети мікроДАО з оркестраторами\n",
"- ✅ Оновлення в реальному часі (кожні 30 секунд)\n",
"- ✅ Управління мікроДАО в кабінеті DAARION"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Service Configuration (UPDATED with Swapper Service + Frontend + Agent Cabinet)\n",
"SERVICES = {\n",
" \"router\": {\"port\": 9102, \"container\": \"dagi-router\", \"health\": \"http://localhost:9102/health\"},\n",
" \"gateway\": {\"port\": 9300, \"container\": \"dagi-gateway\", \"health\": \"http://localhost:9300/health\"},\n",
" \"devtools\": {\"port\": 8008, \"container\": \"dagi-devtools\", \"health\": \"http://localhost:8008/health\"},\n",
" \"crewai\": {\"port\": 9010, \"container\": \"dagi-crewai\", \"health\": \"http://localhost:9010/health\"},\n",
" \"rbac\": {\"port\": 9200, \"container\": \"dagi-rbac\", \"health\": \"http://localhost:9200/health\"},\n",
" \"rag\": {\"port\": 9500, \"container\": \"dagi-rag-service\", \"health\": \"http://localhost:9500/health\"},\n",
" \"memory\": {\"port\": 8000, \"container\": \"dagi-memory-service\", \"health\": \"http://localhost:8000/health\"},\n",
" \"parser\": {\"port\": 9400, \"container\": \"dagi-parser-service\", \"health\": \"http://localhost:9400/health\"},\n",
" \"swapper\": {\"port\": 8890, \"container\": \"swapper-service\", \"health\": \"http://localhost:8890/health\", \"node1\": \"http://144.76.224.179:8890\", \"node2\": \"http://192.168.1.244:8890\"},\n",
" \"frontend\": {\"port\": 8899, \"container\": \"frontend\", \"health\": \"http://localhost:8899\"},\n",
" \"agent_cabinet\": {\"port\": 8898, \"container\": \"agent-cabinet-service\", \"health\": \"http://localhost:8898/health\"},\n",
" \"postgres\": {\"port\": 5432, \"container\": \"dagi-postgres\", \"health\": None},\n",
" \"redis\": {\"port\": 6379, \"container\": \"redis\", \"health\": \"redis-cli PING\"},\n",
" \"neo4j\": {\"port\": 7474, \"container\": \"neo4j\", \"health\": \"http://localhost:7474\"},\n",
" \"qdrant\": {\"port\": 6333, \"container\": \"dagi-qdrant\", \"health\": \"http://localhost:6333/healthz\"},\n",
" \"grafana\": {\"port\": 3000, \"container\": \"grafana\", \"health\": \"http://localhost:3000\"},\n",
" \"prometheus\": {\"port\": 9090, \"container\": \"prometheus\", \"health\": \"http://localhost:9090\"},\n",
" \"ollama\": {\"port\": 11434, \"container\": \"ollama\", \"health\": \"http://localhost:11434/api/tags\"}\n",
"}\n",
"\n",
"print(\"Service\\t\\t\\tPort\\tContainer\\t\\t\\tHealth Endpoint\")\n",
"print(\"=\"*100)\n",
"for name, service in SERVICES.items():\n",
" health = service['health'] or \"N/A\"\n",
" gpu = \" [GPU]\" if service.get('gpu') else \"\"\n",
" print(f\"{name.upper():<20} {service['port']:<7} {service['container']:<30} {health}{gpu}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🖥️ Network Nodes\n",
"\n",
"### Node #1: Production Server (Hetzner)\n",
"- **Node ID:** node-1-hetzner-gex44\n",
"- **IP:** 144.76.224.179\n",
"- **Role:** Production Router + Gateway + All Services (24/7)\n",
"- **Location:** Hetzner Cloud (Germany)\n",
"\n",
"### Node #2: Development Node (MacBook Pro M4 Max)\n",
"- **Node ID:** node-2-macbook-m4max\n",
"- **Local IP:** 192.168.1.244\n",
"- **Role:** Development + Testing + Backup Router\n",
"- **Specs:** M4 Max (16 cores), 64GB RAM, 2TB SSD, 40-core GPU\n",
"- **Location:** Local Network (Ivan's Office)\n",
"- **Docs:** [NODE-2-MACBOOK-SPECS.md](../NODE-2-MACBOOK-SPECS.md)\n",
"\n",
"### Node #3: AI/ML Workstation (Threadripper PRO + RTX 3090)\n",
"- **Node ID:** node-3-threadripper-rtx3090\n",
"- **Hostname:** llm80-che-1-1\n",
"- **IP:** 80.77.35.151:33147\n",
"- **Role:** AI/ML Workloads, GPU Inference, Kubernetes\n",
"- **CPU:** AMD Threadripper PRO 5975WX (32c/64t, 3.6GHz)\n",
"- **RAM:** 128GB DDR4\n",
"- **GPU:** NVIDIA RTX 3090 24GB (CUDA 13.0)\n",
"- **Storage:** Samsung 990 PRO 4TB NVMe\n",
"- **OS:** Ubuntu 24.04 LTS + MicroK8s\n",
"- **Security:** ✅ Clean (verified 2026-01-09)\n",
"\n",
"---"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Network Nodes Configuration\n",
"NODES = {\n",
" \"node-1\": {\n",
" \"name\": \"Hetzner GEX44\",\n",
" \"ip\": \"144.76.224.179\",\n",
" \"local_ip\": None,\n",
" \"role\": \"production\",\n",
" \"uptime\": \"24/7\",\n",
" \"ssh\": \"root@144.76.224.179\",\n",
" \"domain\": \"gateway.daarion.city\",\n",
" \"services\": \"All (17 services)\",\n",
" \"specs\": \"See SYSTEM-INVENTORY.md\"\n",
" },\n",
" \"node-2\": {\n",
" \"name\": \"MacBook Pro M4 Max\",\n",
" \"ip\": None,\n",
" \"local_ip\": \"192.168.1.244\",\n",
" \"role\": \"development\",\n",
" \"uptime\": \"on-demand\",\n",
" \"ssh\": \"apple@192.168.1.244\",\n",
" \"domain\": None,\n",
" \"services\": \"Core only (Router, DevTools, Memory, Ollama)\",\n",
" \"specs\": \"M4 Max, 16 cores, 64GB RAM, 2TB SSD, 40-core GPU\"\n",
" },\n",
" \"node-3\": {\n",
" \"name\": \"Threadripper PRO + RTX 3090\",\n",
" \"ip\": \"80.77.35.151\",\n",
" \"local_ip\": None,\n",
" \"role\": \"ai_ml_workstation\",\n",
" \"uptime\": \"24/7\",\n",
" \"ssh\": \"zevs@80.77.35.151 -p33147\",\n",
" \"hostname\": \"llm80-che-1-1\",\n",
" \"domain\": None,\n",
" \"services\": \"MicroK8s, Ollama (GPU), MongoDB, K8s services\",\n",
" \"specs\": \"Threadripper PRO 5975WX (32c/64t), 128GB RAM, RTX 3090 24GB, Samsung 990 PRO 4TB\",\n",
" \"gpu\": \"NVIDIA RTX 3090 24GB VRAM (CUDA 13.0)\",\n",
" \"os\": \"Ubuntu 24.04 LTS\",\n",
" \"security_status\": \"Clean (verified 2026-01-09)\"\n",
" }\n",
"}\n",
"\n",
"print(\"DAGI Stack Network Nodes:\")\n",
"print(\"=\"*80)\n",
"for node_id, node in NODES.items():\n",
" print(f\"\\n{node_id.upper()}: {node['name']}\")\n",
" print(f\" Role: {node['role']}\")\n",
" print(f\" IP: {node['ip'] or node['local_ip']}\")\n",
" print(f\" SSH: {node['ssh']}\")\n",
" print(f\" Uptime: {node['uptime']}\")\n",
" print(f\" Services: {node['services']}\")\n",
" if node['domain']:\n",
" print(f\" Domain: https://{node['domain']}\")\n",
" print(f\" Specs: {node['specs']}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🐙 GitHub Repositories\n",
"\n",
"### 1. MicroDAO (Current Project)\n",
"- **Repository:** `git@github.com:IvanTytar/microdao-daarion`\n",
"- **HTTPS:** `https://github.com/IvanTytar/microdao-daarion`\n",
"- **Remote Name:** `origin`\n",
"- **Main Branch:** `main`\n",
"- **Purpose:** MicroDAO core code, DAGI Stack, documentation\n",
"\n",
"### 2. DAARION.city\n",
"- **Repository:** `git@github.com:DAARION-DAO/daarion-ai-city.git`\n",
"- **HTTPS:** `https://github.com/DAARION-DAO/daarion-ai-city.git`\n",
"- **Remote Name:** `daarion-city`\n",
"- **Main Branch:** `main`\n",
"- **Purpose:** Official DAARION.city website and integrations\n",
"\n",
"---\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# GitHub Repositories Configuration\n",
"REPOSITORIES = {\n",
" \"microdao-daarion\": {\n",
" \"name\": \"MicroDAO\",\n",
" \"ssh_url\": \"git@github.com:IvanTytar/microdao-daarion\",\n",
" \"https_url\": \"https://github.com/IvanTytar/microdao-daarion\",\n",
" \"remote_name\": \"origin\",\n",
" \"main_branch\": \"main\",\n",
" \"purpose\": \"MicroDAO core code, DAGI Stack, documentation\",\n",
" \"clone_cmd\": \"git clone git@github.com:IvanTytar/microdao-daarion\"\n",
" },\n",
" \"daarion-ai-city\": {\n",
" \"name\": \"DAARION.city\",\n",
" \"ssh_url\": \"git@github.com:DAARION-DAO/daarion-ai-city.git\",\n",
" \"https_url\": \"https://github.com/DAARION-DAO/daarion-ai-city.git\",\n",
" \"remote_name\": \"daarion-city\",\n",
" \"main_branch\": \"main\",\n",
" \"purpose\": \"Official DAARION.city website and integrations\",\n",
" \"clone_cmd\": \"git clone git@github.com:DAARION-DAO/daarion-ai-city.git\"\n",
" }\n",
"}\n",
"\n",
"print(\"GitHub Repositories:\")\n",
"print(\"=\"*80)\n",
"for repo_id, repo in REPOSITORIES.items():\n",
" print(f\"\\n{repo['name']} ({repo_id})\")\n",
" print(f\" SSH URL: {repo['ssh_url']}\")\n",
" print(f\" HTTPS URL: {repo['https_url']}\")\n",
" print(f\" Remote: {repo['remote_name']}\")\n",
" print(f\" Branch: {repo['main_branch']}\")\n",
" print(f\" Purpose: {repo['purpose']}\")\n",
" print(f\" Clone: {repo['clone_cmd']}\")\n",
"\n",
"print(\"\\n\" + \"=\"*80)\n",
"print(\"\\nQuick Commands:\")\n",
"print(\"\\n# Clone MicroDAO:\")\n",
"print(\"git clone git@github.com:IvanTytar/microdao-daarion.git\")\n",
"print(\"\\n# Clone DAARION.city:\")\n",
"print(\"git clone git@github.com:DAARION-DAO/daarion-ai-city.git\")\n",
"print(\"\\n# Add DAARION.city as remote to MicroDAO:\")\n",
"print(\"cd microdao-daarion\")\n",
"print(\"git remote add daarion-city git@github.com:DAARION-DAO/daarion-ai-city.git\")\n",
"print(\"git fetch daarion-city\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"🤖 Для агентів Cursor: SSH доступ до НОДА1\n",
"\n",
"### Підключення до Production Server\n",
"\n",
"**SSH команда:**\n",
"```bash\n",
"ssh root@144.76.224.179\n",
"```\n",
"\n",
"**Робоча директорія:** `/opt/microdao-daarion`\n",
"\n",
"**Важливо:**\n",
"- SSH ключ має бути налаштований локально\n",
"- Працюєте від імені `root`\n",
"- Завжди перевіряйте `hostname` і `pwd` перед виконанням команд\n",
"- Не виконуйте деструктивні команди без підтвердження\n",
"\n",
"**Повна інструкція:** див. `INFRASTRUCTURE.md` → Для агентів Cursor"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# SSH Access for Cursor Agents\n",
"NODE1_ACCESS = {\n",
" \"host\": \"144.76.224.179\",\n",
" \"user\": \"root\",\n",
" \"ssh_command\": \"ssh root@144.76.224.179\",\n",
" \"project_root\": \"/opt/microdao-daarion\",\n",
" \"auth\": \"SSH key (configured locally)\",\n",
" \"common_commands\": [\n",
" \"docker ps\",\n",
" \"docker compose ps\",\n",
" \"docker logs <container_name> --tail 50\",\n",
" \"git status\",\n",
" \"git pull origin main\",\n",
" \"systemctl status docker\"\n",
" ],\n",
" \"safety_checks\": [\n",
" \"Always verify hostname before executing commands\",\n",
" \"Never use 'rm -rf' without confirmation\",\n",
" \"Never use 'docker rm -f' on production containers\",\n",
" \"Always check current directory with 'pwd'\",\n",
" \"Document all changes in git commits\"\n",
" ]\n",
"}\n",
"\n",
"print(\"🔐 SSH Access to NODE1:\")\n",
"print(\"=\"*60)\n",
"print(f\"Host: {NODE1_ACCESS['host']}\")\n",
"print(f\"User: {NODE1_ACCESS['user']}\")\n",
"print(f\"Command: {NODE1_ACCESS['ssh_command']}\")\n",
"print(f\"Project: {NODE1_ACCESS['project_root']}\")\n",
"print(f\"Auth: {NODE1_ACCESS['auth']}\")\n",
"print(\"\\nCommon Commands:\")\n",
"for cmd in NODE1_ACCESS['common_commands']:\n",
" print(f\" - {cmd}\")\n",
"print(\"\\n⚠️ Safety Checks:\")\n",
"for check in NODE1_ACCESS['safety_checks']:\n",
" print(f\" • {check}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"⌘ Vision Encoder Service (NEW)\n",
"\n",
"### Overview\n",
"- **Service:** Vision Encoder (OpenCLIP ViT-L/14)\n",
"- **Port:** 8001\n",
"- **GPU:** Required (NVIDIA CUDA)\n",
"- **Embedding Dimension:** 768\n",
"- **Vector DB:** Qdrant (port 6333/6334)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Vision Encoder Configuration\n",
"VISION_ENCODER = {\n",
" \"service\": \"vision-encoder\",\n",
" \"port\": 8001,\n",
" \"container\": \"dagi-vision-encoder\",\n",
" \"gpu_required\": True,\n",
" \"model\": \"ViT-L-14\",\n",
" \"pretrained\": \"openai\",\n",
" \"embedding_dim\": 768,\n",
" \"endpoints\": {\n",
" \"health\": \"http://localhost:8001/health\",\n",
" \"info\": \"http://localhost:8001/info\",\n",
" \"embed_text\": \"http://localhost:8001/embed/text\",\n",
" \"embed_image\": \"http://localhost:8001/embed/image\",\n",
" \"docs\": \"http://localhost:8001/docs\"\n",
" },\n",
" \"qdrant\": {\n",
" \"host\": \"qdrant\",\n",
" \"port\": 6333,\n",
" \"grpc_port\": 6334,\n",
" \"health\": \"http://localhost:6333/healthz\"\n",
" }\n",
"}\n",
"\n",
"print(\"Vision Encoder Service Configuration:\")\n",
"print(\"=\"*80)\n",
"print(f\"Model: {VISION_ENCODER['model']} ({VISION_ENCODER['pretrained']})\")\n",
"print(f\"Embedding Dimension: {VISION_ENCODER['embedding_dim']}\")\n",
"print(f\"GPU Required: {VISION_ENCODER['gpu_required']}\")\n",
"print(f\"\\nEndpoints:\")\n",
"for name, url in VISION_ENCODER['endpoints'].items():\n",
" print(f\" {name:15} {url}\")\n",
"print(f\"\\nQdrant Vector DB:\")\n",
"print(f\" HTTP: http://localhost:{VISION_ENCODER['qdrant']['port']}\")\n",
"print(f\" gRPC: localhost:{VISION_ENCODER['qdrant']['grpc_port']}\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Vision Encoder Testing Commands\n",
"VISION_ENCODER_TESTS = {\n",
" \"Health Check\": \"curl http://localhost:8001/health\",\n",
" \"Model Info\": \"curl http://localhost:8001/info\",\n",
" \"Text Embedding\": '''curl -X POST http://localhost:8001/embed/text -H \"Content-Type: application/json\" -d '{\"text\": \"DAARION governance\", \"normalize\": true}' ''',\n",
" \"Image Embedding\": '''curl -X POST http://localhost:8001/embed/image -H \"Content-Type: application/json\" -d '{\"image_url\": \"https://example.com/image.jpg\", \"normalize\": true}' ''',\n",
" \"Via Router (Text)\": '''curl -X POST http://localhost:9102/route -H \"Content-Type: application/json\" -d '{\"mode\": \"vision_embed\", \"message\": \"embed text\", \"payload\": {\"operation\": \"embed_text\", \"text\": \"test\", \"normalize\": true}}' ''',\n",
" \"Qdrant Health\": \"curl http://localhost:6333/healthz\",\n",
" \"Run Smoke Tests\": \"./test-vision-encoder.sh\"\n",
"}\n",
"\n",
"print(\"Vision Encoder Testing Commands:\")\n",
"print(\"=\"*80)\n",
"for name, cmd in VISION_ENCODER_TESTS.items():\n",
" print(f\"\\n{name}:\")\n",
" print(f\" {cmd}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 📖 Documentation Links (UPDATED)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Documentation References (UPDATED)\n",
"DOCS = {\n",
" \"Main Guide\": \"../WARP.md\",\n",
" \"Infrastructure\": \"../INFRASTRUCTURE.md\",\n",
" \"Agents Map\": \"../docs/agents.md\",\n",
" \"RAG Ingestion Status\": \"../RAG-INGESTION-STATUS.md\",\n",
" \"HMM Memory Status\": \"../HMM-MEMORY-STATUS.md\",\n",
" \"Crawl4AI Status\": \"../CRAWL4AI-STATUS.md\",\n",
" \"Vision Encoder Status\": \"../VISION-ENCODER-STATUS.md\",\n",
" \"Vision Encoder Deployment\": \"../services/vision-encoder/README.md\",\n",
" \"Repository Management\": \"../DAARION_CITY_REPO.md\",\n",
" \"Server Setup\": \"../SERVER_SETUP_INSTRUCTIONS.md\",\n",
" \"Deployment\": \"../DEPLOY-NOW.md\",\n",
" \"Helion Status\": \"../STATUS-HELION.md\",\n",
" \"Architecture Index\": \"../docs/cursor/README.md\",\n",
" \"API Reference\": \"../docs/api.md\",\n",
" \"Node #2 Specs\": \"../NODE-2-MACBOOK-SPECS.md\"\n",
"}\n",
"\n",
"print(\"Documentation Quick Links:\")\n",
"print(\"=\"*80)\n",
"for name, path in DOCS.items():\n",
" print(f\"{name:<30} {path}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🎤 Мультимодальні Сервіси (НОДА2)\n",
"\n",
"Нові сервіси для розширення можливостей агентів:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"\n",
"multimodal_services = {\n",
" \"STT Service\": {\n",
" \"url\": \"http://192.168.1.244:8895\",\n",
" \"technology\": \"OpenAI Whisper AI\",\n",
" \"features\": [\"Voice→Text\", \"Ukrainian/English/Russian\", \"Telegram integration\"],\n",
" \"endpoints\": [\"/api/stt\", \"/api/stt/upload\", \"/health\"],\n",
" \"status\": \"✅ Ready\"\n",
" },\n",
" \"OCR Service\": {\n",
" \"url\": \"http://192.168.1.244:8896\",\n",
" \"technology\": \"Tesseract + EasyOCR\",\n",
" \"features\": [\"Image→Text\", \"Bounding boxes\", \"6 languages\", \"Confidence scores\"],\n",
" \"endpoints\": [\"/api/ocr\", \"/api/ocr/upload\", \"/health\"],\n",
" \"status\": \"✅ Ready\"\n",
" },\n",
" \"Web Search\": {\n",
" \"url\": \"http://192.168.1.244:8897\",\n",
" \"technology\": \"DuckDuckGo + Google\",\n",
" \"features\": [\"Real-time search\", \"Region-specific\", \"10+ results\"],\n",
" \"endpoints\": [\"/api/search\", \"/health\"],\n",
" \"status\": \"✅ Ready\"\n",
" },\n",
" \"Vector DB\": {\n",
" \"url\": \"http://192.168.1.244:8898\",\n",
" \"technology\": \"ChromaDB + Sentence Transformers\",\n",
" \"features\": [\"Vector database\", \"Semantic search\", \"RAG support\"],\n",
" \"endpoints\": [\"/api/collections\", \"/api/documents\", \"/api/search\", \"/health\"],\n",
" \"status\": \"✅ Ready\"\n",
" }\n",
"}\n",
"\n",
"pd.DataFrame(multimodal_services).T\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"ейсу"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"vision_agents = {\n",
" \"Sofia\": {\n",
" \"model\": \"grok-4.1\",\n",
" \"provider\": \"xAI\",\n",
" \"supports_vision\": True,\n",
" \"supports_files\": True,\n",
" \"description\": \"Vision + Code analysis\"\n",
" },\n",
" \"Spectra\": {\n",
" \"model\": \"qwen3-vl:latest\",\n",
" \"provider\": \"Ollama\",\n",
" \"supports_vision\": True,\n",
" \"supports_files\": False,\n",
" \"description\": \"Vision + Language\"\n",
" }\n",
"}\n",
"\n",
"pd.DataFrame(vision_agents).T\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 📊 Всі порти сервісів (оновлено)\n",
"\n",
"Повний список всіх сервісів з портами:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"all_ports = {\n",
" \"Frontend\": {\"port\": 8899, \"node\": \"Local\", \"status\": \"✅ Active\"},\n",
" \"STT Service\": {\"port\": 8895, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
" \"OCR Service\": {\"port\": 8896, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
" \"Web Search\": {\"port\": 8897, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
" \"Vector DB\": {\"port\": 8898, \"node\": \"НОДА2\", \"status\": \"✅ Ready\"},\n",
" \"Router\": {\"port\": 9102, \"node\": \"NODE1\", \"status\": \"🔄 Multimodal\"},\n",
" \"Telegram Gateway\": {\"port\": 9200, \"node\": \"NODE1\", \"status\": \"🔄 Enhanced\"},\n",
" \"Swapper NODE1\": {\"port\": 8890, \"node\": \"NODE1\", \"status\": \"✅ Active\"},\n",
" \"Swapper NODE2\": {\"port\": 8890, \"node\": \"НОДА2\", \"status\": \"✅ Active\"},\n",
" \"Agent Cabinet\": {\"port\": 8898, \"node\": \"Local\", \"status\": \"✅ Active\"},\n",
" \"Memory Service\": {\"port\": 8000, \"node\": \"NODE1/2\", \"status\": \"✅ Active\"}\n",
"}\n",
"\n",
"pd.DataFrame(all_ports).T\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🔄 Мультимодальні можливості\n",
"\n",
"Статус інтеграції різних типів контенту:\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"multimodal_capabilities = {\n",
" \"Текст\": {\"frontend\": \"✅\", \"telegram\": \"✅\", \"status\": \"ПРАЦЮЄ\"},\n",
" \"Голос→Текст\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
" \"Зображення→Vision\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
" \"Зображення→OCR\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
" \"Документи\": {\"frontend\": \"✅\", \"telegram\": \"⚠️\", \"status\": \"ЧАСТКОВА\"},\n",
" \"Веб-пошук\": {\"frontend\": \"✅\", \"telegram\": \"🔄\", \"status\": \"ІНТЕГРАЦІЯ\"},\n",
" \"Knowledge Base\": {\"frontend\": \"✅\", \"telegram\": \"❌\", \"status\": \"ГОТОВИЙ\"}\n",
"}\n",
"\n",
"pd.DataFrame(multimodal_capabilities).T\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🔒 Security & Incident Response\n",
"\n",
"### Incident #1: Network Scanning & Lockdown (Dec 6, 2025 - Jan 8, 2026)\n",
"\n",
"**Root Cause:** Compromised `daarion-web` container with cryptocurrency miner (`catcal`, `G4NQXBp`)\n",
"**Impact:** Server locked by Hetzner for 33 days due to internal network scanning\n",
"**Resolution:** Container removed, firewall rules implemented, monitoring deployed\n",
"\n",
"### Incident #2: Recurring Compromise (Jan 9, 2026) 🔴 ACTIVE\n",
"\n",
"**Root Cause:** Compromised Docker image auto-restarted after server reboot \n",
"**Malware:** NEW crypto miners (`softirq`, `vrarhpb`) - different from Incident #1 \n",
"**Impact:** \n",
"- ❌ Second abuse report (AbuseID: 10F3971:2A)\n",
"- ❌ Critical CPU load: 25-35 (normal: 1-5)\n",
"- ❌ 1499 zombie processes\n",
"- ⚠️ Deadline: 2026-01-09 12:54 UTC (~3.5 hours remaining)\n",
"\n",
"**Resolution (COMPLETED):** \n",
"1. ✅ Killed all malicious processes (softirq, vrarhpb)\n",
"2. ✅ Stopped and removed `daarion-web` container\n",
"3. ✅ **DELETED Docker images** (78e22c0ee972, 608e203fb5ac) - critical step\n",
"4. ✅ Cleaned 1499 zombie processes → 5 (normal)\n",
"5. ✅ System load normalized: 30+ → 4.19\n",
"6. ✅ Enhanced firewall (SSH rate limiting, port scan blocking)\n",
"7. ✅ Registered retry test with Hetzner\n",
"8. ⏳ **PENDING:** User statement submission (URGENT)\n",
"\n",
"**Why Incident #2 Occurred:** \n",
"- Incident #1 removed container but LEFT Docker image intact\n",
"- Container had `restart: unless-stopped` in docker-compose.yml\n",
"- Server rebooted → docker-compose auto-restarted from compromised image\n",
"- NEW malware variant installed (different miners than Incident #1)\n",
"\n",
"**What is daarion-web?** \n",
"- Next.js frontend (port 3000) - NOT critical for core functionality\n",
"- ✅ Router, Gateway, Telegram bots, API - ALL WORKING\n",
"- Status: DISABLED until secure rebuild completed\n",
"\n",
"**Lessons Learned (Critical):** \n",
"1. 🔴 **ALWAYS delete Docker images, not just containers**\n",
"2. 🟡 **Auto-restart policies are dangerous for compromised containers**\n",
"3. 🟢 **Compromised images can survive container removal**\n",
"4. 🔵 **Complete removal = container + image + restart policy change**\n",
"\n",
"**Next Steps:** \n",
"1. 🔴 **URGENT:** Submit statement to Hetzner before deadline\n",
"2. 🟡 Monitor server for 24 hours post-statement\n",
"3. 🟢 Secure rebuild of daarion-web (see `TASK_REBUILD_DAARION_WEB.md`)\n",
"4. 🔵 Security audit all remaining containers\n",
"\n",
"### Security Measures\n",
"\n",
"1. **Egress Firewall Rules** (блокування внутрішніх мереж Hetzner)\n",
"2. **Monitoring Script** (`/root/monitor_scanning.sh`, runs every 15 min)\n",
"3. **Security Checklist:**\n",
" - [ ] Container vulnerability scanning\n",
" - [ ] Docker Content Trust\n",
" - [ ] Resource limits (CPU/memory)\n",
" - [ ] Network segmentation\n",
" - [ ] Regular security audits\n",
"\n",
"**Full details:** See `INFRASTRUCTURE.md` → Security & Incident Response section\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Security Configuration (UPDATED with Incident #2)\n",
"security_config = {\n",
" \"Firewall Rules\": {\n",
" \"scripts\": [\"/root/prevent_scanning.sh\", \"/root/block_ssh_scanning.sh\"],\n",
" \"status\": \"✅ Enhanced\",\n",
" \"blocks\": [\"10.0.0.0/8\", \"172.16.0.0/12\"],\n",
" \"allows\": [\"80/tcp\", \"443/tcp\"],\n",
" \"features\": [\"SSH rate limiting\", \"Port scan blocking\", \"Enhanced logging\"]\n",
" },\n",
" \"Monitoring\": {\n",
" \"script\": \"/root/monitor_scanning.sh\",\n",
" \"status\": \"✅ Active\",\n",
" \"interval\": \"15 minutes\",\n",
" \"log\": \"/var/log/scan_attempts.log\"\n",
" },\n",
" \"Incident #1\": {\n",
" \"date\": \"2025-12-06\",\n",
" \"malware\": \"catcal, G4NQXBp\",\n",
" \"recovery_time\": \"33 days\",\n",
" \"status\": \"✅ Resolved\"\n",
" },\n",
" \"Incident #2\": {\n",
" \"date\": \"2026-01-09\",\n",
" \"malware\": \"softirq, vrarhpb\",\n",
" \"mitigation_time\": \"30 minutes\",\n",
" \"status\": \"⏳ Statement Pending\",\n",
" \"deadline\": \"2026-01-09 12:54 UTC\",\n",
" \"actions\": [\"Container removed\", \"Images DELETED\", \"Load normalized\", \"Retry test registered\"]\n",
" }\n",
"}\n",
"\n",
"import pandas as pd\n",
"print(\"🔒 Security Configuration:\")\n",
"print(\"=\" * 80)\n",
"pd.DataFrame(security_config).T\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 📝 Notes & Updates\n",
"\n",
"### Recent Changes (2025-11-23)\n",
"- ✅ **Swapper Service інтеграція** в кабінети НОД (тільки в `/nodes/node-1`, `/nodes/node-2`)\n",
"- ✅ **Оновлення в реальному часі** (кожні 30 секунд) для Swapper Service\n",
"- ✅ **Кабінети мікроДАО** з оркестраторами (DAARION, GREENFOOD, ENERGY UNION)\n",
"- ✅ **Управління мікроДАО** в кабінеті DAARION (панель управління всіма мікроДАО)\n",
"- ✅ **Детальні метрики Swapper Service** (моделі, спеціалісти, конфігурація)\n",
"- ✅ **Frontend** (port 8899) з кабінетами НОД та мікроДАО\n",
"- ✅ **Agent Cabinet Service** (port 8898) для метрик агентів\n",
"\n",
"### Network Architecture\n",
"- **Nodes:** 2 (1 production + 1 development)\n",
"- **Total Services:** 19 (додано Frontend + Agent Cabinet)\n",
"- **Swapper Service:** Тільки в кабінетах НОД, оновлення в реальному часі\n",
"- **MicroDAO Cabinets:** 3 (DAARION, GREENFOOD, ENERGY UNION)\n",
"- **Node Cabinets:** 2 (НОДА1, НОДА2)\n",
"\n",
"### Кабінети НОД\n",
"- **НОДА1:** `http://localhost:8899/nodes/node-1`\n",
"- **НОДА2:** `http://localhost:8899/nodes/node-2`\n",
"- **Swapper Service:** Відображається тільки тут, оновлення кожні 30 секунд\n",
"\n",
"### Кабінети МікроДАО\n",
"- **DAARION:** `http://localhost:8899/microdao/daarion` (оркестратор: DAARWIZZ)\n",
"- **GREENFOOD:** `http://localhost:8899/microdao/greenfood` (оркестратор: GREENFOOD)\n",
"- **ENERGY UNION:** `http://localhost:8899/microdao/energy-union` (оркестратор: Helion)\n",
"\n",
"---\n",
"\n",
"**Last Updated:** 2026-01-09 (Security Incident #2 - Emergency mitigation completed) \n",
"**Maintained by:** Ivan Tytar & DAARION Team \n",
"\n",
"---\n",
"\n",
"### 🚨 CRITICAL: Active Security Incident\n",
"- **Incident ID:** 10F3971:2A (Hetzner AbuseID)\n",
"- **Status:** Mitigation completed, statement submission pending\n",
"- **Deadline:** 2026-01-09 12:54:00 UTC (~3.5 hours remaining)\n",
"- **Action Required:** User MUST submit statement at https://statement-abuse.hetzner.com/statements/?token=28b2c7e67a409659f6c823e863887\n",
"- **Task Document:** `/Users/apple/github-projects/microdao-daarion/TASK_REBUILD_DAARION_WEB.md`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 🔴 Incident #4: NODE1 Host Compromise (Jan 10, 2026)\n",
"\n",
"### Summary\n",
"ALL PostgreSQL official images show malware artifacts when run on NODE1.\n",
"This is **NOT** \"Docker Hub compromised\" — this is **NODE1 host compromise**.\n",
"\n",
"### Indicators of Compromise (IOC)\n",
"```\n",
"/tmp/httpd # ~10MB crypto miner (xmrig variant)\n",
"/tmp/.perf.c/ # perfctl malware staging directory\n",
"/tmp/mysql # Another miner variant\n",
"/tmp/cpioshuf # perfctl payload\n",
"/tmp/ipcalc* # perfctl payload\n",
"```\n",
"\n",
"### Affected Images (on NODE1)\n",
"- ❌ postgres:15-alpine\n",
"- ❌ postgres:16-alpine\n",
"- ❌ postgres:14\n",
"- ❌ postgres:16 (Debian)\n",
"\n",
"### Why This is HOST Compromise (not image)\n",
"1. ALL different image variants show same IOC\n",
"2. Previous incidents (#1, #2, #3) already compromised NODE1\n",
"3. `/tmp/.perf.c/` is classic perfctl malware directory\n",
"4. `tmpfs noexec` didn't prevent infection\n",
"\n",
"### Verification Procedure\n",
"```bash\n",
"# Run triage script from MacBook (NOT NODE1!)\n",
"cd /Users/apple/github-projects/microdao-daarion\n",
"./scripts/security/triage-postgres-compromise.sh compare\n",
"\n",
"# Or manually:\n",
"# 1. Get digest from NODE1\n",
"ssh root@144.76.224.179 \"docker inspect --format='{{index .RepoDigests 0}}' postgres:16\"\n",
"\n",
"# 2. Pull same digest on MacBook\n",
"docker pull postgres:16@sha256:<digest>\n",
"\n",
"# 3. Check if clean\n",
"docker run --rm postgres:16@sha256:<digest> ls -la /tmp/\n",
"# If empty → NODE1 compromised, image is clean\n",
"```\n",
"\n",
"### Current Status\n",
"- ⏳ **Verification pending** — Need to test on clean host\n",
"- 🔴 **NODE1 UNSAFE** — Do not deploy PostgreSQL\n",
"- 🟡 **Secrets rotation needed** — Assume all compromised\n",
"\n",
"### Full Documentation\n",
"See `INFRASTRUCTURE.md` → Incident #4"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.0"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in New Issue View Git Blame Copy Permalink