P2: Global multi-node model selection + NCS on NODA1

Architecture for 150+ nodes:
- global_capabilities_client.py: NATS scatter-gather discovery using
  wildcard subject node.*.capabilities.get — zero static node lists.
  New nodes auto-register by deploying NCS and subscribing to NATS.
  Dead nodes expire from cache after 3x TTL automatically.

Multi-node model_select.py:
- ModelSelection now includes node, local, via_nats fields
- select_best_model prefers local candidates, then remote
- Prefer list resolution: local first, remote second
- All logged per request: node, runtime, model, local/remote

NODA1 compose:
- Added node-capabilities service (NCS) to docker-compose.node1.yml
- NATS subscription: node.noda1.capabilities.get
- Router env: NODE_CAPABILITIES_URL + ENABLE_GLOBAL_CAPS_NATS=true

NODA2 compose:
- Router env: ENABLE_GLOBAL_CAPS_NATS=true

Router main.py:
- Startup: initializes global_capabilities_client (NATS connect + first
  discovery). Falls back to local-only capabilities_client if unavailable.
- /infer: uses get_global_capabilities() for cross-node model pool
- Offload support: send_offload_request(node_id, type, payload) via NATS

Verified on NODA2:
- Global caps: 1 node, 14 models (NODA1 not yet deployed)
- Sofiia: cloud_grok → grok-4-1-fast-reasoning (OK)
- Helion: NCS → qwen3:14b local (OK)
- When NODA1 deploys NCS, its models appear automatically via NATS discovery

Made-with: Cursor
This commit is contained in:
Apple
2026-02-27 02:26:12 -08:00
parent 89c3f2ac66
commit a92c424845
5 changed files with 575 additions and 62 deletions

View File

@@ -27,7 +27,7 @@ services:
- DEEPSEEK_API_KEY=sk-0db94e8193ec4a6e9acd593ee8d898e7
- MISTRAL_API_KEY=40Gwjo8nVBx4i4vIkgszvXw9bOwDOu4G
- COHERE_API_KEY=nOdOXnuepLku2ipJWpe6acWgAsJCsDhMO0RnaEJB
- GROK_API_KEY=xai-69zEnDse8qRuQyZATs9jVKgfwdyvkHzgEVrTbV0OTAurZqsjHmvGepXG6H9GhVRYEC7E4NFl6iZeG0ww
- GROK_API_KEY=xai-CpoLMPgw91NP9AEdHPhIrvU4ZnhV1q1P8BJBKCpD5kTPFRXJmTOkgGNHwYdZpXMlRxBgHcgcSlIXccxh
- VISION_ENCODER_URL=http://vision-encoder:8001
- SWAPPER_SERVICE_URL=http://swapper-service:8890
- IMAGE_GEN_URL=http://swapper-service:8890/image/generate
@@ -35,12 +35,22 @@ services:
- STT_SERVICE_UPLOAD_URL=http://swapper-service:8890/stt
- OCR_SERVICE_URL=http://swapper-service:8890
- WEB_SEARCH_SERVICE_URL=http://swapper-service:8890
- REDIS_URL=redis://redis:6379/0
- CREWAI_SERVICE_URL=http://dagi-staging-crewai-service:9010
- NATURE_ID_URL=http://plant-vision-node1:8085
- NATURE_ID_MIN_CONFIDENCE=0.65
- PLANTNET_API_KEY=${PLANTNET_API_KEY}
- ONEOK_CRM_BASE_URL=http://oneok-crm-adapter:8088
- ONEOK_CALC_BASE_URL=http://oneok-calc-adapter:8089
- ONEOK_DOCS_BASE_URL=http://oneok-docs-adapter:8090
- ONEOK_SCHEDULE_BASE_URL=http://oneok-schedule-adapter:8091
- ONEOK_ADAPTER_API_KEY=${ONEOK_ADAPTER_API_KEY}
- ROUTER_TOOL_MAX_ROUNDS=${ROUTER_TOOL_MAX_ROUNDS:-10}
- AGROMATRIX_REVIEW_AUTH_MODE=${AGROMATRIX_REVIEW_AUTH_MODE:-bearer}
- AGROMATRIX_REVIEW_BEARER_TOKENS=${AGROMATRIX_REVIEW_BEARER_TOKENS}
# ── Node Capabilities (multi-node model selection) ──
- NODE_CAPABILITIES_URL=http://node-capabilities:8099/capabilities
- ENABLE_GLOBAL_CAPS_NATS=true
volumes:
- ${DEPLOY_ROOT:-.}/services/router/router_config.yaml:/app/router_config.yaml:ro
- ${DEPLOY_ROOT:-.}/services/router/router-config.yml:/app/router-config.yml:ro
@@ -77,7 +87,7 @@ services:
- CUDA_VISIBLE_DEVICES=0
- CRAWL4AI_URL=http://crawl4ai:11235
# Cloud API keys for video/image generation
- GROK_API_KEY=xai-69zEnDse8qRuQyZATs9jVKgfwdyvkHzgEVrTbV0OTAurZqsjHmvGepXG6H9GhVRYEC7E4NFl6iZeG0ww
- GROK_API_KEY=xai-CpoLMPgw91NP9AEdHPhIrvU4ZnhV1q1P8BJBKCpD5kTPFRXJmTOkgGNHwYdZpXMlRxBgHcgcSlIXccxh
- MISTRAL_API_KEY=40Gwjo8nVBx4i4vIkgszvXw9bOwDOu4G
volumes:
- ${DEPLOY_ROOT:-.}/services/swapper-service/config/swapper_config_node1.yaml:/app/config/swapper_config.yaml:ro
@@ -106,6 +116,28 @@ services:
# Image Generation тепер інтегровано в Swapper Service (lazy loading)
# Endpoint: POST /image/generate на swapper-service:8890
# Plant Vision wrapper (local nature-id CLI -> HTTP)
plant-vision-node1:
build:
context: ./services/plant-vision-node1
dockerfile: Dockerfile
container_name: plant-vision-node1
environment:
- NATURE_ID_CMD=${NATURE_ID_CMD:-python /opt/nature-id/nature_id.py -m plants -l -r 5 -s {image_path}}
- NATURE_ID_TIMEOUT=40
- DOWNLOAD_TIMEOUT=20
networks:
- dagi-network
volumes:
- ${DEPLOY_ROOT:-.}/third_party/nature-id:/opt/nature-id:ro
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8085/health')\""]
interval: 30s
timeout: 10s
retries: 3
start_period: 15s
# Crawl4AI - Advanced Web Crawler with JavaScript support
crawl4ai:
image: unclecode/crawl4ai@sha256:4d8b065bf185962733cb5f9701f4122d03383fa1ab6b5f6a9873f04fa0416a84
@@ -134,7 +166,11 @@ services:
ports:
- "9300:9300"
environment:
- ROUTER_URL=http://router:8000
- ROUTER_URL=${ROUTER_URL:-http://dagi-staging-router:8000}
- GATEWAY_MAX_TOKENS_CONCISE=350
- GATEWAY_MAX_TOKENS_SENPAI_DEFAULT=700
- GATEWAY_MAX_TOKENS_DEFAULT=700
- GATEWAY_MAX_TOKENS_DETAILED=1200
- SERVICE_ID=gateway
- SERVICE_ROLE=gateway
- BRAND_INTAKE_URL=http://brand-intake:9211
@@ -191,12 +227,25 @@ services:
- STT_SERVICE_UPLOAD_URL=http://swapper-service:8890/stt
- OCR_SERVICE_URL=http://swapper-service:8890
- WEB_SEARCH_SERVICE_URL=http://swapper-service:8890
- REDIS_URL=redis://redis:6379/0
- CREWAI_SERVICE_URL=http://dagi-staging-crewai-service:9010
- AGROMATRIX_REVIEW_AUTH_MODE=${AGROMATRIX_REVIEW_AUTH_MODE:-bearer}
- AGROMATRIX_REVIEW_BEARER_TOKENS=${AGROMATRIX_REVIEW_BEARER_TOKENS}
# v4.3 FarmOS integration (fail-closed: якщо пусто — агент повідомить "не налаштований")
- FARMOS_BASE_URL=http://dagi-farmos-node1
- FARMOS_TOKEN=${FARMOS_TOKEN:-}
- FARMOS_USER=${FARMOS_USER:-}
- FARMOS_PASS=${FARMOS_PASS:-}
- FARMOS_CLIENT_ID=${FARMOS_CLIENT_ID:-farm}
env_file:
- .env.stepan.node1
volumes:
- ${DEPLOY_ROOT:-.}/gateway-bot:/app/gateway-bot:ro
- ${DEPLOY_ROOT:-.}/logs:/app/logs
depends_on:
- router
- memory-service
- redis
networks:
- dagi-network
restart: unless-stopped
@@ -207,6 +256,107 @@ services:
retries: 3
start_period: 10s
gateway-worker:
build:
context: ./gateway-bot
dockerfile: Dockerfile
container_name: dagi-gateway-worker-node1
command: ["python", "-m", "daarion_facade.worker"]
environment:
- ROUTER_BASE_URL=http://router:8000
- REDIS_URL=redis://redis:6379/0
- ROUTER_WORKER_TIMEOUT=60
volumes:
- ${DEPLOY_ROOT:-.}/gateway-bot:/app/gateway-bot:ro
- ${DEPLOY_ROOT:-.}/logs:/app/logs
depends_on:
- router
- redis
networks:
- dagi-network
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "print(\"ok\")"]
interval: 30s
timeout: 5s
retries: 3
gateway-reminder-worker:
build:
context: ./gateway-bot
dockerfile: Dockerfile
container_name: dagi-gateway-reminder-worker-node1
command: ["python", "-m", "daarion_facade.reminder_worker"]
environment:
- REDIS_URL=redis://redis:6379/0
- DAARION_REMINDER_POLL_SECONDS=${DAARION_REMINDER_POLL_SECONDS:-2}
- DAARION_REMINDER_TTL_SECONDS=${DAARION_REMINDER_TTL_SECONDS:-2592000}
- DAARION_REMINDER_DEFAULT_TZ=${DAARION_REMINDER_DEFAULT_TZ:-Europe/Kyiv}
- GLOBAL_RELAY_ALLOWED_USER_IDS=${GLOBAL_RELAY_ALLOWED_USER_IDS:-}
- MENTOR_PRIVATE_HANDLES=${MENTOR_PRIVATE_HANDLES:-ivantytar,archenvis,olegarch88}
- MENTOR_PRIVATE_NAMES=${MENTOR_PRIVATE_NAMES:-Іван Титар,Александр Вертій,Олег Ковальчук}
- MENTOR_DISCLOSURE_ALLOWED_USER_IDS=${MENTOR_DISCLOSURE_ALLOWED_USER_IDS:-}
- HELION_MENTOR_CHAT_IDS=${HELION_MENTOR_CHAT_IDS:-}
- HELION_RELAY_ALLOWED_USER_IDS=${HELION_RELAY_ALLOWED_USER_IDS:-}
- DAARWIZZ_TELEGRAM_BOT_TOKEN=${DAARWIZZ_TELEGRAM_BOT_TOKEN:-8323412397:AAGZbAR22LuOiGD8xVC3OXMjahQ8rs2lJwo}
- HELION_TELEGRAM_BOT_TOKEN=${HELION_TELEGRAM_BOT_TOKEN:-8112062582:AAGS-HwRLEI269lDutLtAJTFArsIq31YNhE}
- GREENFOOD_TELEGRAM_BOT_TOKEN=${GREENFOOD_TELEGRAM_BOT_TOKEN:-7495165343:AAGR1XEOzg7DkPFPCzL_eYLCJfxJuonCxug}
- AGROMATRIX_TELEGRAM_BOT_TOKEN=${AGROMATRIX_TELEGRAM_BOT_TOKEN:-8580290441:AAFuDBmFJtpl-3I_WfkH7Hkb59X0fhYNMOE}
- ALATEYA_TELEGRAM_BOT_TOKEN=${ALATEYA_TELEGRAM_BOT_TOKEN:-8436880945:AAEi-HS6GEctddoqBUd37MHfweZQP-OjRlo}
- NUTRA_TELEGRAM_BOT_TOKEN=${NUTRA_TELEGRAM_BOT_TOKEN:-8517315428:AAGTLcKxBAZDsMgx28agKTvl1SqJGi0utH4}
- DRUID_TELEGRAM_BOT_TOKEN=${DRUID_TELEGRAM_BOT_TOKEN:-8145618489:AAFR714mBsNmiuF-rjCw-295iORBReJQZ70}
- CLAN_TELEGRAM_BOT_TOKEN=${CLAN_TELEGRAM_BOT_TOKEN:-8516872152:AAHH26wU8hJZJbSCJXb4vbmPmakTP77ok5E}
- EONARCH_TELEGRAM_BOT_TOKEN=${EONARCH_TELEGRAM_BOT_TOKEN:-7962391584:AAFYkelLRG3VR_Lxuu6pEGG76t4vZdANtz4}
- SENPAI_TELEGRAM_BOT_TOKEN=${SENPAI_TELEGRAM_BOT_TOKEN:-8510265026:AAGFrFBIIEihsLptZSxuKdmW2RoRPQDY9FE}
- ONEOK_TELEGRAM_BOT_TOKEN=${ONEOK_TELEGRAM_BOT_TOKEN}
- SOUL_TELEGRAM_BOT_TOKEN=${SOUL_TELEGRAM_BOT_TOKEN:-8041596416:AAHhpfCtY8paCm_9AD-4stJJg-Vw-CBf6Qk}
- YAROMIR_TELEGRAM_BOT_TOKEN=${YAROMIR_TELEGRAM_BOT_TOKEN:-8128180674:AAGNZdG3LwECI4z_803smsuRHsK3nPdjMLY}
- SOFIIA_TELEGRAM_BOT_TOKEN=${SOFIIA_TELEGRAM_BOT_TOKEN:-8589292566:AAEmPvS6nY9e-Y-TZm04CAHWlaFnWVxajE4}
volumes:
- ${DEPLOY_ROOT:-.}/gateway-bot:/app/gateway-bot:ro
- ${DEPLOY_ROOT:-.}/logs:/app/logs
depends_on:
- redis
networks:
- dagi-network
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "print(\"ok\")"]
interval: 30s
timeout: 5s
retries: 3
metrics-poller-node1:
build:
context: ./gateway-bot
dockerfile: Dockerfile
container_name: dagi-metrics-poller-node1
command: ["python", "-m", "daarion_facade.metrics_poller"]
environment:
- REDIS_URL=redis://redis:6379/0
- MEMORY_SERVICE_URL=http://memory-service:8000
- DAARION_METRICS_POLL_INTERVAL_SECONDS=${DAARION_METRICS_POLL_INTERVAL_SECONDS:-10}
- DAARION_METRICS_TTL_SECONDS=${DAARION_METRICS_TTL_SECONDS:-60}
- DAARION_METRICS_HTTP_CONNECT_TIMEOUT_SECONDS=${DAARION_METRICS_HTTP_CONNECT_TIMEOUT_SECONDS:-2}
- DAARION_METRICS_HTTP_TOTAL_TIMEOUT_SECONDS=${DAARION_METRICS_HTTP_TOTAL_TIMEOUT_SECONDS:-5}
- DAARION_NODE_COUNT=${DAARION_NODE_COUNT:-1}
volumes:
- ${DEPLOY_ROOT:-.}/gateway-bot:/app/gateway-bot:ro
- ${DEPLOY_ROOT:-.}/logs:/app/logs
depends_on:
- redis
- memory-service
networks:
- dagi-network
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "print(\"ok\")"]
interval: 30s
timeout: 5s
retries: 3
# CLAN Consent Outbox Worker (Postgres event-store applier; no execute)
clan-consent-outbox-worker:
build:
@@ -340,6 +490,29 @@ services:
- dagi-network
restart: unless-stopped
# Node Capabilities Service (model inventory for router)
node-capabilities:
build:
context: ./services/node-capabilities
dockerfile: Dockerfile
container_name: node-capabilities-node1
environment:
- NODE_ID=noda1
- OLLAMA_BASE_URL=http://host.docker.internal:11434
- SWAPPER_URL=http://swapper-service:8890
- CACHE_TTL_SEC=15
- ENABLE_NATS_CAPS=true
- NATS_URL=nats://nats:4222
extra_hosts:
- "host.docker.internal:host-gateway"
depends_on:
- nats
networks:
dagi-network:
aliases:
- node-capabilities
restart: unless-stopped
# NATS (JetStream)
nats:
image: nats:2.10-alpine
@@ -736,10 +909,11 @@ services:
ports:
- "9108:9108"
environment:
- GATEWAY_URL=http://172.18.0.18:9300
- GATEWAY_URL=http://gateway:9300
- PROBE_INTERVAL=60
- PROBE_TIMEOUT=30
- METRICS_PORT=9108
- SEMANTIC_AGENTS=clan,sofiia,monitor,helion,agromatrix,senpai
networks:
- dagi-network
restart: unless-stopped
@@ -819,6 +993,72 @@ services:
retries: 5
start_period: 15s
binance-bot-monitor:
build:
context: ./services/binance-bot-monitor
dockerfile: Dockerfile
container_name: dagi-binance-bot-monitor-node1
restart: unless-stopped
environment:
- REDIS_URL=redis://redis:6379/0
- CRAWL4AI_URL=http://crawl4ai:11235
- SWAPPER_URL=http://swapper-service:8890
- BINANCE_CACHE_TTL=3600
- BINANCE_REFRESH_INTERVAL=1800
- BINANCE_API_KEY=${BINANCE_API_KEY:-}
- BINANCE_SECRET_KEY=${BINANCE_SECRET_KEY:-}
networks:
- dagi-network
# ── FarmOS (v4.3 integration) ────────────────────────────────────────────────
# PostgreSQL для farmOS (окрема БД, не чіпає dagi-postgres)
dagi-farmos-db-node1:
image: postgres:16-alpine
container_name: dagi-farmos-db-node1
restart: unless-stopped
environment:
- POSTGRES_DB=farmos
- POSTGRES_USER=farmos
- POSTGRES_PASSWORD=${FARMOS_DB_PASS}
volumes:
- farmos-db-data-node1:/var/lib/postgresql/data
networks:
- dagi-network
healthcheck:
test: ["CMD-SHELL", "pg_isready -U farmos -d farmos"]
interval: 10s
timeout: 5s
retries: 10
start_period: 15s
# farmOS Drupal application (4.x — актуальна стабільна, amd64 для x86_64 сервера)
dagi-farmos-node1:
image: farmos/farmos:4.x-amd64
container_name: dagi-farmos-node1
restart: unless-stopped
depends_on:
dagi-farmos-db-node1:
condition: service_healthy
environment:
- FARMOS_DB_HOST=dagi-farmos-db-node1
- FARMOS_DB_NAME=farmos
- FARMOS_DB_USER=farmos
- FARMOS_DB_PASSWORD=${FARMOS_DB_PASS}
- FARMOS_DB_DRIVER=pgsql
volumes:
- farmos-sites-node1:/opt/drupal/web/sites
networks:
- dagi-network
ports:
# Доступний тільки локально; для браузерного setup — SSH tunnel: ssh -L 8088:localhost:8088
- "127.0.0.1:8088:80"
healthcheck:
test: ["CMD-SHELL", "curl -fsS http://localhost:80 -o /dev/null || exit 1"]
interval: 30s
timeout: 10s
retries: 5
start_period: 60s
volumes:
qdrant-data-node1:
@@ -871,6 +1111,14 @@ volumes:
name: oneok-crm-data-node1
driver: local
# farmOS persistent volumes (v4.3)
farmos-db-data-node1:
name: farmos-db-data-node1
driver: local
farmos-sites-node1:
name: farmos-sites-node1
driver: local
networks:
dagi-network:
external: true