NATS wildcards (node.*.capabilities.get) only work for subscriptions,
not for publish. Switch to a dedicated broadcast subject
(fabric.capabilities.discover) that all NCS instances subscribe to,
enabling proper scatter-gather discovery across nodes.
Made-with: Cursor
Architecture for 150+ nodes:
- global_capabilities_client.py: NATS scatter-gather discovery using
wildcard subject node.*.capabilities.get — zero static node lists.
New nodes auto-register by deploying NCS and subscribing to NATS.
Dead nodes expire from cache after 3x TTL automatically.
Multi-node model_select.py:
- ModelSelection now includes node, local, via_nats fields
- select_best_model prefers local candidates, then remote
- Prefer list resolution: local first, remote second
- All logged per request: node, runtime, model, local/remote
NODA1 compose:
- Added node-capabilities service (NCS) to docker-compose.node1.yml
- NATS subscription: node.noda1.capabilities.get
- Router env: NODE_CAPABILITIES_URL + ENABLE_GLOBAL_CAPS_NATS=true
NODA2 compose:
- Router env: ENABLE_GLOBAL_CAPS_NATS=true
Router main.py:
- Startup: initializes global_capabilities_client (NATS connect + first
discovery). Falls back to local-only capabilities_client if unavailable.
- /infer: uses get_global_capabilities() for cross-node model pool
- Offload support: send_offload_request(node_id, type, payload) via NATS
Verified on NODA2:
- Global caps: 1 node, 14 models (NODA1 not yet deployed)
- Sofiia: cloud_grok → grok-4-1-fast-reasoning (OK)
- Helion: NCS → qwen3:14b local (OK)
- When NODA1 deploys NCS, its models appear automatically via NATS discovery
Made-with: Cursor
