P2.2+P2.3: NATS offload node-worker + router offload integration
Node Worker (services/node-worker/):
- NATS subscriber for node.{NODE_ID}.llm.request / vision.request
- Canonical JobRequest/JobResponse envelope (Pydantic)
- Idempotency cache (TTL 10min) with inflight dedup
- Deadline enforcement (DEADLINE_EXCEEDED on expired jobs)
- Concurrency limiter (semaphore, returns busy)
- Ollama + Swapper vision providers
Router offload (services/router/offload_client.py):
- NATS req/reply with configurable retries
- Circuit breaker per node+type (3 fails/60s → open 120s)
- Concurrency semaphore for remote requests
Model selection (services/router/model_select.py):
- exclude_nodes parameter for circuit-broken nodes
- force_local flag for fallback re-selection
- Integrated circuit breaker state awareness
Router /infer pipeline:
- Remote offload path when NCS selects remote node
- Automatic fallback: exclude failed node → force_local re-select
- Deadline propagation from router to node-worker
Tests: 17 unit tests (idempotency, deadline, circuit breaker)
Docs: ops/offload_routing.md (subjects, envelope, verification)
Made-with: Cursor
This commit is contained in:
34
tests/test_node_worker_deadline.py
Normal file
34
tests/test_node_worker_deadline.py
Normal file
@@ -0,0 +1,34 @@
|
||||
"""Tests for node-worker deadline handling."""
|
||||
import sys
|
||||
import os
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "services", "node-worker"))
|
||||
|
||||
from models import JobRequest
|
||||
|
||||
|
||||
def test_remaining_ms_with_future_deadline():
|
||||
job = JobRequest(deadline_ts=int(time.time() * 1000) + 10000)
|
||||
rem = job.remaining_ms()
|
||||
assert 9000 <= rem <= 10500
|
||||
|
||||
|
||||
def test_remaining_ms_with_past_deadline():
|
||||
job = JobRequest(deadline_ts=int(time.time() * 1000) - 5000)
|
||||
assert job.remaining_ms() == 0
|
||||
|
||||
|
||||
def test_remaining_ms_no_deadline():
|
||||
job = JobRequest(deadline_ts=0)
|
||||
assert job.remaining_ms() == 30_000
|
||||
|
||||
|
||||
def test_effective_idem_key_uses_idempotency_key():
|
||||
job = JobRequest(job_id="j1", idempotency_key="custom-key")
|
||||
assert job.effective_idem_key() == "custom-key"
|
||||
|
||||
|
||||
def test_effective_idem_key_falls_back_to_job_id():
|
||||
job = JobRequest(job_id="j2", idempotency_key="")
|
||||
assert job.effective_idem_key() == "j2"
|
||||
Reference in New Issue
Block a user