09dee24342
Producer (market-data-service):
- Backpressure: smart drop policy (heartbeats→quotes→trades preserved)
- Heartbeat monitor: synthetic HeartbeatEvent on provider silence
- Graceful shutdown: WS→bus→storage→DB engine cleanup sequence
- Bybit V5 public WS provider (backup for Binance, no API key needed)
- FailoverManager: health-based provider switching with recovery
- NATS output adapter: md.events.{type}.{symbol} for SenpAI
- /bus-stats endpoint for backpressure monitoring
- Dockerfile + docker-compose.node1.yml integration
- 36 tests (parsing + bus + failover), requirements.lock
Consumer (senpai-md-consumer):
- NATSConsumer: subscribe md.events.>, queue group senpai-md, backpressure
- State store: LatestState + RollingWindow (deque, 60s)
- Feature engine: 11 features (mid, spread, VWAP, return, vol, latency)
- Rule-based signals: long/short on return+volume+spread conditions
- Publisher: rate-limited features + signals + alerts to NATS
- HTTP API: /health, /metrics, /state/latest, /features/latest, /stats
- 10 Prometheus metrics
- Dockerfile + docker-compose.senpai.yml
- 41 tests (parsing + state + features + rate-limit), requirements.lock
CI: ruff + pytest + smoke import for both services
Tests: 77 total passed, lint clean
Co-authored-by: Cursor <cursoragent@cursor.com>
44 lines
2.2 KiB
Plaintext
44 lines
2.2 KiB
Plaintext
# Market Data Service Configuration
|
|
# Copy to .env and fill in your values
|
|
|
|
# ── Binance (no key needed for public WebSocket) ──────────────────────
|
|
BINANCE_WS_URL=wss://stream.binance.com:9443/ws
|
|
# BINANCE_REST_URL=https://api.binance.com
|
|
|
|
# ── Bybit (backup crypto — no key needed) ────────────────────────────
|
|
BYBIT_WS_URL=wss://stream.bybit.com/v5/public/spot
|
|
|
|
# ── Alpaca (paper trading — free) ─────────────────────────────────────
|
|
# Get free paper keys at: https://app.alpaca.markets/paper/dashboard/overview
|
|
ALPACA_KEY=
|
|
ALPACA_SECRET=
|
|
ALPACA_BASE_URL=https://paper-api.alpaca.markets
|
|
ALPACA_DATA_WS_URL=wss://stream.data.alpaca.markets/v2/iex
|
|
ALPACA_DRY_RUN=true # Set to false when keys are configured
|
|
|
|
# ── Storage ───────────────────────────────────────────────────────────
|
|
SQLITE_URL=sqlite+aiosqlite:///market_data.db
|
|
JSONL_PATH=events.jsonl
|
|
|
|
# ── Reliability ───────────────────────────────────────────────────────
|
|
RECONNECT_MAX_RETRIES=20
|
|
RECONNECT_BASE_DELAY=1.0
|
|
RECONNECT_MAX_DELAY=60.0
|
|
HEARTBEAT_TIMEOUT=30.0
|
|
|
|
# ── HTTP Server / Metrics ─────────────────────────────────────────────
|
|
HTTP_HOST=0.0.0.0
|
|
HTTP_PORT=8891
|
|
METRICS_ENABLED=true
|
|
|
|
# ── NATS Output (SenpAI integration) ─────────────────────────────────
|
|
# Enable to push events to NATS for SenpAI consumption
|
|
# Subject schema: md.events.{type}.{symbol} e.g. md.events.trade.BTCUSDT
|
|
NATS_URL=nats://localhost:4222
|
|
NATS_SUBJECT_PREFIX=md.events
|
|
NATS_ENABLED=false
|
|
|
|
# ── Logging ───────────────────────────────────────────────────────────
|
|
LOG_LEVEL=INFO
|
|
LOG_SAMPLE_RATE=100
|