feat: market-data-service for SenpAI trading agent
New service: real-time market data collection with unified event model. Architecture: - Domain events: TradeEvent, QuoteEvent, BookL2Event, HeartbeatEvent - Provider interface: MarketDataProvider ABC with connect/subscribe/stream/close - Async EventBus with fan-out to multiple consumers Providers: - BinanceProvider: public WebSocket (trades + bookTicker), no API key needed, auto-reconnect with exponential backoff, heartbeat timeout detection - AlpacaProvider: IEX real-time data + paper trading auth, dry-run mode when no keys configured (heartbeats only) Consumers: - StorageConsumer: SQLite (via SQLAlchemy async) + JSONL append-only log - MetricsConsumer: Prometheus counters, latency histograms, events/sec gauge - PrintConsumer: sampled structured logging (1/100 events) CLI: python -m app run --provider binance --symbols BTCUSDT,ETHUSDT HTTP: /health, /metrics (Prometheus), /latest?symbol=XXX Tests: 19/19 passed (Binance parse, Alpaca parse, bus smoke tests) Config: pydantic-settings + .env, all secrets via environment variables. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
89
services/market-data-service/app/domain/events.py
Normal file
89
services/market-data-service/app/domain/events.py
Normal file
@@ -0,0 +1,89 @@
|
||||
"""
|
||||
Unified domain events for market data.
|
||||
|
||||
All providers normalize raw messages into these canonical types.
|
||||
Timestamps are always UTC. ts_exchange may be None if the source
|
||||
doesn't provide exchange timestamps.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class EventType(str, Enum):
|
||||
TRADE = "trade"
|
||||
QUOTE = "quote"
|
||||
BOOK_L2 = "book_l2"
|
||||
HEARTBEAT = "heartbeat"
|
||||
|
||||
|
||||
def _utc_now() -> datetime:
|
||||
return datetime.now(timezone.utc)
|
||||
|
||||
|
||||
def _mono_ns() -> int:
|
||||
"""Monotonic nanoseconds for internal latency measurement."""
|
||||
return time.monotonic_ns()
|
||||
|
||||
|
||||
class BaseEvent(BaseModel):
|
||||
"""Common fields for every market-data event."""
|
||||
|
||||
event_type: EventType
|
||||
provider: str
|
||||
ts_recv: datetime = Field(default_factory=_utc_now)
|
||||
ts_recv_mono_ns: int = Field(default_factory=_mono_ns)
|
||||
|
||||
|
||||
class TradeEvent(BaseEvent):
|
||||
"""A single matched trade (fill)."""
|
||||
|
||||
event_type: EventType = EventType.TRADE
|
||||
symbol: str
|
||||
price: float
|
||||
size: float
|
||||
ts_exchange: Optional[datetime] = None
|
||||
side: Optional[str] = None # "buy" | "sell" | None
|
||||
trade_id: Optional[str] = None
|
||||
|
||||
|
||||
class QuoteEvent(BaseEvent):
|
||||
"""Best bid/ask (top-of-book)."""
|
||||
|
||||
event_type: EventType = EventType.QUOTE
|
||||
symbol: str
|
||||
bid: float
|
||||
ask: float
|
||||
bid_size: float
|
||||
ask_size: float
|
||||
ts_exchange: Optional[datetime] = None
|
||||
|
||||
|
||||
class BookLevel(BaseModel):
|
||||
price: float
|
||||
size: float
|
||||
|
||||
|
||||
class BookL2Event(BaseEvent):
|
||||
"""L2 order-book snapshot (partial depth)."""
|
||||
|
||||
event_type: EventType = EventType.BOOK_L2
|
||||
symbol: str
|
||||
bids: list[BookLevel] = Field(default_factory=list)
|
||||
asks: list[BookLevel] = Field(default_factory=list)
|
||||
ts_exchange: Optional[datetime] = None
|
||||
|
||||
|
||||
class HeartbeatEvent(BaseEvent):
|
||||
"""Provider heartbeat / keep-alive signal."""
|
||||
|
||||
event_type: EventType = EventType.HEARTBEAT
|
||||
|
||||
|
||||
# Union type for type-safe consumers
|
||||
Event = TradeEvent | QuoteEvent | BookL2Event | HeartbeatEvent
|
||||
Reference in New Issue
Block a user