docs: add node1 runbooks, consolidation artifacts, and maintenance scripts

This commit is contained in:
Apple
2026-02-19 00:14:27 -08:00
parent c57e6ed96b
commit 544874d952
586 changed files with 14065 additions and 22 deletions

View File

@@ -0,0 +1,26 @@
# Datastore Inventory
| db | purpose | location | backup | retention | source |
|---|---|---|---|---|---|
| PostgreSQL (`dagi-postgres`) | transactional data, memory tables, rag db | container + volume (`postgres_data_node1` / staging variant) | compose backup container + `scripts/backup/backup_all.sh` | backup keep days/weeks/months in backup compose | `docker-compose.node1.yml`, `docker-compose.backups.yml`, `scripts/backup/backup_all.sh` |
| Qdrant | vector embeddings/index | container + volume (`qdrant-data-node1`) | snapshot copy in backup script | not explicit in policy files | `docker-compose.node1.yml`, `scripts/backup/backup_all.sh` |
| Neo4j | graph memory/relations | container + volumes (`neo4j-data-node1`, logs) | `neo4j-admin dump` in backup script | not explicit in policy files | `docker-compose.node1.yml`, `scripts/backup/backup_all.sh` |
| Redis | cache/session/state | container + volume (`redis-data-node1`) | no dedicated backup script in repo | not explicit | `docker-compose.node1.yml` |
| MinIO | object artifacts (ppt/pdf/doc assets, comfy outputs) | container + volume (`minio-data-node1`) | implicit host-level volume backup only (no dedicated script found) | not explicit | `docker-compose.node1.yml`, `services/artifact-registry/app/main.py` |
| SQLite (`market-data-service`) | local market event storage | `/data/market_data.db` inside market-data volume | host volume snapshot/manual | not explicit | `services/market-data-service/app/config.py`, `docker-compose.node1.yml` |
| NATS JetStream store | event stream persistence | `/data/jetstream` volume | via NATS data volume backup strategy (manual/host-level) | stream-level policies in docs | `nats/nats.conf`, `docker-compose.node1.yml` |
## Schema / Collections Pointers
- Memory schemas/migrations:
- `services/memory-service/migrations/001_create_memory_tables.sql`
- `migrations/046_memory_service_full_schema.sql`
- `migrations/049_memory_v3_human_memory_model.sql`
- `migrations/052_account_linking_schema.sql`
- Market data SQLAlchemy models: `services/market-data-service/app/db/schema.py`
- RAG API/index model contracts: `services/rag-service/app/models.py`
- Qdrant collection access via memory/rag service code:
- `services/memory-service/app/vector_store.py`
- `services/rag-service/app/document_store.py`
## Redaction Note
Any discovered credentials in datastore DSNs/password fields are represented as `<REDACTED>` in inventory narratives.