docs: add node1 runbooks, consolidation artifacts, and maintenance scripts
This commit is contained in:
26
docs/architecture_inventory/inventory_datastores.md
Normal file
26
docs/architecture_inventory/inventory_datastores.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Datastore Inventory
|
||||
|
||||
| db | purpose | location | backup | retention | source |
|
||||
|---|---|---|---|---|---|
|
||||
| PostgreSQL (`dagi-postgres`) | transactional data, memory tables, rag db | container + volume (`postgres_data_node1` / staging variant) | compose backup container + `scripts/backup/backup_all.sh` | backup keep days/weeks/months in backup compose | `docker-compose.node1.yml`, `docker-compose.backups.yml`, `scripts/backup/backup_all.sh` |
|
||||
| Qdrant | vector embeddings/index | container + volume (`qdrant-data-node1`) | snapshot copy in backup script | not explicit in policy files | `docker-compose.node1.yml`, `scripts/backup/backup_all.sh` |
|
||||
| Neo4j | graph memory/relations | container + volumes (`neo4j-data-node1`, logs) | `neo4j-admin dump` in backup script | not explicit in policy files | `docker-compose.node1.yml`, `scripts/backup/backup_all.sh` |
|
||||
| Redis | cache/session/state | container + volume (`redis-data-node1`) | no dedicated backup script in repo | not explicit | `docker-compose.node1.yml` |
|
||||
| MinIO | object artifacts (ppt/pdf/doc assets, comfy outputs) | container + volume (`minio-data-node1`) | implicit host-level volume backup only (no dedicated script found) | not explicit | `docker-compose.node1.yml`, `services/artifact-registry/app/main.py` |
|
||||
| SQLite (`market-data-service`) | local market event storage | `/data/market_data.db` inside market-data volume | host volume snapshot/manual | not explicit | `services/market-data-service/app/config.py`, `docker-compose.node1.yml` |
|
||||
| NATS JetStream store | event stream persistence | `/data/jetstream` volume | via NATS data volume backup strategy (manual/host-level) | stream-level policies in docs | `nats/nats.conf`, `docker-compose.node1.yml` |
|
||||
|
||||
## Schema / Collections Pointers
|
||||
- Memory schemas/migrations:
|
||||
- `services/memory-service/migrations/001_create_memory_tables.sql`
|
||||
- `migrations/046_memory_service_full_schema.sql`
|
||||
- `migrations/049_memory_v3_human_memory_model.sql`
|
||||
- `migrations/052_account_linking_schema.sql`
|
||||
- Market data SQLAlchemy models: `services/market-data-service/app/db/schema.py`
|
||||
- RAG API/index model contracts: `services/rag-service/app/models.py`
|
||||
- Qdrant collection access via memory/rag service code:
|
||||
- `services/memory-service/app/vector_store.py`
|
||||
- `services/rag-service/app/document_store.py`
|
||||
|
||||
## Redaction Note
|
||||
Any discovered credentials in datastore DSNs/password fields are represented as `<REDACTED>` in inventory narratives.
|
||||
Reference in New Issue
Block a user