daarion-admin/microdao-daarion
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
585 Commits 8 Branches 0 Tags
313d777c8451dee6f476b4502fccc5a03fbdaa77
Commit Graph

154 Commits

Author SHA1 Message Date
Apple
67225a39fa docs(platform): add policy configs, runbooks, ops scripts and platform documentation 
Config policies (16 files): alert_routing, architecture_pressure, backlog,
cost_weights, data_governance, incident_escalation, incident_intelligence,
network_allowlist, nodes_registry, observability_sources, rbac_tools_matrix,
release_gate, risk_attribution, risk_policy, slo_policy, tool_limits, tools_rollout

Ops (22 files): Caddyfile, calendar compose, grafana voice dashboard,
deployments/incidents logs, runbooks for alerts/audit/backlog/incidents/sofiia/voice,
cron jobs, scripts (alert_triage, audit_cleanup, migrate_*, governance, schedule),
task_registry, voice alerts/ha/latency/policy

Docs (30+ files): HUMANIZED_STEPAN v2.7-v3 changelogs and runbooks,
NODA1/NODA2 status and setup, audit index and traces, backlog, incident,
supervisor, tools, voice, opencode, release, risk, aistalk, spacebot

Made-with: Cursor
 2026-03-03 07:14:53 -08:00
Apple
e9dedffa48 feat(production): sync all modified production files to git 
Includes updates across gateway, router, node-worker, memory-service,
aurora-service, swapper, sofiia-console UI and node2 infrastructure:

- gateway-bot: Dockerfile, http_api.py, druid/aistalk prompts, doc_service
- services/router: main.py, router-config.yml, fabric_metrics, memory_retrieval,
  offload_client, prompt_builder
- services/node-worker: worker.py, main.py, config.py, fabric_metrics
- services/memory-service: Dockerfile, database.py, main.py, requirements
- services/aurora-service: main.py (+399), kling.py, quality_report.py
- services/swapper-service: main.py, swapper_config_node2.yaml
- services/sofiia-console: static/index.html (console UI update)
- config: agent_registry, crewai_agents/teams, router_agents
- ops/fabric_preflight.sh: updated preflight checks
- router-config.yml, docker-compose.node2.yml: infra updates
- docs: NODA1-AGENT-ARCHITECTURE, fabric_contract updated

Made-with: Cursor
 2026-03-03 07:13:29 -08:00
Apple
2962d33a3b feat(sofiia-console): add artifacts list endpoint + team onboarding doc 
- runbook_artifacts.py: adds list_run_artifacts() returning files with
  names, paths, sizes, mtime_utc from release_artifacts/<run_id>/
- runbook_runs_router.py: adds GET /api/runbooks/runs/{run_id}/artifacts
- docs/runbook/team-onboarding-console.md: one-page team onboarding doc
  covering access, rehearsal run steps, audit auth model (strict, no
  localhost bypass), artifacts location, abort procedure

Made-with: Cursor
 2026-03-03 06:55:49 -08:00
Apple
bddb6cd75a docs(dev): index release evidence template in runbook README 
Made-with: Cursor
 2026-03-03 04:00:15 -08:00
Apple
3c199be6d3 docs(dev): index release and rehearsal runbooks in docs/runbook 
Made-with: Cursor
 2026-03-03 03:55:29 -08:00
Apple
55a5e541df docs(dev): add v1 30-min rehearsal execution checklist 
includes preflight, restart, smoke, observation, evidence steps

defines success criteria and metrics to collect for next-step decision

Made-with: Cursor
 2026-03-03 03:54:53 -08:00
Apple
ad74e4c0ba docs(dev): add sofiia-console post-release review template 
Made-with: Cursor
 2026-03-02 10:20:24 -08:00
Apple
3df414d35a docs(dev): add sofiia-console v1 technical release announcement 
Made-with: Cursor
 2026-03-02 10:17:53 -08:00
Apple
47073ba761 docs(dev): add release runbook for sofiia-console 
Made-with: Cursor
 2026-03-02 10:00:08 -08:00
Apple
1d18634c01 ops(dev): add audit retention pruning script 
Made-with: Cursor
 2026-03-02 09:47:39 -08:00
Apple
9e70fc83d2 ops(dev): add secrets rotation runbook and sofiia-console preflight checks 
Made-with: Cursor
 2026-03-02 09:32:18 -08:00
Apple
d85aa507a2 docs(dev): add redis docker-compose smoke snippet for sofiia-console 
Made-with: Cursor
 2026-03-02 09:11:45 -08:00
Apple
49afb1df99 docs(audit): add NODA2 Sofiia tools audit and full matrix 2026-03-01 01:42:57 -08:00
Apple
9a36020316 P3.5-P3.7: 2-layer inventory, capability routing, STT/TTS adapters, Dev Contract 
NCS:
- _collect_worker_caps() fetches capability flags from node-worker /caps
- _derive_capabilities() merges served model types + worker provider flags
- installed_artifacts replaces inventory_only (disk scan with DISK_SCAN_PATHS env)
- New endpoints: /capabilities/caps, /capabilities/installed

Node Worker:
- STT_PROVIDER, TTS_PROVIDER, OCR_PROVIDER, IMAGE_PROVIDER env flags
- /caps endpoint returns capabilities + providers for NCS aggregation
- STT adapter (providers/stt_mlx_whisper.py) — remote + local mode
- TTS adapter (providers/tts_mlx_kokoro.py) — remote + local mode
- OCR handler via vision_prompted (ollama_vision with OCR prompt)
- NATS subjects: node.{id}.stt/tts/ocr/image.request

Router:
- POST /v1/capability/{stt,tts,ocr,image} — capability-based offload routing
- GET /v1/capabilities — global view with capabilities_by_node
- require_fresh_caps(ttl) preflight guard
- find_nodes_with_capability(cap) + load-based node selection

Ops:
- ops/fabric_snapshot.py — full runtime snapshot collector
- ops/fabric_preflight.sh — quick check + snapshot save + diff
- docs/fabric_contract.md — Dev Contract v0.1 (preflight-first)
- tests/test_fabric_contract.py — CI enforcement (6 tests)

Made-with: Cursor
 2026-02-27 05:24:09 -08:00
Apple
90080c632a fix(fabric): use broadcast subject for NATS capabilities discovery 
NATS wildcards (node.*.capabilities.get) only work for subscriptions,
not for publish. Switch to a dedicated broadcast subject
(fabric.capabilities.discover) that all NCS instances subscribe to,
enabling proper scatter-gather discovery across nodes.

Made-with: Cursor
 2026-02-27 03:20:13 -08:00
NODA1 System
90eff85662 crewai: add agromatrix and plant-intel role packs with updated team config 2026-02-20 17:56:55 +01:00
Apple
544874d952 docs: add node1 runbooks, consolidation artifacts, and maintenance scripts 2026-02-19 00:14:27 -08:00
Apple
de8bb36462 docs+router: formalize runtime policy and remove temporary cloud-first code override 2026-02-18 10:40:40 -08:00
Apple
963813607b Docs sync: align OPENAPI contracts with NODE1 runtime 2026-02-18 05:58:54 -08:00
Apple
7df8cd5882 docs: sync consolidation and session starter 2026-02-16 02:25:54 -08:00
Apple
798c6f88c7 docs: sync consolidation and session starter 2026-02-16 02:21:49 -08:00
Apple
b962d4a288 docs: sync consolidation and session starter 2026-02-16 02:15:59 -08:00
Apple
de3bd8c13f docs: sync consolidation and session starter 2026-02-16 02:15:20 -08:00
Apple
b2be937fbb feat(file-tool): add djvu conversion and extraction actions 2026-02-15 03:11:55 -08:00
Apple
3a565fd910 feat(file-tool): harden svg rendering and add rich pptx/pdf updates 2026-02-15 02:48:35 -08:00
Apple
aad5870e81 feat(file-tool): add image_bundle and svg actions 2026-02-15 02:33:42 -08:00
Apple
36314a871f feat(file-tool): add pptx ods parquet and image actions 2026-02-15 02:30:00 -08:00
Apple
cf6ac778bb feat(file-tool): add text markdown xml html actions 2026-02-15 02:24:11 -08:00
Apple
e91584246d feat(router): implement file_tool excel actions on NODE1 stack 2026-02-15 02:11:28 -08:00
Apple
21576f0ca3 node1: add universal file tool, gateway document delivery, and sync runbook 2026-02-15 01:50:37 -08:00
Apple
ef3473db21 snapshot: NODE1 production state 2026-02-09 
Complete snapshot of /opt/microdao-daarion/ from NODE1 (144.76.224.179).
This represents the actual running production code that has diverged
significantly from the previous main branch.

Key changes from old main:
- Gateway (http_api.py): expanded from ~40KB to 164KB with full agent support
- Router: new /v1/agents/{id}/infer endpoint with vision + DeepSeek routing
- Behavior Policy: SOWA v2.2 (3-level: FULL/ACK/SILENT)
- Agent Registry: config/agent_registry.yml as single source of truth
- 13 agents configured (was 3)
- Memory service integration
- CrewAI teams and roles

Excluded from snapshot: venv/, .env, data/, backups, .tgz archives

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-09 08:46:46 -08:00
Apple
a46a70c014 fix(ops): Add network aliases and stabilize DNS for NODA1 
- docker-compose.node1.yml: Add network aliases (router, gateway,
  memory-service, qdrant, nats, neo4j) to eliminate manual
  `docker network connect --alias` commands
- docker-compose.node1.yml: ROUTER_URL now uses env variable with
  fallback: ${ROUTER_URL:-http://router:8000}
- docker-compose.node1.yml: Increase router healthcheck start_period
  to 30s and retries to 5
- .gitignore: Add noda1-credentials.local.mdc (local-only SSH creds)
- scripts/node1/verify_agents.sh: Improved output with agent list
- docs: Add NODA1-AGENT-VERIFICATION.md, NODA1-AGENT-ARCHITECTURE.md,
  NODA1-VERIFICATION-REPORT-2026-02-03.md
- config/README.md: How to add new agents
- .cursor/rules/, .cursor/skills/: NODA1 operations skill for Cursor

Root cause fixed: Gateway could not resolve 'router' DNS name when
Router container was named 'dagi-staging-router' without alias.

Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-02-03 05:55:56 -08:00
Apple
0c8bef82f4 feat: Add Alateya, Clan, Eonarch agents + fix gateway-router connection 
## Agents Added
- Alateya: R&D, biotech, innovations
- Clan (Spirit): Community spirit agent
- Eonarch: Consciousness evolution agent

## Changes
- docker-compose.node1.yml: Added tokens for all 3 new agents
- gateway-bot/http_api.py: Added configs and webhook endpoints
- gateway-bot/clan_prompt.txt: New prompt file
- gateway-bot/eonarch_prompt.txt: New prompt file

## Fixes
- Fixed ROUTER_URL from :9102 to :8000 (internal container port)
- All 9 Telegram agents now working

## Documentation
- Created PROJECT-MASTER-INDEX.md - single entry point
- Added various status documents and scripts

Tokens configured:
- Helion, NUTRA, Agromatrix (existing)
- Alateya, Clan, Eonarch (new)
- Druid, GreenFood, DAARWIZZ (configured)
 2026-01-28 06:40:34 -08:00
Apple
5290287058 feat: implement TTS, Document processing, and Memory Service /facts API 
- TTS: xtts-v2 integration with voice cloning support
- Document: docling integration for PDF/DOCX/PPTX processing
- Memory Service: added /facts/upsert, /facts/{key}, /facts endpoints
- Added required dependencies (TTS, docling)
 2026-01-17 08:16:37 -08:00
Apple
3478dfce5f 🔒 КРИТИЧНО: Видалено паролі/API ключі з документів + закрито NodePort
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- Видалено всі паролі та API ключі з документів
- Замінено на посилання на Vault
- Закрито NodePort для Memory Service (тільки internal)
- Створено SECURITY-ROTATION-PLAN.md
- Створено ARCHITECTURE-150-NODES.md (план для 150 нод)
- Оновлено config.py (видалено hardcoded Cohere key)
 2026-01-10 09:46:03 -08:00
Apple
f7bf935a21 ✅ NODE3: Memory Service мігровано з Docker в K8s
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- NODE3 додано до K3s кластера як worker (llm80-che-1-1)
- Memory Service працює в K8s на NODE3 (pod: memory-service-node3-*)
- Docker контейнер зупинено та видалено
- Оновлено MEMORY-MODULE-STATUS.md v3.1.0
 2026-01-10 09:26:59 -08:00
Apple
116bf5f3f3 ✅ Memory Service запущено на всіх нодах + Cohere API налаштовано
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- NODE1: Memory Service в K8s (port 30800) ✅
- NODE2: Memory Service в Docker (port 8001) ✅
- NODE3: Memory Service в Docker (port 8001) ✅
- Всі ноди: Cohere API налаштовано для embeddings ✅
- NODE2: ComfyUI перевірено (macOS App, port 8000) ✅
- Оновлено MEMORY-MODULE-STATUS.md v3.0.0
 2026-01-10 09:13:20 -08:00
Apple
6b02349300 🧠 Update Memory Module Status v2.1.0
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- NODE2: PostgreSQL + Agent Memory Schema ✅
- NODE3: ComfyUI installed (v0.8.2, PyTorch+CUDA) ✅
- All nodes now have full memory stack
- Added critical TODOs: Memory Service & Cohere API
 2026-01-10 09:00:17 -08:00
Apple
f4ccf7c570 🧠 Complete Memory Stack setup across all nodes
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- NODE1: Neo4j (K8s), NVIDIA RTX 4000 + CUDA 13.1
- NODE2: Fixed Neo4j & Qdrant containers
- NODE3: Full stack (PostgreSQL + Qdrant + Neo4j)
- Updated MEMORY-MODULE-STATUS.md v2.0.0
 2026-01-10 08:26:42 -08:00
Apple
8aee29d42d 📊 Add Memory Module Status Report across all nodes
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details
2026-01-10 08:11:12 -08:00
Apple
1c247ea40c 📝 Update context docs with session logging system
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- Added Session Logging System section to INFRASTRUCTURE.md
- Added Git Multi-Remote configuration (GitHub + Gitea + GitLab)
- Updated version to 2.5.0
- Added logging commands reference
- Updated infrastructure_quick_ref.ipynb with new features
- Added SSH tunnel instructions for GitLab access
 2026-01-10 04:58:01 -08:00
Apple
744c149300 ✨ Add automated session logging system
Some checks failed
Build and Deploy Docs / build-and-deploy (push) Has been cancelled
Details 
- Created logs/ structure (sessions, operations, incidents)
- Added session-start/log/end scripts
- Installed Git hooks for auto-logging commits/pushes
- Added shell integration for zsh
- Created CHANGELOG.md
- Documented today's session (2026-01-10)
 2026-01-10 04:53:17 -08:00
Apple
778907cf0e docs: add NODE3 (Threadripper PRO + RTX 3090) to infrastructure 
Added NODE3 - AI/ML Workstation Specification:

Hardware:
- CPU: AMD Ryzen Threadripper PRO 5975WX (32 cores / 64 threads, 3.6 GHz boost)
- RAM: 128GB DDR4
- GPU: NVIDIA GeForce RTX 3090 24GB GDDR6X
  - 10496 CUDA cores
  - CUDA 13.0, Driver 580.95.05
- Storage: Samsung SSD 990 PRO 4TB NVMe
  - Root: 100GB (27% used)
  - Available for expansion: 3.5TB

System:
- Hostname: llm80-che-1-1
- IP: 80.77.35.151:33147
- OS: Ubuntu 24.04.3 LTS (Noble Numbat)
- Container Runtime: MicroK8s + containerd
- Uptime: 24/7

Security Status: ✅ CLEAN (verified 2026-01-09)
- No crypto miners detected
- 0 zombie processes
- CPU load: 0.17 (very low)
- GPU utilization: 0% (ready for workloads)

Services Running:
- Port 3000 - Unknown service (needs investigation)
- Port 8080 - Unknown service (needs investigation)
- Port 11434 - Ollama (localhost only)
- Port 27017/27019 - MongoDB (localhost only)
- Kubernetes API: 16443
- K8s services: 10248-10259, 25000

Recommended Use Cases:
- 🤖 Large LLM inference (Llama 70B, Qwen 72B, Mixtral 8x22B)
- 🧠 Model training and fine-tuning
- 🎨 Stable Diffusion XL image generation
- 🔬 AI/ML research and experimentation
- 🚀 Kubernetes-based AI service orchestration

Files Updated:
- INFRASTRUCTURE.md v2.4.0
- docs/infrastructure_quick_ref.ipynb v2.3.0

NODE3 is the most powerful node in the infrastructure:
- Most CPU cores: 32c/64t (vs 16c M4 Max)
- Most RAM: 128GB (vs 64GB)
- Dedicated GPU: RTX 3090 24GB VRAM
- Largest storage: 4TB NVMe (vs 2TB)

Co-Authored-By: Warp <agent@warp.dev>
 2026-01-09 05:53:16 -08:00
Apple
21691aa042 docs: document Security Incident #2 - recurring container compromise 
Security Incident #2 Emergency Response (Jan 9, 2026):
- Documented second compromise with NEW crypto miners (softirq, vrarhpb)
- Root cause: Docker image auto-restarted after server reboot
- Emergency mitigation completed (processes killed, container/images removed, load normalized)
- Created comprehensive rebuild task document: TASK_REBUILD_DAARION_WEB.md
- Updated INFRASTRUCTURE.md v2.3.0 with Incident #2 timeline and lessons learned
- Updated infrastructure_quick_ref.ipynb v2.2.0 with security status

Critical Changes:
- daarion-web container permanently disabled until secure rebuild
- Docker images DELETED (not just container stopped)
- Enhanced firewall rules (SSH rate limiting, port scan blocking)
- Retry test registered with Hetzner
- System load normalized: 30+ → 4.19
- Zombie processes cleaned: 1499 → 5

Files Created/Updated:
1. TASK_REBUILD_DAARION_WEB.md - Detailed rebuild instructions for Cursor agent
2. INFRASTRUCTURE.md - Added Incident #2 to Security section
3. docs/infrastructure_quick_ref.ipynb - Updated security status and version

Lessons Learned:
- ALWAYS delete Docker images, not just containers
- Auto-restart policies are dangerous for compromised containers
- Complete removal = container + image + restart policy change

Status: Emergency mitigation complete, statement submission pending (deadline: 2026-01-09 12:54 UTC)

Hetzner Incident ID: 10F3971:2A (AbuseID)

Co-Authored-By: Warp <agent@warp.dev>
 2026-01-09 02:08:13 -08:00
Apple
a1091b03a3 docs: add Cursor Agent SSH access instructions for NODE1 
- Add detailed SSH connection guide for Cursor agents
- Include common commands, safety checks, and troubleshooting
- Add interactive session example and best practices
- Update INFRASTRUCTURE.md with section for Cursor agents
- Update infrastructure_quick_ref.ipynb with SSH access configuration
- Provide complete workflow examples for remote operations

Co-Authored-By: Warp <agent@warp.dev>
 2026-01-09 02:08:13 -08:00
Apple
e829fe66f2 docs: security incident resolution & firewall implementation 
- Document network scanning incident (Dec 6 2025 - Jan 8 2026)
- Add firewall rules to prevent internal network access
- Deploy monitoring script for scanning attempts
- Update INFRASTRUCTURE.md v2.2.0 with Security section
- Update infrastructure_quick_ref.ipynb v2.1.0
- Root cause: compromised daarion-web container with crypto miner
- Resolution: container removed, firewall applied, monitoring deployed

Co-Authored-By: Warp <agent@warp.dev>
 2026-01-09 02:08:13 -08:00
GitHub Action
e3a8b7464a docs: auto-update repository information [skip ci] 2025-12-08 09:30:23 +00:00
Apple
ad3026e32d docs: Document root cause of daily data loss and fix 2025-12-05 02:42:44 -08:00
Apple
70b528f5cf docs: Add documentation for periodic data loss fix 2025-12-05 02:36:49 -08:00
Apple
db3b74e1ba fix: Integrate asset URL fix into recovery process and update docs 2025-12-03 10:13:19 -08:00