docs(audit): add NODA2 Sofiia tools audit and full matrix

2026-03-01 01:42:57 -08:00
parent 57632699c0
commit 49afb1df99
2 changed files with 229 additions and 0 deletions
--- a/docs/audit/SOFIIA_NODA2_TOOLS_AUDIT_2026-03-01.md
+++ b/docs/audit/SOFIIA_NODA2_TOOLS_AUDIT_2026-03-01.md
@@ -0,0 +1,167 @@
+# Sofiia Tools Audit (NODA2)
+
+Date: 2026-03-01  
+Node: NODA2 (local laptop)  
+Scope: Router tool stack + requested integrations (AgentEmailTool, BrowserTool, SecureVault, SafeCodeExecutor, CalendarTool) + broader Sofiia tool system readiness.
+
+## 1) Inventory and Wiring Integrity
+
+- Tool definitions declared in router: **56 executable tools** (`services/router/tool_manager.py`)
+- Tool dispatch branches in router: **56 tools**
+- Dispatch-to-handler integrity check: **no missing handler definitions**
+
+Evidence:
+- Definitions source: `services/router/tool_manager.py`
+- Dispatch source: `services/router/tool_manager.py`
+
+## 2) NODA2 Infrastructure Readiness
+
+Calendar stack is wired into NODA2 compose:
+- `router` env has `CALENDAR_SERVICE_URL=http://calendar-service:8001`
+- `router` mounts `./tools:/app/tools:ro` (required for local tool modules)
+- `router` depends on `calendar-service`
+- `calendar-service` service present and running
+- `radicale` service present and running
+
+Compose source:
+- `docker-compose.node2-sofiia.yml`
+
+## 3) Requested Tool Audit (Runtime)
+
+### AgentEmailTool
+- Route wiring: present
+- RBAC mapping: present (`tools.email.use`)
+- Limits: present
+- Runtime check: `list_inboxes` -> `ok` (empty list expected on fresh setup)
+
+### BrowserTool
+- Route wiring: present
+- RBAC mapping: present (`tools.browser.use`)
+- Limits: present
+- Runtime check: `start_session/goto/get_current_url/close_session` -> `ok`
+- Async loop blocker resolved via thread offload in router adapter.
+
+### SecureVault
+- Route wiring: present
+- RBAC mapping: present (`tools.vault.manage`)
+- Limits: present
+- Runtime check: `store` -> `ok`
+
+### SafeCodeExecutor
+- Route wiring: present
+- RBAC mapping: present (`tools.exec.safe`)
+- Limits: present
+- Runtime check: `validate` -> `ok` (`python` sample valid)
+
+### CalendarTool (Radicale/CalDAV via calendar-service)
+- Route wiring: present
+- RBAC mapping: present (`tools.calendar.use`)
+- Limits: present
+- Runtime check:
+  - `calendar-service /health` -> healthy
+  - `calendar_tool list_calendars` without `account_id` -> domain error `account_id required` (expected), proving router->service path is live.
+
+## 4) RBAC and Governance Validation
+
+Files present and active:
+- `config/tools_rollout.yml`
+- `config/rbac_tools_matrix.yml`
+- `config/tool_limits.yml`
+
+Validated outcomes:
+- `sofiia` and `admin` mapped to `agent_cto` rollout
+- New tools included in `cto_tools`
+- Role entitlements include calendar/email/browser/executor/vault usage
+- Negative check passed: `monitor` denied on `secure_vault_tool`
+
+## 5) Sofiia CTO Access Audit (repo / notion / git / nodes)
+
+### Repo access
+- `repo_tool` available and callable
+- Runtime check `repo_tool:metadata` -> `ok`
+
+### Notion access
+- `notion_tool` available and callable
+- Runtime check `notion_tool:status` -> `ok` (workspace bot identity returned)
+
+### Git/repo operational tooling
+- `repo_tool`, `pr_reviewer_tool`, `contract_tool`, `kb_tool` are present in tool definitions and dispatch.
+
+### Node visibility/control plane
+- Console endpoint `GET /api/agents?nodes=NODA2` returns healthy agent registry for NODA2.
+- Nodes registry file present: `config/nodes_registry.yml`
+
+## 6) Documentation Coverage
+
+Current docs directories found:
+- `docs/tools/` (tool docs exist for key governance/ops tools)
+- `docs/audit/` and `docs/audits/` (existing system audit artifacts)
+
+Gap observed:
+- Documentation depth is uneven across all 56 tools; some newer tools are wired and working but not yet fully documented in `docs/tools/`.
+
+## 7) Current Risk Register (Audit Findings)
+
+1. **Medium**: Calendar integration is operational, but no account bootstrap in this audit run (no connected calendar account configured yet).
+2. **Low/Medium**: Tool documentation is incomplete relative to actual implemented tool surface (56 tools).
+3. **Low**: Repo is in a very large dirty state; future changes should stay strictly path-scoped to avoid accidental mixed commits.
+
+## 8) Appendix: Executable Tool Set (56)
+
+agent_email_tool
+alert_ingest_tool
+architecture_pressure_tool
+backlog_tool
+binance_account_bots
+binance_bots_top
+browser_tool
+calc_window_quote
+calendar_tool
+comfy_generate_image
+comfy_generate_video
+config_linter_tool
+contract_tool
+cost_analyzer_tool
+crawl4ai_scrape
+crm_create_job
+crm_create_quote
+crm_search_client
+crm_update_quote
+crm_upsert_client
+crm_upsert_site
+crm_upsert_window_unit
+data_governance_tool
+dependency_scanner_tool
+docs_render_invoice_pdf
+docs_render_quote_pdf
+drift_analyzer_tool
+file_tool
+graph_query
+image_generate
+incident_escalation_tool
+incident_intelligence_tool
+job_orchestrator_tool
+kb_tool
+market_data
+memory_search
+notion_tool
+observability_tool
+oncall_tool
+pieces_tool
+pr_reviewer_tool
+presentation_create
+presentation_download
+presentation_status
+remember_fact
+repo_tool
+risk_engine_tool
+risk_history_tool
+safe_code_executor_tool
+schedule_confirm_slot
+schedule_propose_slots
+secure_vault_tool
+threatmodel_tool
+tts_speak
+web_extract
+web_search
+
--- a/docs/audit/SOFIIA_NODA2_TOOL_MATRIX_2026-03-01.md
+++ b/docs/audit/SOFIIA_NODA2_TOOL_MATRIX_2026-03-01.md
@@ -0,0 +1,62 @@
+# Sofiia NODA2 Tool Matrix (2026-03-01)
+
+Legend: `Y`=present/verified, `N`=missing, `NT`=not runtime-tested in this audit run.
+
+| Tool | Wiring | RBAC | Limits | Rollout | Runtime | Risk |
+|---|---|---|---|---|---|---|
+| agent_email_tool | Y | Y | Y | Y | Y (list_inboxes) | low |
+| alert_ingest_tool | Y | Y | Y | N | NT | low |
+| architecture_pressure_tool | Y | Y | Y | N | NT | low |
+| backlog_tool | Y | Y | Y | N | NT | low |
+| binance_account_bots | Y | N | N | N | NT | medium |
+| binance_bots_top | Y | N | N | N | NT | medium |
+| browser_tool | Y | Y | Y | Y | Y (start/goto/get_current_url/close) | low |
+| calc_window_quote | Y | N | N | N | NT | medium |
+| calendar_tool | Y | Y | Y | Y | Y (route live; account missing expected) | medium |
+| comfy_generate_image | Y | Y | Y | Y | NT | low |
+| comfy_generate_video | Y | Y | Y | Y | NT | low |
+| config_linter_tool | Y | Y | Y | Y | NT | low |
+| contract_tool | Y | Y | Y | Y | NT | low |
+| cost_analyzer_tool | Y | Y | Y | Y | NT | low |
+| crawl4ai_scrape | Y | Y | Y | Y | NT | low |
+| crm_create_job | Y | N | N | N | NT | medium |
+| crm_create_quote | Y | N | N | N | NT | medium |
+| crm_search_client | Y | N | N | N | NT | medium |
+| crm_update_quote | Y | N | N | N | NT | medium |
+| crm_upsert_client | Y | N | N | N | NT | medium |
+| crm_upsert_site | Y | N | N | N | NT | medium |
+| crm_upsert_window_unit | Y | N | N | N | NT | medium |
+| data_governance_tool | Y | Y | Y | Y | NT | low |
+| dependency_scanner_tool | Y | Y | Y | Y | NT | low |
+| docs_render_invoice_pdf | Y | N | N | N | NT | medium |
+| docs_render_quote_pdf | Y | N | N | N | NT | medium |
+| drift_analyzer_tool | Y | Y | Y | Y | NT | low |
+| file_tool | Y | Y | Y | Y | NT | low |
+| graph_query | Y | Y | Y | Y | NT | low |
+| image_generate | Y | Y | Y | Y | NT | low |
+| incident_escalation_tool | Y | Y | Y | N | NT | low |
+| incident_intelligence_tool | Y | Y | Y | N | NT | low |
+| job_orchestrator_tool | Y | Y | Y | Y | NT | low |
+| kb_tool | Y | Y | Y | Y | NT | low |
+| market_data | Y | Y | Y | N | NT | low |
+| memory_search | Y | Y | Y | Y | NT | low |
+| notion_tool | Y | N | N | N | Y (status) | medium |
+| observability_tool | Y | Y | Y | Y | NT | low |
+| oncall_tool | Y | Y | Y | Y | NT | low |
+| pieces_tool | Y | N | N | N | NT | medium |
+| pr_reviewer_tool | Y | Y | Y | Y | NT | low |
+| presentation_create | Y | Y | Y | Y | NT | low |
+| presentation_download | Y | Y | N | Y | NT | low |
+| presentation_status | Y | Y | N | Y | NT | low |
+| remember_fact | Y | Y | N | Y | NT | low |
+| repo_tool | Y | Y | Y | Y | Y (metadata) | low |
+| risk_engine_tool | Y | Y | Y | N | NT | low |
+| risk_history_tool | Y | Y | Y | N | NT | low |
+| safe_code_executor_tool | Y | Y | Y | Y | Y (validate) | low |
+| schedule_confirm_slot | Y | N | N | N | NT | medium |
+| schedule_propose_slots | Y | N | N | N | NT | medium |
+| secure_vault_tool | Y | Y | Y | Y | Y (store) | low |
+| threatmodel_tool | Y | Y | Y | Y | NT | low |
+| tts_speak | Y | Y | Y | Y | NT | low |
+| web_extract | Y | Y | Y | Y | NT | low |
+| web_search | Y | Y | Y | Y | NT | low |