docs(audit): add NODA2 Sofiia tools audit and full matrix

This commit is contained in:
Apple
2026-03-01 01:42:57 -08:00
parent 57632699c0
commit 49afb1df99
2 changed files with 229 additions and 0 deletions

View File

@@ -0,0 +1,167 @@
# Sofiia Tools Audit (NODA2)
Date: 2026-03-01
Node: NODA2 (local laptop)
Scope: Router tool stack + requested integrations (AgentEmailTool, BrowserTool, SecureVault, SafeCodeExecutor, CalendarTool) + broader Sofiia tool system readiness.
## 1) Inventory and Wiring Integrity
- Tool definitions declared in router: **56 executable tools** (`services/router/tool_manager.py`)
- Tool dispatch branches in router: **56 tools**
- Dispatch-to-handler integrity check: **no missing handler definitions**
Evidence:
- Definitions source: `services/router/tool_manager.py`
- Dispatch source: `services/router/tool_manager.py`
## 2) NODA2 Infrastructure Readiness
Calendar stack is wired into NODA2 compose:
- `router` env has `CALENDAR_SERVICE_URL=http://calendar-service:8001`
- `router` mounts `./tools:/app/tools:ro` (required for local tool modules)
- `router` depends on `calendar-service`
- `calendar-service` service present and running
- `radicale` service present and running
Compose source:
- `docker-compose.node2-sofiia.yml`
## 3) Requested Tool Audit (Runtime)
### AgentEmailTool
- Route wiring: present
- RBAC mapping: present (`tools.email.use`)
- Limits: present
- Runtime check: `list_inboxes` -> `ok` (empty list expected on fresh setup)
### BrowserTool
- Route wiring: present
- RBAC mapping: present (`tools.browser.use`)
- Limits: present
- Runtime check: `start_session/goto/get_current_url/close_session` -> `ok`
- Async loop blocker resolved via thread offload in router adapter.
### SecureVault
- Route wiring: present
- RBAC mapping: present (`tools.vault.manage`)
- Limits: present
- Runtime check: `store` -> `ok`
### SafeCodeExecutor
- Route wiring: present
- RBAC mapping: present (`tools.exec.safe`)
- Limits: present
- Runtime check: `validate` -> `ok` (`python` sample valid)
### CalendarTool (Radicale/CalDAV via calendar-service)
- Route wiring: present
- RBAC mapping: present (`tools.calendar.use`)
- Limits: present
- Runtime check:
- `calendar-service /health` -> healthy
- `calendar_tool list_calendars` without `account_id` -> domain error `account_id required` (expected), proving router->service path is live.
## 4) RBAC and Governance Validation
Files present and active:
- `config/tools_rollout.yml`
- `config/rbac_tools_matrix.yml`
- `config/tool_limits.yml`
Validated outcomes:
- `sofiia` and `admin` mapped to `agent_cto` rollout
- New tools included in `cto_tools`
- Role entitlements include calendar/email/browser/executor/vault usage
- Negative check passed: `monitor` denied on `secure_vault_tool`
## 5) Sofiia CTO Access Audit (repo / notion / git / nodes)
### Repo access
- `repo_tool` available and callable
- Runtime check `repo_tool:metadata` -> `ok`
### Notion access
- `notion_tool` available and callable
- Runtime check `notion_tool:status` -> `ok` (workspace bot identity returned)
### Git/repo operational tooling
- `repo_tool`, `pr_reviewer_tool`, `contract_tool`, `kb_tool` are present in tool definitions and dispatch.
### Node visibility/control plane
- Console endpoint `GET /api/agents?nodes=NODA2` returns healthy agent registry for NODA2.
- Nodes registry file present: `config/nodes_registry.yml`
## 6) Documentation Coverage
Current docs directories found:
- `docs/tools/` (tool docs exist for key governance/ops tools)
- `docs/audit/` and `docs/audits/` (existing system audit artifacts)
Gap observed:
- Documentation depth is uneven across all 56 tools; some newer tools are wired and working but not yet fully documented in `docs/tools/`.
## 7) Current Risk Register (Audit Findings)
1. **Medium**: Calendar integration is operational, but no account bootstrap in this audit run (no connected calendar account configured yet).
2. **Low/Medium**: Tool documentation is incomplete relative to actual implemented tool surface (56 tools).
3. **Low**: Repo is in a very large dirty state; future changes should stay strictly path-scoped to avoid accidental mixed commits.
## 8) Appendix: Executable Tool Set (56)
agent_email_tool
alert_ingest_tool
architecture_pressure_tool
backlog_tool
binance_account_bots
binance_bots_top
browser_tool
calc_window_quote
calendar_tool
comfy_generate_image
comfy_generate_video
config_linter_tool
contract_tool
cost_analyzer_tool
crawl4ai_scrape
crm_create_job
crm_create_quote
crm_search_client
crm_update_quote
crm_upsert_client
crm_upsert_site
crm_upsert_window_unit
data_governance_tool
dependency_scanner_tool
docs_render_invoice_pdf
docs_render_quote_pdf
drift_analyzer_tool
file_tool
graph_query
image_generate
incident_escalation_tool
incident_intelligence_tool
job_orchestrator_tool
kb_tool
market_data
memory_search
notion_tool
observability_tool
oncall_tool
pieces_tool
pr_reviewer_tool
presentation_create
presentation_download
presentation_status
remember_fact
repo_tool
risk_engine_tool
risk_history_tool
safe_code_executor_tool
schedule_confirm_slot
schedule_propose_slots
secure_vault_tool
threatmodel_tool
tts_speak
web_extract
web_search

View File

@@ -0,0 +1,62 @@
# Sofiia NODA2 Tool Matrix (2026-03-01)
Legend: `Y`=present/verified, `N`=missing, `NT`=not runtime-tested in this audit run.
| Tool | Wiring | RBAC | Limits | Rollout | Runtime | Risk |
|---|---|---|---|---|---|---|
| agent_email_tool | Y | Y | Y | Y | Y (list_inboxes) | low |
| alert_ingest_tool | Y | Y | Y | N | NT | low |
| architecture_pressure_tool | Y | Y | Y | N | NT | low |
| backlog_tool | Y | Y | Y | N | NT | low |
| binance_account_bots | Y | N | N | N | NT | medium |
| binance_bots_top | Y | N | N | N | NT | medium |
| browser_tool | Y | Y | Y | Y | Y (start/goto/get_current_url/close) | low |
| calc_window_quote | Y | N | N | N | NT | medium |
| calendar_tool | Y | Y | Y | Y | Y (route live; account missing expected) | medium |
| comfy_generate_image | Y | Y | Y | Y | NT | low |
| comfy_generate_video | Y | Y | Y | Y | NT | low |
| config_linter_tool | Y | Y | Y | Y | NT | low |
| contract_tool | Y | Y | Y | Y | NT | low |
| cost_analyzer_tool | Y | Y | Y | Y | NT | low |
| crawl4ai_scrape | Y | Y | Y | Y | NT | low |
| crm_create_job | Y | N | N | N | NT | medium |
| crm_create_quote | Y | N | N | N | NT | medium |
| crm_search_client | Y | N | N | N | NT | medium |
| crm_update_quote | Y | N | N | N | NT | medium |
| crm_upsert_client | Y | N | N | N | NT | medium |
| crm_upsert_site | Y | N | N | N | NT | medium |
| crm_upsert_window_unit | Y | N | N | N | NT | medium |
| data_governance_tool | Y | Y | Y | Y | NT | low |
| dependency_scanner_tool | Y | Y | Y | Y | NT | low |
| docs_render_invoice_pdf | Y | N | N | N | NT | medium |
| docs_render_quote_pdf | Y | N | N | N | NT | medium |
| drift_analyzer_tool | Y | Y | Y | Y | NT | low |
| file_tool | Y | Y | Y | Y | NT | low |
| graph_query | Y | Y | Y | Y | NT | low |
| image_generate | Y | Y | Y | Y | NT | low |
| incident_escalation_tool | Y | Y | Y | N | NT | low |
| incident_intelligence_tool | Y | Y | Y | N | NT | low |
| job_orchestrator_tool | Y | Y | Y | Y | NT | low |
| kb_tool | Y | Y | Y | Y | NT | low |
| market_data | Y | Y | Y | N | NT | low |
| memory_search | Y | Y | Y | Y | NT | low |
| notion_tool | Y | N | N | N | Y (status) | medium |
| observability_tool | Y | Y | Y | Y | NT | low |
| oncall_tool | Y | Y | Y | Y | NT | low |
| pieces_tool | Y | N | N | N | NT | medium |
| pr_reviewer_tool | Y | Y | Y | Y | NT | low |
| presentation_create | Y | Y | Y | Y | NT | low |
| presentation_download | Y | Y | N | Y | NT | low |
| presentation_status | Y | Y | N | Y | NT | low |
| remember_fact | Y | Y | N | Y | NT | low |
| repo_tool | Y | Y | Y | Y | Y (metadata) | low |
| risk_engine_tool | Y | Y | Y | N | NT | low |
| risk_history_tool | Y | Y | Y | N | NT | low |
| safe_code_executor_tool | Y | Y | Y | Y | Y (validate) | low |
| schedule_confirm_slot | Y | N | N | N | NT | medium |
| schedule_propose_slots | Y | N | N | N | NT | medium |
| secure_vault_tool | Y | Y | Y | Y | Y (store) | low |
| threatmodel_tool | Y | Y | Y | Y | NT | low |
| tts_speak | Y | Y | Y | Y | NT | low |
| web_extract | Y | Y | Y | Y | NT | low |
| web_search | Y | Y | Y | Y | NT | low |