docs(audit): add NODA2 Sofiia tools audit and full matrix
This commit is contained in:
167
docs/audit/SOFIIA_NODA2_TOOLS_AUDIT_2026-03-01.md
Normal file
167
docs/audit/SOFIIA_NODA2_TOOLS_AUDIT_2026-03-01.md
Normal file
@@ -0,0 +1,167 @@
|
|||||||
|
# Sofiia Tools Audit (NODA2)
|
||||||
|
|
||||||
|
Date: 2026-03-01
|
||||||
|
Node: NODA2 (local laptop)
|
||||||
|
Scope: Router tool stack + requested integrations (AgentEmailTool, BrowserTool, SecureVault, SafeCodeExecutor, CalendarTool) + broader Sofiia tool system readiness.
|
||||||
|
|
||||||
|
## 1) Inventory and Wiring Integrity
|
||||||
|
|
||||||
|
- Tool definitions declared in router: **56 executable tools** (`services/router/tool_manager.py`)
|
||||||
|
- Tool dispatch branches in router: **56 tools**
|
||||||
|
- Dispatch-to-handler integrity check: **no missing handler definitions**
|
||||||
|
|
||||||
|
Evidence:
|
||||||
|
- Definitions source: `services/router/tool_manager.py`
|
||||||
|
- Dispatch source: `services/router/tool_manager.py`
|
||||||
|
|
||||||
|
## 2) NODA2 Infrastructure Readiness
|
||||||
|
|
||||||
|
Calendar stack is wired into NODA2 compose:
|
||||||
|
- `router` env has `CALENDAR_SERVICE_URL=http://calendar-service:8001`
|
||||||
|
- `router` mounts `./tools:/app/tools:ro` (required for local tool modules)
|
||||||
|
- `router` depends on `calendar-service`
|
||||||
|
- `calendar-service` service present and running
|
||||||
|
- `radicale` service present and running
|
||||||
|
|
||||||
|
Compose source:
|
||||||
|
- `docker-compose.node2-sofiia.yml`
|
||||||
|
|
||||||
|
## 3) Requested Tool Audit (Runtime)
|
||||||
|
|
||||||
|
### AgentEmailTool
|
||||||
|
- Route wiring: present
|
||||||
|
- RBAC mapping: present (`tools.email.use`)
|
||||||
|
- Limits: present
|
||||||
|
- Runtime check: `list_inboxes` -> `ok` (empty list expected on fresh setup)
|
||||||
|
|
||||||
|
### BrowserTool
|
||||||
|
- Route wiring: present
|
||||||
|
- RBAC mapping: present (`tools.browser.use`)
|
||||||
|
- Limits: present
|
||||||
|
- Runtime check: `start_session/goto/get_current_url/close_session` -> `ok`
|
||||||
|
- Async loop blocker resolved via thread offload in router adapter.
|
||||||
|
|
||||||
|
### SecureVault
|
||||||
|
- Route wiring: present
|
||||||
|
- RBAC mapping: present (`tools.vault.manage`)
|
||||||
|
- Limits: present
|
||||||
|
- Runtime check: `store` -> `ok`
|
||||||
|
|
||||||
|
### SafeCodeExecutor
|
||||||
|
- Route wiring: present
|
||||||
|
- RBAC mapping: present (`tools.exec.safe`)
|
||||||
|
- Limits: present
|
||||||
|
- Runtime check: `validate` -> `ok` (`python` sample valid)
|
||||||
|
|
||||||
|
### CalendarTool (Radicale/CalDAV via calendar-service)
|
||||||
|
- Route wiring: present
|
||||||
|
- RBAC mapping: present (`tools.calendar.use`)
|
||||||
|
- Limits: present
|
||||||
|
- Runtime check:
|
||||||
|
- `calendar-service /health` -> healthy
|
||||||
|
- `calendar_tool list_calendars` without `account_id` -> domain error `account_id required` (expected), proving router->service path is live.
|
||||||
|
|
||||||
|
## 4) RBAC and Governance Validation
|
||||||
|
|
||||||
|
Files present and active:
|
||||||
|
- `config/tools_rollout.yml`
|
||||||
|
- `config/rbac_tools_matrix.yml`
|
||||||
|
- `config/tool_limits.yml`
|
||||||
|
|
||||||
|
Validated outcomes:
|
||||||
|
- `sofiia` and `admin` mapped to `agent_cto` rollout
|
||||||
|
- New tools included in `cto_tools`
|
||||||
|
- Role entitlements include calendar/email/browser/executor/vault usage
|
||||||
|
- Negative check passed: `monitor` denied on `secure_vault_tool`
|
||||||
|
|
||||||
|
## 5) Sofiia CTO Access Audit (repo / notion / git / nodes)
|
||||||
|
|
||||||
|
### Repo access
|
||||||
|
- `repo_tool` available and callable
|
||||||
|
- Runtime check `repo_tool:metadata` -> `ok`
|
||||||
|
|
||||||
|
### Notion access
|
||||||
|
- `notion_tool` available and callable
|
||||||
|
- Runtime check `notion_tool:status` -> `ok` (workspace bot identity returned)
|
||||||
|
|
||||||
|
### Git/repo operational tooling
|
||||||
|
- `repo_tool`, `pr_reviewer_tool`, `contract_tool`, `kb_tool` are present in tool definitions and dispatch.
|
||||||
|
|
||||||
|
### Node visibility/control plane
|
||||||
|
- Console endpoint `GET /api/agents?nodes=NODA2` returns healthy agent registry for NODA2.
|
||||||
|
- Nodes registry file present: `config/nodes_registry.yml`
|
||||||
|
|
||||||
|
## 6) Documentation Coverage
|
||||||
|
|
||||||
|
Current docs directories found:
|
||||||
|
- `docs/tools/` (tool docs exist for key governance/ops tools)
|
||||||
|
- `docs/audit/` and `docs/audits/` (existing system audit artifacts)
|
||||||
|
|
||||||
|
Gap observed:
|
||||||
|
- Documentation depth is uneven across all 56 tools; some newer tools are wired and working but not yet fully documented in `docs/tools/`.
|
||||||
|
|
||||||
|
## 7) Current Risk Register (Audit Findings)
|
||||||
|
|
||||||
|
1. **Medium**: Calendar integration is operational, but no account bootstrap in this audit run (no connected calendar account configured yet).
|
||||||
|
2. **Low/Medium**: Tool documentation is incomplete relative to actual implemented tool surface (56 tools).
|
||||||
|
3. **Low**: Repo is in a very large dirty state; future changes should stay strictly path-scoped to avoid accidental mixed commits.
|
||||||
|
|
||||||
|
## 8) Appendix: Executable Tool Set (56)
|
||||||
|
|
||||||
|
agent_email_tool
|
||||||
|
alert_ingest_tool
|
||||||
|
architecture_pressure_tool
|
||||||
|
backlog_tool
|
||||||
|
binance_account_bots
|
||||||
|
binance_bots_top
|
||||||
|
browser_tool
|
||||||
|
calc_window_quote
|
||||||
|
calendar_tool
|
||||||
|
comfy_generate_image
|
||||||
|
comfy_generate_video
|
||||||
|
config_linter_tool
|
||||||
|
contract_tool
|
||||||
|
cost_analyzer_tool
|
||||||
|
crawl4ai_scrape
|
||||||
|
crm_create_job
|
||||||
|
crm_create_quote
|
||||||
|
crm_search_client
|
||||||
|
crm_update_quote
|
||||||
|
crm_upsert_client
|
||||||
|
crm_upsert_site
|
||||||
|
crm_upsert_window_unit
|
||||||
|
data_governance_tool
|
||||||
|
dependency_scanner_tool
|
||||||
|
docs_render_invoice_pdf
|
||||||
|
docs_render_quote_pdf
|
||||||
|
drift_analyzer_tool
|
||||||
|
file_tool
|
||||||
|
graph_query
|
||||||
|
image_generate
|
||||||
|
incident_escalation_tool
|
||||||
|
incident_intelligence_tool
|
||||||
|
job_orchestrator_tool
|
||||||
|
kb_tool
|
||||||
|
market_data
|
||||||
|
memory_search
|
||||||
|
notion_tool
|
||||||
|
observability_tool
|
||||||
|
oncall_tool
|
||||||
|
pieces_tool
|
||||||
|
pr_reviewer_tool
|
||||||
|
presentation_create
|
||||||
|
presentation_download
|
||||||
|
presentation_status
|
||||||
|
remember_fact
|
||||||
|
repo_tool
|
||||||
|
risk_engine_tool
|
||||||
|
risk_history_tool
|
||||||
|
safe_code_executor_tool
|
||||||
|
schedule_confirm_slot
|
||||||
|
schedule_propose_slots
|
||||||
|
secure_vault_tool
|
||||||
|
threatmodel_tool
|
||||||
|
tts_speak
|
||||||
|
web_extract
|
||||||
|
web_search
|
||||||
|
|
||||||
62
docs/audit/SOFIIA_NODA2_TOOL_MATRIX_2026-03-01.md
Normal file
62
docs/audit/SOFIIA_NODA2_TOOL_MATRIX_2026-03-01.md
Normal file
@@ -0,0 +1,62 @@
|
|||||||
|
# Sofiia NODA2 Tool Matrix (2026-03-01)
|
||||||
|
|
||||||
|
Legend: `Y`=present/verified, `N`=missing, `NT`=not runtime-tested in this audit run.
|
||||||
|
|
||||||
|
| Tool | Wiring | RBAC | Limits | Rollout | Runtime | Risk |
|
||||||
|
|---|---|---|---|---|---|---|
|
||||||
|
| agent_email_tool | Y | Y | Y | Y | Y (list_inboxes) | low |
|
||||||
|
| alert_ingest_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| architecture_pressure_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| backlog_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| binance_account_bots | Y | N | N | N | NT | medium |
|
||||||
|
| binance_bots_top | Y | N | N | N | NT | medium |
|
||||||
|
| browser_tool | Y | Y | Y | Y | Y (start/goto/get_current_url/close) | low |
|
||||||
|
| calc_window_quote | Y | N | N | N | NT | medium |
|
||||||
|
| calendar_tool | Y | Y | Y | Y | Y (route live; account missing expected) | medium |
|
||||||
|
| comfy_generate_image | Y | Y | Y | Y | NT | low |
|
||||||
|
| comfy_generate_video | Y | Y | Y | Y | NT | low |
|
||||||
|
| config_linter_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| contract_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| cost_analyzer_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| crawl4ai_scrape | Y | Y | Y | Y | NT | low |
|
||||||
|
| crm_create_job | Y | N | N | N | NT | medium |
|
||||||
|
| crm_create_quote | Y | N | N | N | NT | medium |
|
||||||
|
| crm_search_client | Y | N | N | N | NT | medium |
|
||||||
|
| crm_update_quote | Y | N | N | N | NT | medium |
|
||||||
|
| crm_upsert_client | Y | N | N | N | NT | medium |
|
||||||
|
| crm_upsert_site | Y | N | N | N | NT | medium |
|
||||||
|
| crm_upsert_window_unit | Y | N | N | N | NT | medium |
|
||||||
|
| data_governance_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| dependency_scanner_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| docs_render_invoice_pdf | Y | N | N | N | NT | medium |
|
||||||
|
| docs_render_quote_pdf | Y | N | N | N | NT | medium |
|
||||||
|
| drift_analyzer_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| file_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| graph_query | Y | Y | Y | Y | NT | low |
|
||||||
|
| image_generate | Y | Y | Y | Y | NT | low |
|
||||||
|
| incident_escalation_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| incident_intelligence_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| job_orchestrator_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| kb_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| market_data | Y | Y | Y | N | NT | low |
|
||||||
|
| memory_search | Y | Y | Y | Y | NT | low |
|
||||||
|
| notion_tool | Y | N | N | N | Y (status) | medium |
|
||||||
|
| observability_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| oncall_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| pieces_tool | Y | N | N | N | NT | medium |
|
||||||
|
| pr_reviewer_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| presentation_create | Y | Y | Y | Y | NT | low |
|
||||||
|
| presentation_download | Y | Y | N | Y | NT | low |
|
||||||
|
| presentation_status | Y | Y | N | Y | NT | low |
|
||||||
|
| remember_fact | Y | Y | N | Y | NT | low |
|
||||||
|
| repo_tool | Y | Y | Y | Y | Y (metadata) | low |
|
||||||
|
| risk_engine_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| risk_history_tool | Y | Y | Y | N | NT | low |
|
||||||
|
| safe_code_executor_tool | Y | Y | Y | Y | Y (validate) | low |
|
||||||
|
| schedule_confirm_slot | Y | N | N | N | NT | medium |
|
||||||
|
| schedule_propose_slots | Y | N | N | N | NT | medium |
|
||||||
|
| secure_vault_tool | Y | Y | Y | Y | Y (store) | low |
|
||||||
|
| threatmodel_tool | Y | Y | Y | Y | NT | low |
|
||||||
|
| tts_speak | Y | Y | Y | Y | NT | low |
|
||||||
|
| web_extract | Y | Y | Y | Y | NT | low |
|
||||||
|
| web_search | Y | Y | Y | Y | NT | low |
|
||||||
Reference in New Issue
Block a user