feat: Implement Matrix Rooms Bridge

- MATRIX_ROOMS_BRIDGE_SPEC.md documentation
- Migration 012: Add matrix_room_id/alias to city_rooms
- Matrix Gateway service (port 7025)
- City-service: auto-create Matrix rooms on room creation
- Backfill endpoint for existing rooms
- API returns matrix_room_id/alias in room responses
This commit is contained in:
Apple
2025-11-26 12:33:54 -08:00
parent 29febee464
commit 984f67c26e
19 changed files with 3356 additions and 0 deletions

View File

@@ -0,0 +1,646 @@
# Matrix Gateway API Specification
**Version:** 1.0.0
**Service:** matrix-gateway
**Port:** 7003 (internal only)
**Purpose:** Internal service for Matrix protocol integration
---
## Overview
The **matrix-gateway** is an internal service that:
- Wraps Matrix Client-Server API
- Provides simplified endpoints for DAARION services
- Handles Matrix authentication and session management
- Transforms Matrix events ↔ DAARION internal DTOs
- Manages webhook subscriptions for real-time events
**NOT exposed to public internet** — internal service mesh only.
---
## Architecture
```
┌─────────────────┐
│ messaging-service│
│ │
│ (DAARION API) │
└────────┬────────┘
│ HTTP (internal)
┌─────────────────┐
│ matrix-gateway │
│ │
│ (Matrix API) │
└────────┬────────┘
│ Matrix C-S API
┌─────────────────┐
│ Matrix Homeserver│
│ (Synapse) │
└─────────────────┘
```
---
## Authentication
All requests must include:
```
X-Internal-Service-Token: <shared-secret>
```
Services authenticate via shared secret (not Matrix access tokens).
---
## Endpoints
### 1. Room Management
#### **POST /internal/matrix/create-room**
Create a new Matrix room.
**Request:**
```json
{
"name": "General Discussion",
"topic": "Main DAARION.city channel",
"visibility": "public",
"room_alias_name": "general",
"preset": "public_chat",
"initial_state": [
{
"type": "m.room.guest_access",
"state_key": "",
"content": { "guest_access": "can_join" }
}
],
"power_level_content_override": {
"users": {
"@daarion:daarion.city": 100
},
"events": {
"m.room.name": 50,
"m.room.topic": 50
}
},
"creation_content": {
"m.federate": true
}
}
```
**Response:**
```json
{
"room_id": "!general:daarion.city",
"room_alias": "#general:daarion.city"
}
```
**Errors:**
- 400: Invalid request
- 500: Matrix API error
---
#### **GET /internal/matrix/rooms/{roomId}**
Get room details.
**Response:**
```json
{
"room_id": "!general:daarion.city",
"name": "General Discussion",
"topic": "Main DAARION.city channel",
"avatar_url": "mxc://daarion.city/avatar123",
"canonical_alias": "#general:daarion.city",
"member_count": 42,
"joined_members": 38,
"encryption": null
}
```
---
#### **PUT /internal/matrix/rooms/{roomId}/name**
Update room name.
**Request:**
```json
{
"name": "Updated Room Name"
}
```
**Response:**
```json
{
"event_id": "$event123:daarion.city"
}
```
---
### 2. Message Sending
#### **POST /internal/matrix/send**
Send a message to a room.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"sender": "agent:sofia",
"sender_matrix_id": "@sofia:daarion.city",
"msgtype": "m.text",
"body": "Hello from DAARION!",
"format": "org.matrix.custom.html",
"formatted_body": "<p>Hello from <strong>DAARION</strong>!</p>",
"relates_to": {
"m.in_reply_to": {
"event_id": "$parent_event:daarion.city"
}
}
}
```
**Response:**
```json
{
"event_id": "$event456:daarion.city",
"sent_at": "2025-11-24T10:30:00Z"
}
```
**Supported msgtypes:**
- `m.text` — plain text
- `m.image` — image
- `m.file` — file attachment
- `m.audio` — audio
- `m.video` — video
- `m.notice` — bot/agent notice
---
#### **POST /internal/matrix/send-reaction**
React to a message.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"event_id": "$target_event:daarion.city",
"reactor": "user:alice",
"reactor_matrix_id": "@alice:daarion.city",
"emoji": "👍"
}
```
**Response:**
```json
{
"event_id": "$reaction789:daarion.city"
}
```
---
#### **POST /internal/matrix/redact**
Redact (delete) a message.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"event_id": "$event_to_delete:daarion.city",
"reason": "Spam"
}
```
**Response:**
```json
{
"event_id": "$redaction999:daarion.city"
}
```
---
### 3. Membership
#### **POST /internal/matrix/invite**
Invite a user/agent to a room.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"user_id": "@alice:daarion.city",
"inviter": "user:admin",
"inviter_matrix_id": "@admin:daarion.city"
}
```
**Response:**
```json
{
"status": "invited"
}
```
---
#### **POST /internal/matrix/join**
Join a room (on behalf of user/agent).
**Request:**
```json
{
"room_id": "!general:daarion.city",
"user_id": "@alice:daarion.city"
}
```
**Response:**
```json
{
"status": "joined",
"room_id": "!general:daarion.city"
}
```
---
#### **POST /internal/matrix/leave**
Leave a room.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"user_id": "@alice:daarion.city"
}
```
**Response:**
```json
{
"status": "left"
}
```
---
#### **POST /internal/matrix/kick**
Kick a user from a room.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"user_id": "@spammer:daarion.city",
"kicker": "@admin:daarion.city",
"reason": "Violation of rules"
}
```
**Response:**
```json
{
"status": "kicked"
}
```
---
### 4. Event Sync
#### **GET /internal/matrix/sync**
Get recent events (polling mode).
**Query params:**
- `since` — sync token (optional)
- `timeout` — long-polling timeout in ms (default 30000)
- `filter` — JSON filter (optional)
**Response:**
```json
{
"next_batch": "s1234_567_8_9_10",
"rooms": {
"join": {
"!general:daarion.city": {
"timeline": {
"events": [
{
"type": "m.room.message",
"event_id": "$event123:daarion.city",
"sender": "@alice:daarion.city",
"content": {
"msgtype": "m.text",
"body": "Hello!"
},
"origin_server_ts": 1732445400000
}
],
"limited": false,
"prev_batch": "p1234_567"
}
}
}
}
}
```
---
#### **POST /internal/matrix/webhook/subscribe**
Subscribe to room events via webhook.
**Request:**
```json
{
"room_id": "!general:daarion.city",
"webhook_url": "http://messaging-service:7004/webhooks/matrix-events",
"events": ["m.room.message", "m.room.member"]
}
```
**Response:**
```json
{
"subscription_id": "sub-abc123"
}
```
When events occur, matrix-gateway will POST to webhook_url:
```json
{
"subscription_id": "sub-abc123",
"room_id": "!general:daarion.city",
"event": {
"type": "m.room.message",
"event_id": "$event456:daarion.city",
"sender": "@bob:daarion.city",
"content": {
"msgtype": "m.text",
"body": "Hi there"
},
"origin_server_ts": 1732445500000
}
}
```
---
#### **DELETE /internal/matrix/webhook/subscribe/{subscriptionId}**
Unsubscribe from webhook.
**Response:**
```json
{
"status": "unsubscribed"
}
```
---
### 5. User Management
#### **POST /internal/matrix/register-user**
Register a new Matrix user (for agent or human).
**Request:**
```json
{
"username": "alice",
"password": "generated-secure-password",
"display_name": "Alice",
"avatar_url": "mxc://daarion.city/avatar456",
"admin": false
}
```
**Response:**
```json
{
"user_id": "@alice:daarion.city",
"access_token": "syt_...",
"device_id": "DEVICE123"
}
```
---
#### **PUT /internal/matrix/users/{userId}/display-name**
Update user display name.
**Request:**
```json
{
"display_name": "Alice (Updated)"
}
```
**Response:**
```json
{
"status": "updated"
}
```
---
#### **PUT /internal/matrix/users/{userId}/avatar**
Update user avatar.
**Request:**
```json
{
"avatar_url": "mxc://daarion.city/new-avatar"
}
```
**Response:**
```json
{
"status": "updated"
}
```
---
### 6. Media Upload
#### **POST /internal/matrix/upload**
Upload media (for messages with images/files).
**Request:** `multipart/form-data`
- `file` — file to upload
**Response:**
```json
{
"content_uri": "mxc://daarion.city/file123",
"content_type": "image/png",
"size": 102400
}
```
---
### 7. Room History
#### **GET /internal/matrix/rooms/{roomId}/messages**
Get paginated message history.
**Query params:**
- `from` — pagination token (required)
- `dir``b` (backwards) or `f` (forwards), default `b`
- `limit` — max events, default 10
**Response:**
```json
{
"start": "t1234_567",
"end": "t1234_500",
"chunk": [
{
"type": "m.room.message",
"event_id": "$event789:daarion.city",
"sender": "@charlie:daarion.city",
"content": {
"msgtype": "m.text",
"body": "Previous message"
},
"origin_server_ts": 1732445300000
}
]
}
```
---
## Event Types (Matrix → DAARION mapping)
| Matrix Event Type | DAARION Internal Event |
|-------------------|------------------------|
| `m.room.message` (msgtype=m.text) | `messaging.message.created` |
| `m.room.message` (msgtype=m.image) | `messaging.media.uploaded` |
| `m.room.member` (join) | `messaging.member.joined` |
| `m.room.member` (leave) | `messaging.member.left` |
| `m.room.member` (invite) | `messaging.member.invited` |
| `m.room.name` | `messaging.channel.updated` |
| `m.room.topic` | `messaging.channel.updated` |
| `m.reaction` | `messaging.reaction.added` |
| `m.room.redaction` | `messaging.message.deleted` |
---
## Error Responses
All errors follow the format:
```json
{
"error": "M_FORBIDDEN",
"message": "You are not allowed to send messages in this room"
}
```
Common error codes:
- `M_FORBIDDEN` — Insufficient permissions
- `M_NOT_FOUND` — Room/user not found
- `M_UNKNOWN` — Generic Matrix error
- `M_BAD_JSON` — Invalid request payload
- `INTERNAL_ERROR` — matrix-gateway internal error
---
## Configuration
Environment variables:
- `MATRIX_HOMESERVER_URL` — e.g. `https://matrix.daarion.city`
- `MATRIX_ADMIN_TOKEN` — admin access token for homeserver operations
- `INTERNAL_SERVICE_SECRET` — shared secret for service-to-service auth
- `WEBHOOK_TIMEOUT_MS` — timeout for webhook deliveries (default 5000)
- `SYNC_TIMEOUT_MS` — long-polling timeout (default 30000)
---
## Implementation Notes
1. **User impersonation**: matrix-gateway can send messages on behalf of any user/agent (using admin privileges or shared secret registration).
2. **Event transformation**: All Matrix events are enriched with DAARION entity IDs (user:..., agent:...) before forwarding to services.
3. **Webhook reliability**: Webhooks are retried 3 times with exponential backoff. Failed events are logged but not re-queued.
4. **Rate limiting**: matrix-gateway implements internal rate limiting to avoid overwhelming the homeserver (max 100 req/s per service).
5. **Caching**: Room metadata (name, topic, members) is cached for 5 minutes to reduce load on Matrix homeserver.
---
## Testing
Use the provided `matrix-gateway-test.http` file for manual testing:
```http
### Create room
POST http://localhost:7003/internal/matrix/create-room
X-Internal-Service-Token: dev-secret-token
Content-Type: application/json
{
"name": "Test Room",
"topic": "Testing",
"visibility": "public"
}
### Send message
POST http://localhost:7003/internal/matrix/send
X-Internal-Service-Token: dev-secret-token
Content-Type: application/json
{
"room_id": "!test:daarion.city",
"sender": "agent:test",
"sender_matrix_id": "@test:daarion.city",
"msgtype": "m.text",
"body": "Hello from test!"
}
```
---
**Version:** 1.0.0
**Last Updated:** 2025-11-24
**Maintainer:** DAARION Platform Team

View File

@@ -0,0 +1,17 @@
FROM python:3.11-slim
WORKDIR /app
# Install dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application
COPY . .
# Expose port
EXPOSE 7025
# Run
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "7025"]

View File

@@ -0,0 +1,29 @@
"""
Matrix Gateway Configuration
"""
from pydantic_settings import BaseSettings
from functools import lru_cache
class Settings(BaseSettings):
# Service
service_name: str = "matrix-gateway"
service_version: str = "1.0.0"
port: int = 7025
# Synapse
synapse_url: str = "http://daarion-synapse:8008"
synapse_admin_token: str = ""
matrix_server_name: str = "daarion.space"
# Registration secret (for creating rooms as admin)
synapse_registration_secret: str = "daarion_reg_secret_2024"
class Config:
env_prefix = "MATRIX_GATEWAY_"
@lru_cache()
def get_settings() -> Settings:
return Settings()

View File

@@ -0,0 +1,324 @@
"""
Matrix Gateway Service
Provides internal API for Matrix operations (room creation, lookup, etc.)
"""
from fastapi import FastAPI, HTTPException, Query
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from typing import Optional
import httpx
import hashlib
import hmac
import logging
from config import get_settings
# Setup logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
settings = get_settings()
app = FastAPI(
title="DAARION Matrix Gateway",
description="Internal API for Matrix operations",
version=settings.service_version
)
# CORS (internal service, but add for flexibility)
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Admin access token (will be set on first request)
_admin_token: Optional[str] = None
# Models
class CreateRoomRequest(BaseModel):
slug: str
name: str
visibility: str = "public"
topic: Optional[str] = None
class CreateRoomResponse(BaseModel):
matrix_room_id: str
matrix_room_alias: str
class FindRoomResponse(BaseModel):
matrix_room_id: str
matrix_room_alias: str
class ErrorResponse(BaseModel):
error: str
detail: Optional[str] = None
class HealthResponse(BaseModel):
status: str
synapse: str
server_name: str
async def get_admin_token() -> str:
"""Get or create admin access token for Matrix operations."""
global _admin_token
if _admin_token and settings.synapse_admin_token:
return settings.synapse_admin_token
if _admin_token:
return _admin_token
# Try to use provided token
if settings.synapse_admin_token:
_admin_token = settings.synapse_admin_token
return _admin_token
# Create admin user and get token
try:
async with httpx.AsyncClient() as client:
# Get nonce
nonce_resp = await client.get(
f"{settings.synapse_url}/_synapse/admin/v1/register"
)
nonce_resp.raise_for_status()
nonce = nonce_resp.json()["nonce"]
# Generate MAC
mac = hmac.new(
key=settings.synapse_registration_secret.encode('utf-8'),
digestmod=hashlib.sha1
)
mac.update(nonce.encode('utf-8'))
mac.update(b"\x00")
mac.update(b"daarion_admin")
mac.update(b"\x00")
mac.update(b"admin_password_2024")
mac.update(b"\x00")
mac.update(b"admin")
# Register admin
register_resp = await client.post(
f"{settings.synapse_url}/_synapse/admin/v1/register",
json={
"nonce": nonce,
"username": "daarion_admin",
"password": "admin_password_2024",
"admin": True,
"mac": mac.hexdigest()
}
)
if register_resp.status_code == 200:
result = register_resp.json()
_admin_token = result.get("access_token")
logger.info("Admin user created successfully")
return _admin_token
elif register_resp.status_code == 400:
# User already exists, try to login
login_resp = await client.post(
f"{settings.synapse_url}/_matrix/client/v3/login",
json={
"type": "m.login.password",
"user": "daarion_admin",
"password": "admin_password_2024"
}
)
login_resp.raise_for_status()
result = login_resp.json()
_admin_token = result.get("access_token")
logger.info("Admin user logged in successfully")
return _admin_token
else:
raise Exception(f"Failed to create admin: {register_resp.text}")
except Exception as e:
logger.error(f"Failed to get admin token: {e}")
raise HTTPException(status_code=500, detail=f"Failed to get admin token: {e}")
@app.get("/healthz", response_model=HealthResponse)
async def health_check():
"""Health check endpoint."""
synapse_status = "unknown"
try:
async with httpx.AsyncClient(timeout=5.0) as client:
resp = await client.get(f"{settings.synapse_url}/_matrix/client/versions")
if resp.status_code == 200:
synapse_status = "connected"
else:
synapse_status = "error"
except Exception:
synapse_status = "unavailable"
return HealthResponse(
status="ok" if synapse_status == "connected" else "degraded",
synapse=synapse_status,
server_name=settings.matrix_server_name
)
@app.post("/internal/matrix/rooms/create", response_model=CreateRoomResponse)
async def create_room(request: CreateRoomRequest):
"""
Create a Matrix room for a City Room.
This is an internal endpoint - should only be called by city-service.
"""
admin_token = await get_admin_token()
room_alias_name = f"city_{request.slug}"
room_name = f"DAARION City — {request.name}"
async with httpx.AsyncClient() as client:
try:
# Create room
create_resp = await client.post(
f"{settings.synapse_url}/_matrix/client/v3/createRoom",
headers={"Authorization": f"Bearer {admin_token}"},
json={
"name": room_name,
"room_alias_name": room_alias_name,
"topic": request.topic or f"City room: {request.name}",
"preset": "public_chat" if request.visibility == "public" else "private_chat",
"visibility": "public" if request.visibility == "public" else "private",
"creation_content": {
"m.federate": False # Don't federate for now
},
"initial_state": [
{
"type": "m.room.history_visibility",
"content": {"history_visibility": "shared"}
},
{
"type": "m.room.guest_access",
"content": {"guest_access": "can_join"}
}
]
}
)
if create_resp.status_code == 200:
result = create_resp.json()
matrix_room_id = result["room_id"]
matrix_room_alias = f"#city_{request.slug}:{settings.matrix_server_name}"
logger.info(f"Created Matrix room: {matrix_room_id} ({matrix_room_alias})")
return CreateRoomResponse(
matrix_room_id=matrix_room_id,
matrix_room_alias=matrix_room_alias
)
elif create_resp.status_code == 400:
error = create_resp.json()
if "M_ROOM_IN_USE" in str(error):
# Room already exists, find it
alias = f"#city_{request.slug}:{settings.matrix_server_name}"
find_resp = await client.get(
f"{settings.synapse_url}/_matrix/client/v3/directory/room/{alias.replace('#', '%23').replace(':', '%3A')}",
headers={"Authorization": f"Bearer {admin_token}"}
)
if find_resp.status_code == 200:
room_info = find_resp.json()
return CreateRoomResponse(
matrix_room_id=room_info["room_id"],
matrix_room_alias=alias
)
logger.error(f"Failed to create room: {create_resp.text}")
raise HTTPException(status_code=400, detail=f"Matrix error: {error.get('error', 'Unknown')}")
else:
logger.error(f"Failed to create room: {create_resp.text}")
raise HTTPException(status_code=500, detail="Failed to create Matrix room")
except httpx.RequestError as e:
logger.error(f"Matrix request error: {e}")
raise HTTPException(status_code=503, detail="Matrix unavailable")
@app.get("/internal/matrix/rooms/find-by-alias", response_model=FindRoomResponse)
async def find_room_by_alias(alias: str = Query(..., description="Matrix room alias")):
"""
Find a Matrix room by its alias.
Example: ?alias=#city_general:daarion.space
"""
admin_token = await get_admin_token()
# URL encode the alias
encoded_alias = alias.replace("#", "%23").replace(":", "%3A")
async with httpx.AsyncClient() as client:
try:
resp = await client.get(
f"{settings.synapse_url}/_matrix/client/v3/directory/room/{encoded_alias}",
headers={"Authorization": f"Bearer {admin_token}"}
)
if resp.status_code == 200:
result = resp.json()
return FindRoomResponse(
matrix_room_id=result["room_id"],
matrix_room_alias=alias
)
elif resp.status_code == 404:
raise HTTPException(status_code=404, detail="Room not found")
else:
logger.error(f"Failed to find room: {resp.text}")
raise HTTPException(status_code=500, detail="Failed to find room")
except httpx.RequestError as e:
logger.error(f"Matrix request error: {e}")
raise HTTPException(status_code=503, detail="Matrix unavailable")
@app.get("/internal/matrix/rooms/{room_id}")
async def get_room_info(room_id: str):
"""Get information about a Matrix room."""
admin_token = await get_admin_token()
async with httpx.AsyncClient() as client:
try:
resp = await client.get(
f"{settings.synapse_url}/_matrix/client/v3/rooms/{room_id}/state",
headers={"Authorization": f"Bearer {admin_token}"}
)
if resp.status_code == 200:
state = resp.json()
# Extract room info from state
name = None
topic = None
for event in state:
if event.get("type") == "m.room.name":
name = event.get("content", {}).get("name")
elif event.get("type") == "m.room.topic":
topic = event.get("content", {}).get("topic")
return {
"room_id": room_id,
"name": name,
"topic": topic
}
else:
raise HTTPException(status_code=resp.status_code, detail="Failed to get room info")
except httpx.RequestError as e:
logger.error(f"Matrix request error: {e}")
raise HTTPException(status_code=503, detail="Matrix unavailable")
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=settings.port)

View File

@@ -0,0 +1,6 @@
fastapi==0.109.0
uvicorn[standard]==0.27.0
httpx==0.26.0
pydantic==2.5.3
pydantic-settings==2.1.0