commit 77d7a50fa9697864caab79abfdfeac0dd70eda84 Author: Samuel Amar Date: Wed Jul 29 14:00:22 2026 +0200 Initial commit: API gateway with admin console - FastAPI async gateway with httpx proxying to multiple upstreams - SQLite database with SQLAlchemy ORM - Admin console: manage services, users, API keys, endpoint access - Per-key, per-endpoint granular access control - OpenAPI document sync and caching (5-minute TTL) - Request/response logging with full transaction inspection - In-memory rate limiting (per-key, fixed-window) - Tiered log retention (7d payloads, 90d rows, incremental vacuum) - TLS verification toggle per service (for self-signed certificates) - Service connectivity validation with automatic endpoint refresh - Request browser with filters and deep-link inspection - Docker setup with persistent volume - Modal forms for create/edit flows Co-Authored-By: Claude Haiku 4.5 diff --git a/.claude/launch.json b/.claude/launch.json new file mode 100644 index 0000000..ea31998 --- /dev/null +++ b/.claude/launch.json @@ -0,0 +1,11 @@ +{ + "version": "0.0.1", + "configurations": [ + { + "name": "gateway", + "runtimeExecutable": ".venv\\Scripts\\python.exe", + "runtimeArgs": ["-m", "uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8090"], + "port": 8090 + } + ] +} diff --git a/.dockerignore b/.dockerignore new file mode 100644 index 0000000..e74696f --- /dev/null +++ b/.dockerignore @@ -0,0 +1,11 @@ +.venv/ +__pycache__/ +*.pyc +gateway.db +cookies.txt +.claude/ +.git/ +.gitignore +docker-compose.yml +Dockerfile +README.md diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e16edbb --- /dev/null +++ b/.gitignore @@ -0,0 +1,5 @@ +.venv/ +__pycache__/ +*.pyc +gateway.db +cookies.txt diff --git a/.project b/.project new file mode 100644 index 0000000..5ca6a9a --- /dev/null +++ b/.project @@ -0,0 +1,11 @@ + + + API gatweay + + + + + + + + diff --git a/Dockerfile b/Dockerfile new file mode 100644 index 0000000..ad6473b --- /dev/null +++ b/Dockerfile @@ -0,0 +1,25 @@ +FROM python:3.14-slim + +ENV PYTHONDONTWRITEBYTECODE=1 \ + PYTHONUNBUFFERED=1 + +WORKDIR /app + +COPY requirements.txt . +RUN pip install --no-cache-dir -r requirements.txt + +COPY app ./app + +# Non-root user; the SQLite database lives on the /data volume +RUN useradd --create-home gateway && mkdir /data && chown gateway:gateway /data +USER gateway + +ENV GATEWAY_DATABASE_URL=sqlite:////data/gateway.db + +EXPOSE 8000 + +HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \ + CMD python -c "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=2)" + +# Single worker on purpose: the rate limiter and sync cache are in-memory. +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"] diff --git a/README.md b/README.md new file mode 100644 index 0000000..4c2e3b4 --- /dev/null +++ b/README.md @@ -0,0 +1,99 @@ +# API Gateway + +A FastAPI-based gateway that exposes a single entrypoint for multiple upstream APIs, +with a built-in web management console. + +## Features + +- **Single entrypoint** — consumers call `//` and the gateway + proxies to the registered upstream (all HTTP methods, query strings, bodies, + headers). `/admin`, `/static`, `/docs` and `/health` are reserved. +- **Endpoint catalog, auto-synced** — each service's callable endpoints are + mirrored from its OpenAPI document (`/openapi.json`, `/swagger.json` or + `/api-docs`) whenever the API Keys page loads: new operations appear, removed + ones are deleted (their grants with them). Services without a reachable spec + keep their existing catalog. Patterns support `{param}` (one segment), `*` + (within a segment) and `**` (any depth). +- **Per-key access rights, hierarchical picker** — every API key is granted a + specific set of endpoints, chosen in a path tree (service → path segments → + operations) where a parent checkbox selects everything beneath it. A request + must match a listed endpoint AND the key must hold a grant on it; otherwise + 404 (`unknown_endpoint`) or 403 (`access_denied`). Keys are hashed at rest, + shown once at creation, revocable, with per-key rate limits. +- **Web management console** (`/admin`) — manage connected APIs, users, and API + keys; usage dashboards with charts. Every console user has full management + access. +- **Transaction capture & request browser** — every gateway request is recorded + with service, endpoint, key, status, latency, client IP, query string, and + the request/response payloads (textual bodies up to 64 KB; binary summarized). + The Requests page filters by service / user / key / status class / path and + links to a per-request detail view showing the complete transaction. +- **Usage monitoring** — charts: traffic and average latency over time (adaptive + buckets: 5 min / 1 h / 1 day), volume by service / user / endpoint, status + codes, latency by service, top keys — filterable by time range (1 h – 30 d), + service, user and key. + +- **Retention** — a background purge runs every 6 hours: request/response + payloads are blanked after 7 days (the log row stays inspectable), whole log + rows are deleted after 90 days, and freed pages are returned to the + filesystem via incremental vacuum. Both windows are configurable; `0` + disables that tier. + +## Quick start + +```bash +pip install -r requirements.txt +uvicorn app.main:app --host 0.0.0.0 --port 8000 +``` + +Open http://localhost:8000/admin — first startup seeds an admin account +(`admin` / `admin` by default — **change it immediately**). + +## Docker + +```bash +docker compose up -d +``` + +The console is served on http://localhost:8000/admin; the SQLite database +persists in the `gateway-data` volume. Set `GATEWAY_SECRET_KEY` (and the admin +credentials) in `docker-compose.yml` before deploying anywhere real. The image +runs a single uvicorn worker on purpose — the rate limiter and the OpenAPI +sync cache are in-memory. + +## Configuration (environment variables) + +| Variable | Default | Purpose | +|---|---|---| +| `GATEWAY_DATABASE_URL` | `sqlite:///./gateway.db` | SQLAlchemy database URL | +| `GATEWAY_SECRET_KEY` | dev value | Session cookie signing key — set in production | +| `GATEWAY_ADMIN_USER` / `GATEWAY_ADMIN_PASSWORD` | `admin` / `admin` | Seeded admin credentials | +| `GATEWAY_PAYLOAD_RETENTION_DAYS` | `7` | Blank stored payloads after N days (0 = keep forever) | +| `GATEWAY_LOG_RETENTION_DAYS` | `90` | Delete log rows after N days (0 = keep forever) | + +## Using the gateway as a consumer + +1. Register a service (e.g. slug `weather` → `https://api.example.com/v1`) and + list its endpoints (import from OpenAPI or add manually). +2. Issue an API key and tick the endpoints it may call. +3. The consumer calls: + +```bash +curl -H "X-API-Key: gw_..." "http://localhost:8000/weather/forecast?city=Paris" +``` + +The gateway authenticates the key, matches the request against the service's +endpoint list, checks the key's grant and the rate limit, proxies the request to +`https://api.example.com/v1/forecast?city=Paris`, logs it, and returns the +upstream response. Gateway-generated errors are JSON with an `error` code: +`missing_api_key` (401), `invalid_api_key` / `access_denied` (403), +`unknown_service` / `unknown_endpoint` (404), `rate_limited` (429), +`upstream_unreachable` (502), `upstream_timeout` (504). + +## Notes & limits + +- The stats queries use SQLite date functions; if you point `GATEWAY_DATABASE_URL` + at Postgres/MySQL, adapt `app/admin/stats.py` (`strftime`, `iif`). +- The rate limiter is in-memory (single process). Run one worker, or swap in Redis + for multi-worker deployments. +- Streaming responses are buffered (fine for JSON APIs; not for large file proxying). diff --git a/app/__init__.py b/app/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/admin/__init__.py b/app/admin/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/app/admin/deps.py b/app/admin/deps.py new file mode 100644 index 0000000..995d4ea --- /dev/null +++ b/app/admin/deps.py @@ -0,0 +1,29 @@ +from fastapi import Depends, HTTPException, Request, status +from fastapi.responses import RedirectResponse +from sqlalchemy.orm import Session + +from app import config, security +from app.database import get_db +from app.models import User + + +class LoginRequired(HTTPException): + """Raised when there is no valid session; handled by redirecting to /admin/login.""" + + def __init__(self): + super().__init__(status_code=status.HTTP_303_SEE_OTHER) + + +def login_redirect_handler(request: Request, exc: LoginRequired): + return RedirectResponse("/admin/login", status_code=303) + + +def current_user(request: Request, db: Session = Depends(get_db)) -> User: + token = request.cookies.get(config.SESSION_COOKIE) + user_id = security.read_session_token(token) if token else None + if user_id is None: + raise LoginRequired() + user = db.get(User, user_id) + if user is None or not user.is_active: + raise LoginRequired() + return user diff --git a/app/admin/routes.py b/app/admin/routes.py new file mode 100644 index 0000000..e626abd --- /dev/null +++ b/app/admin/routes.py @@ -0,0 +1,423 @@ +from fastapi import APIRouter, Depends, Form, HTTPException, Request +from fastapi.responses import HTMLResponse, RedirectResponse +from fastapi.templating import Jinja2Templates +from sqlalchemy import func +from sqlalchemy.orm import Session + +from app import config, discovery, security +from app.admin.deps import current_user +from app.database import get_db +from app.models import ApiKey, Endpoint, RequestLog, Service, User + +router = APIRouter(prefix="/admin") +templates = Jinja2Templates(directory=str(config.BASE_DIR / "app" / "templates")) + +# Slugs the proxy catch-all must never claim +RESERVED_SLUGS = {"admin", "static", "health", "docs", "redoc", "openapi.json"} + + +def build_tree(endpoints: list[Endpoint]) -> dict: + """Nest endpoints by path segment for the hierarchical access picker. + Node: {name, children: {segment: node}, endpoints: [Endpoint]}. + Chains of empty single-child nodes are compressed ('api' + 'v1' -> 'api/v1').""" + root = {"name": "", "children": {}, "endpoints": []} + for e in endpoints: + node = root + for part in (p for p in e.path.split("/") if p): + node = node["children"].setdefault( + part, {"name": part, "children": {}, "endpoints": []}) + node["endpoints"].append(e) + + def compress(node: dict) -> None: + for key in list(node["children"]): + child = node["children"][key] + while not child["endpoints"] and len(child["children"]) == 1: + (grandchild,) = child["children"].values() + child["name"] = child["name"] + "/" + grandchild["name"] + child["endpoints"] = grandchild["endpoints"] + child["children"] = grandchild["children"] + compress(child) + if child["name"] != key: + node["children"][child["name"]] = node["children"].pop(key) + + compress(root) + return root + + +def render(request: Request, name: str, user: User | None = None, **ctx): + return templates.TemplateResponse( + request, name, {"user": user, "active": name.split(".")[0], **ctx} + ) + + +def _redirect(url: str) -> RedirectResponse: + return RedirectResponse(url, status_code=303) + + +# ---------- auth ---------- + +@router.get("/login", response_class=HTMLResponse) +def login_page(request: Request): + return render(request, "login.html") + + +@router.post("/login") +def login(request: Request, username: str = Form(...), password: str = Form(...), + db: Session = Depends(get_db)): + user = db.query(User).filter(User.username == username).one_or_none() + if not user or not user.is_active or not security.verify_password(password, user.password_hash): + return render(request, "login.html", error="Invalid username or password.") + response = _redirect("/admin") + response.set_cookie( + config.SESSION_COOKIE, + security.create_session_token(user.id), + max_age=config.SESSION_MAX_AGE, + httponly=True, + samesite="lax", + ) + return response + + +@router.get("/logout") +def logout(): + response = _redirect("/admin/login") + response.delete_cookie(config.SESSION_COOKIE) + return response + + +# ---------- dashboard ---------- + +@router.get("", response_class=HTMLResponse) +@router.get("/", response_class=HTMLResponse) +def dashboard(request: Request, user: User = Depends(current_user), + db: Session = Depends(get_db)): + recent = ( + db.query(RequestLog).order_by(RequestLog.timestamp.desc()).limit(15).all() + ) + return render(request, "dashboard.html", user, recent=recent) + + +# ---------- services & endpoints ---------- + +@router.get("/services", response_class=HTMLResponse) +def services_page(request: Request, user: User = Depends(current_user), + db: Session = Depends(get_db)): + services = db.query(Service).order_by(Service.name).all() + counts = dict( + db.query(RequestLog.service_id, func.count(RequestLog.id)) + .group_by(RequestLog.service_id).all() + ) + return render(request, "services.html", user, services=services, counts=counts) + + +@router.post("/services") +def create_service(name: str = Form(...), slug: str = Form(...), base_url: str = Form(...), + description: str = Form(""), timeout_seconds: float = Form(30.0), + verify_tls: bool = Form(False), + user: User = Depends(current_user), db: Session = Depends(get_db)): + slug = slug.strip().lower() + if slug in RESERVED_SLUGS: + raise HTTPException(400, f"Slug '{slug}' is reserved by the gateway itself.") + if db.query(Service).filter(Service.slug == slug).count(): + raise HTTPException(400, f"Slug '{slug}' is already taken.") + service = Service(name=name.strip(), slug=slug, base_url=base_url.strip().rstrip("/"), + description=description.strip(), timeout_seconds=timeout_seconds, + verify_tls=verify_tls) + db.add(service) + db.commit() + # The page auto-validates the new service (which also caches its endpoints). + return _redirect(f"/admin/services?validate={service.id}") + + +@router.post("/services/{service_id}/update") +def update_service(service_id: int, name: str = Form(...), base_url: str = Form(...), + description: str = Form(""), timeout_seconds: float = Form(30.0), + verify_tls: bool = Form(False), + user: User = Depends(current_user), db: Session = Depends(get_db)): + service = db.get(Service, service_id) + if not service: + raise HTTPException(404) + service.name, service.base_url = name.strip(), base_url.strip().rstrip("/") + service.description, service.timeout_seconds = description.strip(), timeout_seconds + service.verify_tls = verify_tls + db.commit() + return _redirect(f"/admin/services?validate={service.id}") + + +@router.post("/services/{service_id}/validate") +def validate_service(service_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + """Probe the upstream; if it publishes an OpenAPI document this also + refreshes the endpoint cache.""" + service = db.get(Service, service_id) + if not service: + raise HTTPException(404) + return discovery.validate_service(db, service) + + +@router.post("/services/{service_id}/toggle") +def toggle_service(service_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + service = db.get(Service, service_id) + if not service: + raise HTTPException(404) + service.is_active = not service.is_active + db.commit() + return _redirect("/admin/services") + + +@router.post("/services/{service_id}/delete") +def delete_service(service_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + service = db.get(Service, service_id) + if service: + db.delete(service) + db.commit() + return _redirect("/admin/services") + + +# ---------- users ---------- + +@router.get("/users", response_class=HTMLResponse) +def users_page(request: Request, user: User = Depends(current_user), + db: Session = Depends(get_db)): + users = db.query(User).order_by(User.username).all() + return render(request, "users.html", user, users=users) + + +@router.post("/users") +def create_user(username: str = Form(...), password: str = Form(...), + user: User = Depends(current_user), db: Session = Depends(get_db)): + username = username.strip() + if db.query(User).filter(User.username == username).count(): + raise HTTPException(400, f"Username '{username}' is already taken.") + db.add(User(username=username, password_hash=security.hash_password(password))) + db.commit() + return _redirect("/admin/users") + + +@router.post("/users/{user_id}/toggle") +def toggle_user(user_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + target = db.get(User, user_id) + if not target: + raise HTTPException(404) + if target.id == user.id: + raise HTTPException(400, "You cannot deactivate your own account.") + target.is_active = not target.is_active + db.commit() + return _redirect("/admin/users") + + +@router.post("/users/{user_id}/password") +def reset_password(user_id: int, password: str = Form(...), + user: User = Depends(current_user), db: Session = Depends(get_db)): + target = db.get(User, user_id) + if not target: + raise HTTPException(404) + target.password_hash = security.hash_password(password) + db.commit() + return _redirect("/admin/users") + + +@router.post("/users/{user_id}/delete") +def delete_user(user_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + if user_id == user.id: + raise HTTPException(400, "You cannot delete your own account.") + target = db.get(User, user_id) + if target: + db.delete(target) + db.commit() + return _redirect("/admin/users") + + +# ---------- API keys / endpoint access ---------- + +@router.get("/keys", response_class=HTMLResponse) +def keys_page(request: Request, user: User = Depends(current_user), + db: Session = Depends(get_db)): + services = db.query(Service).order_by(Service.name).all() + # Rendered straight from the database — the OpenAPI sync runs in the + # background via /admin/api/endpoints/sync, triggered by the page's JS. + trees = {s.id: build_tree(s.endpoints) for s in services} + synced = {s.id: discovery.spec_status(s.id) for s in services} + keys = db.query(ApiKey).order_by(ApiKey.created_at.desc()).all() + users = db.query(User).filter(User.is_active).order_by(User.username).all() + new_key = request.query_params.get("new_key") + return render(request, "keys.html", user, keys=keys, users=users, + services=services, trees=trees, synced=synced, new_key=new_key) + + +@router.get("/api/endpoints/sync") +def sync_endpoints(force: bool = False, user: User = Depends(current_user), + db: Session = Depends(get_db)): + """Refresh all endpoint catalogs from their OpenAPI documents (TTL-cached). + The keys page calls this in the background and reloads if anything changed.""" + changed_any = False + specs = {} + for service in db.query(Service).order_by(Service.name).all(): + found, changed = discovery.sync_service(db, service, force=force) + specs[service.id] = found + changed_any = changed_any or changed + return {"changed": changed_any, "specs": specs} + + +@router.post("/keys") +def create_key(request: Request, name: str = Form(...), user_id: int = Form(...), + rate_limit_per_minute: int = Form(60), + endpoint_ids: list[int] = Form([]), + user: User = Depends(current_user), db: Session = Depends(get_db)): + owner = db.get(User, user_id) + if not owner: + raise HTTPException(400, "Unknown user.") + plain, prefix, key_hash = security.generate_api_key() + key = ApiKey(user_id=owner.id, name=name.strip(), prefix=prefix, key_hash=key_hash, + rate_limit_per_minute=max(0, rate_limit_per_minute)) + key.endpoints = db.query(Endpoint).filter( + Endpoint.id.in_(endpoint_ids)).all() if endpoint_ids else [] + db.add(key) + db.commit() + # Shown once on the next page load; never stored in plain text. + return _redirect(f"/admin/keys?new_key={plain}") + + +@router.post("/keys/{key_id}/access") +def update_key_access(key_id: int, endpoint_ids: list[int] = Form([]), + rate_limit_per_minute: int = Form(60), + user: User = Depends(current_user), db: Session = Depends(get_db)): + key = db.get(ApiKey, key_id) + if not key: + raise HTTPException(404) + key.endpoints = db.query(Endpoint).filter( + Endpoint.id.in_(endpoint_ids)).all() if endpoint_ids else [] + key.rate_limit_per_minute = max(0, rate_limit_per_minute) + db.commit() + return _redirect("/admin/keys") + + +@router.post("/keys/{key_id}/toggle") +def toggle_key(key_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + key = db.get(ApiKey, key_id) + if not key: + raise HTTPException(404) + key.is_active = not key.is_active + db.commit() + return _redirect("/admin/keys") + + +@router.post("/keys/{key_id}/delete") +def delete_key(key_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + key = db.get(ApiKey, key_id) + if key: + db.delete(key) + db.commit() + return _redirect("/admin/keys") + + +# ---------- request browser ---------- + +PAGE_SIZE = 50 + + +def _int_or_none(value: str | None) -> int | None: + """HTML GET forms submit empty strings for untouched fields — treat + anything non-numeric as 'no filter' instead of a validation error.""" + try: + return int(value) if value else None + except ValueError: + return None + + +@router.get("/requests", response_class=HTMLResponse) +def requests_page(request: Request, user: User = Depends(current_user), + db: Session = Depends(get_db), + service_id: str | None = None, user_id: str | None = None, + key_id: str | None = None, status_class: str | None = None, + q: str | None = None, page: str | None = None): + service_id = _int_or_none(service_id) + user_id = _int_or_none(user_id) + key_id = _int_or_none(key_id) + page = _int_or_none(page) or 1 + query = db.query(RequestLog) + if service_id: + query = query.filter(RequestLog.service_id == service_id) + if key_id: + query = query.filter(RequestLog.api_key_id == key_id) + if user_id: + query = query.join(ApiKey, RequestLog.api_key_id == ApiKey.id).filter( + ApiKey.user_id == user_id) + if status_class in ("2", "3", "4", "5"): + low = int(status_class) * 100 + query = query.filter(RequestLog.status_code >= low, + RequestLog.status_code < low + 100) + if q: + query = query.filter(RequestLog.path.contains(q)) + + total = query.count() + page = max(1, page) + logs = (query.order_by(RequestLog.timestamp.desc()) + .offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE).all()) + + services = db.query(Service).order_by(Service.name).all() + users = db.query(User).order_by(User.username).all() + keys = db.query(ApiKey).order_by(ApiKey.name).all() + return render(request, "requests.html", user, logs=logs, total=total, + page=page, pages=max(1, -(-total // PAGE_SIZE)), + services=services, users=users, keys=keys, + f={"service_id": service_id, "user_id": user_id, "key_id": key_id, + "status_class": status_class or "", "q": q or ""}) + + +def _pretty_json(text: str) -> str: + import json + try: + return json.dumps(json.loads(text), indent=2, ensure_ascii=False) + except (ValueError, TypeError): + return text + + +@router.get("/requests/{log_id}/data") +def request_data(log_id: int, user: User = Depends(current_user), + db: Session = Depends(get_db)): + """Everything the inline request inspector needs, as JSON.""" + log = db.get(RequestLog, log_id) + if not log: + raise HTTPException(404) + return { + "id": log.id, + "time": log.timestamp.strftime("%Y-%m-%d %H:%M:%S"), + "status": log.status_code, + "latency_ms": round(log.latency_ms, 1), + "method": log.method, + "path": log.path, + "query_string": log.query_string, + "service": log.service.name if log.service else None, + "slug": log.service.slug if log.service else None, + "forwarded_to": (log.service.base_url + log.path + + ("?" + log.query_string if log.query_string else "")) + if log.service else None, + "endpoint": f"{log.endpoint.method} {log.endpoint.path}" if log.endpoint else None, + "endpoint_description": log.endpoint.description if log.endpoint else "", + "key": log.api_key.name if log.api_key else None, + "key_prefix": log.api_key.prefix if log.api_key else "", + "user": log.api_key.user.username if log.api_key else None, + "client_ip": log.client_ip, + "request_body": _pretty_json(log.request_body), + "response_body": _pretty_json(log.response_body), + } + + +# ---------- monitoring ---------- + +@router.get("/monitoring", response_class=HTMLResponse) +def monitoring_page(request: Request, user: User = Depends(current_user), + db: Session = Depends(get_db)): + logs = db.query(RequestLog).order_by(RequestLog.timestamp.desc()).limit(100).all() + services = db.query(Service).order_by(Service.name).all() + users = db.query(User).order_by(User.username).all() + keys = db.query(ApiKey).order_by(ApiKey.name).all() + return render(request, "monitoring.html", user, logs=logs, + services=services, users=users, keys=keys) diff --git a/app/admin/stats.py b/app/admin/stats.py new file mode 100644 index 0000000..661bd87 --- /dev/null +++ b/app/admin/stats.py @@ -0,0 +1,237 @@ +"""JSON endpoints backing the monitoring charts. + +All endpoints accept: + hours — window size (1..720) + service_id — restrict to one service + user_id — restrict to keys owned by one user + key_id — restrict to one API key + +Time series use adaptive buckets: 5 min (<=6 h), 1 h (<=48 h), 1 day beyond. +SQLite-specific date functions are used (see README). +""" +import time +from datetime import datetime, timedelta, timezone + +from fastapi import APIRouter, Depends, Query +from sqlalchemy import Integer, func +from sqlalchemy.orm import Query as OrmQuery, Session + +from app.admin.deps import current_user +from app.database import get_db +from app.models import ApiKey, Endpoint, RequestLog, Service, User + +router = APIRouter(prefix="/admin/api", dependencies=[Depends(current_user)]) + + +class Filters: + def __init__( + self, + hours: int = Query(24, ge=1, le=720), + service_id: int | None = Query(None), + user_id: int | None = Query(None), + key_id: int | None = Query(None), + ): + self.hours = hours + self.since = datetime.now(timezone.utc) - timedelta(hours=hours) + self.service_id = service_id + self.user_id = user_id + self.key_id = key_id + + def apply(self, q: OrmQuery, joined_key: bool = False) -> OrmQuery: + """Apply window + filters. Pass joined_key when the query already + joins ApiKey so we don't join twice.""" + q = q.filter(RequestLog.timestamp >= self.since) + if self.service_id: + q = q.filter(RequestLog.service_id == self.service_id) + if self.key_id: + q = q.filter(RequestLog.api_key_id == self.key_id) + if self.user_id: + if not joined_key: + q = q.join(ApiKey, RequestLog.api_key_id == ApiKey.id) + q = q.filter(ApiKey.user_id == self.user_id) + return q + + +def _bucket_seconds(hours: int) -> int: + if hours <= 6: + return 300 + if hours <= 48: + return 3600 + return 86400 + + +def _label_format(hours: int, bucket: int) -> str: + if bucket < 3600: + return "%H:%M" + if bucket == 3600: + return "%H:00" if hours <= 24 else "%m-%d %H:00" + return "%Y-%m-%d" + + +def _bucket_expr(bucket: int): + # epoch - (epoch % bucket) floors to the bucket start; plain `/` must be + # avoided because SQLAlchemy renders it as true (float) division. + epoch = func.cast(func.strftime("%s", RequestLog.timestamp), Integer) + return epoch - (epoch % bucket) + + +def _series(f: Filters, db: Session, *aggregates): + """Group the filtered logs into time buckets and fill gaps with None. + Returns (labels, [values-per-aggregate]).""" + bucket = _bucket_seconds(f.hours) + fmt = _label_format(f.hours, bucket) + expr = _bucket_expr(bucket).label("bucket") + rows = ( + f.apply(db.query(expr, *aggregates)) + .group_by("bucket").order_by("bucket").all() + ) + found = {int(r[0]): r for r in rows} + start = int(f.since.timestamp()) // bucket * bucket + now = int(time.time()) + labels: list[str] = [] + series: list[list] = [[] for _ in aggregates] + for t in range(start, now + 1, bucket): + labels.append(datetime.fromtimestamp(t, timezone.utc).strftime(fmt)) + row = found.get(t) + for i in range(len(aggregates)): + series[i].append(row[i + 1] if row else None) + return labels, series + + +@router.get("/stats/summary") +def summary(f: Filters = Depends(), db: Session = Depends(get_db)): + total = f.apply(db.query(func.count(RequestLog.id))).scalar() or 0 + errors = f.apply(db.query(func.count(RequestLog.id))).filter( + RequestLog.status_code >= 500).scalar() or 0 + avg_latency = f.apply(db.query(func.avg(RequestLog.latency_ms))).scalar() or 0 + return { + "total_requests": total, + "error_count": errors, + "error_rate": round(errors / total * 100, 2) if total else 0, + "avg_latency_ms": round(avg_latency, 1), + "active_services": db.query(Service).filter(Service.is_active).count(), + "active_keys": db.query(ApiKey).filter(ApiKey.is_active).count(), + } + + +@router.get("/stats/timeseries") +def timeseries(f: Filters = Depends(), db: Session = Depends(get_db)): + labels, (ok, errors) = _series( + f, db, + func.sum(func.iif(RequestLog.status_code < 500, 1, 0)), + func.sum(func.iif(RequestLog.status_code >= 500, 1, 0)), + ) + return { + "labels": labels, + "ok": [int(v) if v is not None else 0 for v in ok], + "errors": [int(v) if v is not None else 0 for v in errors], + } + + +@router.get("/stats/latency-timeseries") +def latency_timeseries(f: Filters = Depends(), db: Session = Depends(get_db)): + labels, (avg_ms,) = _series(f, db, func.avg(RequestLog.latency_ms)) + return { + "labels": labels, + "avg_ms": [round(v, 1) if v is not None else None for v in avg_ms], + } + + +@router.get("/stats/by-service") +def by_service(f: Filters = Depends(), db: Session = Depends(get_db)): + q = ( + db.query(Service.name, func.count(RequestLog.id).label("count")) + .join(RequestLog, RequestLog.service_id == Service.id) + ) + rows = ( + f.apply(q) + .group_by(Service.id) + .order_by(func.count(RequestLog.id).desc()) + .all() + ) + return {"labels": [r.name for r in rows], "counts": [r.count for r in rows]} + + +@router.get("/stats/by-endpoint") +def by_endpoint(f: Filters = Depends(), db: Session = Depends(get_db)): + q = ( + db.query(Service.slug, Endpoint.method, Endpoint.path, + func.count(RequestLog.id).label("count")) + .select_from(RequestLog) + .join(Endpoint, RequestLog.endpoint_id == Endpoint.id) + .join(Service, Endpoint.service_id == Service.id) + ) + rows = ( + f.apply(q) + .group_by(Endpoint.id) + .order_by(func.count(RequestLog.id).desc()) + .limit(15) + .all() + ) + return { + "labels": [f"{r.slug}: {r.method} {r.path}" for r in rows], + "counts": [r.count for r in rows], + } + + +@router.get("/stats/by-status") +def by_status(f: Filters = Depends(), db: Session = Depends(get_db)): + q = ( + db.query(RequestLog.status_code, func.count(RequestLog.id)) + ) + rows = ( + f.apply(q) + .group_by(RequestLog.status_code) + .order_by(RequestLog.status_code) + .all() + ) + return {"labels": [str(r[0]) for r in rows], "counts": [r[1] for r in rows]} + + +@router.get("/stats/latency-by-service") +def latency_by_service(f: Filters = Depends(), db: Session = Depends(get_db)): + q = ( + db.query(Service.name, func.avg(RequestLog.latency_ms).label("avg_ms")) + .join(RequestLog, RequestLog.service_id == Service.id) + ) + rows = ( + f.apply(q) + .group_by(Service.id) + .order_by(func.avg(RequestLog.latency_ms).desc()) + .all() + ) + return {"labels": [r.name for r in rows], "avg_ms": [round(r.avg_ms, 1) for r in rows]} + + +@router.get("/stats/top-keys") +def top_keys(f: Filters = Depends(), db: Session = Depends(get_db)): + q = ( + db.query(ApiKey.name, func.count(RequestLog.id).label("count")) + .join(RequestLog, RequestLog.api_key_id == ApiKey.id) + ) + rows = ( + f.apply(q, joined_key=True) + .group_by(ApiKey.id) + .order_by(func.count(RequestLog.id).desc()) + .limit(10) + .all() + ) + return {"labels": [r.name for r in rows], "counts": [r.count for r in rows]} + + +@router.get("/stats/by-user") +def by_user(f: Filters = Depends(), db: Session = Depends(get_db)): + q = ( + db.query(User.username, func.count(RequestLog.id).label("count")) + .select_from(RequestLog) + .join(ApiKey, RequestLog.api_key_id == ApiKey.id) + .join(User, ApiKey.user_id == User.id) + ) + rows = ( + f.apply(q, joined_key=True) + .group_by(User.id) + .order_by(func.count(RequestLog.id).desc()) + .limit(15) + .all() + ) + return {"labels": [r.username for r in rows], "counts": [r.count for r in rows]} diff --git a/app/config.py b/app/config.py new file mode 100644 index 0000000..4d638e8 --- /dev/null +++ b/app/config.py @@ -0,0 +1,21 @@ +import os +from pathlib import Path + +BASE_DIR = Path(__file__).resolve().parent.parent + +DATABASE_URL = os.environ.get("GATEWAY_DATABASE_URL", f"sqlite:///{BASE_DIR / 'gateway.db'}") +SECRET_KEY = os.environ.get("GATEWAY_SECRET_KEY", "dev-secret-change-me") +SESSION_COOKIE = "gw_session" +SESSION_MAX_AGE = 60 * 60 * 8 # 8 hours + +# Default admin credentials seeded on first startup (change after first login) +DEFAULT_ADMIN_USERNAME = os.environ.get("GATEWAY_ADMIN_USER", "admin") +DEFAULT_ADMIN_PASSWORD = os.environ.get("GATEWAY_ADMIN_PASSWORD", "admin") + +PROXY_DEFAULT_TIMEOUT = 30.0 +API_KEY_HEADER = "X-API-Key" + +# Retention (0 = keep forever). Payloads are blanked after the first window; +# whole log rows are deleted after the second. +PAYLOAD_RETENTION_DAYS = int(os.environ.get("GATEWAY_PAYLOAD_RETENTION_DAYS", "7")) +LOG_RETENTION_DAYS = int(os.environ.get("GATEWAY_LOG_RETENTION_DAYS", "90")) diff --git a/app/database.py b/app/database.py new file mode 100644 index 0000000..9004ca5 --- /dev/null +++ b/app/database.py @@ -0,0 +1,22 @@ +from sqlalchemy import create_engine +from sqlalchemy.orm import DeclarativeBase, sessionmaker + +from app import config + +engine = create_engine( + config.DATABASE_URL, + connect_args={"check_same_thread": False} if config.DATABASE_URL.startswith("sqlite") else {}, +) +SessionLocal = sessionmaker(bind=engine, autoflush=False, expire_on_commit=False) + + +class Base(DeclarativeBase): + pass + + +def get_db(): + db = SessionLocal() + try: + yield db + finally: + db.close() diff --git a/app/discovery.py b/app/discovery.py new file mode 100644 index 0000000..891b570 --- /dev/null +++ b/app/discovery.py @@ -0,0 +1,113 @@ +"""Keep service endpoint catalogs in sync with the upstream's OpenAPI document.""" +import time + +import httpx +from sqlalchemy.orm import Session + +from app.models import Endpoint, Service + +METHODS = {"GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS"} + +SPEC_PATHS = ("/openapi.json", "/swagger.json", "/api-docs") + +# Syncing hits the upstream over HTTP, so results are cached per service and +# only refreshed after the TTL (or on demand with force=True). +SYNC_TTL_SECONDS = 300 +_last_sync: dict[int, float] = {} # service id -> monotonic time of last attempt +_spec_found: dict[int, bool] = {} # service id -> did the last attempt find a spec + + +def fetch_spec(base_url: str, verify_tls: bool = True) -> dict | None: + for candidate in SPEC_PATHS: + try: + resp = httpx.get(base_url.rstrip("/") + candidate, timeout=2, + follow_redirects=True, verify=verify_tls) + if resp.status_code == 200: + data = resp.json() + if isinstance(data, dict) and "paths" in data: + return data + except (httpx.HTTPError, ValueError): + continue + return None + + +def spec_status(service_id: int) -> bool | None: + """Did the last sync attempt find an OpenAPI document? None = never tried.""" + return _spec_found.get(service_id) + + +def sync_service(db: Session, service: Service, force: bool = False) -> tuple[bool, bool]: + """Mirror the upstream OpenAPI paths into the endpoint catalog: add new + operations, refresh descriptions, delete operations that disappeared + (their key grants go with them). Services without a reachable spec are + left untouched. Returns (spec_found, catalog_changed); results are cached + between syncs.""" + now = time.monotonic() + if not force and now - _last_sync.get(service.id, float("-inf")) < SYNC_TTL_SECONDS: + return _spec_found.get(service.id, False), False + _last_sync[service.id] = now + + spec = fetch_spec(service.base_url, service.verify_tls) + _spec_found[service.id] = spec is not None + if spec is None: + return False, False + return True, apply_spec(db, service, spec) + + +def validate_service(db: Session, service: Service) -> dict: + """Connectivity probe for the admin UI. Prefers the OpenAPI document — + finding one both proves reachability and refreshes the endpoint cache — + and falls back to a plain GET on the base URL otherwise.""" + start = time.perf_counter() + spec = fetch_spec(service.base_url, service.verify_tls) + if spec is not None: + _last_sync[service.id] = time.monotonic() + _spec_found[service.id] = True + apply_spec(db, service, spec) + return {"ok": True, "spec_found": True, + "endpoints": len(service.endpoints), + "latency_ms": round((time.perf_counter() - start) * 1000, 1)} + try: + resp = httpx.get(service.base_url, timeout=5, follow_redirects=True, + verify=service.verify_tls) + return {"ok": True, "spec_found": False, "status_code": resp.status_code, + "endpoints": len(service.endpoints), + "latency_ms": round((time.perf_counter() - start) * 1000, 1)} + except httpx.HTTPError as exc: + return {"ok": False, "spec_found": False, + "error": str(exc) or type(exc).__name__, + "latency_ms": round((time.perf_counter() - start) * 1000, 1)} + + +def apply_spec(db: Session, service: Service, spec: dict) -> bool: + """Write an OpenAPI document's paths into the endpoint catalog. + Returns whether anything changed.""" + wanted: dict[tuple[str, str], str] = {} + for path, operations in spec["paths"].items(): + if not isinstance(operations, dict): + continue + for method, op in operations.items(): + method = method.upper() + if method not in METHODS: + continue + summary = op.get("summary", "") if isinstance(op, dict) else "" + wanted[(method, path)] = summary + + existing = {(e.method, e.path): e for e in service.endpoints} + changed = False + for (method, path), summary in wanted.items(): + endpoint = existing.get((method, path)) + if endpoint is None: + db.add(Endpoint(service_id=service.id, method=method, path=path, + description=summary)) + changed = True + elif endpoint.description != summary: + endpoint.description = summary + changed = True + for key, endpoint in existing.items(): + if key not in wanted: + db.delete(endpoint) + changed = True + db.commit() + db.expire(service, ["endpoints"]) + return changed diff --git a/app/main.py b/app/main.py new file mode 100644 index 0000000..c133d69 --- /dev/null +++ b/app/main.py @@ -0,0 +1,133 @@ +import asyncio +from contextlib import asynccontextmanager, suppress + +from fastapi import FastAPI +from fastapi.staticfiles import StaticFiles +from fastapi.responses import RedirectResponse +from sqlalchemy import text + +from app import config, proxy, retention, security +from app.admin import routes as admin_routes +from app.admin import stats as admin_stats +from app.admin.deps import LoginRequired, login_redirect_handler +from app.database import Base, SessionLocal, engine +from app.models import User + + +def enable_incremental_vacuum() -> None: + """Deleted log rows should hand their pages back to the filesystem; + switching auto_vacuum requires a one-time VACUUM (cannot run in a + transaction, hence the raw connection).""" + raw = engine.raw_connection() + try: + cursor = raw.cursor() + mode = cursor.execute("PRAGMA auto_vacuum").fetchone()[0] + if mode != 2: # 2 = INCREMENTAL + cursor.execute("PRAGMA auto_vacuum=INCREMENTAL") + raw.commit() + cursor.execute("VACUUM") + raw.commit() + finally: + raw.close() + + +def _table_exists(db, name: str) -> bool: + return db.execute(text( + "SELECT 1 FROM sqlite_master WHERE type='table' AND name=:n" + ), {"n": name}).first() is not None + + +def migrate_schema() -> None: + """Bring older databases up to date (SQLite).""" + with engine.begin() as conn: + cols = [row[1] for row in conn.exec_driver_sql("PRAGMA table_info(request_logs)")] + if cols and "endpoint_id" not in cols: + conn.exec_driver_sql( + "ALTER TABLE request_logs ADD COLUMN endpoint_id INTEGER " + "REFERENCES endpoints(id) ON DELETE SET NULL" + ) + for column, ddl in ( + ("query_string", "VARCHAR(2048) DEFAULT ''"), + ("request_body", "TEXT DEFAULT ''"), + ("response_body", "TEXT DEFAULT ''"), + ): + if cols and column not in cols: + conn.exec_driver_sql(f"ALTER TABLE request_logs ADD COLUMN {column} {ddl}") + service_cols = [row[1] for row in conn.exec_driver_sql("PRAGMA table_info(services)")] + if service_cols and "verify_tls" not in service_cols: + conn.exec_driver_sql("ALTER TABLE services ADD COLUMN verify_tls BOOLEAN DEFAULT 1") + + +def seed_and_migrate() -> None: + db = SessionLocal() + try: + # Role-era databases: replace role->service grants with per-key grants + # on a service-wide catch-all endpoint, then drop the role tables. + if _table_exists(db, "roles"): + db.execute(text( + "INSERT INTO endpoints (service_id, method, path, description) " + "SELECT id, '*', '/**', 'Migrated: full service access' FROM services" + )) + if _table_exists(db, "role_service_access"): + db.execute(text( + "INSERT OR IGNORE INTO key_endpoint_access (api_key_id, endpoint_id) " + "SELECT k.id, e.id " + "FROM api_keys k " + "JOIN users u ON u.id = k.user_id " + "JOIN role_service_access rsa ON rsa.role_id = u.role_id " + "JOIN endpoints e ON e.service_id = rsa.service_id AND e.path = '/**'" + )) + db.execute(text("DROP TABLE role_service_access")) + if _table_exists(db, "role_permissions"): + db.execute(text("DROP TABLE role_permissions")) + db.execute(text("ALTER TABLE users DROP COLUMN role_id")) + db.execute(text("DROP TABLE roles")) + db.commit() + print("Migrated role-based access to per-key endpoint grants.") + + if db.query(User).count() == 0: + db.add(User( + username=config.DEFAULT_ADMIN_USERNAME, + password_hash=security.hash_password(config.DEFAULT_ADMIN_PASSWORD), + )) + db.commit() + print(f"Seeded admin user '{config.DEFAULT_ADMIN_USERNAME}' " + f"(password: '{config.DEFAULT_ADMIN_PASSWORD}' — change it after first login).") + finally: + db.close() + + +@asynccontextmanager +async def lifespan(app: FastAPI): + Base.metadata.create_all(engine) + migrate_schema() + seed_and_migrate() + enable_incremental_vacuum() + retention_task = asyncio.create_task(retention.retention_loop()) + yield + retention_task.cancel() + with suppress(asyncio.CancelledError): + await retention_task + await proxy.close_client() + + +app = FastAPI(title="API Gateway", version="2.0.0", lifespan=lifespan) +app.add_exception_handler(LoginRequired, login_redirect_handler) + + +@app.get("/", include_in_schema=False) +def root(): + return RedirectResponse("/admin") + + +@app.get("/health", include_in_schema=False) +def health(): + return {"status": "ok"} + + +app.include_router(admin_routes.router) +app.include_router(admin_stats.router) +app.mount("/static", StaticFiles(directory=str(config.BASE_DIR / "app" / "static")), name="static") +# The proxy catch-all (/{slug}/...) must come last so it never shadows +# /admin, /static, /docs or /health. +app.include_router(proxy.router) diff --git a/app/models.py b/app/models.py new file mode 100644 index 0000000..c229c31 --- /dev/null +++ b/app/models.py @@ -0,0 +1,126 @@ +from datetime import datetime, timezone + +from sqlalchemy import ( + Boolean, Column, DateTime, Float, ForeignKey, Integer, String, Table, Text, +) +from sqlalchemy.orm import Mapped, mapped_column, relationship + +from app.database import Base + + +def utcnow() -> datetime: + return datetime.now(timezone.utc) + + +key_endpoint_access = Table( + "key_endpoint_access", + Base.metadata, + Column("api_key_id", ForeignKey("api_keys.id", ondelete="CASCADE"), primary_key=True), + Column("endpoint_id", ForeignKey("endpoints.id", ondelete="CASCADE"), primary_key=True), +) + + +class User(Base): + __tablename__ = "users" + + id: Mapped[int] = mapped_column(primary_key=True) + username: Mapped[str] = mapped_column(String(64), unique=True, index=True) + password_hash: Mapped[str] = mapped_column(String(256)) + is_active: Mapped[bool] = mapped_column(Boolean, default=True) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) + + api_keys: Mapped[list["ApiKey"]] = relationship( + back_populates="user", cascade="all, delete-orphan" + ) + + +class Service(Base): + __tablename__ = "services" + + id: Mapped[int] = mapped_column(primary_key=True) + name: Mapped[str] = mapped_column(String(128)) + slug: Mapped[str] = mapped_column(String(64), unique=True, index=True) + base_url: Mapped[str] = mapped_column(String(512)) + description: Mapped[str] = mapped_column(Text, default="") + timeout_seconds: Mapped[float] = mapped_column(Float, default=30.0) + # Disable for upstreams with self-signed / internal-CA certificates. + verify_tls: Mapped[bool] = mapped_column(Boolean, default=True) + is_active: Mapped[bool] = mapped_column(Boolean, default=True) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) + + endpoints: Mapped[list["Endpoint"]] = relationship( + back_populates="service", cascade="all, delete-orphan", + order_by="Endpoint.path", + ) + + +class Endpoint(Base): + """A callable route of an upstream service. + + `method` is an HTTP verb or '*' (any). `path` starts with '/' and may use + '{param}' (one segment), '*' (any within a segment) and '**' (any depth), + e.g. '/orders/{id}', '/reports/**'. + """ + __tablename__ = "endpoints" + + id: Mapped[int] = mapped_column(primary_key=True) + service_id: Mapped[int] = mapped_column(ForeignKey("services.id", ondelete="CASCADE")) + method: Mapped[str] = mapped_column(String(10), default="*") + path: Mapped[str] = mapped_column(String(512)) + description: Mapped[str] = mapped_column(Text, default="") + + service: Mapped[Service] = relationship(back_populates="endpoints") + api_keys: Mapped[list["ApiKey"]] = relationship( + secondary=key_endpoint_access, back_populates="endpoints" + ) + + @property + def label(self) -> str: + return f"{self.method} {self.path}" + + +class ApiKey(Base): + __tablename__ = "api_keys" + + id: Mapped[int] = mapped_column(primary_key=True) + user_id: Mapped[int] = mapped_column(ForeignKey("users.id", ondelete="CASCADE")) + name: Mapped[str] = mapped_column(String(128)) + prefix: Mapped[str] = mapped_column(String(16), index=True) # first chars, for display + key_hash: Mapped[str] = mapped_column(String(128), unique=True, index=True) + rate_limit_per_minute: Mapped[int] = mapped_column(Integer, default=60) # 0 = unlimited + is_active: Mapped[bool] = mapped_column(Boolean, default=True) + created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow) + last_used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) + + user: Mapped[User] = relationship(back_populates="api_keys") + endpoints: Mapped[list[Endpoint]] = relationship( + secondary=key_endpoint_access, back_populates="api_keys" + ) + + +class RequestLog(Base): + __tablename__ = "request_logs" + + id: Mapped[int] = mapped_column(primary_key=True) + timestamp: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=utcnow, index=True) + api_key_id: Mapped[int | None] = mapped_column( + ForeignKey("api_keys.id", ondelete="SET NULL"), nullable=True, index=True + ) + service_id: Mapped[int | None] = mapped_column( + ForeignKey("services.id", ondelete="SET NULL"), nullable=True, index=True + ) + endpoint_id: Mapped[int | None] = mapped_column( + ForeignKey("endpoints.id", ondelete="SET NULL"), nullable=True, index=True + ) + method: Mapped[str] = mapped_column(String(10)) + path: Mapped[str] = mapped_column(String(1024)) + query_string: Mapped[str] = mapped_column(String(2048), default="") + status_code: Mapped[int] = mapped_column(Integer, index=True) + latency_ms: Mapped[float] = mapped_column(Float) + client_ip: Mapped[str] = mapped_column(String(64), default="") + request_body: Mapped[str] = mapped_column(Text, default="") + response_body: Mapped[str] = mapped_column(Text, default="") + + api_key: Mapped[ApiKey | None] = relationship() + service: Mapped[Service | None] = relationship() + endpoint: Mapped[Endpoint | None] = relationship() diff --git a/app/proxy.py b/app/proxy.py new file mode 100644 index 0000000..6451524 --- /dev/null +++ b/app/proxy.py @@ -0,0 +1,208 @@ +import re +import time +from functools import lru_cache + +import httpx +from fastapi import APIRouter, Depends, Request, Response +from fastapi.responses import JSONResponse +from sqlalchemy.orm import Session + +from app import config, ratelimit, security +from app.database import get_db +from app.models import ApiKey, Endpoint, RequestLog, Service, utcnow + +router = APIRouter() + +# Hop-by-hop headers must not be forwarded either direction +_HOP_BY_HOP = { + "connection", "keep-alive", "proxy-authenticate", "proxy-authorization", + "te", "trailers", "transfer-encoding", "upgrade", "host", "content-length", +} + +# One client per TLS-verification mode (verify is a client-level setting in httpx). +_clients: dict[bool, httpx.AsyncClient] = {} + + +async def get_client(verify_tls: bool = True) -> httpx.AsyncClient: + client = _clients.get(verify_tls) + if client is None: + client = _clients[verify_tls] = httpx.AsyncClient( + follow_redirects=False, verify=verify_tls) + return client + + +async def close_client() -> None: + for client in _clients.values(): + await client.aclose() + _clients.clear() + + +@lru_cache(maxsize=1024) +def _pattern_to_regex(pattern: str) -> re.Pattern: + """Endpoint path pattern -> regex. '{param}' = one segment, '*' = any + within a segment, '**' = any depth.""" + regex = "" + for token in re.split(r"(\{[^}]*\}|\*\*|\*)", pattern): + if not token: + continue + if token == "**": + regex += ".*" + elif token == "*": + regex += "[^/]*" + elif token.startswith("{") and token.endswith("}"): + regex += "[^/]+" + else: + regex += re.escape(token) + return re.compile("^" + regex + "$") + + +def match_endpoints(endpoints: list[Endpoint], method: str, path: str) -> list[Endpoint]: + """All endpoints of a service that match this request, most specific + (exact method) first.""" + matches = [ + e for e in endpoints + if e.method in ("*", method) and _pattern_to_regex(e.path).match(path) + ] + return sorted(matches, key=lambda e: e.method == "*") + + +def _error(status: int, code: str, message: str) -> JSONResponse: + return JSONResponse(status_code=status, content={"error": code, "message": message}) + + +# Payload capture for transaction inspection +MAX_STORED_BODY = 64 * 1024 +_TEXTUAL_TYPES = ("application/json", "application/xml", + "application/x-www-form-urlencoded", "text/", "+json", "+xml") + + +def _readable(body: bytes, content_type: str) -> str: + """Body as storable text: textual payloads are kept (truncated at 64 KB), + binary ones are summarized.""" + if not body: + return "" + ct = (content_type or "").lower() + if not any(t in ct for t in _TEXTUAL_TYPES): + return f"<{len(body)} bytes of {ct or 'unknown content type'}>" + text = body[:MAX_STORED_BODY].decode("utf-8", "replace") + if len(body) > MAX_STORED_BODY: + text += "\n… (truncated)" + return text + + +@router.api_route( + "/{slug}/{path:path}", + methods=["GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS"], +) +@router.api_route( + "/{slug}", + methods=["GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS"], +) +async def gateway(slug: str, request: Request, path: str = "", + db: Session = Depends(get_db)): + start = time.perf_counter() + + # --- authenticate --- + plain_key = request.headers.get(config.API_KEY_HEADER) + if not plain_key: + return _error(401, "missing_api_key", f"Provide your API key in the {config.API_KEY_HEADER} header.") + + api_key = ( + db.query(ApiKey) + .filter(ApiKey.key_hash == security.hash_api_key(plain_key)) + .one_or_none() + ) + if api_key is None or not api_key.is_active or not api_key.user.is_active: + return _error(403, "invalid_api_key", "API key is unknown or has been revoked.") + + # --- resolve service --- + service = db.query(Service).filter(Service.slug == slug).one_or_none() + if service is None or not service.is_active: + return _error(404, "unknown_service", f"No active service registered under '{slug}'.") + + # --- read the request payload once (forwarded and logged) --- + body = await request.body() + stored_request = _readable(body, request.headers.get("content-type", "")) + + def deny(status: int, code: str, message: str, endpoint: Endpoint | None) -> JSONResponse: + response = _error(status, code, message) + _log(db, api_key, service, endpoint, request, request_path, status, start, + stored_request, response.body.decode()) + return response + + # --- resolve endpoint & authorize --- + request_path = "/" + path + matched = match_endpoints(service.endpoints, request.method, request_path) + if not matched: + return deny(404, "unknown_endpoint", + f"No endpoint of '{slug}' matches {request.method} {request_path}.", None) + + granted_ids = {e.id for e in api_key.endpoints} + endpoint = next((e for e in matched if e.id in granted_ids), None) + if endpoint is None: + return deny(403, "access_denied", + f"This API key has no access to {request.method} {request_path} on '{slug}'.", + matched[0]) + + # --- rate limit --- + if not ratelimit.check(api_key.id, api_key.rate_limit_per_minute): + return deny(429, "rate_limited", + f"Rate limit of {api_key.rate_limit_per_minute} requests/minute exceeded.", + endpoint) + + # --- proxy upstream --- + upstream_url = service.base_url.rstrip("/") + request_path + headers = { + k: v for k, v in request.headers.items() + if k.lower() not in _HOP_BY_HOP and k.lower() != config.API_KEY_HEADER.lower() + } + headers["x-forwarded-for"] = request.client.host if request.client else "" + headers["x-forwarded-proto"] = request.url.scheme + + client = await get_client(service.verify_tls) + try: + upstream = await client.request( + request.method, + upstream_url, + params=request.query_params, + content=body, + headers=headers, + timeout=service.timeout_seconds or config.PROXY_DEFAULT_TIMEOUT, + ) + except httpx.TimeoutException: + return deny(504, "upstream_timeout", "The upstream API did not respond in time.", endpoint) + except httpx.HTTPError: + return deny(502, "upstream_unreachable", "Could not reach the upstream API.", endpoint) + + _log(db, api_key, service, endpoint, request, request_path, upstream.status_code, start, + stored_request, _readable(upstream.content, upstream.headers.get("content-type", ""))) + + response_headers = { + k: v for k, v in upstream.headers.items() if k.lower() not in _HOP_BY_HOP + } + return Response( + content=upstream.content, + status_code=upstream.status_code, + headers=response_headers, + media_type=upstream.headers.get("content-type"), + ) + + +def _log(db: Session, api_key: ApiKey, service: Service, endpoint: Endpoint | None, + request: Request, request_path: str, status: int, start: float, + request_body: str = "", response_body: str = "") -> None: + api_key.last_used_at = utcnow() + db.add(RequestLog( + api_key_id=api_key.id, + service_id=service.id, + endpoint_id=endpoint.id if endpoint else None, + method=request.method, + path=request_path, + query_string=request.url.query, + status_code=status, + latency_ms=round((time.perf_counter() - start) * 1000, 2), + client_ip=request.client.host if request.client else "", + request_body=request_body, + response_body=response_body, + )) + db.commit() diff --git a/app/ratelimit.py b/app/ratelimit.py new file mode 100644 index 0000000..4810bb8 --- /dev/null +++ b/app/ratelimit.py @@ -0,0 +1,26 @@ +"""In-memory fixed-window rate limiter, keyed per API key. + +Good enough for a single-process gateway; swap for Redis if you scale out. +""" +import time +from collections import defaultdict +from threading import Lock + +_windows: dict[int, tuple[int, int]] = defaultdict(lambda: (0, 0)) # key_id -> (window_start, count) +_lock = Lock() + + +def check(key_id: int, limit_per_minute: int) -> bool: + """Returns True if the request is allowed.""" + if limit_per_minute <= 0: + return True + now = int(time.time() // 60) + with _lock: + window, count = _windows[key_id] + if window != now: + _windows[key_id] = (now, 1) + return True + if count >= limit_per_minute: + return False + _windows[key_id] = (window, count + 1) + return True diff --git a/app/retention.py b/app/retention.py new file mode 100644 index 0000000..b8c856e --- /dev/null +++ b/app/retention.py @@ -0,0 +1,77 @@ +"""Tiered log retention. + +Tier 1: request/response payloads are blanked after PAYLOAD_RETENTION_DAYS — + the log row stays inspectable, it just loses the bodies. +Tier 2: whole log rows are deleted after LOG_RETENTION_DAYS. + +Both run in batches so a large backlog never locks the database against the +proxy's own log writes, followed by an incremental vacuum to hand freed pages +back to the filesystem. A background loop triggers this every 6 hours. +""" +import asyncio +from datetime import datetime, timedelta, timezone + +from sqlalchemy import text + +from app import config +from app.database import SessionLocal + +PURGE_INTERVAL_SECONDS = 6 * 3600 +BATCH_SIZE = 5000 + + +def _cutoff(days: int) -> str: + """Naive-UTC timestamp string, matching how rows are stored.""" + moment = datetime.now(timezone.utc).replace(tzinfo=None) - timedelta(days=days) + return moment.strftime("%Y-%m-%d %H:%M:%S") + + +def purge_once() -> dict: + stats = {"payloads_blanked": 0, "rows_deleted": 0} + db = SessionLocal() + try: + if config.PAYLOAD_RETENTION_DAYS > 0: + cutoff = _cutoff(config.PAYLOAD_RETENTION_DAYS) + while True: + result = db.execute(text( + "UPDATE request_logs SET request_body = '', response_body = '' " + "WHERE id IN (SELECT id FROM request_logs " + " WHERE timestamp < :cutoff " + " AND (request_body != '' OR response_body != '') " + " LIMIT :batch)" + ), {"cutoff": cutoff, "batch": BATCH_SIZE}) + db.commit() + stats["payloads_blanked"] += result.rowcount + if result.rowcount < BATCH_SIZE: + break + + if config.LOG_RETENTION_DAYS > 0: + cutoff = _cutoff(config.LOG_RETENTION_DAYS) + while True: + result = db.execute(text( + "DELETE FROM request_logs WHERE id IN " + "(SELECT id FROM request_logs WHERE timestamp < :cutoff LIMIT :batch)" + ), {"cutoff": cutoff, "batch": BATCH_SIZE}) + db.commit() + stats["rows_deleted"] += result.rowcount + if result.rowcount < BATCH_SIZE: + break + + if stats["rows_deleted"] or stats["payloads_blanked"]: + db.execute(text("PRAGMA incremental_vacuum")) + db.commit() + finally: + db.close() + return stats + + +async def retention_loop() -> None: + while True: + try: + stats = await asyncio.to_thread(purge_once) + if stats["payloads_blanked"] or stats["rows_deleted"]: + print(f"Retention: deleted {stats['rows_deleted']} log rows, " + f"blanked {stats['payloads_blanked']} payloads.") + except Exception as exc: # never let a purge failure kill the loop + print(f"Retention run failed: {exc}") + await asyncio.sleep(PURGE_INTERVAL_SECONDS) diff --git a/app/security.py b/app/security.py new file mode 100644 index 0000000..9a27385 --- /dev/null +++ b/app/security.py @@ -0,0 +1,58 @@ +import hashlib +import hmac +import secrets + +from itsdangerous import BadSignature, URLSafeTimedSerializer + +from app import config + +_serializer = URLSafeTimedSerializer(config.SECRET_KEY, salt="gw-session") + +_PBKDF2_ITERATIONS = 600_000 + + +# ---- password hashing (PBKDF2-SHA256) ---- + +def hash_password(password: str) -> str: + salt = secrets.token_hex(16) + digest = hashlib.pbkdf2_hmac( + "sha256", password.encode(), bytes.fromhex(salt), _PBKDF2_ITERATIONS + ).hex() + return f"pbkdf2${_PBKDF2_ITERATIONS}${salt}${digest}" + + +def verify_password(password: str, stored: str) -> bool: + try: + _, iterations, salt, digest = stored.split("$") + computed = hashlib.pbkdf2_hmac( + "sha256", password.encode(), bytes.fromhex(salt), int(iterations) + ).hex() + return hmac.compare_digest(computed, digest) + except (ValueError, TypeError): + return False + + +# ---- API keys ---- + +def generate_api_key() -> tuple[str, str, str]: + """Returns (plain_key, prefix, key_hash). Plain key is shown once.""" + plain = "gw_" + secrets.token_urlsafe(32) + return plain, plain[:11], hash_api_key(plain) + + +def hash_api_key(plain: str) -> str: + return hashlib.sha256(plain.encode()).hexdigest() + + +# ---- session cookies ---- + +def create_session_token(user_id: int) -> str: + return _serializer.dumps({"uid": user_id}) + + +def read_session_token(token: str) -> int | None: + try: + data = _serializer.loads(token, max_age=config.SESSION_MAX_AGE) + return int(data["uid"]) + except (BadSignature, KeyError, ValueError, TypeError): + return None diff --git a/app/static/style.css b/app/static/style.css new file mode 100644 index 0000000..c54b1ab --- /dev/null +++ b/app/static/style.css @@ -0,0 +1,218 @@ +/* Dark admin theme — colors from the validated reference palette (dark mode). */ +:root { + color-scheme: dark; + --page: #0d0d0d; + --surface: #1a1a19; + --surface-2: #232322; + --ink: #ffffff; + --ink-2: #c3c2b7; + --muted: #898781; + --grid: #2c2c2a; + --border: rgba(255, 255, 255, 0.10); + --series-1: #3987e5; /* blue */ + --series-8: #e66767; /* red */ + --good: #0ca30c; + --warning: #fab219; + --critical: #d03b3b; + --radius: 10px; + font-family: system-ui, -apple-system, "Segoe UI", sans-serif; +} + +* { box-sizing: border-box; } +body { margin: 0; background: var(--page); color: var(--ink); font-size: 14px; } +a { color: var(--series-1); text-decoration: none; } + +.layout { display: flex; min-height: 100vh; } + +/* ---- sidebar ---- */ +.sidebar { + width: 220px; flex-shrink: 0; background: var(--surface); + border-right: 1px solid var(--border); padding: 20px 12px; + display: flex; flex-direction: column; gap: 4px; position: sticky; top: 0; height: 100vh; +} +.sidebar .brand { font-size: 16px; font-weight: 700; padding: 0 10px 16px; } +.sidebar .brand span { color: var(--series-1); } +.sidebar a.nav-item { + color: var(--ink-2); padding: 9px 10px; border-radius: 8px; display: block; +} +.sidebar a.nav-item:hover { background: var(--surface-2); color: var(--ink); } +.sidebar a.nav-item.active { background: var(--surface-2); color: var(--ink); font-weight: 600; } +.sidebar .spacer { flex: 1; } +.sidebar .whoami { color: var(--muted); font-size: 12px; padding: 0 10px 8px; } + +/* ---- main ---- */ +.main { flex: 1; padding: 28px 32px; max-width: 1200px; } +h1 { font-size: 20px; margin: 0 0 20px; } +h2 { font-size: 15px; margin: 0 0 12px; color: var(--ink-2); font-weight: 600; } + +.card { + background: var(--surface); border: 1px solid var(--border); + border-radius: var(--radius); padding: 18px; margin-bottom: 20px; +} +.grid { display: grid; gap: 16px; } +.grid.cols-2 { grid-template-columns: 1fr 1fr; } +.grid.cols-3 { grid-template-columns: repeat(3, 1fr); } +@media (max-width: 900px) { .grid.cols-2, .grid.cols-3 { grid-template-columns: 1fr; } } + +/* ---- stat tiles ---- */ +.tiles { display: grid; grid-template-columns: repeat(auto-fit, minmax(160px, 1fr)); gap: 16px; margin-bottom: 20px; } +.tile { background: var(--surface); border: 1px solid var(--border); border-radius: var(--radius); padding: 16px 18px; } +.tile .label { color: var(--muted); font-size: 12px; margin-bottom: 6px; } +.tile .value { font-size: 26px; font-weight: 700; } +.tile .value small { font-size: 14px; color: var(--ink-2); font-weight: 400; } + +/* ---- tables ---- */ +table { width: 100%; border-collapse: collapse; } +th { text-align: left; color: var(--muted); font-size: 12px; font-weight: 600; padding: 8px 10px; border-bottom: 1px solid var(--grid); } +td { padding: 9px 10px; border-bottom: 1px solid var(--grid); color: var(--ink-2); font-variant-numeric: tabular-nums; } +tr:last-child td { border-bottom: none; } +td.strong { color: var(--ink); font-weight: 600; } +code { background: var(--surface-2); padding: 2px 6px; border-radius: 5px; font-size: 12.5px; } +pre { + background: var(--surface-2); border: 1px solid var(--border); border-radius: 8px; + padding: 12px; font-size: 12.5px; line-height: 1.5; overflow: auto; max-height: 480px; + white-space: pre-wrap; word-break: break-word; color: var(--ink-2); margin: 0; +} + +/* ---- badges ---- */ +.badge { display: inline-flex; align-items: center; gap: 5px; font-size: 12px; padding: 2px 9px; border-radius: 999px; border: 1px solid var(--border); } +.badge.on { color: var(--good); } +.badge.off { color: var(--muted); } +.badge.admin { color: var(--series-1); } +.status-2xx { color: var(--good); } +.status-4xx { color: var(--warning); } +.status-5xx { color: var(--critical); } + +/* ---- forms ---- */ +form.inline { display: inline; } +label { display: block; color: var(--muted); font-size: 12px; margin: 10px 0 4px; } +input[type=text], input[type=password], input[type=url], input[type=number], select, textarea { + width: 100%; background: var(--surface-2); border: 1px solid var(--border); + border-radius: 8px; color: var(--ink); padding: 8px 10px; font: inherit; +} +input:focus, select:focus, textarea:focus { outline: 2px solid var(--series-1); outline-offset: -1px; } +.checks { display: flex; flex-wrap: wrap; gap: 10px; margin-top: 6px; } +.checks label { display: flex; align-items: center; gap: 6px; margin: 0; color: var(--ink-2); font-size: 13px; } + +button, .btn { + background: var(--series-1); color: #fff; border: none; border-radius: 8px; + padding: 8px 14px; font: inherit; font-weight: 600; cursor: pointer; +} +button:hover { filter: brightness(1.1); } +button.ghost { background: transparent; border: 1px solid var(--border); color: var(--ink-2); font-weight: 500; padding: 5px 10px; font-size: 12.5px; } +button.ghost:hover { background: var(--surface-2); color: var(--ink); } +button.danger { background: transparent; border: 1px solid var(--border); color: var(--critical); font-weight: 500; padding: 5px 10px; font-size: 12.5px; } +button.danger:hover { background: rgba(208, 59, 59, 0.12); } + +/* ---- misc ---- */ +.row { display: flex; gap: 10px; align-items: center; flex-wrap: wrap; } +.right { margin-left: auto; } +.alert { border-radius: 8px; padding: 12px 14px; margin-bottom: 16px; border: 1px solid var(--border); } +.alert.error { color: var(--critical); } +.alert.success { color: var(--good); background: rgba(12, 163, 12, 0.08); } +.alert.success code { user-select: all; } +.hint { color: var(--muted); font-size: 12px; margin-top: 6px; } +.chart-box { position: relative; height: 260px; } +details.editor { margin-top: 8px; } +details.editor summary { cursor: pointer; color: var(--muted); font-size: 12.5px; } +.range-picker { display: flex; gap: 6px; margin-bottom: 16px; } +.range-picker button.active { background: var(--surface-2); color: var(--ink); } + +/* ---- request browser ---- */ +tr.req-row { cursor: pointer; } +tr.req-row:hover td { background: var(--surface-2); } +tr.req-row.selected td { background: var(--surface-2); box-shadow: inset 2px 0 0 var(--series-1); } +tr.req-row:focus-visible { outline: 2px solid var(--series-1); outline-offset: -2px; } + +/* Split view: list on top, inspector below, each scrolling internally. */ +.req-split { display: flex; flex-direction: column; gap: 16px; height: calc(100vh - 130px); } +.req-split .card { margin-bottom: 0; } +.req-list { flex: 1 1 0; min-height: 0; display: flex; flex-direction: column; } +.req-list .table-scroll { flex: 1; min-height: 0; overflow-y: auto; } +.req-list .table-scroll thead th { position: sticky; top: 0; background: var(--surface); z-index: 1; } +#inspector { flex: 1 1 0; min-height: 0; display: flex; flex-direction: column; } +#inspector[hidden] { display: none; } +#inspector .inspector-body { flex: 1; min-height: 0; overflow-y: auto; } +#inspector .inspector-body pre { max-height: none; } + +/* ---- on/off switch ---- */ +.switch { position: relative; display: inline-block; width: 36px; height: 20px; vertical-align: middle; } +.switch input { opacity: 0; width: 0; height: 0; } +.switch .slider { + position: absolute; inset: 0; cursor: pointer; border-radius: 999px; + background: var(--surface-2); border: 1px solid var(--border); transition: background .15s; +} +.switch .slider::before { + content: ""; position: absolute; width: 14px; height: 14px; border-radius: 50%; + left: 2px; top: 2px; background: var(--muted); transition: transform .15s, background .15s; +} +.switch input:checked + .slider { background: var(--good); border-color: transparent; } +.switch input:checked + .slider::before { transform: translateX(16px); background: #fff; } +.switch input:focus-visible + .slider { outline: 2px solid var(--series-1); } + +/* ---- icon buttons ---- */ +button.icon { + background: transparent; border: 1px solid var(--border); color: var(--muted); + padding: 4px 7px; line-height: 0; border-radius: 7px; +} +button.icon:hover { color: var(--critical); background: rgba(208, 59, 59, 0.12); } +button.armed { + color: #fff !important; background: var(--critical) !important; + border-color: var(--critical) !important; line-height: 1.2; + font-size: 12px; font-weight: 600; +} +button.icon.edit:hover { color: var(--series-1); background: rgba(57, 135, 229, 0.12); } + +/* ---- connectivity validation ---- */ +.vresult { font-size: 12px; margin-left: 6px; } +.vresult.ok { color: var(--good); } +.vresult.warn { color: var(--warning); } +.vresult.err { color: var(--critical); } + +/* ---- modal overlay ---- */ +.modal-overlay { + position: fixed; inset: 0; z-index: 100; display: none; + background: rgba(0, 0, 0, 0.55); backdrop-filter: blur(2px); + align-items: flex-start; justify-content: center; padding: 7vh 20px 40px; +} +.modal-overlay.open { display: flex; } +.modal { + position: relative; width: 100%; max-width: 720px; + max-height: 86vh; overflow-y: auto; + background: var(--surface); border: 1px solid var(--border); + border-radius: var(--radius); padding: 20px 22px; +} +.modal > h2 { margin: 0 32px 14px 0; color: var(--ink); } +.modal .modal-close { position: absolute; top: 14px; right: 14px; } + +/* ---- spinner ---- */ +.spinner, .tree.syncing::after { + display: inline-block; width: 12px; height: 12px; border-radius: 50%; + border: 2px solid var(--grid); border-top-color: var(--series-1); + animation: spin .8s linear infinite; vertical-align: -2px; +} +@keyframes spin { to { transform: rotate(360deg); } } + +/* ---- endpoint tree picker ---- */ +.tree { position: relative; } +.tree.syncing::after { content: ""; position: absolute; top: 4px; right: 4px; } +.tree.syncing { opacity: .75; } +.tree details { margin: 0; } +.tree summary { list-style: none; cursor: pointer; padding: 3px 0; } +.tree summary::-webkit-details-marker { display: none; } +.tree summary::before { content: "▸"; display: inline-block; width: 14px; color: var(--muted); transition: transform .1s; } +.tree details[open] > summary::before { transform: rotate(90deg); } +.tree .tree-children { margin-left: 22px; border-left: 1px solid var(--grid); padding-left: 12px; } +.tree label { display: flex; align-items: center; gap: 7px; margin: 0; padding: 3px 0; color: var(--ink-2); font-size: 13px; cursor: pointer; } +.tree summary label { display: inline-flex; } +.tree .seg { color: var(--ink); font-weight: 600; font-size: 13px; } +.mchip { + display: inline-block; min-width: 42px; text-align: center; font-size: 10.5px; font-weight: 700; + padding: 1px 6px; border-radius: 5px; background: var(--surface-2); color: var(--series-1); + border: 1px solid var(--border); letter-spacing: .3px; +} +.tree .ep-desc { color: var(--muted); font-size: 12px; } + +/* ---- login ---- */ +.login-wrap { min-height: 100vh; display: grid; place-items: center; } +.login-card { width: 340px; } diff --git a/app/templates/base.html b/app/templates/base.html new file mode 100644 index 0000000..74402ee --- /dev/null +++ b/app/templates/base.html @@ -0,0 +1,90 @@ + + + + + + {% block title %}API Gateway{% endblock %} + + + +
+ +
+ {% block content %}{% endblock %} +
+
+ + +{% block scripts %}{% endblock %} + + diff --git a/app/templates/dashboard.html b/app/templates/dashboard.html new file mode 100644 index 0000000..f9a132e --- /dev/null +++ b/app/templates/dashboard.html @@ -0,0 +1,95 @@ +{% extends "base.html" %} +{% block title %}Dashboard · API Gateway{% endblock %} +{% block content %} +

Dashboard

+ +
+
Requests (24h)
+
Error rate (24h)
+
Avg latency (24h)
+
Active services
+
Active API keys
+
+ +
+

Traffic — last 24 hours

+
+
+ +
+
+

Requests by service (24h)

+
+
+
+

Recent requests

+ + + + {% for log in recent %} + + + + + + + + {% else %} + + {% endfor %} + +
Time (UTC)ServicePathStatusms
{{ log.timestamp.strftime("%H:%M:%S") }}{{ log.service.name if log.service else "–" }}{{ log.method }} {{ log.path[:40] }}{{ log.status_code }}{{ "%.0f" | format(log.latency_ms) }}
No traffic yet. Send a request through /gw/<service>/<path>.
+
+
+{% endblock %} + +{% block scripts %} + +{% endblock %} diff --git a/app/templates/keys.html b/app/templates/keys.html new file mode 100644 index 0000000..f491e81 --- /dev/null +++ b/app/templates/keys.html @@ -0,0 +1,239 @@ +{% extends "base.html" %} +{% block title %}API Keys · API Gateway{% endblock %} + +{% macro trash(label) %} + +{% endmacro %} + +{% macro pencil(label, modal) %} + +{% endmacro %} + +{% macro close_button() %} + +{% endmacro %} + +{% macro tree_node(node, granted) %} + {% if node.children %} +
+ + + +
+ {% for e in node.endpoints | sort(attribute='method') %} + + {% endfor %} + {% for name, child in node.children | dictsort %} + {{ tree_node(child, granted) }} + {% endfor %} +
+
+ {% else %} + {% for e in node.endpoints | sort(attribute='method') %} + + {% endfor %} + {% endif %} +{% endmacro %} + +{% macro endpoint_picker(services, trees, synced, granted) %} +
+ {% for s in services %} +
+ + + +
+ {% for e in trees[s.id].endpoints | sort(attribute='method') %} + + {% endfor %} + {% for name, child in trees[s.id].children | dictsort %} + {{ tree_node(child, granted) }} + {% endfor %} + {% if not trees[s.id].children and not trees[s.id].endpoints %} + No endpoints known for this service. + {% endif %} +
+
+ {% else %} + Register a service first. + {% endfor %} +
+{% endmacro %} + +{% block content %} +
+

API Keys & Access

+ +
+ +{% if new_key %} +
+ New API key created — copy it now, it will not be shown again:
+ {{ new_key }} +
+{% endif %} + +
+ + + + {% for k in keys %} + + + + + + + + + + + {% else %} + + {% endfor %} + +
NameOwnerKeyAccessRate limitLast usedOn
{{ k.name }}{{ k.user.username }}{{ k.prefix }}… + {% if k.endpoints %}{{ k.endpoints | length }} endpoint{{ '' if k.endpoints | length == 1 else 's' }} + {% else %}none{% endif %} + {{ k.rate_limit_per_minute if k.rate_limit_per_minute else '∞' }}/min{{ k.last_used_at.strftime("%Y-%m-%d %H:%M") if k.last_used_at else 'never' }} +
+ +
+
+ {{ pencil('Edit access for ' ~ k.name, 'modal-key-' ~ k.id) }} +
+ {{ trash('Delete key ' ~ k.name) }} +
+
No API keys yet.
+
Endpoint catalogs are mirrored from each service's OpenAPI document + (refreshed at most every 5 minutes; grants on removed endpoints are cleaned up). + Refresh now +
+
+ +
+

How consumers call the gateway

+

Send requests to /<service-slug>/<path> with the header X-API-Key: <key>. The request must match one of the service's endpoints and the key must hold a grant on it.

+
+ +{% for k in keys %} + +{% endfor %} + + +{% endblock %} + +{% block scripts %} + +{% endblock %} diff --git a/app/templates/login.html b/app/templates/login.html new file mode 100644 index 0000000..a372a0d --- /dev/null +++ b/app/templates/login.html @@ -0,0 +1,26 @@ + + + + + + Sign in · API Gateway + + + + + + diff --git a/app/templates/monitoring.html b/app/templates/monitoring.html new file mode 100644 index 0000000..fc8d6fe --- /dev/null +++ b/app/templates/monitoring.html @@ -0,0 +1,187 @@ +{% extends "base.html" %} +{% block title %}Monitoring · API Gateway{% endblock %} +{% block content %} +

Usage Monitoring

+ +
+
+ + + + + + +
+ + + +
+ +
+
Requests
+
Errors (5xx)
+
Error rate
+
Avg latency
+
+ +
+
+

Requests over time

+
+
+
+

Average latency over time (ms)

+
+
+
+

Requests by service

+
+
+
+

Responses by status code

+
+
+
+

Requests by user

+
+
+
+

Requests by endpoint

+
+
+
+

Average latency by service (ms)

+
+
+
+

Top API keys by usage

+
+
+
+ +
+

Latest 100 requests

+ + + + {% for log in logs %} + + + + + + + + + + + + {% else %} + + {% endfor %} + +
Time (UTC)ServiceKeyUserEndpointRequestStatusLatencyClient IP
{{ log.timestamp.strftime("%Y-%m-%d %H:%M:%S") }}{{ log.service.name if log.service else "–" }}{{ log.api_key.name if log.api_key else "–" }}{{ log.api_key.user.username if log.api_key else "–" }}{% if log.endpoint %}{{ log.endpoint.path }}{% else %}–{% endif %}{{ log.method }} {{ log.path[:50] }}{{ log.status_code }}{{ "%.1f" | format(log.latency_ms) }} ms{{ log.client_ip }}
No requests logged yet.
+
+{% endblock %} + +{% block scripts %} + +{% endblock %} diff --git a/app/templates/requests.html b/app/templates/requests.html new file mode 100644 index 0000000..8e92baf --- /dev/null +++ b/app/templates/requests.html @@ -0,0 +1,173 @@ +{% extends "base.html" %} +{% block title %}Requests · API Gateway{% endblock %} +{% block content %} +

Requests

+ +
+
+
+ + + + + + {{ total }} request{{ '' if total == 1 else 's' }} +
+
+ +
+
+ + + + {% for log in logs %} + + + + + + + + + {% else %} + + {% endfor %} + +
Time (UTC)ServiceKeyRequestStatusLatency
{{ log.timestamp.strftime("%Y-%m-%d %H:%M:%S") }}{{ log.service.name if log.service else "–" }}{{ log.api_key.name if log.api_key else "–" }}{{ log.method }} {{ log.path[:60] }}{% if log.query_string %}?{{ log.query_string[:30] }}{% endif %}{{ log.status_code }}{{ "%.1f" | format(log.latency_ms) }} ms
No requests match these filters.
+
+ + {% if pages > 1 %} + {% set base = '/admin/requests?service_id=' ~ (f.service_id or '') ~ '&user_id=' ~ (f.user_id or '') ~ '&key_id=' ~ (f.key_id or '') ~ '&status_class=' ~ f.status_class ~ '&q=' ~ f.q %} +
+ {% if page > 1 %}← Newer{% endif %} + Page {{ page }} of {{ pages }} + {% if page < pages %}Older →{% endif %} +
+ {% endif %} +
+ + +
+{% endblock %} + +{% block scripts %} + +{% endblock %} diff --git a/app/templates/services.html b/app/templates/services.html new file mode 100644 index 0000000..4962c2b --- /dev/null +++ b/app/templates/services.html @@ -0,0 +1,147 @@ +{% extends "base.html" %} +{% block title %}Services · API Gateway{% endblock %} + +{% macro trash(label) %} + +{% endmacro %} + +{% macro pencil(label, modal) %} + +{% endmacro %} + +{% block content %} +
+

Connected APIs

+ +
+ +
+ + + + + + {% for s in services %} + + + + + + + + + + + {% else %} + + {% endfor %} + +
NameGateway routeUpstreamTimeoutRequestsConnectivityOn
{{ s.name }}/{{ s.slug }}/…{{ s.base_url }}{{ s.timeout_seconds }}s{{ counts.get(s.id, 0) }} + + + +
+ +
+
+ {{ pencil('Edit ' ~ s.name, 'modal-service-' ~ s.id) }} +
+ {{ trash('Delete ' ~ s.name) }} +
+
No services registered yet.
+
Endpoints are managed on the API Keys page, + where each service's catalog is kept in sync with its OpenAPI document.
+
+ +{% for s in services %} + +{% endfor %} + + +{% endblock %} + +{% block scripts %} + +{% endblock %} diff --git a/app/templates/users.html b/app/templates/users.html new file mode 100644 index 0000000..0b2ae03 --- /dev/null +++ b/app/templates/users.html @@ -0,0 +1,91 @@ +{% extends "base.html" %} +{% block title %}Users · API Gateway{% endblock %} + +{% macro trash(label) %} + +{% endmacro %} + +{% macro pencil(label, modal) %} + +{% endmacro %} + +{% block content %} +
+

Users

+ +
+ +
+ + + + {% for u in users %} + + + + + + + + {% endfor %} + +
UsernameAPI keysCreatedOn
{{ u.username }}{{ u.api_keys | length }}{{ u.created_at.strftime("%Y-%m-%d") }} + {% if u.id != user.id %} +
+ +
+ {% else %} + + {% endif %} +
+ {{ pencil('Edit ' ~ u.username, 'modal-user-' ~ u.id) }} + {% if u.id != user.id %} +
+ {{ trash('Delete ' ~ u.username) }} +
+ {% endif %} +
+
Every user can sign in to this console and own API keys. + Gateway access itself is granted per key on the API Keys page.
+
+ +{% for u in users %} + +{% endfor %} + + +{% endblock %} diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..633a8e4 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,19 @@ +services: + gateway: + build: . + ports: + - "8000:8000" + environment: + # Set a strong random value in production — signs the admin session cookies. + GATEWAY_SECRET_KEY: change-me-in-production + GATEWAY_ADMIN_USER: admin + GATEWAY_ADMIN_PASSWORD: admin + # Retention (days; 0 = keep forever) + GATEWAY_PAYLOAD_RETENTION_DAYS: "7" + GATEWAY_LOG_RETENTION_DAYS: "90" + volumes: + - gateway-data:/data + restart: unless-stopped + +volumes: + gateway-data: diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..bd973d1 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,7 @@ +fastapi>=0.115 +uvicorn[standard]>=0.32 +httpx>=0.28 +SQLAlchemy>=2.0.36 +jinja2>=3.1 +python-multipart>=0.0.12 +itsdangerous>=2.2