Skip to main content

​
AI Usage Ops Dashboard Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.
Goal: Build an owner-only global AI usage dashboard backed by the existing ai_usage table. Architecture: Add a bot API route that validates x-api-secret, parses range/filter query params, queries global AI usage aggregates, and resolves guild names from the Discord client cache. Add a Next.js global-admin-gated proxy route and a dashboard page that renders summary cards, charts, and comparison tables using existing dashboard primitives. Tech Stack: Node 22, Express 5, PostgreSQL, Vitest, Next.js 16 App Router, React 19, TypeScript, Tailwind 4, Recharts via StableResponsiveContainer.

​
Task 1: Backend AI Usage Aggregation

Files:
  • Create: src/api/repositories/aiUsageRepository.js
  • Test: tests/api/repositories/aiUsageRepository.test.js
  • Step 1: Write failing repository tests
Add tests for: 
import { describe, expect, it, vi } from 'vitest';
import {
  fetchAiUsageOpsSnapshot,
  parseAiUsageOpsQuery,
  parseProviderFromModel,
} from '../../../src/api/repositories/aiUsageRepository.js';

it('parses supported ranges and rejects invalid filters', () => {
  expect(parseAiUsageOpsQuery({ range: '24h' }).preset).toBe('24h');
  expect(parseAiUsageOpsQuery({ range: '7d' }).interval).toBe('hour');
  expect(parseAiUsageOpsQuery({ range: '30d' }).interval).toBe('day');
  expect(() => parseAiUsageOpsQuery({ range: '999d' })).toThrow('Invalid range');
  expect(() => parseAiUsageOpsQuery({ type: 'delete' })).toThrow('Invalid type');
});

it('parses provider from provider:model and groups bare models as unknown', () => {
  expect(parseProviderFromModel('z-ai:glm-5.1')).toEqual({ provider: 'z-ai', model: 'glm-5.1' });
  expect(parseProviderFromModel('bare-model')).toEqual({ provider: 'unknown', model: 'bare-model' });
});

it('returns empty aggregates when ai_usage has no rows', async () => {
  const dbPool = { query: vi.fn().mockResolvedValue({ rows: [] }) };
  const result = await fetchAiUsageOpsSnapshot({ dbPool, client: { guilds: { cache: new Map() } }, query: {} });
  expect(result.summary.requests).toBe(0);
  expect(result.summary.cacheReadRate).toBeNull();
  expect(result.byProvider).toEqual([]);
});

it('aggregates totals, cache rates, weighted TPS, p95 latency, and guild names', async () => {
  const rows = [
    { id: 1, guild_id: 'guild-1', channel_id: 'chan-1', type: 'respond', model: 'z-ai:glm-5.1', input_tokens: 1000, output_tokens: 250, cache_read_tokens: 300, cache_creation_tokens: 50, cost_usd: '0.012', duration_ms: 500, search_count: 2, created_at: '2026-06-07T10:00:00.000Z' },
    { id: 2, guild_id: 'guild-2', channel_id: 'chan-2', type: 'safety', model: 'bare-model', input_tokens: 500, output_tokens: 100, cache_read_tokens: 0, cache_creation_tokens: 0, cost_usd: '0.002', duration_ms: 1000, search_count: 0, created_at: '2026-06-07T10:10:00.000Z' },
  ];
  const dbPool = { query: vi.fn().mockResolvedValue({ rows }) };
  const client = { guilds: { cache: new Map([['guild-1', { name: 'Alpha' }]]) } };
  const result = await fetchAiUsageOpsSnapshot({ dbPool, client, query: { range: '24h' } });
  expect(result.summary.requests).toBe(2);
  expect(result.summary.totalTokens).toBe(1850);
  expect(result.summary.cacheReadRate).toBeCloseTo(0.2);
  expect(result.summary.avgTokensPerSecond).toBeCloseTo(1233.333, 2);
  expect(result.byProvider.map((row) => row.name)).toEqual(['z-ai', 'unknown']);
  expect(result.topGuilds[0]).toMatchObject({ guildId: 'guild-1', guildName: 'Alpha' });
});
  • Step 2: Run repository tests to verify RED
Run: pnpm test -- tests/api/repositories/aiUsageRepository.test.js Expected: fail because src/api/repositories/aiUsageRepository.js does not exist.
  • Step 3: Implement repository
Create src/api/repositories/aiUsageRepository.js with:
  • parseAiUsageOpsQuery(query) supporting 24h, 7d, 30d, 90d, and type in classify/respond/safety.
  • parseProviderFromModel(modelString) using the first colon.
  • fetchAiUsageOpsSnapshot({ dbPool, client, query }) issuing one parameterized SELECT from ai_usage for the requested range and optional filters, then aggregating rows in JavaScript.
  • helper functions for totals, cache rates, weighted TPS, p95 latency, provider/model/type/guild grouping, time buckets, and recent expensive rows.
  • Step 4: Run repository tests to verify GREEN
Run: pnpm test -- tests/api/repositories/aiUsageRepository.test.js Expected: pass.
  • Step 5: Commit backend repository
git add src/api/repositories/aiUsageRepository.js tests/api/repositories/aiUsageRepository.test.js
git commit -m "feat(api): aggregate global ai usage"

​
Task 2: Bot API Route

Files:
  • Create: src/api/routes/aiUsage.js
  • Modify: src/api/index.js
  • Test: tests/api/routes/aiUsage.test.js
  • Step 1: Write failing route tests
Add tests that build the Express app with a mocked pool and verify: 
it('returns 401 without x-api-secret', async () => {
  const res = await request(app).get('/api/v1/ai-usage');
  expect(res.status).toBe(401);
});

it('returns 400 for an invalid range', async () => {
  const res = await request(app).get('/api/v1/ai-usage?range=bad').set('x-api-secret', TEST_SECRET);
  expect(res.status).toBe(400);
});

it('returns a snapshot with valid auth', async () => {
  const res = await request(app).get('/api/v1/ai-usage?range=24h').set('x-api-secret', TEST_SECRET);
  expect(res.status).toBe(200);
  expect(res.body).toHaveProperty('summary');
  expect(res.body).toHaveProperty('byProvider');
  expect(res.body.range.preset).toBe('24h');
});
  • Step 2: Run route tests to verify RED
Run: pnpm test -- tests/api/routes/aiUsage.test.js Expected: fail with 404 or missing route.
  • Step 3: Implement route and mount it
Create src/api/routes/aiUsage.js using Router(), isValidSecret(req.headers['x-api-secret']), and fetchAiUsageOpsSnapshot({ dbPool: req.app.locals.dbPool, client: req.app.locals.client, query: req.query }). Mount it in src/api/index.js near /performance: 
import aiUsageRouter from './routes/aiUsage.js';
router.use('/ai-usage', aiUsageRouter);
  • Step 4: Run route tests to verify GREEN
Run: pnpm test -- tests/api/routes/aiUsage.test.js Expected: pass.
  • Step 5: Commit bot route
git add src/api/routes/aiUsage.js src/api/index.js tests/api/routes/aiUsage.test.js
git commit -m "feat(api): expose owner ai usage snapshot"

​
Task 3: Web Proxy, Types, and Page Titles

Files:
  • Create: web/src/app/api/ai-usage/route.ts
  • Create: web/src/types/ai-usage.ts
  • Modify: web/src/lib/page-titles.ts
  • Test: web/tests/api/ai-usage-route.test.ts
  • Test: web/tests/lib/page-titles.test.ts
  • Step 1: Write failing web API and title tests
Add route tests that mock authorizeRequestGlobalAdmin and proxyBotApiEndpoint: 
it('returns the global admin auth error without proxying', async () => {
  mockAuthorizeRequestGlobalAdmin.mockResolvedValueOnce(NextResponse.json({ error: 'Forbidden' }, { status: 403 }));
  const response = await GET(new NextRequest('http://localhost/api/ai-usage'));
  expect(response.status).toBe(403);
  expect(mockProxyBotApiEndpoint).not.toHaveBeenCalled();
});

it('proxies range and filters to the bot API after owner auth', async () => {
  mockAuthorizeRequestGlobalAdmin.mockResolvedValueOnce(null);
  await GET(new NextRequest('http://localhost/api/ai-usage?range=24h&type=respond'));
  expect(mockProxyBotApiEndpoint).toHaveBeenCalledWith('/ai-usage?range=24h&type=respond', '[api/ai-usage]', 'Failed to fetch AI usage data');
});
Update web/tests/lib/page-titles.test.ts to expect AI Usage and docs href for /dashboard/ai-usage.
  • Step 2: Run web API/title tests to verify RED
Run: pnpm --filter volvox-bot-web test -- tests/api/ai-usage-route.test.ts tests/lib/page-titles.test.ts Expected: fail because route/title wiring is missing.
  • Step 3: Implement web route, types, and title wiring
Create web/src/app/api/ai-usage/route.ts with dynamic = 'force-dynamic', owner auth, URL search forwarding, and proxyBotApiEndpoint. Create web/src/types/ai-usage.ts matching the spec response. Add /dashboard/ai-usage title/docs matcher to web/src/lib/page-titles.ts.
  • Step 4: Run web API/title tests to verify GREEN
Run: pnpm --filter volvox-bot-web test -- tests/api/ai-usage-route.test.ts tests/lib/page-titles.test.ts Expected: pass.
  • Step 5: Commit web plumbing
git add web/src/app/api/ai-usage/route.ts web/src/types/ai-usage.ts web/src/lib/page-titles.ts web/tests/api/ai-usage-route.test.ts web/tests/lib/page-titles.test.ts
git commit -m "feat(web): proxy owner ai usage data"

​
Task 4: Dashboard AI Usage UI

Files:
  • Create: web/src/components/dashboard/ai-usage-dashboard.tsx
  • Create: web/src/app/dashboard/ai-usage/page.tsx
  • Modify: web/src/components/layout/sidebar.tsx
  • Test: web/tests/components/dashboard/ai-usage-dashboard.test.tsx
  • Test: web/tests/components/layout/sidebar.test.tsx
  • Step 1: Write failing component and sidebar tests
Add component tests that mock fetch and verify loading, empty, populated, filter changes, and failed response states. Add sidebar expectations: 
mockGlobalAdminStatus.value = { isGlobalAdmin: true, isLoading: false, status: 'allowed' };
expect(screen.getByRole('link', { name: /ai usage/i })).toHaveAttribute('href', '/dashboard/ai-usage');

mockGlobalAdminStatus.value = { isGlobalAdmin: false, isLoading: false, status: 'denied' };
expect(screen.queryByRole('link', { name: /ai usage/i })).not.toBeInTheDocument();
  • Step 2: Run UI tests to verify RED
Run: pnpm --filter volvox-bot-web test -- tests/components/dashboard/ai-usage-dashboard.test.tsx tests/components/layout/sidebar.test.tsx Expected: fail because component and nav item do not exist.
  • Step 3: Implement dashboard page and component
Create AiUsageDashboard with:
  • range/provider/model/type filter controls.
  • summary metric cards.
  • time-series chart using StableResponsiveContainer.
  • provider/model/type/top-guild/recent-request tables.
  • loading, empty, and error states.
Create web/src/app/dashboard/ai-usage/page.tsx with metadata, isDashboardGlobalAdmin() redirect, ErrorBoundary, and <AiUsageDashboard />. Add sidebar AI Usage with a lucide icon under System Ops and add /dashboard/ai-usage to GLOBAL_ADMIN_ONLY_HREFS.
  • Step 4: Run UI tests to verify GREEN
Run: pnpm --filter volvox-bot-web test -- tests/components/dashboard/ai-usage-dashboard.test.tsx tests/components/layout/sidebar.test.tsx Expected: pass.
  • Step 5: Commit UI
git add web/src/components/dashboard/ai-usage-dashboard.tsx web/src/app/dashboard/ai-usage/page.tsx web/src/components/layout/sidebar.tsx web/tests/components/dashboard/ai-usage-dashboard.test.tsx web/tests/components/layout/sidebar.test.tsx
git commit -m "feat(dashboard): add owner ai usage page"

​
Task 5: Docs, AGENTS Check, and Verification

Files:
  • Modify: docs/features/analytics.mdx
  • Modify: docs/dashboard.mdx
  • Read/check: AGENTS.md
  • Step 1: Update docs
Document that global admins can use System Ops -> AI Usage for deployment-wide token, cache, cost, model/provider, and top-server usage. Do not edit docs/changelog.mdx except to repair generated output required by docs validation.
  • Step 2: Run targeted tests
Run: 
pnpm test -- tests/api/repositories/aiUsageRepository.test.js tests/api/routes/aiUsage.test.js
pnpm --filter volvox-bot-web test -- tests/api/ai-usage-route.test.ts tests/lib/page-titles.test.ts tests/components/dashboard/ai-usage-dashboard.test.tsx tests/components/layout/sidebar.test.tsx
Expected: all targeted tests pass.
  • Step 3: Run broader checks
Run: 
pnpm lint
pnpm test -- tests/docs/docs.test.js
Expected: both pass. Record any pre-existing warnings separately.
  • Step 4: Browser verification
Start local dev: 
pnpm mono:dev
Use agent-browser to open /dashboard/ai-usage, verify owner access, desktop layout, mobile layout, filters, and no visible overflow. If browser tooling cannot run, state that plainly.
  • Step 5: Final commit
git add docs/features/analytics.mdx docs/dashboard.mdx
git commit -m "docs(dashboard): document owner ai usage ops"
Last modified on June 8, 2026