Exports Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Let users export all of an organisation's data as a single .zip containing full-fidelity data.json + data.yaml and an OKF (Open Knowledge Format) cross-linked Markdown bundle — whole-org (admin+) and per-connection (viewer+).

Architecture: A collect → render → package pipeline in src/lib/export/. collectOrgData(orgId) reads every org-owned table via Drizzle into one plain, typed object graph (OrgExport). The JSON/YAML/OKF renderers and the zip archiver are pure functions of that graph — no DB — so they are unit-testable. Two thin route handlers assemble + stream the zip; two small client components trigger the download via fetch→blob (a plain <a download> can't send the x-organisation-id header).

Tech Stack: Next.js 15 App Router, Drizzle ORM (Neon), Vitest, yaml (serialization), fflate (zip). Reference design: docs/plans/2026-07-09-data-portability-design.md.

Testing note: This repo has no test database (jsdom/DB test envs are broken — see MISTAKES.md). Therefore: renderers/archiver are TDD'd as pure functions over a hand-built sample OrgExport; collectOrgData (thin DB glue) is covered by a type-level "every table represented" guard test plus a manual authenticated smoke, not a DB unit test.

Schema note: collectOrgData selects whole rows (db.select().from(table)), so any columns added later (e.g. the follow-up-reminder dueAt/sourceMomentId fields on another branch) flow into the export automatically once merged — no rework.

Task 1: Add dependencies

Files: - Modify: package.json

Step 1: Install the two approved deps.

Run: npm install yaml fflate Expected: both added to dependencies; npm run build still passes.

Step 2: Verify.

Run: node -e "require('yaml'); require('fflate'); console.log('ok')" Expected: ok

Step 3: Commit

git add package.json package-lock.json
git commit -m "chore: add yaml and fflate for exports"

Task 2: Define the export graph type

Files: - Create: src/lib/export/types.ts

Step 1: Define OrgExport — the plain graph every renderer consumes. Use InferSelectModel so it tracks the schema.

import type { InferSelectModel } from "drizzle-orm";
import type {
  organisations, connections, moments, momentConnections,
  spaces, connectionSpaces, qualities, observations,
  networkLinks, organisationMemberships, users,
} from "@/lib/db/schema";

type Row<T> = InferSelectModel<T>;

/** A member with the user fields we include (never secrets). */
export interface ExportMember {
  role: Row<typeof organisationMemberships>["role"];
  name: string | null;
  email: string;
}

/** The complete, plain, in-memory mirror of one organisation's data. */
export interface OrgExport {
  exportedAt: string; // ISO
  organisation: Pick<
    Row<typeof organisations>,
    "id" | "name" | "slug" | "plan" | "createdAt"
  >;
  connections: Row<typeof connections>[];
  moments: Row<typeof moments>[];
  momentConnections: Row<typeof momentConnections>[];
  spaces: Row<typeof spaces>[];
  connectionSpaces: Row<typeof connectionSpaces>[];
  qualities: Row<typeof qualities>[];
  observations: Row<typeof observations>[];
  networkLinks: Row<typeof networkLinks>[];
  members: ExportMember[];
}

/** File tree emitted by a renderer: repo-relative path → file contents. */
export type FileTree = Record<string, string>;

Step 2: Typecheck.

Run: npx tsc --noEmit Expected: no errors in src/lib/export/types.ts (pre-existing with-fallback.test.ts error may remain — ignore it).

Step 3: Commit

git add src/lib/export/types.ts
git commit -m "feat(export): add OrgExport graph type"

Task 3: JSON renderer (pure, TDD)

Files: - Create: src/lib/export/json.ts - Test: src/lib/export/json.test.ts

Step 1: Write the failing test

import { describe, it, expect } from "vitest";
import { renderJson } from "./json";
import { sampleExport } from "./sample.fixture";

describe("renderJson", () => {
  it("round-trips: parse(render(x)) deep-equals x (dates as ISO strings)", () => {
    const json = renderJson(sampleExport());
    const parsed = JSON.parse(json);
    expect(parsed.organisation.slug).toBe("acme");
    expect(parsed.connections).toHaveLength(sampleExport().connections.length);
  });

  it("pretty-prints with 2-space indent", () => {
    expect(renderJson(sampleExport())).toContain('\n  "exportedAt"');
  });
});

Step 2: Create the shared fixture src/lib/export/sample.fixture.ts returning a small hand-built OrgExport (1 org, 2 connections, 2 moments, 1 space, 1 quality, 1 observation, 1 network link, 1 member, and momentConnections linking both connections to a shared moment — this shared moment exercises OKF cross-linking later). Keep it deterministic (fixed dates).

Step 3: Run test to verify it fails

Run: npx vitest run src/lib/export/json.test.ts Expected: FAIL ("renderJson is not a function").

Step 4: Implement

import type { OrgExport } from "./types";

/** Full-fidelity JSON. Dates serialize to ISO strings via JSON.stringify. */
export function renderJson(data: OrgExport): string {
  return JSON.stringify(data, null, 2);
}

Step 5: Run test to verify it passes

Run: npx vitest run src/lib/export/json.test.ts Expected: PASS.

Step 6: Commit

git add src/lib/export/json.ts src/lib/export/json.test.ts src/lib/export/sample.fixture.ts
git commit -m "feat(export): JSON renderer"

Task 4: YAML renderer (pure, TDD)

Files: - Create: src/lib/export/yaml.ts - Test: src/lib/export/yaml.test.ts

Step 1: Write the failing test

import { describe, it, expect } from "vitest";
import { parse } from "yaml";
import { renderYaml } from "./yaml";
import { renderJson } from "./json";
import { sampleExport } from "./sample.fixture";

describe("renderYaml", () => {
  it("parses back to the same object graph as JSON", () => {
    const fromYaml = parse(renderYaml(sampleExport()));
    const fromJson = JSON.parse(renderJson(sampleExport()));
    expect(fromYaml).toEqual(fromJson);
  });
});

Step 2: Run to verify it fails

Run: npx vitest run src/lib/export/yaml.test.ts Expected: FAIL.

Step 3: Implement

import { stringify } from "yaml";
import type { OrgExport } from "./types";

/** Same graph as JSON, YAML syntax. JSON round-trip first normalises Dates
 *  to ISO strings so YAML and JSON compare equal. */
export function renderYaml(data: OrgExport): string {
  return stringify(JSON.parse(JSON.stringify(data)));
}

Step 4: Run to verify it passes

Run: npx vitest run src/lib/export/yaml.test.ts Expected: PASS.

Step 5: Commit

git add src/lib/export/yaml.ts src/lib/export/yaml.test.ts
git commit -m "feat(export): YAML renderer"

Task 5: OKF Markdown renderer (pure, TDD)

Files: - Create: src/lib/export/okf.ts - Test: src/lib/export/okf.test.ts

Step 1: Write the failing test — assert the bundle shape, frontmatter, and cross-links.

import { describe, it, expect } from "vitest";
import { renderOkf } from "./okf";
import { sampleExport } from "./sample.fixture";

describe("renderOkf", () => {
  const tree = renderOkf(sampleExport());

  it("emits an index and one file per connection/space/moment", () => {
    expect(tree["okf/index.md"]).toBeDefined();
    expect(Object.keys(tree).some((p) => p.startsWith("okf/connections/"))).toBe(true);
    expect(Object.keys(tree).some((p) => p.startsWith("okf/moments/"))).toBe(true);
    expect(tree["okf/members.md"]).toBeDefined();
  });

  it("connection docs carry YAML frontmatter with type + provenance", () => {
    const conn = Object.entries(tree).find(([p]) => p.startsWith("okf/connections/"))![1];
    expect(conn).toMatch(/^---\n/);
    expect(conn).toContain("type:");
    expect(conn).toContain("id:");
  });

  it("a shared moment is cross-linked from both its connections (relative md links)", () => {
    const connDocs = Object.entries(tree)
      .filter(([p]) => p.startsWith("okf/connections/"))
      .map(([, c]) => c);
    const linkers = connDocs.filter((c) => c.includes("](../moments/"));
    expect(linkers.length).toBeGreaterThanOrEqual(2);
  });

  it("marks AI-inferred thread summaries with provenance, never as human text", () => {
    const withSummary = Object.values(tree).find((c) => c.includes("thread_summary_source:"));
    expect(withSummary).toBeDefined();
  });
});

Step 2: Run to verify it fails

Run: npx vitest run src/lib/export/okf.test.ts Expected: FAIL.

Step 3: Implement renderOkf(data): FileTree. Include small pure helpers in the same file: - frontmatter(obj) → YAML frontmatter block delimited by --- (reuse yaml stringify). - slugify(name) → filesystem-safe, collision-suffixed with a short id fragment. - index.md: org name, exportedAt, counts, and a links section to each sub-index (progressive disclosure). - connections/<slug>.md: frontmatter (type, id, tags from metadata, contact fields, created, thread_summary_source: inferred when a summary exists); body = thread summary (labelled AI-written), a Qualities list, Moments as - [<date>](../moments/<slug>.md) links, Spaces links. - moments/<slug>.md: frontmatter (id, source, event_date, created, author); body = content + backlinks to connections/space. - spaces/<slug>.md: members + moments as links. - observations/index.md: one section per observation (type, content, status). - members.md: table of role/name/email.

Keep every inter-doc reference a relative markdown link so the bundle is a self-contained OKF graph.

Step 4: Run to verify it passes

Run: npx vitest run src/lib/export/okf.test.ts Expected: PASS.

Step 5: Commit

git add src/lib/export/okf.ts src/lib/export/okf.test.ts
git commit -m "feat(export): OKF markdown bundle renderer"

Task 6: Archive (zip) assembler (pure, TDD)

Files: - Create: src/lib/export/archive.ts - Test: src/lib/export/archive.test.ts

Step 1: Write the failing test — build the full bundle and unzip it back.

import { describe, it, expect } from "vitest";
import { unzipSync, strFromU8 } from "fflate";
import { buildExportArchive } from "./archive";
import { sampleExport } from "./sample.fixture";

describe("buildExportArchive", () => {
  it("produces a zip containing README, data.json, data.yaml and the okf tree", () => {
    const zip = buildExportArchive(sampleExport());
    const files = unzipSync(zip);
    expect(Object.keys(files)).toEqual(
      expect.arrayContaining(["README.md", "data.json", "data.yaml", "okf/index.md"]),
    );
    expect(JSON.parse(strFromU8(files["data.json"])).organisation.slug).toBe("acme");
  });
});

Step 2: Run to verify it fails

Run: npx vitest run src/lib/export/archive.test.ts Expected: FAIL.

Step 3: Implement

import { zipSync, strToU8 } from "fflate";
import type { OrgExport } from "./types";
import { renderJson } from "./json";
import { renderYaml } from "./yaml";
import { renderOkf } from "./okf";

function readme(data: OrgExport): string {
  return `# Tending export — ${data.organisation.name}\n\n` +
    `Generated ${data.exportedAt}.\n\n` +
    `- \`data.json\` / \`data.yaml\` — full-fidelity structured mirror.\n` +
    `- \`okf/\` — Open Knowledge Format bundle: cross-linked Markdown, human- and AI-readable. Start at \`okf/index.md\`.\n`;
}

/** Assemble the full export as a zip (Uint8Array). Pure — no DB, no IO. */
export function buildExportArchive(data: OrgExport): Uint8Array {
  const files: Record<string, Uint8Array> = {
    "README.md": strToU8(readme(data)),
    "data.json": strToU8(renderJson(data)),
    "data.yaml": strToU8(renderYaml(data)),
  };
  for (const [path, contents] of Object.entries(renderOkf(data))) {
    files[path] = strToU8(contents);
  }
  return zipSync(files);
}

Step 4: Run to verify it passes

Run: npx vitest run src/lib/export/archive.test.ts Expected: PASS.

Step 5: Commit

git add src/lib/export/archive.ts src/lib/export/archive.test.ts
git commit -m "feat(export): zip archive assembler"

Task 7: `collectOrgData` + "every table" guard

Files: - Create: src/lib/export/collect.ts - Test: src/lib/export/collect.test.ts

Step 1: Write the failing guard test — a compile/shape test that fails if a schema table is added but not collected. It asserts the set of OrgExport data keys matches an explicit expected set; when someone adds a table they must update both, forcing a conscious decision.

import { describe, it, expect } from "vitest";
import { EXPORTED_ENTITY_KEYS } from "./collect";

describe("collect coverage", () => {
  it("exports every org-owned entity we intend to (update deliberately when schema grows)", () => {
    expect([...EXPORTED_ENTITY_KEYS].sort()).toEqual(
      [
        "connections", "moments", "momentConnections", "spaces",
        "connectionSpaces", "qualities", "observations",
        "networkLinks", "members",
      ].sort(),
    );
  });
});

Step 2: Run to verify it fails

Run: npx vitest run src/lib/export/collect.test.ts Expected: FAIL.

Step 3: Implement collect.ts. Export EXPORTED_ENTITY_KEYS and collectOrgData(orgId). Read each table with db.select().from(table).where(eq(table.organisationId, orgId)); for join tables (momentConnections, connectionSpaces) scope via the org's connection ids; members joins organisationMemberships→users selecting only role/name/email. Assemble and return OrgExport with exportedAt: new Date().toISOString(). Any read throwing aborts the whole collect (no partial mirror).

export const EXPORTED_ENTITY_KEYS = new Set([
  "connections", "moments", "momentConnections", "spaces",
  "connectionSpaces", "qualities", "observations", "networkLinks", "members",
] as const);
// ... collectOrgData(orgId): Promise<OrgExport>

Step 4: Run to verify it passes

Run: npx vitest run src/lib/export/collect.test.ts Expected: PASS. Then npx tsc --noEmit clean on the export module.

Step 5: Commit

git add src/lib/export/collect.ts src/lib/export/collect.test.ts
git commit -m "feat(export): collectOrgData with coverage guard"

Task 8: Whole-org export route (admin+)

Files: - Create: src/app/api/export/route.ts - Reference: src/lib/utils/api.ts (getOrgContext, errorResponse), src/lib/auth/permissions.ts (hasMinRole)

Step 1: Implement GET. Auth via getOrgContext(request); require hasMinRole(membership.role, "admin") else 403. const data = await collectOrgData(organisationId); const zip = buildExportArchive(data); Return a Response with the Uint8Array body, headers Content-Type: application/zip and Content-Disposition: attachment; filename="tending-export-<slug>-<YYYY-MM-DD>.zip". Map errors like sibling routes (401/403/402/500).

Step 2: Manual verification (no DB unit test). With the dev server running and an authenticated admin session:

Run: curl -s -H "x-organisation-id: <org>" -H "Cookie: <session>" http://localhost:3000/api/export -o /tmp/export.zip && unzip -l /tmp/export.zip Expected: lists README.md, data.json, data.yaml, okf/.... Also verify a viewer/contributor session gets 403.

Step 3: Commit

git add src/app/api/export/route.ts
git commit -m "feat(export): whole-org export route (admin+)"

Task 9: Per-connection export (viewer+)

Files: - Create: src/lib/export/collect-connection.ts (+ test collect-connection.test.ts) - Create: src/app/api/connections/[connectionId]/export/route.ts

Step 1: Write the failing test for a pure scopeToConnection(data, connectionId): OrgExport that filters a full OrgExport down to one connection's subgraph (the connection, its moments + their momentConnections, its qualities, its spaces, network links touching it, org + members). Test with sampleExport() that the shared moment and the co-linked connection are retained.

Step 2: Run → fail; implement scopeToConnection (pure); Step 3: Run → pass.

Step 4: Add the route GET — getOrgContext, require viewer, verify the connection belongs to the org (404 else), scopeToConnection(await collectOrgData(orgId), connectionId), buildExportArchive, stream zip named tending-<connection-slug>-<date>.zip.

Step 5: Commit

git add src/lib/export/collect-connection.ts src/lib/export/collect-connection.test.ts "src/app/api/connections/[connectionId]/export/route.ts"
git commit -m "feat(export): per-connection export (viewer+)"

Task 10: Download UI — Settings + connection page

Files: - Create: src/components/export/export-button.tsx (client) - Modify: the Settings page (add whole-org button, admin+ only) and src/app/(dashboard)/[orgSlug]/connections/[connectionId]/page.tsx (add per-connection button)

Step 1: Implement a reusable client ExportButton that fetches the export URL with the x-organisation-id header, reads the blob, and triggers a download via a temporary object URL. Show a "Preparing…" state; surface errors inline. (No <a download> — it can't set the header.)

"use client";
import { useState } from "react";
export function ExportButton({ url, organisationId, filename, label = "Export", canExport = true }: {
  url: string; organisationId: string; filename: string; label?: string; canExport?: boolean;
}) {
  const [busy, setBusy] = useState(false);
  const [error, setError] = useState<string | null>(null);
  if (!canExport) return null;
  async function run() {
    setBusy(true); setError(null);
    try {
      const res = await fetch(url, { headers: { "x-organisation-id": organisationId } });
      if (!res.ok) { setError("Export failed"); return; }
      const blob = await res.blob();
      const href = URL.createObjectURL(blob);
      const a = document.createElement("a");
      a.href = href; a.download = filename; a.click();
      URL.revokeObjectURL(href);
    } catch { setError("Export failed"); } finally { setBusy(false); }
  }
  return (/* button + optional error text, warm-theme classes */);
}

Step 2: Wire it into Settings (whole-org, gated to admin+ via the page's known role) pointing at /api/export, and into the connection page pointing at /api/connections/<id>/export.

Step 3: Verify npx tsc --noEmit, npm run lint, npm run build all clean; drive both buttons in the running app and confirm the downloaded zip opens and the OKF index.md cross-links resolve.

Step 4: Commit

git add src/components/export/export-button.tsx "src/app/(dashboard)/[orgSlug]/connections/[connectionId]/page.tsx" <settings-file>
git commit -m "feat(export): download buttons in settings and connection page"

Task 11: Final verification

Run: npx tsc --noEmit (ignore pre-existing with-fallback.test.ts error), npm run lint, npm run test, npm run build — all green.
Manual: whole-org export as admin (open zip, read okf/index.md, follow a connection→moment link), per-connection export as viewer, and confirm a viewer is blocked from the whole-org route.
Update STATE.md (exports shipped) and, if the pattern was non-obvious, note anything in MISTAKES.md.
Commit any doc updates.

Out of scope (later plans)

Phase 2 (webhooks) and Phase 3 (/api/v1) per docs/plans/2026-07-09-data-portability-design.md.
Async/background generation for very large orgs (documented scale path).

Exports Implementation Plan

Task 1: Add dependencies

Task 2: Define the export graph type

Task 3: JSON renderer (pure, TDD)

Task 4: YAML renderer (pure, TDD)

Task 5: OKF Markdown renderer (pure, TDD)

Task 6: Archive (zip) assembler (pure, TDD)

Task 7: collectOrgData + "every table" guard

Task 8: Whole-org export route (admin+)

Task 9: Per-connection export (viewer+)

Task 10: Download UI — Settings + connection page

Task 11: Final verification

Out of scope (later plans)

Task 7: `collectOrgData` + "every table" guard