docs: align implementation plan with behavior tests

This commit is contained in:
Wenxin Duan
2026-09-07 10:23:10 +08:00
parent d866c16c50
commit 9933672c2e
@@ -37,7 +37,6 @@
- Create: `packages/shared/package.json` - Create: `packages/shared/package.json`
- Create: `packages/shared/tsconfig.json` - Create: `packages/shared/tsconfig.json`
- Create: `packages/shared/src/index.ts` - Create: `packages/shared/src/index.ts`
- Create: `packages/shared/tests/smoke.test.ts`
- Create: `packages/harness/package.json` - Create: `packages/harness/package.json`
- Create: `packages/harness/tsconfig.json` - Create: `packages/harness/tsconfig.json`
- Create: `apps/server/package.json` - Create: `apps/server/package.json`
@@ -51,22 +50,11 @@
- Consumes: Node.js 20+ and pnpm 11 available in the development environment. - Consumes: Node.js 20+ and pnpm 11 available in the development environment.
- Produces: root commands `dev`, `lint`, `typecheck`, `test`, and `build`; workspace package names `@agent-studio/shared` and `@agent-studio/harness`. - Produces: root commands `dev`, `lint`, `typecheck`, `test`, and `build`; workspace package names `@agent-studio/shared` and `@agent-studio/harness`.
- [ ] **Step 1: Add the workspace smoke test** - [ ] **Step 1: Create strict workspace configuration and package scripts**
```ts
// packages/shared/tests/smoke.test.ts
import { describe, expect, it } from "vitest";
describe("workspace", () => {
it("runs TypeScript tests", () => expect(true).toBe(true));
});
```
- [ ] **Step 2: Create strict workspace configuration and package scripts**
Root scripts must delegate with `pnpm -r`, keep `dev` parallel, and make `test` use Vitest in run mode. `tsconfig.base.json` must enable `strict`, `noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, `useUnknownInCatchVariables`, `verbatimModuleSyntax`, and `noEmit` for checking. `.env.example` must contain only the empty/default values from the design spec. Root scripts must delegate with `pnpm -r`, keep `dev` parallel, and make `test` use Vitest in run mode. `tsconfig.base.json` must enable `strict`, `noUncheckedIndexedAccess`, `exactOptionalPropertyTypes`, `useUnknownInCatchVariables`, `verbatimModuleSyntax`, and `noEmit` for checking. `.env.example` must contain only the empty/default values from the design spec.
- [ ] **Step 3: Install current stable dependencies** - [ ] **Step 2: Install current stable dependencies**
Run: Run:
@@ -79,7 +67,7 @@ pnpm --filter @agent-studio/web add @agent-studio/shared react react-dom @assist
pnpm --filter @agent-studio/web add -D vite @vitejs/plugin-react jsdom @testing-library/react @testing-library/user-event @testing-library/jest-dom pnpm --filter @agent-studio/web add -D vite @vitejs/plugin-react jsdom @testing-library/react @testing-library/user-event @testing-library/jest-dom
``` ```
- [ ] **Step 4: Record and inspect the installed public API** - [ ] **Step 3: Record and inspect the installed public API**
Run: Run:
@@ -91,13 +79,13 @@ rg "useExternalStoreRuntime|AttachmentAdapter|ThreadPrimitive|ComposerPrimitive"
Write exact dependency versions and any material signature differences into a temporary commit note in `README.md`; Task 16 expands it into the final version/API section. Write exact dependency versions and any material signature differences into a temporary commit note in `README.md`; Task 16 expands it into the final version/API section.
- [ ] **Step 5: Verify the baseline** - [ ] **Step 4: Verify the baseline**
Run: `pnpm test && pnpm typecheck && pnpm lint` Run: `pnpm test && pnpm typecheck && pnpm lint`
Expected: the smoke test passes and all packages typecheck/lint without ignored errors. Expected: all packages typecheck/lint without ignored errors; packages that do not yet contain tests exit cleanly through their explicit `--passWithNoTests` baseline configuration.
- [ ] **Step 6: Commit** - [ ] **Step 5: Commit**
```bash ```bash
git add package.json pnpm-lock.yaml pnpm-workspace.yaml tsconfig.base.json eslint.config.js .gitignore .env.example packages apps README.md git add package.json pnpm-lock.yaml pnpm-workspace.yaml tsconfig.base.json eslint.config.js .gitignore .env.example packages apps README.md
@@ -1035,36 +1023,25 @@ git commit -m "test: cover complete Agent Studio flow"
- Create: `docs/security-boundaries.md` - Create: `docs/security-boundaries.md`
- Create: `docs/classroom-demo.md` - Create: `docs/classroom-demo.md`
- Create: `docs/production-roadmap.md` - Create: `docs/production-roadmap.md`
- Create: `packages/harness/tests/documentation.test.ts`
**Interfaces:** **Interfaces:**
- Consumes: actual dependency versions, commands, routes and observed behavior from Tasks 1–16. - Consumes: actual dependency versions, commands, routes and observed behavior from Tasks 1–16.
- Produces: complete operator/trainer documentation with Mermaid diagrams and no unverified claims. - Produces: complete operator/trainer documentation with Mermaid diagrams and no unverified claims.
- [ ] **Step 1: Write documentation assertions** - [ ] **Step 1: Write README and five guides**
Create a focused Vitest that reads the Markdown files and asserts README contains the exact phrases `Pi does not provide an OS sandbox`, `Project Trust is not a sandbox`, `skill.requested`, `skill.loaded`, all root commands, and every required documentation link.
- [ ] **Step 2: Verify red state**
Run: `pnpm --filter @agent-studio/harness test -- documentation`
Expected: FAIL because final documentation is incomplete.
- [ ] **Step 3: Write README and five guides**
Document purpose, actual versions/API differences, architecture and Agent Loop diagrams, boundaries, directory tree, setup, environment and Settings model configuration, credential lifetime, image chain, exact demo prompt, Approve/Deny behavior, progressive disclosure, inferred Skill events, in-memory/single-process limits, test commands and production roadmap. Security guide explicitly explains Node process inheritance and the need for Docker/VM/micro-VM/remote sandbox for untrusted work. Document purpose, actual versions/API differences, architecture and Agent Loop diagrams, boundaries, directory tree, setup, environment and Settings model configuration, credential lifetime, image chain, exact demo prompt, Approve/Deny behavior, progressive disclosure, inferred Skill events, in-memory/single-process limits, test commands and production roadmap. Security guide explicitly explains Node process inheritance and the need for Docker/VM/micro-VM/remote sandbox for untrusted work.
- [ ] **Step 4: Verify documentation** - [ ] **Step 2: Verify documentation against actual behavior**
Run: `pnpm --filter @agent-studio/harness test -- documentation` Run every documented setup, test and build command. Compare documented versions with `pnpm list -r --depth 0`, routes with the Fastify route registry, and environment names with `apps/server/src/config.ts`. Render Mermaid blocks in a compatible preview and inspect every internal file link.
Expected: PASS and no stale commands, versions or route names. Expected: all commands succeed and no documented version, route, environment name, link or security claim disagrees with the implementation.
- [ ] **Step 5: Commit** - [ ] **Step 3: Commit**
```bash ```bash
git add README.md docs packages/harness/tests git add README.md docs
git commit -m "docs: add Agent Studio teaching guide" git commit -m "docs: add Agent Studio teaching guide"
``` ```