[FEATURE] Targets & Deps & Extras #3
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "ep/May13-2026/TargetsAndExtrasCleanup"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Schema rewrite (heph/src/config/schema.ts): - targets: top-level block; explicit deploy sets composed via includes: - manifests: new block (raw + sealed-secret + sops + age kinds) — split out from extras - charts: new block (helm) — split out from extras - services: now includes prebuilt-image runtimes that were extras.kind:image - services.kind: image renamed to kind: build (build-only artifact) - bootstrap: extras → charts + manifests - overlay.target: string OR { base, add, remove } patch form - dropped: extras, services.optional, services.optionalDependencies, services.dependsOn, jobs.dependsOn, jobs.optionalDependencies, overlay.use/with/without/skipDeps/runJobs Resolver rewrite (heph/src/config/resolve.ts, replaces deps.ts): - target closure via includes: chain (cycle-checked, diamond-flattened) - dependencies: now means "must be Ready/Complete before this starts" NOT set-expansion. Target is the explicit deploy list. - Verification edges: every name in node.dependencies / node.attach / job.service must be in target closure; error includes `heph target add <name> to <target>` fix-it hint. - Unified-graph cycle detection across services/manifests/charts/jobs including target:<name> deps (flattened to member edges). - Timing-constraint validation: service ↛ post-deploy job (deadlock) pre-deploy job → app service (not yet exist) — but pre-deploy job → infra service (prebuilt image, applied early) IS OK pre-deploy ↛ post-deploy (timing inversion) - --with / --without CLI patches handled as ephemeral target ops. Tests: schema.test.ts + resolve.test.ts rewritten. 59 tests / 97 assertions green. Test matrix covers: target closure, includes cycles, diamond inheritance, missing dep/attach/service-ref edges, mixed-kind cycle, target.add/remove patches, --with/--without, target-as-dep flattening, all timing-constraint rows, overlay-replaces-base-target. NOT yet done (subsequent commits on this branch): - merge.ts still uses old shape (extras, overlay.use, etc.) - cli/* still uses old CreateFlags + overlay fields - operator/cr.ts still references eff.extras - operator/src/* CR shape unchanged - 40+ other test files still import deps.ts / reference dropped fields - docs not yet updated tsc will fail across consumers. Clean break per user direction (no migration logic — ephemeral clusters, fresh deploy).CLI flags (heph/src/cli/parse.ts): - CreateFlags reshaped: target / targetAdd / targetRemove / withNames / withoutNames replace addServices / withDeps / withoutDeps / skipDeps / optionalPolicy / runJobs / noJobs - parseCreate accepts --target / --target-add / --target-remove / --with / --without; drops --add-service / --run-job / --no-deps / -y / --no-prompt / --no-jobs Config merge (heph/src/config/merge.ts): - resolveAndApply: top-level orchestration — wraps resolveDeploySet + applyResolution; throws with fix-it hint if overlay.target missing - merge: produces new EffectiveConfig (services + manifests + charts + jobs, no extras) - applyResolution: filters merged eff by resolver result for all four kinds - overlayFromFlags: writes overlay.target as string OR { base, add, remove } patch, depending on whether --with/--without/--target-add/-remove were used - classifyTargetOps: classifies CLI patch names into per-kind buckets via classifyDep (rejects target:<name> in CLI patches — only members) - mergeServiceCatalog / mergeManifestCatalog / mergeChartCatalog: exported catalog helpers (replaces extras merge logic) Local overlay (heph/src/config/local-overlay.ts): - extras: → manifests: + charts: - bootstrap.extras: → bootstrap.charts: + bootstrap.manifests: - drops services.<n>.{dependencies,optionalDependencies,dependsOn,optional} from the allow-list (project-level concerns; can't be per-dev) - rejects targets:/jobs: at always-rejected list CLI orchestrator (heph/src/cli/index.ts): - create case: validates --target against base.targets, classifies --with/--without/--target-add/-remove via classifyTargetOps, writes overlay.target as bare-string or patch object - up case: --with/--without as ephemeral CLI patches (not persisted); drops --yes/--no-prompt/--skip-deps/--run-job/--no-jobs handling - ensureBootstrap: checks both base.bootstrap.charts AND .manifests - drops deps-prompt import Removed: - heph/src/cli/deps-prompt.ts (optional-dep interactive prompts — no more optional deps) - heph/test/deps-prompt-no-tty.test.ts NOT yet done (subsequent commits on this branch): - 12+ source files still reference deleted/renamed fields: kind: "image" → "build" (cli/shell.ts, status.ts, logs.ts, wait-ready.ts, operator/cr.ts; ALSO operator/src/environment/environment.types.ts + operator/src/render/manifests.ts on operator side) eff.extras → eff.manifests + eff.charts (operator/cr.ts, extras/apply.ts, operator/rbac.ts, tls/inject.ts, mcp/server.ts, cli/shell.ts) base.bootstrap.extras → .charts + .manifests (bootstrap/reconcile.ts, operator/rbac.ts, tls/inject.ts) svc.dependsOn → svc.dependencies (cli/wait-ready.ts, operator/cr.ts, jobs/apply.ts) Various dropped flags in mcp/server.ts (addServices, etc.) - extras/apply.ts needs split into manifests/apply.ts + charts/apply.ts - operator/src/environment/environment.types.ts CR shape unchanged - operator/src/render/manifests.ts initContainer rendering still reads deleted dependsOn - 5+ test files still use old shape (test/bootstrap.test.ts, test/merge.test.ts, test/resolve-apply.test.ts, test/local-overlay.test.ts, test/operator-cr.test.ts, test/wait-ready.test.ts, test/jobs.test.ts, test/e2e/*) - docs not yet updated tsc still fails across consumers (~50 errors); test suite still fails at import time. Foundation (schema + resolver + merge + 59 tests) is solid.Apply path (extras/apply.ts → split into three): - heph/src/manifests/apply.ts: raw + sealed-secret + sops + age handlers, shared between tier-1 (bootstrap.manifests) and tier-2 (env-scoped). Also hosts ensureEnvNamespace (used to live in extras/apply.ts). - heph/src/charts/apply.ts: helm install handler, shared between tier-1 and tier-2. - heph/src/services/apply-infra.ts: deploys prebuilt-image services (kind: service with image: set, no source: — postgres/redis/vendor) as Deployment + Service directly via kubectl, BEFORE pre-deploy jobs land. Source-built services continue going through the Environment CR. Infra/app distinction kept the pre-deploy → app-services ordering working. - heph/src/extras/ directory deleted. cli/up.ts apply order: ensureEnvNamespace → applyInfraServices → applyEnvManifests → applyEnvCharts → applyPreDeployJobs → applyEnvironmentCR → waitForReady → applyPostDeployJobs Operator CR (heph/src/operator/cr.ts): - EnvironmentServiceSpec.kind: "service" | "build" (renamed from "image") - buildEnvironmentCR skips: kind: build (no Pod, build-only) infra services (handled by apply-infra.ts) leaving only source-built app services in the CR for the operator. - svc.dependsOn dropped — initContainer wait list now derived from svc.dependencies via deriveDependsOn() (resolves service names + looks up port from schema; manifest/chart/job/target deps skip). - eff.extras → eff.manifests + attach reads manifests[attachName] now. Only kind: raw manifests have produces:; sealed-secret/sops/age skip. Bootstrap + tls (bootstrap/reconcile.ts, tls/inject.ts): - bootstrap.extras → bootstrap.charts + bootstrap.manifests, applied separately via applyChartToNamespace / applyManifestToNamespace - Catalog hash now covers both (renamed extrasHash → bootstrapHash). - tls/inject.ts: cert-manager + reflector + ingress-nginx go to charts; ClusterIssuer + wildcard Certificate go to manifests (kind: raw). - User-declared entries still win on name collision. Operator RBAC (operator/rbac.ts): - collectManifestEntries: walks bootstrap.manifests + base.manifests (was bootstrap.extras + base.extras) - Now only kind: raw manifests participate (sealed-secret/sops/age are CLI-applied, operator never sees them; no operator-side RBAC needed). - Legacy type aliases ExtraRbacRule / ExtraManifest kept (point to ManifestRbacRule / ManifestRaw) for test compat. CLI surface (logs/shell/status/wait-ready): - kind: "image" → "build" everywhere (services.kind enum rename) - svc.dependsOn → svc.dependencies (filtered to declared services + ports for blocked-on hints in wait-ready's tree renderer) Jobs (jobs/apply.ts): - renderJob takes optional eff: EffectiveConfig for port resolution - deriveJobDependsOn: translates dependencies → host:port for initContainer TCP wait; service/extra → service.port lookup; manifest/chart/job/target skip (not TCP peers) - jobSpecForHash: dependsOn → dependencies in hash material mcp/server.ts: - heph_create tool rewritten for new CreateFlags shape: target / targetAdd / targetRemove / withNames / withoutNames; classifies via classifyTargetOps before calling overlayFromFlags; throws when target missing or unknown. Status: heph/src/ is **fully tsc-green**. test/ files (~10 files) still reference old shape (base.extras / extras/apply.ts / CreateFlags.addServices / overlay.use / kind: image service entries / etc.) and need rewriting in the next pass. New schema + resolve tests (59 from phase 1-2 commit) still green; everything else still fails at import time. Remaining work (subsequent commits): - test/bootstrap.test.ts — update for bootstrap.charts + .manifests split - test/local-overlay.test.ts — manifests/charts in LocalOverlay (was extras) - test/merge.test.ts — new CreateFlags + overlayFromFlags signature, drop ov.use - test/operator-cr.test.ts — kind: image extras → kind: service+image services - test/resolve-apply.test.ts — delete (deps.ts is gone; resolve.test.ts covers) - test/wait-ready.test.ts — kind: image → build - test/jobs.test.ts — dependsOn references - test/e2e/* — extras/apply.ts import + extras shape - Operator side (operator/src/): CR shape kind: image → build - Docs: 02-config-schema.md + ~6 other doc filesCR enum (operator/src/environment/environment.types.ts): - ServiceSpec.kind: "service" | "build" (was "service" | "image") - Keeps the operator's CR shape in lockstep with heph/src/operator/cr.ts's EnvironmentServiceSpec (CRD wire format must match). Consumer updates: operator/src/{environment.controller,render/manifests, render/registry-creds,share/types}.ts — `kind === "image"` / `!== "image"` references all flipped to "build". Behavior is byte-identical (the rename is purely a vocabulary shift — `build` = build-only artifact with no Pod). Status: heph/src/ AND operator/src/ both tsc-green. Test files still need rewriting (~10 in heph/test, none in operator/test from this pass).Test file updates to match new schema/resolver/CR shape: heph/test/bootstrap.test.ts rewritten for bootstrap.charts + bootstrap.manifests (was bootstrap.extras); hashExtras → hashBootstrap. heph/test/local-overlay.test.ts LocalOverlay.extras → .manifests + .charts; bootstrap.extras override tests updated. heph/test/merge.test.ts new CreateFlags shape + overlayFromFlags signature (now takes buckets); ov.use → overlay.target; manifest add+remove via overlay; full rewrite of overlayFromFlags tests covering bare-target + patch-target. heph/test/operator-cr.test.ts extras: → manifests:; kind: manifest → kind: raw; eff.extras → eff.manifests; dropped tests that relied on kind: image extras (postgres now an infra service); `passes through dependsOn` test rewritten to derive dependsOn from dependencies + schema port lookup. heph/test/operator-rbac.test.ts collectManifestEntries test: walks bootstrap.manifests + manifests (was bootstrap.extras + extras). heph/test/tls.test.ts every bootstrap.extras check split into bootstrap.charts + bootstrap.manifests (cert-manager+reflector+ingress-nginx in charts; ClusterIssuer + Certificate in manifests). heph/test/jobs.test.ts dependsOn → dependencies (bare-name fallback when no eff provided). heph/test/wait-ready.test.ts kind: image → build; dropped service fields no longer on schema (dependsOn, optionalDependencies, optional); rewrote "blocked on deps" fixture to use dependencies + a postgres infra service. heph/test/up-output.test.ts duplicate dependencies keys (sed artifact) removed; optionalDependencies dropped. heph/test/e2e/_harness.ts ProjectSpec: extras → manifests + charts + targets; ExtraFixture types replaced. heph/test/e2e/inproc-lifecycle.test.ts extras/apply.ts import → manifests/apply + services/apply-infra; applyEnvExtras test replaced with applyInfraServices covering kind: service + image fixtures. heph/test/e2e/extras-secrets.test.ts extras.kind:image postgres → services entry; new target: "default" overlay; CLI flags --add-service postgres + --no- prompt → --target default. heph/test/resolve-apply.test.ts deleted (covered by resolve.test.ts). Final status: - heph/src/ + operator/src/ tsc clean - heph/test/ : 513 pass / 0 fail / 40 skip / 1158 assertions across 53 files - Phases 1-6 (schema, resolver, CLI, apply path, operator CR, tests) complete. Remaining (Phase 7): docs rewrite — 02-config-schema.md (major), plus deltas in 04-lifecycle, 11-operator, 12-webhook, 13-cli-operator, 14-architecture, 17-roadmap.Three remaining failures in operator/test/render.test.ts were the same kind-enum rename — 3 tests referencing `atlas: { kind: "image" }` in fixtures. After the s/image/build/ rename: 147/147 operator tests pass. Project-wide test status: 660 tests pass, 0 fail across both packages.Full rewrite reflecting new schema: - Top-level taxonomy: services / manifests / charts / jobs / targets (was services / extras / jobs) - Targets block — explicit deploy sets composable via includes: - dependencies semantic: readiness gate (was set expansion) - infra-vs-app service distinction (prebuilt-image vs source-built) - Bootstrap split into bootstrap.charts + bootstrap.manifests - Overlay.target field (string or patch form) - CLI flag mapping table: drops --add-service / --run-job / --no-jobs / --no-prompt / --yes / --no-deps; adds --target / --target-add / --target-remove / --with / --without - Resolver order of operations - Per-dev overlay: explicit reject list (jobs / targets / dependencies) - Manifests catalog: discriminated union (raw / sealed-secret / sops / age) Remaining docs touched but not rewritten in this commit: - docs/00-overview.md, 04-lifecycle.md, 11-operator.md, 12-webhook.md, 13-cli-operator.md, 14-architecture.md, 17-roadmap.md — may have surface-level references to extras / dependencies-as-expansion that benefit from a follow-up sweep. The canonical schema doc (02) is the load-bearing one.Updated docs whose vocabulary changed materially after the targets/manifests/ charts refactor: README.md quickstart heph.yaml example: services replaces extras for postgres; targets block declared; --add-service → --target. Repo tree (src/) lists new manifests/, charts/, services/ folders. AGENTS.md dep-graph entry references resolve.ts:NodeKind (was deps.ts); manifest rendering points at the right paths; "add a secret-flavored manifest kind" walkthrough. "tier-1 cluster resource" entry points at bootstrap.charts + bootstrap.manifests (was bootstrap.extras). docs/00-overview.md hard-rules + solution shape updated for targets-block model; example heph create flag → --target backend. docs/03-cli.md heph create flag table fully rewritten: --target (required), --target-add / --target-remove, --with / --without (bare-name synonyms). Removed flags called out: --add-service, --run-job, --no-jobs, --yes / --no-prompt, --no-deps / --skip-deps. heph patch examples updated (target.add / target.remove ops replace use+= / dependsOn-=). heph config: rejected-project-fields list updated. docs/17-roadmap.md new resume-context note dated 2026-05-13 covering the full refactor (schema split, targets, dependencies semantic, infra-vs-app services, deleted flags); test counts bumped 335 → 660. NOT touched in this commit (low-priority deep-detail docs that may still reference the old vocabulary): docs/06-gotchas.md, 08-next-steps.md, 09-research-summary.md, 10-tls.md, 11-operator.md, 12-webhook.md, 13-cli-operator.md, 14-architecture.md, 15-navigation.md, 16-local.md, 18-integration.md, 22-secrets.md. These can be swept in a follow-up PR; the canonical schema doc (02) + load-bearing surface (README / AGENTS / 00 / 03 / 17) is the source of truth for the new model.Self-review (Dunemask-voice pass): - Remove dead `dedupe<T>()` helper in heph/src/cli/index.ts (last caller was the optional-dep-prompt result merging, deleted in phase 3). - Document apply-infra.ts carve-outs explicitly: no imagePullSecrets injection (private-registry infra images need user pre-staging — same as the pre-refactor extras.kind:image behavior), no readinessProbe (Pod-Ready relies on the container's own healthcheck). Recorded inline so the next agent doesn't try to "fix" them silently. Biome format pass: 13 files reformatted (no semantic changes). Final verification after format: - heph tsc clean - operator tsc clean - web tsc clean - heph test: 513 pass / 0 fail / 40 skip / 1158 assertions - operator test: 147 pass / 0 fail / 265 assertions