API Schema Versioning

Some target APIs evolve per release: fields are introduced, graduate from alpha to GA, get renamed, or are removed. A single static schema can't honestly model "this field is valid against API v1.32 but not v1.28." This page describes the contract between formae core and a plugin that ships per-version schemas, and the reference implementation in formae-plugin-kubernetes.

The pattern was first developed for the Kubernetes plugin (where the cluster's minor version determines which fields are accepted by the API server). It generalizes to any provider whose API evolves on a version axis you can detect at apply time.

When you need this

You need versioned schemas when all of the following hold:

  1. The target API's accepted field set changes between releases your users will run.
  2. Users target different versions concurrently (e.g. one cluster on 1.31, another on 1.34).
  3. Submitting an unsupported field is a hard error at the provider, not a no-op.

If only (1) and (2) apply but unsupported fields are silently ignored, you can often get away with a single schema plus runtime-side preflight checks. The full versioning pipeline below is what you reach for when authoring against the wrong field set rots in production.

The pattern in one paragraph

Ship a schema package containing per-version subtrees (schema/pkl/<pkg>/v<X.Y>/...). Expose an ApiVersion field on the plugin's Config class so users can declare which version their target speaks. At extract / apply time, formae reads that field, narrows the Pkl wildcard import globs to the matching v<X.Y>/** subtree, and rewrites import aliases in the generated .pkl so only version-appropriate symbols are referenced. The plugin author's job is to publish the per-version subtrees and the ApiVersion field; everything that follows is handled by core.

How formae core handles versioning

This section describes the SDK side — the mechanisms in the formae binary itself that plugin authors get for free once they conform to the contract. Knowing this lets you debug strange extract output, understand which knobs you control, and recognize what you don't have to reinvent.

The contract (what plugins must provide)

Contract point Plugin responsibility
Per-version subtree layout Ship schema/pkl/<pkg>/v<X.Y>/... under the plugin's installed schema directory.
Version key in Config Expose ApiVersion (or apiVersion) as a top-level field on the plugin's Pkl Config class.
Optional manifest default Filesystem layout: the highest v*/ subdir is the default when no target specifies a version.
Runtime resolution The plugin Go side maps ApiVersion to a concrete runtime client and runs preflight checks.

That's it. Every other piece below is handled by core.

Schema manifest discovery

When the formae agent starts and a plugin is installed, the package resolver scans the plugin's schema/pkl/<pkg>/ directory for v*/ subdirectories. The result is a SchemaManifest:

```go title="internal/schema/pkl/package_resolver.go" type SchemaManifest struct { Versions []string // e.g. ["v1.21", "v1.22", ..., "v1.36"] Default string // the highest semver-aware entry } 


`SchemaManifestForNamespace(ns)` returns this manifest. Plugins that don't ship `v*/` subdirs return `nil` and get the legacy unrestricted-glob behavior — versioning is opt-in via filesystem layout.

### Version resolution per Forma

At extract time, core walks every `Target` in the Forma and resolves the schema version per namespace using `resolveSchemaVersions`:

```go title="internal/schema/pkl/pkl_serialize.go"
// Priority:
//  1. ApiVersion field inside Target.Config (the plugin's own Config schema
//     declares it; formae reads the JSON blob and looks for the convention key).
//  2. Filesystem scan — SchemaManifest.Default for the namespace.
//  3. Unresolved — namespace falls back to the unrestricted "@<pkg>/**/*.pkl" glob.
func resolveSchemaVersions(data *model.Forma, options *schema.SerializeOptions) (map[string]string, error)

The plugin doesn't write this code. It writes the ApiVersion field into its Config schema. Core reads the JSON-serialized Target.Config blob, accepts both ApiVersion and apiVersion, and produces a map[namespace]version (e.g. {"K8S": "v1.34"}) for the extract pipeline.

!!! warning "One version per namespace per extract" resolveSchemaVersions errors when two targets in the same namespace declare different ApiVersion values. The Pkl ImportsGenerator can only narrow each package one way per extract pass, so honoring conflicting versions would silently corrupt resources bound to the losing target. Split into separate Formae or align versions. Per-target schema dispatch is a future redesign.

Local-package swap

When extract runs in SchemaLocationLocal mode, core rewrites the generated PklProject dependency specs so each versioned namespace points at the local plugin install instead of the hub-published package URI:

```go title="internal/schema/pkl/pkl_serialize.go" // swapVersionedDepsToLocal rewrites the PklProject dep specs so that any // namespace with a resolved schema version is pulled from its local install // (where v/ subtrees live) instead of a hub-published package. func swapVersionedDepsToLocal(includes []string, versions map[string]string, opts schema.SerializeOptions) []string 


Local dev gets per-version dispatch, and remote runs use whatever the hub-published package ships. This is intentional: versioned dispatch is a local-plugin-development feature, and remote (the production default) is never silently flipped to local because a CLI-host plugin tree happens to exist.

### Pkl-side glob narrowing

Core invokes the Pkl `ImportsGenerator` with a `schemaVersions` external property — a comma-separated `pkg=ver,pkg=ver` string. The generator narrows the per-package wildcard glob:

```pkl title="internal/schema/pkl/generator/ImportsGenerator.pkl"
// Without versions: import everything from the package
["@k8s"] = new Mapping {
    ...import*("@k8s/*.pkl"); ...import*("@k8s/**/*.pkl")
}

// With versions: only root files + the chosen per-version subtree
["@k8s"] = new Mapping {
    ...import*("@k8s/*.pkl"); ...import*("@k8s/v1.34/**/*.pkl")
}

The narrowed packages mapping flows into ResourcesGenerator (which finds all @ResourceHint-annotated classes) and ultimately into the extract codegen's typeMap. Only resources from the chosen version (plus version-agnostic root files like target.pkl, shared.pkl) end up in the generated .pkl.

Collision-aware import aliasing

Extract emits import headers like import "<path>" as <alias>. The alias is derived by modulePathToAlias in internal/schema/pkl/generator/pklGenerator.pkl:

Case Alias
No basename collision <baseName> (e.g. vpc, queue)
Cross-plugin collision (@aws/ecs/service.pkl + @gcp/cloudrun/service.pkl) <parent>_<baseName> → ecs_service / cloudrun_service
Versioned collision (@k8s/v1.34/k8s.pkl + @k8s/v1.35/k8s.pkl) <baseName>_<sanitizedVersion> → k8s_v1_34 / k8s_v1_35
Self-collision via package root (@k8s/k8s.pkl + @k8s/v1.34/k8s.pkl) Root collapses to <baseName> → k8s; per-version takes version branch → k8s_v1_34

Two sanitizers run on the parent segment to keep the alias a legal Pkl identifier:

  • Strip the leading @ (@k8s → k8s).
  • Replace any non-[A-Za-z0-9_] character with _ so dotted directory names (v1.34) become valid identifiers (v1_34).

The version branch fires when the parent segment matches ^v\d+(\..+)?$. This places the meaningful symbol first (k8s_v1_34 reads as "k8s for v1.34" — natural at use sites like new k8s_v1_34.PolicyRule).

Filtering speculative namespace-base imports

When extract collects allImports for collision-aware aliasing, it historically synthesized an @<pkg>/<pkg>.pkl import for every resource (for cross-package Tag types and similar). Core now filters this through the resolved package graph: if @<pkg>/<pkg>.pkl doesn't actually exist as a file in the package, the synthesized import is skipped.

Without this filter, a phantom @k8s/k8s.pkl (which doesn't exist in the published kubernetes plugin — that package ships target.pkl at the root) would collide with the real @k8s/v1.34/k8s.pkl, forcing the version branch to fire (k8s_v1_34) while the body renderer still emitted the bare basename (k8s). Header and body would disagree and the extracted Forma would fail to evaluate. The filter prevents that.

What you get end-to-end

For a Forma with target.config.kubernetesVersion = "1.34", when a user runs formae extract:

  1. Core resolves K8S → v1.34 via resolveSchemaVersions.
  2. PklProject deps swap to the local plugin install when running in SchemaLocationLocal mode.
  3. ImportsGenerator narrows the @k8s wildcard to root + v1.34/**.
  4. ResourcesGenerator finds only resources defined in @k8s/v1.34/....
  5. Extract codegen emits a .pkl whose imports are aliased correctly (collision-aware, version-friendly).
  6. The resulting .pkl evaluates against the same v1.34/ subtree the original Forma referenced.

The plugin author writes the ApiVersion Config field + the per-version subtrees. Core handles the rest.

Reference implementation: plugin-k8s

This section walks through how formae-plugin-kubernetes conforms to the core contract. The same recipe applies to any plugin whose target API evolves per version.

Published schema layout

This is the schema/pkl/ tree that ends up under ~/.pel/formae/plugins/k8s/ after formae plugin install. It's also the package layout consumers see when they pull the package from hub.platform.engineering.

schema/pkl/
├── PklProject                         # package root; declares "@k8s" + deps
├── shared.pkl                         # version-agnostic typealiases + K8sVersion class
├── target.pkl                         # Config + Auth (Config carries kubernetesVersion)
├── helm/                              # helm-chart wrappers (per-version + shared)
│   └── v1.34/...
├── v1.21/
│   ├── PklProject
│   ├── k8s.pkl                        # SubResource classes valid in 1.21
│   ├── apps/Deployment.pkl
│   ├── core/Pod.pkl
│   └── ...
├── v1.22/
├── v1.23/
├── …
└── v1.36/                             # latest supported minor
    ├── PklProject
    ├── k8s.pkl
    ├── apps/Deployment.pkl
    ├── core/Pod.pkl
    └── …                              # all api-group subdirs

Important properties of this layout:

  • No top-level k8s.pkl. The package root is target.pkl (Config + Auth only). Per-version subdirs each hold their own k8s.pkl containing the SubResource classes valid for that minor. Avoids the basename collision that would otherwise force <baseName>_<sanitizedVersion> aliases.
  • shared.pkl at the root. Version-agnostic typealiases (KubernetesMinorVersion, etc.) and the K8sVersion annotation class live here. Per-version files reference them through the package root, so the version dropped doesn't strand orphan imports.
  • Per-version PklProject. Each v<X.Y>/ ships its own PklProject so the per-version subtree can stand alone if needed (Pkl resolves @k8s/v1.34/... paths through that nested project).
  • helm/ is parallel. Helm-chart wrappers also ship per-version, in their own subtree, because they import per-version subresource files (@k8s/v1.34/k8s.pkl) and therefore need to be regenerated when versions change.

The kubernetes plugin includes 16 minor versions today (v1.21 → v1.36). Each new minor adds one subdir; an unsupported minor would be dropped from this list. CI runs a conformance matrix that exercises every minor against a real kind cluster.

The Config schema's apiVersion field

```pkl title="schema/pkl/target.pkl" typealias KubernetesMinorVersion = String(matches(Regex(#"^\d+.\d+$"#)))

open class Config { kubernetesVersion: KubernetesMinorVersion? // <-- The contract field auth: Auth // ... } 


Core picks up `kubernetesVersion` (matched by the case-insensitive `ApiVersion` / `apiVersion` lookup) and feeds it into `resolveSchemaVersions`. Users write:

```pkl
new k8s.Config {
    kubernetesVersion = "1.34"
    auth = new k8s.KubeconfigAuth {}
}

The version annotation class

``pkl title="schema/pkl/shared.pkl" /// API version a field/class is gated to. /// ///introducedInis the first K8s minor in which the field is valid. ///removedInis the minor in which the API was removed. ///deprecatedIn` is the minor in which the field was deprecated. class K8sVersion { introducedIn: KubernetesMinorVersion? deprecatedIn: KubernetesMinorVersion? removedIn: KubernetesMinorVersion? reference: String? // KEP URL, RFC, vendor doc } 


Inside each per-version `v<X.Y>/` subtree, fields that were added in a later minor are simply absent from the file. The annotation is what drives that selection at publish time, so when a consumer imports `@k8s/v1.30/core/Service.pkl`, they only see fields that were available in K8s 1.30 — `trafficDistribution` (Beta in 1.30) is present, while `tolerance` (Beta in 1.35) is not.

Three granularities of gating, all visible in the per-version trees as additions or omissions:

```pkl title="Property-level — field landed in 1.30, beta then GA"
@k8s.FieldHint {}
@k8s.K8sVersion { introducedIn = "1.30"; reference = "https://kep.k8s.io/4444" }
trafficDistribution: String?

```pkl title="Module-level — whole resource available from 1.29" @K8sVersion { introducedIn = "1.29"; reference = "https://github.com/kubernetes/enhancements/issues/3000" } module flowschema extends "../k8s.pkl" 


```pkl title="Class-level — nested class gated independently"
@SubResourceHint {}
@K8sVersion { introducedIn = "1.32"; reference = "https://kep.k8s.io/2837" }
open class ContainerResizePolicy extends formae.SubResource {
    // ...
}

Runtime resolution (Go side)

Beyond the Config field, the plugin also resolves the runtime version (for client construction + preflight). Priority order:

```go title="pkg/config/version.go" const ( MinSupportedK8sVersion = "1.31" MaxSupportedK8sVersion = "1.36" ClientGoVersion = "0.36.0" // keep in sync with go.mod )

// ResolveK8sVersion: cfg.KubernetesVersion -> FORMAE_K8S_VERSION env -> // live discovery via the cluster's /version endpoint. func ResolveK8sVersion(ctx context.Context, cfg *Config, disc discovery.DiscoveryInterface) (string, error) 


A preflight `CheckField` helper walks the per-version gate metadata and refuses to apply when the user pins a gated field on a cluster that doesn't accept it. Catches the "I added `trafficDistribution` to my Service.pkl but my cluster is 1.29" mistake before any RPC is sent.

### Authoring against versioned schemas

Users in their `forma.pkl` import paths matching the version they're targeting:

```pkl title="examples/nginx-v1.34.pkl"
amends "@formae/forma.pkl"

import "@formae/formae.pkl"
import "@k8s/target.pkl" as k8s
import "@k8s/v1.34/core/Namespace.pkl" as ns
import "@k8s/v1.34/apps/Deployment.pkl" as dep

forma {
  new formae.Target {
    label = "k8s-local"
    config = new k8s.Config {
      kubernetesVersion = "1.34"
      auth = new k8s.KubeconfigAuth {}
    }
  }
  // ...
}

The @k8s/v1.34/... paths are the per-version subtrees in the published package. The package root @k8s/target.pkl carries only Config + Auth (version-agnostic). When the Forma is extracted later, core's ImportsGenerator narrowing makes sure only v1.34-relevant resources show up in the regenerated file.

Pitfalls

Don't name per-version files the same as the package root

If your package root contains a <pkg>.pkl AND your per-version directory contains another <pkg>.pkl, every Forma using both will go through the collision-disambiguation path. The result is correct — core handles it — but the aliases are uglier than they need to be.

The kubernetes plugin ships its root as target.pkl (Config + Auth only), so there is no @k8s/k8s.pkl to collide with @k8s/v1.34/k8s.pkl. Clean aliases without disambiguation. If your plugin can avoid the basename collision the same way, do so. If not — and matching apple/pkl-k8s convention often means using k8s.pkl at both layers — the SDK handles it.

Version directory naming

Directory names like v1.34 contain ., which is forbidden in Pkl identifiers. Core sanitizes non-identifier characters (. → _) so the synthesized alias remains a legal Pkl identifier. The convention to follow: name version directories v<major>.<minor> (matching the API's own version string). Don't name them 1.34/ (bare), 1_34/ (preempting sanitization), or v1_34/ (preempting sanitization at a different layer) — those force your users' imports to look unfamiliar.

Pin the API client library in lockstep

If the plugin uses an upstream client library (client-go for Kubernetes, official SDKs for cloud APIs), the supported API minors in the published schema layout should match the minor your client version actually understands the wire protocol for. The kubernetes plugin pins MaxSupportedK8sVersion to the highest minor client-go understands; drift means client-go can't deserialize a new field even though the schema says it's valid.

Local-only versioning today

Per-version dispatch only fires in SchemaLocationLocal mode. The published hub package today ships the unified schema — no per-version narrowing during the swap step. So:

  • Local plugin development → versioned dispatch + per-version subtrees.
  • Remote (production default) → unified schema, schemaVersions empty, unrestricted glob.

This is intentional. It also means that to test the full version-dispatch path you need a local plugin install (make install) — formae extract against a hub-published plugin won't exercise the narrowing.

Checklist

To add API schema versioning to a plugin:

  • [ ] Add an apiVersion (or similarly-named) field on your Config class so core can read it from extracted Forma targets.
  • [ ] Publish per-version subtrees under schema/pkl/<pkg>/v<X.Y>/ containing only the symbols valid in each minor.
  • [ ] Keep the package root version-agnostic (Config + Auth + version-aware typealiases).
  • [ ] Add a runtime version resolver (in Go) that maps apiVersion → live client + preflight checks.
  • [ ] Wire a CI matrix that runs conformance against every supported version.
  • [ ] Bump the API client library and any Max…Version constant in lockstep when a new minor lands.
  • [ ] Ship example formae files for each supported version so users have a working @<pkg>/v<X.Y>/... reference.