feat(gcp): auto-merge resources across API versions (#561) #38
Loading…
Reference in a new issue
No description provided.
Delete branch "issue-561-auto-merge-gcp-versions"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
ADDITIONAL_VERSIONS/ADDITIONAL_VERSION_RESOURCE_FILTERwith automatic discovery of all stable (non-alpha/beta) API versions per GCP serviceHow it works
Fetch phase: For every API with multiple stable versions, non-preferred versions are fetched alongside the preferred one and saved as
{name}-{version}.json.Generation phase: Schema files are sorted so preferred versions load first. A seen-set keyed on
service.resourcePathdeduplicates across versions — preferred version resources always win, and older versions only contribute resources that don't exist in the preferred version.Verification
ADDITIONAL_VERSIONSdeno check/deno lint/deno fmtall passTest plan
deno check,deno lint,deno fmtpass on codegen/Closes #561
🤖 Generated with Claude Code
Replace the manual ADDITIONAL_VERSIONS / ADDITIONAL_VERSION_RESOURCE_FILTER pattern with automatic discovery and merging of all stable API versions. The fetch phase now discovers every stable (non-alpha/beta) version per API and saves non-preferred versions as {name}-{version}.json. The generation phase sorts schema files deterministically (preferred before additional) and applies cross-version resource deduplication so preferred-version resources always win. This adds 167 new resources across 22 services (admin, cloudbuild, connectors, dialogflow, drive, gkehub, jobs, monitoring, notebooks, osconfig, run, tpu, and others) that were previously only available in older API versions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>Adversarial Review
Scope:
codegen/gcp/pipeline.tsandcodegen/designs/gcp.md(allmodel/files are auto-generated and excluded per CLAUDE.md).This PR replaces the manually curated
ADDITIONAL_VERSIONS/ADDITIONAL_VERSION_RESOURCE_FILTERmechanism with automatic discovery and merging of all stable (non-alpha/beta) API versions. The cross-version deduplication uses aseenResourceKeysSet with deterministic sort order ensuring preferred versions always win.Medium
codegen/gcp/pipeline.ts:46— RegexADDITIONAL_VERSION_FILENAME_REcan misidentify preferred-version files as additional-version files.The regex
/^(.+)-(v\d+)$/matches any stem ending with-v\d+. If a GCP API'snamefield itself ends with a version-like suffix (hypothetical:my-v2), its preferred version filemy-v2.jsonwould be misidentified as an additional version of APImywith versionv2. This would cause it to be sorted after all non-additional files, so its resources would be processed as "additional" — meaning any resource keys already seen from an unrelatedmy.jsonwould shadow its resources.Breaking example: If the GCP Discovery Directory ever adds an API named
foo-v3, the preferred schemafoo-v3.jsonwould be classified as additional versionv3of APIfoo. Iffoo.jsonalso exists, resources fromfoo-v3that share a key withfoowould be silently dropped.Current risk: No known GCP API name currently ends with
-v\d+, so this is latent. But the auto-discovery nature of this change means new APIs added to the Discovery Directory could trigger it without any code changes.Suggested fix: Cross-reference against the known set of preferred-version filenames. During generation, after collecting
schemaFiles, build aSet<string>of preferred stems (all files that don't match the regex). Then for each regex match, verify thatgroup1(the extracted base name) exists in the preferred set before treating the file as additional. Files whose extracted base doesn't correspond to a known preferred file should be treated as preferred themselves.Low
codegen/gcp/pipeline.ts:439— APIs without apreferredversion get no cross-version merging.When
versions.find((v) => v.preferred)returns undefined, the additional-version loopcontinues, skipping the API entirely.selectBestVersion(used in the main loop) has its own fallback for no-preferred APIs (highest stable version). So these APIs get a single schema file with no cross-version merge — potentially missing resources from other stable versions.This matches the old behavior (which also required explicit listing), so it's not a regression. Noting it as a design choice.
codegen/gcp/pipeline.ts:461— Removed per-fetch log line for additional versions reduces debuggability.The old code logged each additional version fetch:
Fetched additional version: iam v1. The new code only logs a final count. When investigating why a particular API version wasn't fetched, the summaryFetched 47 additional API version(s)is less useful than individual lines. Minor observability loss.Verdict
PASS — The logic is sound. Sort-then-dedup correctly implements preferred-wins semantics, the service filter interaction with additional versions is correct (matching by base service name), and the auto-discovery approach is a clean simplification over the manual curation. The regex misidentification risk (Medium #1) is theoretical given current GCP API naming and doesn't warrant blocking.
Code Review
Blocking Issues
None.
Suggestions
Sorting edge case for non-standard GCP version strings (
codegen/gcp/pipeline.ts:500–509): TheADDITIONAL_VERSION_FILENAME_REregex (/^(.+)-(v\d+)$/) only matches standard version suffixes like-v1,-v2. GCP APIs with non-standard version strings — such asadminwhose additional versions aredatatransfer_v1andreports_v1— produce filenames likeadmin-datatransfer_v1.jsonthat don't match this regex. Those files are classified as non-additional in the sort, and since-(ASCII 45) sorts before.(ASCII 46),admin-datatransfer_v1.jsonends up sorted beforeadmin.json, reversing the intended "preferred version first" order for those APIs.In practice this is harmless today — resource paths across
adminversions are disjoint, soseenResourceKeysdedup causes no conflicts regardless of order. But the invariant "preferred version always wins" silently doesn't hold for any API whose version suffix isn'tv\d+. Worth tightening the regex or the sort comparator to handle the general case before this becomes observable.Design doc section numbering (
codegen/designs/gcp.md): Section §10a ("Idempotent Create") and §13a ("List Factory Method") usea-suffixes that break the sequential numbering. Minor, but worth renumbering to keep the doc easy to navigate.