Skip to content

fix: add missing target/target-repo/allowed-repos to safe output schemas#36636

Merged
pelikhan merged 6 commits into
mainfrom
fix/safe-outputs-target-schema-gaps
Jun 3, 2026
Merged

fix: add missing target/target-repo/allowed-repos to safe output schemas#36636
pelikhan merged 6 commits into
mainfrom
fix/safe-outputs-target-schema-gaps

Conversation

@dsyme
Copy link
Copy Markdown
Collaborator

@dsyme dsyme commented Jun 3, 2026

Summary

Adds missing target, target-repo, and allowed-repos properties to the safe-output JSON schema entries that already support them at the Go and CJS runtime layer. Before this fix, valid cross-repository configurations (e.g. target: "*", target-repo: owner/repo) were rejected at compile time with Unknown properties errors because the schema declared additionalProperties: false but omitted those fields.

Changes

pkg/parser/schemas/main_workflow_schema.json

Added missing schema properties to the following safe output types:

Safe Output Type Added Fields
resolve-pull-request-review-thread target, target-repo, allowed-repos
assign-milestone target, allowed-repos
hide-comment target, allowed-repos
link-sub-issue target, allowed-repos
close-discussion allowed-repos
update-discussion allowed-repos
close-pull-request allowed-repos
mark-pull-request-as-ready-for-review allowed-repos
add-reviewer allowed-repos
assign-to-agent allowed-repos
update-pull-request allowed-repos

Also updated the resolve-pull-request-review-thread description to document cross-repository support.

pkg/parser/schema_safe_outputs_target_test.go (new)

Adds two test functions:

  • TestMainWorkflowSchema_SafeOutputsTargetProperties — 25 sub-tests asserting each affected safe output type accepts target, target-repo, and allowed-repos in the schema.
  • TestMainWorkflowSchema_SafeOutputsRejectsUnknownProperties — 4 sub-tests asserting additionalProperties: false is still enforced (misspelled or unknown keys are rejected).

docs/adr/36636-sync-safe-output-schema-with-target-config.md (new)

Draft ADR documenting the decision to keep the JSON schema as a faithful mirror of SafeOutputTargetConfig, the alternatives considered (relaxing additionalProperties, schema generation from Go structs), and the consequences.

Root Cause

The Go parser (SafeOutputTargetConfig / ParseTargetConfig) and the CJS runtime (repo_helpers.cjs) already accepted target, target-repo, and allowed-repos for cross-repository operations. The JSON schema — validated with additionalProperties: false — was the only layer that rejected them, causing spurious compile-time failures for valid workflow configurations.

Impact

  • No runtime behavior changes; the fix is purely additive to the schema.
  • Existing workflows are unaffected.
  • Valid cross-repository safe-output configurations now compile without errors.

Generated by PR Description Updater for issue #36636 · sonnet46 599.3K ·

The JSON schema for several safe output types was missing properties that
the Go code (SafeOutputTargetConfig) and CJS runtime already supported,
causing 'Unknown properties' validation errors at compile time.

Fixed safe outputs:
- resolve-pull-request-review-thread: added target, target-repo, allowed-repos
- assign-milestone: added target, allowed-repos
- hide-comment: added target, allowed-repos
- link-sub-issue: added target, allowed-repos
- close-discussion: added allowed-repos
- close-pull-request: added allowed-repos
- update-discussion: added allowed-repos
- update-pull-request: added allowed-repos
- mark-pull-request-as-ready-for-review: added allowed-repos
- add-reviewer: added allowed-repos
- assign-to-agent: added allowed-repos

Added comprehensive schema validation test that exercises target,
target-repo, and allowed-repos for all 25 safe output types that embed
SafeOutputTargetConfig, plus negative tests confirming unknown properties
are still rejected.
Copilot AI review requested due to automatic review settings June 3, 2026 12:27
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes JSON schema drift for several safe-outputs configurations by adding missing target, target-repo, and/or allowed-repos properties that the Go workflow compiler/runtime already supports, preventing compile-time "Unknown properties" validation errors for otherwise-valid workflows.

Changes:

  • Extend pkg/parser/schemas/main_workflow_schema.json to include target / target-repo / allowed-repos on the affected safe output schemas (while keeping additionalProperties: false behavior).
  • Add unit tests to ensure safe outputs that support these fields are accepted by schema validation and that unknown fields are still rejected.
Show a summary per file
File Description
pkg/parser/schemas/main_workflow_schema.json Adds missing target-related properties to multiple safe-output schema definitions to align with existing Go/runtime support.
pkg/parser/schema_safe_outputs_target_test.go Adds regression tests ensuring schemas accept target / target-repo / allowed-repos where supported and still reject unknown fields.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 2/2 changed files
  • Comments generated: 2

Comment thread pkg/parser/schema_safe_outputs_target_test.go Outdated
Comment thread pkg/parser/schema_safe_outputs_target_test.go Outdated
dsyme and others added 2 commits June 3, 2026 13:38
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

✅ smoke-ci: safeoutputs CLI comment + comment-memory run (26886641653)

Generated by 🧪 Smoke CI for issue #36636 ·

@pelikhan pelikhan marked this pull request as draft June 3, 2026 14:38
@pelikhan pelikhan marked this pull request as ready for review June 3, 2026 14:38
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

⚠️ PR Code Quality Reviewer failed during code quality review.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

🧪 Test Quality Sentinel completed test quality analysis.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

Design Decision Gate 🏗️ completed the design decision gate check.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

🧪 Test Quality Sentinel Report

Test Quality Score: 80/100 — Excellent

Analyzed 2 test function(s) with 28 table-driven subtests: 2 design tests (behavioral contracts), 0 implementation tests, 0 guideline violations.

📊 Metrics & Test Classification (2 tests analyzed)
Metric Value
New/modified tests analyzed 2 (28 subtests across both table-driven tests)
✅ Design tests (behavioral contracts) 2 (100%)
⚠️ Implementation tests (low value) 0 (0%)
Tests with error/edge cases 2 (100%)
Duplicate test clusters 0
Test inflation detected ⚠️ Yes — 370 test lines vs 99 schema lines (≈3.7:1); justified by multi-output regression coverage
🚨 Coding-guideline violations 0

Test Classification Details

Test File Classification Issues Detected
TestMainWorkflowSchema_SafeOutputsTargetProperties pkg/parser/schema_safe_outputs_target_test.go:14 ✅ Design 24 table rows; verifies target/target-repo/allowed-repos accepted by schema for each safe output type
TestMainWorkflowSchema_SafeOutputsRejectsUnknownProperties pkg/parser/schema_safe_outputs_target_test.go:296 ✅ Design 4 table rows; verifies additionalProperties: false constraint rejects unknown fields

Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 2 tests — unit (//go:build !integration)
  • 🟨 JavaScript (*.test.cjs, *.test.js): 0 tests

Verdict

Check passed. 0% of new tests are implementation tests (threshold: 30%). Both test functions enforce behavioral contracts: schema acceptance of valid fields and schema rejection of unknown fields. The //go:build !integration tag is present. No mock libraries used. Assertion messages are descriptive. The 3.7:1 inflation ratio is accepted because the test covers 24+ safe output types against a JSON schema change with no corresponding production .go file.

📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · sonnet46 860.6K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 80/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). Both tests enforce behavioral contracts with table-driven coverage of happy paths and error cases.

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skills-Based Review 🧠

Applied /diagnose and /tdd — approving. Root cause is properly addressed (schema–implementation drift), fix is additive with low regression risk, and a purpose-built regression test suite accompanies the change.

📋 Key Themes & Highlights

Key Themes

  • Root cause fixed, not just symptom: The target/target-repo/allowed-repos fields were already supported by SafeOutputTargetConfig in Go and the CJS runtime; adding them to the JSON schema is the correct fix.
  • Comprehensive positive coverage: All 25 affected safe output types get regression tests — any future schema drift will be caught immediately.
  • Minor gap in negative tests: The additionalProperties: false guard is only verified for 4 of 25 types (see inline comment).
  • add-comment test slightly under-specified: Only target is tested; target-repo and allowed-repos are schema-supported but not exercised (see inline comment).

Positive Highlights

  • t.Parallel() throughout — fast test execution
  • ✅ Clear comment header explaining the regression scenario and why the test exists
  • ✅ Both positive and negative test functions in the same file — easy to find
  • ✅ Schema changes are surgical and well-described

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · sonnet46 1.4M

{
name: "update-issue with target and target-repo",
safeOutputs: map[string]any{
"update-issue": map[string]any{
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The add-comment test only exercises target, but the schema also defines target-repo and allowed-repos for this type. Extending the test case to include all three fields would give full parity with the other 24 entries and close a small coverage gap.

💡 Suggested extension
{
    name: "add-comment with target, target-repo, and allowed-repos",
    safeOutputs: map[string]any{
        "add-comment": map[string]any{
            "max":           5,
            "target":        "*",
            "target-repo":   "github/github",
            "allowed-repos": []any{"github/docs"},
        },
    },
},

This also serves as a live spec that add-comment supports cross-repo targeting.

}
})
}
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[/tdd] The negative-property test (TestMainWorkflowSchema_SafeOutputsRejectsUnknownProperties) only covers 4 of the 25 types exercised in the positive test. While sampling is pragmatic, the 4 chosen types (resolve-pull-request-review-thread, assign-milestone, hide-comment, link-sub-issue) are a bit arbitrary. Consider adding a few more from distinct schema shapes (e.g. add-comment, push-to-pull-request-branch) to increase confidence that additionalProperties: false holds broadly.

💡 Why this matters

Schema definitions are hand-edited JSON, so it's easy to accidentally omit "additionalProperties": false when adding new property blocks. A broader sample catches that class of drift early.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 3, 2026

🏗️ Design Decision Gate — ADR Required

This PR adds 469 new lines in business-logic directories (pkg/parser/...) but did not have a linked Architecture Decision Record (ADR).

📄 Draft ADR committed: docs/adr/36636-sync-safe-output-schema-with-target-config.md — review and complete it before merging.

🔒 This PR cannot merge until an ADR is linked in the PR body.

📋 What to do next
  1. Review the draft ADR committed to your branch — it was generated from the PR diff and PR description.
  2. Complete the missing sections — confirm the decision rationale, refine the alternatives (especially whether schema auto-generation should become a follow-up), and adjust consequences as needed.
  3. Commit the finalized ADR to docs/adr/ on your branch.
  4. Reference the ADR in this PR body by adding a line such as:

    ADR: ADR-36636: Keep the Safe-Output JSON Schema in Sync with SafeOutputTargetConfig

Once an ADR is linked in the PR body, this gate will re-run and verify the implementation matches the decision.

❓ Why ADRs Matter

This change closes a drift between the JSON schema and the Go/CJS runtime — exactly the kind of "why does the schema look this way?" decision future contributors will want recorded. ADRs create a searchable, permanent record of why the codebase looks the way it does.

📋 Michael Nygard ADR Format Reference

An ADR must contain these four sections to be considered complete:

  • Context — What is the problem? What forces are at play?
  • Decision — What did you decide? Why?
  • Alternatives Considered — What else could have been done?
  • Consequences — What are the trade-offs (positive and negative)?

ADRs are stored in docs/adr/ numbered by PR number (e.g., 36636-...md).

🔒 This gate remains blocking until an ADR is linked in the PR body.

🏗️ ADR gate enforced by Design Decision Gate 🏗️ · opus48 4.1M ·

@pelikhan pelikhan merged commit 863db5b into main Jun 3, 2026
23 checks passed
@pelikhan pelikhan deleted the fix/safe-outputs-target-schema-gaps branch June 3, 2026 14:48
dsyme added a commit that referenced this pull request Jun 3, 2026
…mas (#36636)

* fix: add missing target/target-repo/allowed-repos to safe output schemas

The JSON schema for several safe output types was missing properties that
the Go code (SafeOutputTargetConfig) and CJS runtime already supported,
causing 'Unknown properties' validation errors at compile time.

Fixed safe outputs:
- resolve-pull-request-review-thread: added target, target-repo, allowed-repos
- assign-milestone: added target, allowed-repos
- hide-comment: added target, allowed-repos
- link-sub-issue: added target, allowed-repos
- close-discussion: added allowed-repos
- close-pull-request: added allowed-repos
- update-discussion: added allowed-repos
- update-pull-request: added allowed-repos
- mark-pull-request-as-ready-for-review: added allowed-repos
- add-reviewer: added allowed-repos
- assign-to-agent: added allowed-repos

Added comprehensive schema validation test that exercises target,
target-repo, and allowed-repos for all 25 safe output types that embed
SafeOutputTargetConfig, plus negative tests confirming unknown properties
are still rejected.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* docs(adr): add draft ADR-36636 for safe-output schema sync

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Peli de Halleux <pelikhan@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants