---
name: harvest-upstream-patterns
description: Harvest generalizable patterns from client repos and propose them as code PRs into a Gemma template/reference repo. Use when refreshing a canonical template (e.g. airflow3-best-practice) with real innovations from client deployments, processing nominated client PRs, or running the scheduled upstream-harvest feedback loop.
argument-hint: "[path-to-template-repo]"
disable-model-invocation: true
---

# Harvest Upstream Patterns

Orchestrate the client→template feedback loop: detect generalizable patterns that exist
in client repos but not (yet) in a canonical template repo, generalize them (strip every
client-specific detail), and open one reviewable code PR per pattern back into the template.

This is the inverse of `gemma-deployment-security`: instead of pushing a fixed checklist
*down* into client repos, it harvests innovation *up* into our references.

It runs two phases, backed by two sub-skills:

1. **`detect-upstream-candidates`** — read-only. Compares each client source against the
   template, collects nominated PRs, applies the denylist + generalizability discriminator,
   scores **cross-client corroboration**, dedups against the ledger, and emits a structured
   candidate list.
2. **`generalize-pattern`** — for one candidate: strips client specifics, applies the
   generalized change to the template files, writes the PR body, and appends a ledger entry.

> **Execution context.** In CI these phases are split across two jobs (detect = producer,
> generalize = matrix → one PR per candidate). Run locally/manually, this skill performs
> them sequentially. The logic is identical; only the orchestration differs.

## Arguments

`$ARGUMENTS` — optional path to the **template repo** root (the harvest target, where PRs
land and where the ledger lives). Defaults to the current working directory.

```
/harvest-upstream-patterns /path/to/airflow3-best-practice
/harvest-upstream-patterns                                    # template = cwd
```

The set of client **source repos** and the run **mode** are provided by the caller
(the CI workflow inputs, or interactively when run locally):

- `sources` — list of `org/repo` client repos to harvest from.
- `mode` — `scan` (diff sources vs template + collect nominated PRs), or `pr` (harvest a
  single nominated client PR, used for the manual override / testing path).
- `ledger_path` — defaults to `.github/harvest-ledger.yml` in the template repo.

## Step 1 — Resolve inputs

- **Template root** (`$TEMPLATE_ROOT`): `$ARGUMENTS` or cwd. Verify it exists and is the
  canonical reference (it is the baseline every candidate is measured against).
- **Sources**: the client repos to harvest from. In CI they are shallow-cloned read-only
  under a working directory; locally, ask for paths or `org/repo` refs.
- **Ledger**: read `$LEDGER_PATH` (default `.github/harvest-ledger.yml`). If it is missing,
  treat the ledger as empty and create it from
  `${CLAUDE_SKILL_DIR}/assets/harvest-ledger-template.yml` when the first entry is written.

Announce the template target, the sources, the mode, and the ledger path before proceeding.

## Step 2 — Detect candidates

Invoke **`detect-upstream-candidates`** with `$TEMPLATE_ROOT`, the sources, the mode, the
denylist, and the current ledger. It returns a list conforming to
`${CLAUDE_SKILL_DIR}/assets/candidate-schema.json`.

Each candidate includes: a stable `id`, the `intent`, `target_files`, `sources[]`,
`corroboration_count`, `confidence` (`single` | `corroborated`), and a `rationale`.

Print the candidate table before applying anything:

```
## Harvest Candidates

| id | confidence | sources | intent |
|---|---|---|---|
| cap-drop-all-services | 🔁 corroborated (2) | example-org#41, another-org#88 | drop all Linux caps on compose services |
| pin-uv-base-image-sha | single | example-org#41 | pin uv base image by digest |
```

**Ordering rule:** corroborated candidates (`corroboration_count >= 2`) come first — they
are the strongest signal (independent convergence across clients). Within each confidence
tier, preserve detection order.

## Step 3 — Generalize and propose, one candidate at a time

For each candidate, in the order above, invoke **`generalize-pattern`** with the candidate
and `$TEMPLATE_ROOT`. That sub-skill:

1. Reads the source snippet(s) and the corresponding template file(s).
2. **Strips every client specific** per
   `${CLAUDE_SKILL_DIR}/references/generalization-contract.md` (names, hosts, values,
   credentials, business logic — replaced with template placeholders).
3. Applies the generalized change to the template files only.
4. Fills `${CLAUDE_SKILL_DIR}/assets/pr-body-template.md` — including the
   **"What was generalized / stripped"** section and, for corroborated candidates, the
   **🔁 Corroborated by N clients** banner listing every source.
5. Appends a `proposed` entry to the ledger.

In CI, each candidate is a separate matrix leg → its own branch `harvest/<id>` and its own
PR. Locally, produce one branch + change set per candidate so they stay independently
reviewable; never bundle unrelated candidates into one change.

## Step 4 — Dedup and ledger discipline

A candidate is **skipped** (not re-proposed) if any holds:

- Its `id` is already in the ledger with status `ported` or `declined`.
- An open PR on branch `harvest/<id>` already exists.

When a candidate is proposed, its ledger entry is written with `status: proposed`,
`confidence`, `corroboration_count`, and the `sources`. Merging the PR is the human's signal
to flip it to `ported`. Closing it unmerged **must** be recorded as `declined` so the
candidate is not re-proposed — this is the job of the close-handler workflow the orchestrating
repo is required to run (specified in `${CLAUDE_PLUGIN_ROOT}/skills/harvest-upstream-patterns/references/required-ci-gates.md`;
the workflow lives in `Gemma-Analytics/.github`, not this plugin). If that workflow is not
yet in place, declines must be recorded in the ledger manually, or declined candidates will
recur every run.

## Step 5 — Summary

After processing all candidates, print:

```
Harvest complete for <template-repo>.

Proposed: N PRs (M corroborated)
Skipped:  K (already in ledger / open PR)
Aborted:  A (could not be generalized cleanly — see reasons below)

Corroborated (strong signal — review first):
1. [id] <intent> — corroborated by <N> independent deployments
2. ...

Single-source:
1. [id] <intent>

Aborted (dropped, not proposed):
1. [id] <intent> — <reason it could not be generalized>
```

Never silently drop a candidate: anything `generalize-pattern` aborted must appear in the
Aborted bucket with its reason, so a human knows it was seen and why it was not proposed.
(Source counts are aggregate only — do not name client repos in this summary; precise refs
live in the ledger.)

## Rules

- **Never auto-merge.** Every harvested change is a PR for human review.
- **Never harvest denylisted content** (`.env*`, `*secret*`, credentials, keys, `data/`,
  `dags/` business logic, `seeds/`). See the generalization contract for the full list.
- **Never copy a client specific verbatim** into the template. If a value cannot be
  generalized to a placeholder, the candidate is dropped, not proposed with the specific.
- **Corroboration changes the bar, not the rigor.** A corroborated candidate may be
  proposed even if borderline-general, but it is still fully generalized and still
  human-reviewed — corroboration lowers the propose threshold, never the stripping standard.
- One PR per candidate. Independent review, independent merge, clean ledger mapping.
- The template repo is the baseline. A delta is only a candidate if the template does not
  already do it (or does it worse).