# sickle: stale products, renamed 2026-08-04

Sickle carries **two complete lineages** of every frame, and which one is live
flips per filter:

| filters | live lineage | superseded lineage |
|---|---|---|
| F187N, F210M | `destreak_o007_crf` | `align_o007_crf` |
| F335M, F470N, F480M | `align_o007_crf` | `destreak_o007_crf` |

The rule is `jwst_gc_pipeline.reduction.destreak_policy.SICKLE_SHORTWAVE_FILTERS`
— sickle's short-wavelength filters destreak, its long-wavelength filters stay on
the plain aligned copy. Nothing on disk said which was which, and cataloging is
pointed at one lineage by a single `--each-suffix`, so the wrong value silently
photometers the other one.

## What was renamed

**2763 files** in F335M, F470N and F480M — the `destreak_o007_crf` lineage and
everything derived from it, plus its `_destreak.fits` Image3 inputs. Dates run
2026-04-24 to 2026-06-16; the live `align` products are 2026-07-27.

The pattern is `{basename}_stale_{YYYYMMDD}{ext}`, the date being the file's own
mtime, e.g.

    jw03958007001_03106_00001_nrcblong_destreak_o007_crf.fits
    -> jw03958007001_03106_00001_nrcblong_destreak_o007_crf_stale_20260430.fits

Every pipeline glob is anchored on a fixed trailing token (`_crf.fits`,
`_daophot_basic.fits`, `_i2d.fits`, `{suffix}.fits`), so inserting `_stale_<date>`
before the extension makes a file invisible to all of them. Verified after the
rename:

```
   24 hits,    0 stale  | cataloging get_filenames (LW, live)
    0 hits,    0 stale  | cataloging get_filenames (LW, stale token)
    0 hits,    0 stale  | Image3 product-crf copy
   24 hits,    0 stale  | overlap gate jw*_crf.fits
    1 hits,    0 stale  | mosaic i2d
```

The four unanchored globs in the tree are all in `scripts/analysis/` — one-off
diagnostics a person runs deliberately, not pipeline steps. They will still see
these files.

## What was NOT renamed, and why

**F187N and F210M are held.** The `SICKLE_SHORTWAVE_FILTERS` rule landed
2026-07-31 (commit `de06c3b`); before it sickle was uniformly `align`, so those
two filters' `align` products (2026-06-15 frames, 2026-07-27/28 catalogs) are the
**last good short-wavelength products**, made under the previous policy.

The only `destreak` products they have are from the 2026-08-04 run that issue
\#270 showed produced crf which were stale copies carrying the old GNS pointing.
So for the short-wavelength half the *live* lineage currently holds corrupt files
and the *superseded* lineage holds the last trustworthy ones. Renaming there
would leave F187N/F210M with nothing good.

Clear the `HOLD` entries in
`scratchpad/sickle_stale_rename.py` once a clean re-reduce has produced real
`destreak` products, then re-run it for those two.

## Also moved earlier

`_stale_product_crf_20260627/` — 264 product-named crf from 2026-06-27 that a
copying step was feeding forward on every run. See that directory's README, issue
\#270 and PR #271.

## Reversing

Renames only; nothing was deleted.

    cd /orange/adamginsburg/jwst/sickle
    for f in F335M F470N F480M; do
        cd $f/pipeline
        for x in *_stale_????????.*; do
            mv "$x" "$(echo "$x" | sed -E 's/_stale_[0-9]{8}(\.)/\1/')"
        done
        cd ../..
    done

Restoring them re-arms the ambiguity this rename exists to remove.