# Exact-probability optotagging sensitivity analysis

**Exact sign-flip probabilities are now the main sensitivity analysis. The original calculation is preserved as a separate reference.** This report supersedes the Monte Carlo-based sensitivity conclusions where they differ; it does not replace the historical reference result.

The original **31/995 selected SST neurons at the highest light level** is unchanged. The **13-neuron stable set under per-window correction is also unchanged**. However, the earlier absence of an always-selected set under correction across windows was numerical: exact probabilities reveal **11 always-selected neurons** under that more stringent scope.

## Why the strict corrections were limited

The original implementation uses exact sign-flip probabilities when at most 20 paired trial differences are nonzero, and otherwise uses 99,999 randomizations with the correction (exceedances+1)/(99,999+1). Its Monte Carlo branch therefore cannot return p<0.00001.

There are only 25 trials per SST light level. Exact sign-flip probabilities can be as small as 2^-25 = **0.0000000298023223876953125**, about 336 times smaller than the Monte Carlo floor. We computed exact probabilities for all **11,940 SST unit × level × window tests**. Of these, 110 had used Monte Carlo in the original implementation; 107 of those were at its floor. At the highest level, 60 of 61 Monte Carlo tests were at the floor. These are tests, not distinct neuron counts.

Primary correction across four windows includes 995×4=3980 tests. Holm at alpha=0.01 starts at 0.01/3980 ≈ **0.000002513**, below the original floor. Many strongly responding units could not pass this threshold using the original calculation. Some other units still passed because their probabilities came from its exact branch.

For the highest light level, keeping increase≥10 Hz and response≥2×baseline, strict Holm results are:

| Response window (ms) | Original, per window | Exact, per window | Original, across windows | Exact, across windows |
|---|---:|---:|---:|---:|
| [2,8) | 25 | 25 | 3 | **22** |
| [3,8) | 22 | 22 | 4 | **20** |
| [2,6) | 20 | 20 | 5 | **17** |
| [4,8) | 16 | 17 | 2 | **15** |

All use alpha=0.01. “Original” means the preserved exact/Monte Carlo hybrid, not Monte Carlo for every unit. The small per-window change can reflect Monte Carlo estimation error as well as finite resolution; the large across-window changes demonstrate the floor problem.

## Exact method

For each unit, light level, and window, let d_i be post-minus-pre spike counts on a trial. Zero differences can be removed because either sign gives the same contribution. With k nonzero differences, the null distribution assigns equal weight to the 2^k sign configurations of the magnitudes |d_i|.

We accumulate integer multiplicities of attainable signed sums using dynamic programming, starting with one way to make zero. Each magnitude v adds the current multiplicity to both sums s+v and s−v. The one-sided p-value is the count of configurations with sum **greater than or equal to** the observed sum, divided by 2^k. Ties are included; this is not a mid-p calculation. If all differences are zero, p=1.

This avoids enumerating 2^25 individual configurations. Integer multiplicities preserve exact tail counts, and the denominator is a power of two; for these sample sizes the resulting probabilities are exactly representable in binary floating point. We save numerator, denominator, and nonzero-trial count. No random seed or Monte Carlo +1 correction is used for the exact calculation.

This changes numerical evaluation of the existing null distribution, not its scientific assumptions. Exchangeability/symmetry required by the paired sign-flip test remains an assumption.

## Reference result retained separately

The supplied original code, original full results, and reproduced rate/test/selection stage are retained in the reproduction bundle. Recomputed original rates, p/q values, and selected identities match the supplied reference at every level.

The historical primary rule remains response [2,8) ms versus baseline [-8,-2) ms, BH≤0.05, increase≥10 Hz, and response≥2×baseline. It selects **31 neurons** with either probability method, with identical identities. Original lower-level counts, 3 at level 1.3 and 24 at level 1.7, are likewise unchanged at the reference setting.

Using exact probabilities, the primary BH count at the reference window is **28, 31, 31** at cutoffs 0.01, 0.05, 0.10. All tested effect combinations (5/10/20 Hz and 1.5/2/3-fold) still select 31 at BH=0.05 and the reference window.

Exact counts at cutoff 0.05 and the reference effects are:

| Window (ms) | BH FDR | BY FDR | Holm FWER | Uncorrected |
|---|---:|---:|---:|---:|
| [2,8) | 31 | 28 | 27 | 41 |
| [3,8) | 30 | 27 | 26 | 41 |
| [2,6) | 24 | 24 | 22 | 36 |
| [4,8) | 27 | 23 | 21 | 36 |

This table is unchanged from the earlier calculation. Uncorrected results remain diagnostic and are excluded from the corrected stability sets.

## Do the stable-neuron conclusions change?

Each correction scope includes 324 settings per level: four windows × three correction methods (BH/BY/Holm) × three cutoffs × three absolute effects × three fold requirements.

At the highest SST level:

| Quantity | Original probabilities | Exact probabilities |
|---|---:|---:|
| Per-window corrected count range | 16–32 | **17–32** |
| Always selected, per window | 13 | **13** |
| Changing classification, per window | 20 | **20** |
| Never selected, per window | 962 | **962** |
| Across-window corrected count range | 2–32 | **15–32** |
| Always selected, across windows | 0 | **11** |
| Changing classification, across windows | 33 | **22** |
| Never selected, across windows | 962 | **962** |

The **same 13 neurons** remain selected in every per-window corrected setting:
951083206, 951083231, 951083397, 951083876, 951087429, 951087843, 951087886, 951088523, 951094157, 951094682, 951101812, 951105085, 951105613.

The **11 stable across both correction scopes** are this set excluding **951083231 and 951088523**. Those two are selected in every per-window setting but not every across-window setting even with exact probabilities. Thus their remaining sensitivity is not eliminated by correcting numerical resolution.

The union remains 33 units. In the per-window grid, 18 of the 31 historical reference selections change classification, while units **951083349 and 951083371** are selected only under some alternative settings. Stable/changing/never-selected category identities are unchanged for this scope, although some selection frequencies change. The neuron CSV supplies both old and exact frequencies and classifications at every level.

Lower levels also show numerical effects because their two levels are jointly corrected as the secondary family. Exact corrected count ranges are:

| Light level | Per-window range | Across-window range | Always selected: per window / across windows |
|---|---:|---:|---:|
| 1.3 | 0–4 | 0–4 | 0 / 0 |
| 1.7 | 10–27 | 8–26 | 10 / 8 |
| 2.0 | 17–32 | 15–32 | 13 / 11 |

The earlier level-1.7 lower bound of zero therefore does not survive exact calculation. Across all tested settings, switching to exact probabilities caused gains but no losses relative to the original implementation. This is an observed result here, not a general guarantee about Monte Carlo estimates.

## Fixed analysis choices and limitations

The same source recordings and QC filters are retained: Allen SST session 794812542 and wild-type 767871931; quality=good, amplitude_cutoff<0.1, presence_ratio>0.95, isi_violations<0.5, finite QC values, and valid peak electrode. There are 995 eligible SST and 702 wild-type units.

Windows are [2,8), [3,8), [2,6), and [4,8) ms, with equal-duration baselines ending at −2 ms. Cutoffs are 0.01/0.05/0.10; minimum increases 5/10/20 Hz; fold requirements 1.5/2/3. Fold requirements use multiplication and handle zero baselines without infinite ratios. All windows respect the original artifact masks.

Corrections are computed over all eligible tests before effect filtering. The primary family has 995 tests per window, or 3980 across windows; the pooled secondary family has 1990 per window, or 7960 across windows. Levels remain separately reported. There are 2268 settings including uncorrected diagnostics. No setting is promoted to a new primary analysis based on its outcome.

Coverage remains conditional on acquisition bounds and complete extraction; internal acquisition gaps and continuous unit-level observation are not independently established. Wild-type has 15 trials per level and remains descriptive under the original minimum of 20. It receives no inferential labels. There is one recording per group, and neuron counts are not animal-level replication.

These analyses are exploratory relative to the previously examined data. Selection fractions describe the chosen grid, not probabilities of activation. Exact p-values remove simulation error; they do not establish the biological null assumptions, exclude light artifacts, distinguish direct from indirect network responses, or prove direct optogenetic activation. The early artifact mask also prevents evaluating responses before 2 ms.

## Validation and reproduction

Checks passed for probability mass conservation, inclusive tails and zero cases, brute-force enumeration of small signed-count examples, binomial special cases through 25 trials, and agreement with every test already computed exactly by the original implementation. Input checksums, QC alignment, stimulus isolation, and supplied coverage bounds were checked during reconstruction. Selection counts obeyed expected threshold monotonicity.

The saved **reproduce_exact.py** runs from the original input ZIP and recreates **all final CSV tables and all three final PNG figures**, with no notebook or Skill dependency. It also reproduces the original reference as a separate comparison. The full script was executed in a fresh namespace; exact settings, neuron tables, and all selected-unit sets matched. Final figures were regenerated, geometrically checked, and visually inspected.

Run in a writable output directory:
`python reproduce_exact.py /path/to/optotagging-reference-study.zip`

The reproduction bundle includes the script, preserved original code/results, original-versus-exact settings, exact numerators/denominators and floor flags, sparse membership tables, per-setting gained/lost identities, unchanged QC and trials, wild-type descriptions, validation records, environment versions, figures, and checksums. Sparse membership tables list selected pairs only; unlisted eligible unit/setting pairs are unselected.

Use **exact-settings.csv** and **exact-neuron-stability.csv** for the final sensitivity conclusions. Files prefixed **original-** are historical comparisons. The exact report and outputs supersede the earlier Monte Carlo sensitivity summary, while preserving its reference result.
