**SR-MMP activity from molecular structure — Tox21 assessment**

**Result.** A regularized logistic model using molecular fingerprints provides useful but imperfect ranking of SR-MMP activity in a scaffold-held-out part of the public Tox21 benchmark. ROC AUC is **0.797** (95% scaffold-bootstrap interval **0.734–0.849**), and average precision is **0.516** (**0.393–0.626**), against a test active fraction of **0.178**. Separate sigmoid calibration improves held-out probability accuracy. These results support prioritizing compounds for further SR-MMP assay work within the studied chemical setting; they do not establish universal chemical coverage or predict human liver toxicity.

This is an assessment of the benchmark's **binary in-vitro mitochondrial membrane potential (MMP) assay label**. It is not a reconstruction of concentration–response curves, a mechanistic diagnosis, or an in-vivo safety model. The binary table cannot resolve potency, exposure dependence, assay interference, or the reason for an individual activity call.

**Public evidence and data accounting.** The source is the public Tox21 CSV distributed by DeepChem/MoleculeNet, retrieved on 28 September 2026. The original NIH challenge data page explicitly defines 1 as active and 0 as inactive and links MMP to PubChem AID 720637. The challenge About page expands MMP as mitochondrial membrane potential. The source has 7,831 rows and 12 assay columns; only structure and SR-MMP labels enter this model. Other assay labels are not predictors.

Original SR-MMP counts are 918 active, 4,892 inactive, and 2,021 missing. Missing labels remain unknown, not inactive. The source CSV SHA-256 is 45d09792492ce049039dd24aa27b07fc79ce20c573187d4d90bcd178c0c0d360. Downloaded source bytes, documentation snapshots, URLs, UTC retrieval times, and checksums are included in the bundle. Primary concentration–response measurements and assay protocol details were not independently re-curated.

| Final row disposition | Source rows |
|---|---:|
| Included unique labeled representative | 5,395 |
| Missing SR-MMP label | 2,021 |
| Consistent duplicate collapsed | 161 |
| More than one carbon-containing fragment | 144 |
| No carbon-containing parent | 64 |
| Unsupported element in retained parent | 20 |
| Conflicting standardized-identity labels | 20 |
| Invalid SMILES | 6 |
| Total | 7,831 |

Missing label takes precedence in this disposition table; a separate structure-status field retains the chemistry audit. Conflicting labels occurred in nine standardized identity groups. The retained set has 854 actives and 4,541 inactives. Full row-level exclusions, conflicts, original and standardized SMILES, identifiers, and source-row mappings are provided.

**Structure processing and leakage controls.** RDKit cleanup is followed by selection of the sole carbon-containing fragment, charge neutralization where possible, canonical tautomer selection, and removal of stereochemistry for modeling identity. Unambiguous non-carbon counterions can be removed. Multi-carbon-fragment records are excluded conservatively, including some potentially legitimate salts or combinations, rather than guessing which component explains activity. Retained parent atoms are restricted to H, B, C, N, O, F, Si, P, S, Cl, Se, Br, and I. Permanent charge can remain.

Identical modeling identities with consistent labels are collapsed to the earliest source row; conflicting labeled groups are excluded. This intentionally pools stereoisomers and canonical tautomers, reducing leakage while losing distinctions that may matter biologically. RDKit defaults, including tautomer-enumeration limits, apply.

Non-chiral Bemis–Murcko scaffolds define groups. All acyclic structures share one group. A single seeded five-fold StratifiedGroupKFold assignment uses folds 0–2 for training, fold 3 for calibration, and fold 4 for final testing. No split was selected by model performance. Identity and scaffold sets are disjoint across all three partitions.

| Partition | Compounds | Active | Inactive | Scaffold groups | Active fraction |
|---|---:|---:|---:|---:|---:|
| Training | 3,508 | 518 | 2,990 | 602 | 14.77% |
| Calibration | 942 | 168 | 774 | 505 | 17.83% |
| Test | 945 | 168 | 777 | 505 | 17.78% |

The target 60/20/20 allocation becomes approximately 65.0/17.5/17.5 because large groups remain intact. The 1,313 acyclic compounds and 1,119 compounds with the benzene-only scaffold are both in training. The final test therefore does not directly validate performance within those two common groups. This is a new scaffold split of the benchmark, not the official Tox21 challenge final evaluation.

**Model and development decisions.** Inputs are 2,048-bit Morgan fingerprints of radius 2, without chirality. The classifier is L2-regularized logistic regression with no class weighting or resampling. Four C values (0.01, 0.1, 1, 10) were compared using four scaffold-grouped folds exclusively within training. Selection used mean fold average precision, with smaller C breaking ties. C = 1 won (mean AP 0.423; fold SD 0.040). All fits converged without warnings. Fingerprints are deterministic; no learned preprocessing sees calibration or test data.

The model fit on all training compounds supplies decision scores to a separate sigmoid calibrator fit only on calibration compounds. Its intercept is −0.721927 and slope is 0.439902. The positive slope preserves ranking while reducing extreme probabilities. Test predictions were not used to select the model, fit the calibrator, choose the operating threshold, or define the coverage screen.

**Discrimination and probability accuracy.** Average precision uses sklearn's non-interpolated definition, not trapezoidal PR area. The prevalence baseline predicts the training active fraction (0.147662) for every test compound. Its test AP equals the test active fraction; its probability scores also reflect the training-to-test prevalence shift.

| Test metric | Prevalence baseline | Raw model | Calibrated model |
|---|---:|---:|---:|
| Average precision | 0.1778 | 0.5159 | 0.5159 |
| ROC AUC | 0.5000 | 0.7972 | 0.7972 |
| Brier score, lower is better | 0.14708 | 0.12964 | 0.11528 |
| Log loss, lower is better | 0.47143 | 0.44437 | 0.37503 |
| ECE, ten fixed-width bins | 0.03012 | 0.08099 | 0.02379 |
| Mean predicted probability | 0.14766 | 0.20850 | 0.18572 |
| Observed active fraction | 0.17778 | 0.17778 | 0.17778 |

For calibrated probabilities, the Brier interval is 0.09404–0.13936 and the log-loss interval is 0.31807–0.43664. Paired bootstrap differences (calibrated minus raw) are −0.01436 for Brier (−0.02542 to −0.00414) and −0.06934 for log loss (−0.11137 to −0.03146). Both favor calibration. Relative to the prevalence baseline, calibrated Brier improves by 0.03180 and log loss by 0.09640.

The raw model is overconfident on the test set. Evaluation-only logistic calibration diagnostics give raw slope 0.452 and intercept −0.775, versus calibrated slope 1.029 and intercept −0.033 (ideal values 1 and 0). These test-fitted diagnostic coefficients were not applied to predictions.

Calibration is improved, not established as perfect or portable. The calibrated 0.1–0.2 bin has mean prediction 0.143 but observed activity 25/255 = 0.098. Only 29 test compounds receive calibrated probability at least 0.6. Extreme-probability reliability is weakly supported. ECE depends on binning and sample size and is not used alone as evidence of calibration.

Intervals use 1,000 seeded percentile bootstrap replicates of the 505 test scaffold groups, retaining all compounds within sampled groups. Whole-test metrics had 1,000 valid replicates. These are compound-weighted metrics with scaffold-resampled uncertainty; replicate sizes can vary. The model, calibrator, threshold and split remain fixed. Intervals do not include training-set selection, refitting, alternative partitions, or external-library shift. Full bootstrap tables and invalid-resample counts are included.

**Operating thresholds.** Probability 0.5 is not automatically an appropriate screening threshold. The alternative threshold, 0.183040, maximizes F1 on calibration data only; it is illustrative, not a utility-optimized decision rule.

| Calibrated threshold | TP | FP | FN | TN | Precision | Recall | Specificity |
|---|---:|---:|---:|---:|---:|---:|---:|
| 0.5 | 40 | 24 | 128 | 753 | 0.625 | 0.238 | 0.969 |
| 0.183040, development-selected | 125 | 219 | 43 | 558 | 0.363 | 0.744 | 0.718 |

The lower threshold retrieves more actives at the cost of more inactive follow-ups. A practical threshold needs the intended screen's false-positive/false-negative costs and prevalence, neither of which this study optimizes.

**Chemical coverage.** Each calibration and test compound was compared only against training fingerprints. Test median maximum training Tanimoto similarity is 0.40 (10th percentile 0.249; 90th percentile 0.594). Similarity describes representation support, not a validated probability of correctness.

| Maximum training Tanimoto | Test n | Active | Active fraction | Calibrated AP (95% interval) | Brier |
|---|---:|---:|---:|---:|---:|
| <0.3 | 211 | 19 | 9.0% | 0.549 (0.190–0.785) | 0.0652 |
| 0.3 to <0.5 | 479 | 87 | 18.2% | 0.437 (0.280–0.582) | 0.1270 |
| 0.5 to <0.7 | 227 | 60 | 26.4% | 0.674 (0.524–0.800) | 0.1476 |
| ≥0.7 | 28 | 2 | 7.1% | Not reported: too sparse | 0.0304 |

Ranking estimates require at least 50 compounds, 10 actives, 10 inactives, and 10 scaffolds. Higher similarity does not monotonically improve every metric. AP and Brier depend on activity prevalence; low Brier in a low-prevalence band does not establish that distant chemistry is more predictable.

A prespecified heuristic screen requires similarity at least the calibration-set 10th percentile (0.250243) and all descriptors within their training 1st–99th percentile envelopes:

| Descriptor | Training central 98% envelope |
|---|---:|
| Molecular weight (g/mol) | 71.15–800.93 |
| Calculated logP | −2.01–7.72 |
| Topological polar surface area (Å²) | 0–206.04 |
| H-bond donors | 0–6 |
| H-bond acceptors | 0–13 |
| Rotatable bonds | 0–21 |
| Ring count | 0–6 |
| Heavy atoms | 4–55 |

The screen marks 773/945 as supported and flags 172/945 for review. “Supported” is only the name of this heuristic flag, not a certificate of validity. The flagged subset has just 10 actives and a wide AP interval (0.168–0.807), and its Brier score is lower than the supported subset's. This experiment therefore does **not** validate the screen as a way to remove high-error predictions. No flagged compounds were removed from primary results.

Six test structures have identical bit fingerprints to a training structure despite different identities and scaffolds. They are ring-size variants with the same unfolded local feature presence but different feature counts, exposing a limitation of radius-2 presence/absence fingerprints. All six are inactive. Excluding them in a disclosed sensitivity analysis changes AP from 0.51586 to 0.51619 and AUC from 0.79719 to 0.79658; the primary conclusion is unchanged. The model cannot distinguish every chemical difference in the original structures, including stereochemical differences removed during preparation.

The defensible scope is standardized, single-carbon-parent compounds resembling the retained Tox21 collection under the same assay-label context. Excluded mixtures, non-carbon compounds and unsupported-element parents lack model evidence. Extreme properties, unfamiliar chemistry, altered prevalence, or different assay conditions need further validation. Even within retained chemistry, the test supports unseen scaffolds from this collection, not all future compounds. External validation should sample the intended deployment library and use matching assay conditions.

**Figures and full evidence.** The discrimination figure shows PR and ROC curves. The calibration figure shows active rates in ten fixed-width bins with scaffold-bootstrap intervals and counts; sparse bins (fewer than 20 compounds or 10 scaffolds) are marked and their intervals omitted from the plot. The coverage figure shows similarity distributions and stratified performance. The illustrative compound grid contains three most-confident examples in each outcome category at the development threshold. These deliberately selected examples are neither a representative random sample nor mechanistic explanations.

Full tables include row audit, exclusions by label, conflicts, outer split and training CV assignments, model selection, out-of-fold predictions, calibration threshold selection, all held-out probabilities, nearest training structures, descriptors, bootstrap results, reliability bins, curves, coverage strata and fingerprint-overlap sensitivity. Compound IDs are hashes of modeling identity; source_row is zero-based. Source molecule IDs and SMILES remain traceable.

**Reproduction.** The bundle contains offline Python code, frozen source bytes, source/software provenance, configuration, coefficient tables, figures, and full tables. Recorded versions are Python 3.12.3, RDKit 2026.03.5, scikit-learn 1.9.0, pandas 3.0.5, NumPy 2.4.6, SciPy 1.17.0, and Matplotlib 3.8.4. The existing local runtime was used without installing packages or modifying the application. README.md describes execution and replay validation. The code verifies input checksums, recreates the grouped split, fits and evaluates the model, and checks predictions and metrics against frozen outputs.

**Verified source references**

1. Public Tox21 CSV: https://deepchemdata.s3-us-west-1.amazonaws.com/datasets/tox21.csv.gz
2. DeepChem loader documentation, captured from a mutable master URL: https://raw.githubusercontent.com/deepchem/deepchem/master/deepchem/molnet/load_function/tox21_datasets.py
3. NIH challenge data and label definitions: https://tripod.nih.gov/tox21/challenge/data.jsp
4. NIH endpoint definitions: https://tripod.nih.gov/tox21/challenge/about.jsp
5. PubChem assay 720637, linked by source 3: https://pubchem.ncbi.nlm.nih.gov/bioassay/720637 — the linkage is verified; this assay record itself was not downloaded through the available connector.

All conclusions concern **SR-MMP in-vitro activity**, not human liver toxicity, clinical injury, or overall compound safety.
