Sensitivity benchmark: the existing observables the model must reproduce

The sensitivity pages forecast what the forward model will measure under stated survey premises. This page asks the prior question, in the spirit of the UniverseMachine programme [Behroozi2019]:

Which already-published measurements must the forward model reproduce before its forecasts mean anything — and, given the signal-to-noise of the data that exist today, which part of the full model can be fit right now?

It has three parts: the benchmark compilation itself (the UniverseMachine set, then the multi-wavelength expansion matched to this pipeline’s observables), the run-today assessment (§ What part of the model can be run today), and the missing-data table — what is absent, which experiment delivers it, and when (§ What is missing, and which experiment delivers it). Every entry cites a published measurement; the reference list with links is on Bibliography.

The data live in the data repository

The machine-readable companion of this page is $HOD_MOD_DATA_DIR/benchmark_observables/ — one JSON per (reference, observable, sample), organised per wavelength and tracer, with full metadata, uncertainties and provenance. See The benchmark data tree below for the layout, the file schema and the operator workflow.

The UniverseMachine precedent 

UniverseMachine [Behroozi2019] fits an empirical galaxy–halo model to a deliberately complete compilation of one-point and two-point statistics (their Table 1; all datasets re-normalised to uniform stellar-population, dust and SFH assumptions, their Appendix C):

The UniverseMachine constraint set.
Observable	Redshift	Data sources (as compiled in [Behroozi2019])
Stellar-mass functions	0 < z < 4	SDSS, PRIMUS [Moustakas2013], UltraVISTA [Muzzin2013], CANDELS, ZFOURGE
Cosmic SFR density	0 < z < 10	UV / 24 μm / radio / Hα / SED compilations
Specific SFRs	0 < z < 8	same surveys as the CSFR compilation
Quenched fractions	0 < z < 3.5	Bauer et al. 2013, [Moustakas2013], [Muzzin2013]
UV luminosity functions	4 < z < 10	[Finkelstein2015], Bouwens et al. 2016
Galaxy autocorrelations (all / SF / quenched)	z ≈ 0; z ≈ 0.5	SDSS (re-measured); PRIMUS (Coil et al. 2017)
Massive × MW-mass cross-correlation	z ≈ 0	SDSS (constrains satellite disruption)
Environmental quenching of centrals	z ≈ 0	SDSS (quenched fraction vs. neighbour density)
UV–M* relations	4 < z < 8	[Song2016] (re-derived SED stacks)
IRX–UV dust relations	4 < z < 7	Bouwens et al. 2016 (ALMA)

Two lessons carry over. First, completeness is the constraint: no single statistic pins the galaxy–halo connection; the compilation does. Second, re-normalisation to shared assumptions (IMF, SPS, dust) is part of the data model — the same discipline the tier-2/3 pages apply through their SED-calibration parameters (ml_nir, luv_norm, …).

The expanded multi-wavelength benchmark set 

The forward model predicts strictly more than the UniverseMachine set: gas, AGN and lensing observables join the stellar ones. For every observable in the tier-1/2/3 vector, the table lists the current best published measurement and its precision — this is the data vector an actual fit (as opposed to a Fisher forecast) would use today.

Benchmark measurements per model observable.
Model observable	Current best measurement	Precision / signal-to-noise today
`wp` per (z, M*) cell	SDSS DR7 per-sample w_p [Zehavi2011]; DESI DR1 small-scale clustering + lensing (arXiv:2512.15962)	few-% per point at z < 0.2; DESI extends to z ≈ 1 (LRG)
`ds` (ΔΣ)	SDSS [Mandelbaum2006], [Leauthaud2017]; DESI×DES/KiDS lensing [Heydenreich2025] [Lange2025]	5–15 % per radial bin per sample
`n_gal` / SMF (grid densities)	GAMA DR4 low-z SMF [Driver2022]; COSMOS2020 to z = 5.5 [Weaver2023]	few-% (wide, low z) to ~0.1 dex (deep, high z)
SF/quenched split (`sfq` sector)	COSMOS2020 quiescent SMFs [Weaver2023]; PRIMUS quenched fractions [Moustakas2013]	~0.05–0.1 dex per (z, M*) bin
sSFR main sequence (`ssfr_ms_*`)	MS compilation 0 < z < 6 [Popesso2023]	~0.1 dex normalisation, ~0.2 dex scatter
`sfrd`	Madau–Dickinson compilation [MadauDickinson2014]; LoTSS radio view	~12 % per Δz shell (the tier-3 noise assumption is current-data)
shear tomography (`cl_kk`)	KiDS-Legacy, 1347 deg² [Wright2025] (+ DES Y3 joint)	S8 to 2.3 %
`cl_kCMB` (CMB-lensing auto)	ACT DR6 [Qu2024] / Planck PR4	43σ (2.3 % amplitude)
`cl_gkCMB`	DESI LRG × ACT DR6 + Planck, 4 tomographic bins [Kim2024]	38–50σ; S8 to 2.7 %
`cl_gy` (galaxy × tSZ)	ACT × BOSS CMASS stacked tSZ + kSZ profiles [Schaan2021] [Amodeo2021]	~10σ-level profile detections per sample
`cl_gX` (galaxy × soft X-ray, bands)	eROSITA × legacy galaxy samples: band-resolved X-ray profiles and scaling relations [Comparat2025]	L_X–M slope to ±0.09 (one broad band + coarse sub-bands)
`cl_XX` (X-ray auto, bands)	no published tomographic soft-band auto at the assumed depth	— (eRASS-depth CXB fluctuation analyses only)
`xlf` (z-resolved AGN XLF)	Chandra/deep-fields compilation 0 < z < 7 [Aird2015]	~0.1 dex at L_X > 10⁴³; faint end (10⁴²) pencil-beam only at z ≈ 1
`wp_agn` / AGN bias	eROSITA eRASS1 AGN clustering [Comparat2023]; local BASS AGN [Powell2022]	bias per L_X bin at ~10 % (broad L_X bins)
AGN sector external pins	M_BH census [KormendyHo2013] [Greene2020]; BASS DR2 ERDF [Ananna2022]	μ_BH to ~0.1 dex (local); ERDF break/slopes measured at z ≈ 0
`qlf_uv` / `qlf_opt`	homogenised type-1 QLF 0 < z < 7.5 [Kulkarni2019]	~0.05–0.1 dex over the bright end
`rlf` (radio LFs)	LoTSS Deep: SF radio LF / L₁₅₀–SFR [Bonato2021], jet-mode AGN LF to z ≈ 2.5 [Kondapally2022]	~0.1 dex per (L, z) bin
`cl_gR` / `cl_RR` (radio maps)	no µJy-depth wide-area maps at 0.95–3 GHz yet (SKA premise)	—
`ilf` / `cl_gI` (IR)	WISE all-sky 3.4/4.6/12 μm imaging [Wright2010] (the tier-3 band choice); SPHEREx spectral maps arriving [Dore2014]	WISE crosses measurable now; spectro-IR from 2026–27
`uvlf`	GALEX local [Wyder2005]; z = 4–8 [Finkelstein2015]; z = 9–16 JWST [Harikane2023]	~0.1–0.2 dex per bin
`half` (Hα LF)	HiZELS z = 0.4–2.2 [Sobral2013]	~0.1 dex per bin
`oiilf`	[OII] LF compilation to z ≈ 1.6 [Comparat2015OII]	~0.1 dex per bin
`himf`	ALFALFA final HIMF [Jones2018ALFALFA]; MeerKAT MIGHTEE-HI [Ponomareva2023]	knee mass ±0.01 (stat), slope ±0.02 — local only
`cl_gHI` (21 cm × galaxies)	CHIME × eBOSS stacking [CHIME2023]	11σ (z ≈ 0.8–1.4, coarse scales)
`cl_HIHI` (21 cm auto)	first CHIME auto-power detection [CHIMEauto2025]	detection-level (foreground-limited)
`ncl` (cluster counts)	eRASS1 abundances, 5259 clusters / 12 791 deg² [Ghirardini2024]	σ8 = 0.88 ± 0.02, S8 = 0.86 ± 0.01 — a live benchmark (and a ~2σ-high S8, i.e. a real test, not a formality)
geometry for (w0, wa, h)	DESI DR2 BAO [DESIDR2]; DR1 full-shape [DESI2024FS]	w0waCDM preferred at 3.1σ (with CMB); σ8 = 0.842 ± 0.034 alone

What part of the model can be run today 

Matching the table against the forecast premises, sector by sector:

Tier-1 (31 parameters, 12 observables) — runnable in full. Every tier-1 observable has a published counterpart at usable S/N: clustering and g-g lensing [Zehavi2011] [Mandelbaum2006], abundances [Driver2022], shear [Wright2025], CMB lensing and its crosses [Qu2024] [Kim2024], tSZ [Schaan2021], X-ray cross [Comparat2025] and XLF [Aird2015]. The attainable cosmology is today’s LSS state of the art — S8 at the 2–3 % level (KiDS 2.3 %, LRG×κ_CMB 2.7 %, eRASS1 1.2 %) — not the 0.1 % of the Stage-IV error model; the structure of the study (degeneracy breaking, baryon calibration) is testable now, the headline precision is not.
Tier-2 galaxy grid — roughly the z < 0.5 half. The premise is a volume-limited M* > 10¹⁰ grid to z = 1 over f_sky = 0.5. Today DESI (~14 000 deg²) with 4MOST now in survey operations covers wide-area spectroscopy to z ≈ 0.5–0.6 for these masses; beyond that the samples are colour-selected (LRG) or deep-pencil-beam (COSMOS). About half the 80 (z, M*) cells — and with COSMOS2020-style deep fields, the SF/Q split [Weaver2023] — can be populated with real data now; the z-evolution slopes lose roughly half their lever arm.
X-ray band spectroscopy — collapses to the 1-band control. eROSITA provides the all-sky broad band and coarse sub-bands at t·A_eff two orders of magnitude short of the Athena premise (F_lim ≈ 2×10⁻¹⁶ erg/s/cm²); the tier-2 --n-bands 1 control run documents exactly what survives: Γ_AGN, f_abs and the ICM-metallicity triple fall back to their priors, kT_norm becomes flat. The gas sector is fittable today at broad-band level [Comparat2025]; its spectroscopic refinement is not.
Shear/CMB-lensing sector — at ~4× the forecast noise. KiDS-Legacy’s 1347 deg² is f_sky ≈ 0.033 against the assumed 0.5, so cosmic-variance errors are ~3.9× the tier-2 assumption; ACT DR6 κκ is already within reach of the assumed S4-like noise floor. Rubin (survey started 30 June 2026) and Euclid DR1 close the shear gap from late 2026.
AGN sector — runnable at z < 1 with today’s pins. XLF [Aird2015], clustering [Comparat2023] [Powell2022], the local M_BH census [KormendyHo2013] [Greene2020] and the BASS ERDF [Ananna2022] exist; the per-L_X-bin completeness to L_X = 10⁴² at z = 1 (the Athena premise) does not — eROSITA is complete there only above ~10⁴³·⁵.
Radio / IR / HI (tier-3) — LFs yes, maps partly. The radio and [OII]/Hα/UV/QLF luminosity functions all have published counterparts ([Bonato2021] [Kondapally2022] [Comparat2015OII] [Sobral2013] [Wyder2005] [Kulkarni2019]), so the SED-calibration parameters are fittable now. Of the map crosses, the IR ones are available today — the tier-3 3.4/4.9/12 μm bands are the WISE bands [Wright2010] — while the 0.95–3 GHz radio maps at µJy depth await the SKA, and the HI sector has only the local HIMF [Jones2018ALFALFA] plus low-S/N 21 cm crosses [CHIME2023] [CHIMEauto2025].
Extended cosmology — geometry yes, growth machinery not yet. DESI DR2 BAO already delivers the (w0, wa) geometry [DESIDR2] and full-shape σ8 [DESI2024FS]; on the model side the freed (w0, wa, Σm_ν) act through the CPL growth ODE only, so a differentiable P(k)-shape upgrade remains the prerequisite for fitting them against these data (see the tier-2 caveats).

Summed up: all 12 tier-1 observables and roughly two thirds of the tier-2/3 data rows have published counterparts today, at per-point errors 3–30× the end-of-decade assumptions; the sectors with no data path today are the multi-band X-ray spectroscopy, the deep radio-map crosses, and tomographic 21 cm.

What is missing, and which experiment delivers it 

Missing data → delivering experiment → expected availability.
Missing ingredient	Experiment / release	When
Volume-limited M* > 10¹⁰ grid to z = 1, wide sky	Euclid DR1 (1900 deg² spectroscopy + photometry, 21 Oct 2026); DESI extended operations; 4MOST surveys (in operations since Q2 2026)	2026 →
Stage-IV shear at f_sky ≈ 0.4–0.5	Rubin LSST (survey started 30 June 2026; DR1 ≈ one year of survey + one year of processing); Euclid wide releases; Roman (launch 30 Aug 2026) for the high-z calibration	2027–2030
S4-like CMB lensing + tSZ depth over half the sky	Simons Observatory full facility (operating since 2025); enhanced LAT upgrade (30 000 extra detectors)	upgrade complete ≈ 2028
Multi-band X-ray spectra at F_lim ≈ 2×10⁻¹⁶, all-sky; AGN completeness L_X > 10⁴² to z = 1	NewAthena (ESA adoption Q1 2027)	launch 2037
µJy radio intensity maps at 0.95–3 GHz	SKA-Mid (science verification 2029; Cycle-0 PI observations 2032); SKA-Low science verification 2027 [Bacon2018]	2029–2032
All-sky spectro-photometric IR maps (102 bands)	SPHEREx (launched Mar 2025; weekly quick releases since Jul 2025; full-depth all-sky spectral maps over the 2-yr mission) [Dore2014]	2026–2027
HIMF beyond z ≈ 0.05 and tomographic 21 cm	MeerKAT/MIGHTEE-HI [Ponomareva2023] deep pointings; SKA-Mid HI surveys [SKA2019]	late 2020s → 2030s
Time-domain M_BH census at scale (breaks the σ_lm triple)	SDSS-V Black Hole Mapper [Kollmeier2017] (ongoing)	ongoing → late 2020s

Literature-grounded extensions 

Following the citation graph of the papers behind this benchmark (each proposal is a published measurement the pipeline does not yet predict, or a published relation it does not yet exploit):

kSZ profiles. ACT × CMASS kSZ measurements exist now [Schaan2021] and were converted into gas thermodynamic profiles in [Amodeo2021] — a momentum-weighted (n_e × v) observable that breaks the density–temperature degeneracy of the X-ray/tSZ pair. The model already carries n_e(r|M); it needs only the linear-theory velocity weight.
FRB dispersion measures. The Macquart relation [Macquart2020] is a published, direct census of the same ionised baryons the f_b(M) sector redistributes — an absolute calibration of Σ n_e that neither X-ray (n_e²) nor tSZ (n_e T) provides.
BAO / full-shape multipoles. With (w0, wa, h) now free, the DESI DR2 BAO distances [DESIDR2] and DR1 full-shape multipoles [DESI2024FS] are the natural geometric data block — contingent on the P(k) upgrade.
Tomographic galaxy × CMB-lensing data. The published 4-bin LRG × κ_CMB data vector [Kim2024] is exactly the model’s cl_gkCMB — a benchmark fit that needs no new machinery at all.
External AGN pins. The local M_BH census [KormendyHo2013] [Greene2020] and the BASS DR2 ERDF [Ananna2022] pin (μ_BH, σ_BH, λ*, δ₁, δ₂) outside the σ_lm kernel — the published route to breaking the tier-2 σ_lm triple, with SDSS-V [Kollmeier2017] extending the census.
Environmental quenching and satellite-disruption statistics. The UniverseMachine appendices publish central-quenching-vs-density and massive×MW cross-correlations [Behroozi2019] that discriminate one-halo conformity and satellite disruption — observables the SF/Q split (sfq sector) could predict with a per-halo environment kernel.
UV–M* + dust relations. The z = 4–8 UV–M* stacks [Song2016] and IRX-type relations directly constrain luv_norm/tau_uv_mslope outside 0 < z < 2 — the published path for extending the grid in redshift, together with the JWST UVLF frontier [Harikane2023].
Jet-mode AGN demographics. The LoTSS LERG luminosity functions and their quiescent/SF host split [Kondapally2022] benchmark the wave-3 radio-loud sector (f_loud0, beta_loud, b_jet) as a function of host type, not just in aggregate.
Cluster-count cosmology as a consistency test. eRASS1 abundances [Ghirardini2024] give S8 = 0.86 ± 0.01 — ~2σ above the lensing probes; the model’s ncl observable with its free L_X–M relation is positioned to test whether scaling-relation freedom absorbs the offset.
21 cm auto-spectrum. The first CHIME auto-power detection [CHIMEauto2025] upgrades cl_HIHI from forecast-only to benchmarkable.

The benchmark data tree 

The compilation above is materialised as a JSON tree in the data repository, $HOD_MOD_DATA_DIR/benchmark_observables/ — 83 files, one per (bibliographic reference, observable, sample):

benchmark_observables/
  README.md                # schema + operator workflow
  index.json               # every file with provenance + extraction flag
  <wavelength>/<tracer>/<RefKey>__<observable>[__<sample>].json

with wavelength ∈ {radio, infrared, optical, uv, xray, microwave, multiwavelength} and tracer ∈ {galaxies, agn, clusters, hi, gas, lensing, cmb_lensing, blackholes}. The tree is generated (and regenerated after any change) by:

python -m hod_mod.scripts.data.make_benchmark_observables \
    --out $HOD_MOD_DATA_DIR/benchmark_observables

File schema (`schema_version` 1)

Field	Content
`wavelength`, `tracer`, `observable`, `sample_id`	position in the taxonomy; `observable` uses the model’s names (`wp`, `ds`, `xlf`, `himf`, …)
`reference`	`key` (matching the entries on Bibliography), `citation`, `arxiv` link, optional `doi` and `note`. For datasets ingested from the curated `data/{paper}` folders the arXiv/DOI come from the folder’s `metadata.json` (authoritative)
`provenance`	`type` (see the classes below), `origin` (file path or formula), optional `extraction_method`, and the `needs_operator_extraction` flag with its `extraction_hint`
`sample`	survey, redshift, selection and the measurement’s own cosmology
`units`	per-column units (h-conventions stated where they differ from the model’s)
`data`	column arrays including uncertainties; `null` marks a masked or non-finite entry

Provenance classes 

observed (45) — real measurements ingested from the curated local sources: the nine benchmark folders under the repo’s data/ (Zehavi/SDSS wp, Guo 2018/2019, Leauthaud 2012 ΔΣ, More 2015 CMASS ×3 samples, van Uitert 2016 ΔΣ, Zacharegkas 2025 DES, Zu & Mandelbaum 2015 per-M*-bin wp/ΔΣ, Lange 2025 DESI DR1 BGS/LRG — see Data Formats for the folder convention), the [Comparat2025] broad-band w(θ) for S1–S7, and the 15×100 eV energy-band w(θ) FITS sets for all seven volume-limited samples (the 16th per-sample FITS, the broad-band sum, is already represented by the broad-band entries).
observed_derived_fit (2) — points evaluated from a published fitting function with published parameters: the [MadauDickinson2014] SFRD (their Eq. 15, Salpeter IMF) and the ALFALFA HIMF Schechter fit [Jones2018ALFALFA]. The binned points behind the fits remain flagged for extraction.
simulated (26) — the forward-model fiducial prediction with the forecast survey noise, dumped from the tier-2 production npz (90 parameters, full grid) or the tier-3 smoke npz (reduced grid — to be refreshed from the tier-3 production npz once that run completes). These are stand-ins: y_err is the Stage-IV noise model, not current-survey errors, and every one names the published table that should replace it (e.g. the [Aird2015] electronic XLF tables, the KiDS-Legacy band powers [Wright2025], the [Kulkarni2019] QLF tables).
placeholder (10) — no local stand-in is meaningful (data outside the model grid such as the z > 4 UV LFs, scalar relations such as the M_BH census, or external catalogues such as the eRASS1 cluster products); reference + description + extraction hint only.

Operator worklist 

38 files carry needs_operator_extraction: true (all simulated and placeholder entries plus the two derived fits). index.json lists them:

python -c "
import json
idx = json.load(open('index.json'))
for k, v in idx.items():
    if v['needs_operator_extraction']:
        print(v['provenance'].ljust(22), k)"

When a published table has been extracted, replace the entry in place (same file name), set provenance.type = 'observed' with the extraction_method, record the source table/figure in provenance.origin, and clear the flag. One curated source is an intentional empty placeholder: van Uitert 2016 measures only lensing, so its wp CSV documents the GAMA clustering alternatives in its header instead of data.

Relation to the other pages 

Sensitivity study: differentiable pipeline, scale cuts and degeneracy breaking establishes where the information lives under flat errors; Stage-IV forecast: expected cosmology from the combined multi-wavelength surveys prices it with Stage-IV noise; Tier-2 forecast: nothing fixed (90 parameters) / Tier-3 forecast: multi-wavelength maps, band LFs, z < 2, M* > 10⁹ free the full parameter vector. This page grounds all four: the same data vector, restricted to the rows published today at their measured S/N, is the fit that can be run now — and the missing-data table is the schedule on which the forecasts become fits.