Informative interaction control protocol
The six-case informative control is complete, and the media-interaction feature remains experimental. All six cases pass the prespecified diagnostic and recovery gate. The optional larger replication proposal was withdrawn on 22 October 2026; it is not an active prerequisite or next step. No larger study or remote execution infrastructure was started.
This page records the executed design and its close-out decision. The original pre-sampling protocol, source snapshots, fits and independent reviews remain unchanged in the local evidence directory .planning/tv-radio-informative-control-20261022/, including control/protocol-at-launch.md. The numerical design, sampling controls and six-case acceptance criteria below are retained. These controls provide no empirical TV/radio or causal evidence.
Aim and estimand
Estimate the original-target-unit coefficient gamma of the product of radio and TV pre-amplitude exposure bases. Compare fixed generating adstock/saturation shapes with sampled shapes; main-effect amplitudes, intercept, noise scale and signed gamma remain unknown in both stages. Positive, zero and negative gamma truths test sign recovery and a null case.
The earlier two-history ablation recovered positive gamma only when both shapes and amplitudes were fixed. Its eligible synchronised free-amplitude interval included zero. This protocol changes the spend design explicitly instead of treating those results as a software failure or changing their eligibility.
Frozen data-generating mechanism
Use 104 weekly dates starting 1 January 2024, no controls, zero prehistory and no outside-history response tail. Build the complete factorial grid of TV/radio spend levels {0, 0.35, 1}. Shuffle its nine blocks once using seed_for('informative-design-order') from scripts.qualify_media_interaction. Prefix five all-zero weeks and repeat each block for 11 weeks. The realised block order is (TV, radio):
(0, 1), (0, 0), (1, 0.35), (0.35, 0), (0.35, 0.35),
(1, 1), (1, 0), (0, 0.35), (0.35, 1)
Each block is longer than the four-lag carryover window. Both marginal spend maxima equal one. In canonical radio/TV order, normalised trailing geometric alpha is [0.3, 0.6], logistic lambda is [2, 1.5] and original-unit beta is [0.2, 0.3]. The target is 1 + beta_radio*z_radio + beta_tv*z_tv + gamma*z_radio*z_tv + epsilon, with independent Normal(0, 0.035) errors. Use one error vector seeded by seed_for('informative-control-noise', 0) across all three gamma values. These paired histories are not independent replications. The design is frozen without looking at generated outcomes or sampled results.
Construct X/y/truth using the unchanged independent NumPy exposure/response helpers. Native fitting uses the existing training-only maximum scalers. Original-unit gamma equals retained interaction_gamma * target_scale; fixed-shape beta remains sampled with its ordinary prior. The scaled-target prior specification is identical between stages for a given dataset; changing gamma truth changes the target maximum and therefore its original-unit prior scale. This is part of the existing fitting procedure, not a constant-original-prior comparison across signals.
Design admission
Before inference, residualise z_radio*z_tv against the intercept and both main bases. Record its residual variance fraction, residual norm, design rank and standardised condition number, together with the known-shape, known-noise unconstrained likelihood SE 0.035 / norm(residual). This SE is a likelihood diagnostic, not a Bayesian posterior interval or a prediction of the constrained-prior estimator’s power.
Require full four-column design rank and residual interaction variance fraction >=0.20. The frozen schedule gives fraction approximately 0.28821 and conditional SE approximately 0.04400. The original independent and synchronised replicate-zero histories remain descriptive design comparators, with fractions approximately 0.07505 and 0.05400. Their fitted results are historical evidence with different sampling budgets; no new difficult-history fit is admitted here.
Exactly six new attempts
Use gamma truths +0.4, 0 and -0.4, in that order. For each, first fit fixed_shapes, then free_shapes. The former calls the existing ablation factory’s free_amplitudes stage; the latter calls the unchanged fully free nonlinear factory. Use interaction Normal(0, 0.5) in scaled-target units. Other priors remain intercept Normal(0.7,0.3), beta HalfNormal(0.5), sampled alpha Beta(2,2), sampled lambda LogNormal(log(2),0.5), sigma HalfNormal(0.1). Fixed alpha/lambda replace only their own priors.
Every attempt uses the unchanged BENCHMARK_CONTROLS profile: native compiled float64 PyMC NUTS, four chains/cores, 1,000 tuning and 1,000 retained draws per chain, target acceptance 0.95, maximum tree depth 10, jitter+adapt_diag, blas_cores=4, no progress bar. BLAS/OpenMP shell threads are one. Do not use the Python-only test linker. Seed each fit with seed_for('informative-control-fit', gamma, stage, 0); the existing helper uses seed+1 for 100 prior predictive draws and seed+2 for posterior prediction.
Run a 1,800-second admission budget from before the first model build, including new inference, prediction, individual reporting and recovery summaries. Admit a subsequent fit only if remaining time exceeds the longest completed end-to-end attempt. An admitted attempt may overrun; record that overrun. Record errors, saved draws, skips and failures explicitly. No outcome-dependent seed, prior, parametrisation, sampler or schedule retry is permitted.
Metrics and progression gate
Retain native fits before postprocessing, prior/posterior predictive draws, observed-target checks, complete-response checks, original-unit gamma/beta summaries, correlations, whole-history zero-spend removal summaries, exact configs, data/source/runtime hashes and controls. Fixed shapes have no inferred intervals or sampler diagnostics. Use 94% HDIs and the existing eligibility gate on all actual free scalars and derived gamma/removal estimands: finite values/MCSE/R-hat/ESS/BFMI, max R-hat <=1.01, min bulk/tail ESS >=400, min BFMI >=0.3, zero divergences and maximum-depth hits.
The bounded control passes only if all six declared cases complete and pass diagnostics, all gamma HDIs include their generating value, positive/negative cases exclude zero with the correct sign, and both null intervals include zero. Every failed condition is reported separately. These strict case-specific requirements are a progression rule; their failure can occur through finite-sample or Monte Carlo variation and is not automatically an implementation defect. Passing does not establish nominal coverage or power. Ineligible intervals are descriptive only; zero inclusion is inconclusive as to sign rather than evidence of no effect.
A passing gate establishes the stated six-case result. The original protocol allowed preparation of an optional larger study after a pass, without authorising its execution. That follow-up proposal has now been withdrawn; the gate is not a requirement to start further sampling.
Close-out decision
The current implementation and bounded checks are complete. Independent reviews accepted the implementation and retained six-case evidence with no material findings. All six fits pass diagnostics; positive and negative intervals exclude zero with the correct sign, and every interval contains its generating value. Both null intervals contain zero, so their estimated signs are inconclusive. These paired outcomes do not establish nominal interval coverage, detection power or robustness across spend histories.
The withdrawn proposal used 400 noise replications in each of 18 cells to target worst-case Monte Carlo standard error 0.025 for proportions. Its 7,200-fit workload was an optional precision choice, not a requirement for the experimental feature. Its plan and resource estimate remain historical evidence. Neither local nor remote preparation or execution is an active follow-up.
The next practical use is an additive-versus-interaction comparison on genuine client data with justified controls, prior sensitivity and held-out evaluation, as described in the interaction guide. The anonymous teaching aliases do not satisfy that data requirement. Any further simulation should answer a specific unresolved question and have its own agreed scope.
General statistical qualification remains outside the completed work. Bayesian posterior uncertainty and repeated-sampling Monte Carlo uncertainty answer different questions; the latter requires a separately designed study if those claims become necessary. See Morris, White and Crowther (2019) for simulation-study design principles.
Review and execution requirements
Before sampling, independent review must accept this protocol and its implementation; focused computational checks, lint, formatting and configured typing must pass. Freeze the reviewed document and exact implementation/helper/native hashes before launch. Reuse earlier evidence only as immutable historical references. After execution, independently audit raw-draw units, likelihood/response calculations, diagnostics, six-case accounting and progression classification. Keep engineering completion, sampler eligibility, statistical qualification and causal identification distinct.