Calibration

Calibration adds external information to the likelihood of a built PanelMMM. It does not overwrite the configured parameter priors. ammm 3.3.2 provides different interfaces for a local saturation-curve measurement and a complete experimental intervention. The distinction determines what the evidence can constrain.

ammm supports these calibration paths:

• lift-test measurements through add_lift_test_measurements(...)
• cost-per-target calibration through add_cost_per_target_calibration(...)
• a complete experimental intervention through add_experiment_evidence(...)

For GeoLift results, start with Experimental evidence. A study estimate is experimental evidence with uncertainty. Its validity still depends on the study design, counterfactual estimator and applicability to the MMM. It is not an error-free parameter value or an automatic prior for a channel.

Choose the interface by its estimand

Named FE, CRE and RE do not admit these calibration methods. The older lift method can index supported low-level dimensioned models; this does not make it available for the named FE/CRE estimator contracts.

What “calibrating priors” means here

Let $D$ denote the MMM observations, $E$ the external evidence, and $\theta$ the joint model parameters. When the two sources are conditionally independent under the stated model, the update is

The external likelihood $L_E$ changes the posterior. The configured $p(\theta)$ remains in the model. Sampling estimates the parameters jointly; it does not set a saturation coefficient equal to the measured lift.

One could instead form a joint evidence-informed prior proportional to $p(\theta)L_E(\theta)$ and update it with independent MMM data. That is a different implementation workflow. ammm does not automatically convert a GeoLift estimate into separate coefficient, adstock and saturation priors. Applying the same study through both an evidence-informed prior and an added likelihood would count it twice.

General rule

Calibration methods operate on a built model, before fitting.

Typical sequence:

mmm.build_model(X, y)

# optional calibration step(s) here

idata = mmm.fit(X, y)

If you try to add calibration before the model graph exists, ammm raises an error.

Lift-test calibration

Use the existing add_lift_test_measurements(...) method when the measurement can be represented as a difference on the modelled saturation curve. Its input and outcome quantities must refer to that same local contrast. A multi-period GeoLift total does not acquire that meaning merely by being named delta_y.

df_lift_test = pd.DataFrame(
    {
        "channel": ["tv", "search"],
        "x": [100.0, 80.0],
        "delta_x": [20.0, 10.0],
        "delta_y": [15.0, 6.0],
        "sigma": [3.0, 2.0],
    }
)

mmm.build_model(X, y)
mmm.add_lift_test_measurements(df_lift_test)

Required columns for lift tests

Lift-test data always needs:

• channel
• x
• delta_x
• delta_y
• sigma

It also needs:

• every configured entry in dims
• any additional coordinate columns required by the calibrated variables

In practice, time-varying media models usually require date, because the time-varying multiplier is indexed by date.

Coordinate values must match the built graph. For date-indexed media, provide the date column expected by the calibration helper, even when the input dataset uses another public date-column name.

The older helper indexes registered saturation parameter variables by name. Fixed numeric saturation parameters that do not create named graph variables can therefore raise a missing-variable error. The new experimental-evidence implementation separately supports existing sampled or fixed response parameters within its admitted profile.

What ammm does

add_lift_test_measurements(...):

1. validates the mapping columns
2. scales the lift-test channel and target values to the model scale
3. maps the rows to the model coordinates
4. adds a likelihood term named lift_measurements by default

If time_varying_media is enabled, ammm includes the media temporal multiplier in the calibrated saturation function automatically.

Mathematical definition of the existing lift method

For row $i$ , let $j(i)$ identify the channel and let the supplied coordinate labels select its other parameter dimensions. Write $c_{x,i}>0$ and $c_{y,i}>0$ for the corresponding stored channel and target scales. The wrapper constructs

It divides both x and delta_x by the channel scale. It divides both delta_y and sigma by the target scale. Supply values in the intended original units; applying this scaling yourself as well would scale them twice. The input coordinates are the values presented to the saturation function. The wrapper applies scaling, but no carryover transformation.

Let $S_j(u;\phi_j)$ be the configured saturation function, including its amplitude parameter. The predicted scaled difference is

Here $q_i=1$ for static media. For supported time-varying media, $q_i$ is the existing media_temporal_latent_multiplier selected using the row’s coordinates, including its date. The date selects a multiplier; it does not define an experiment start, end or carryover window.

For the default logistic saturation,

The lift likelihood therefore depends directly on the selected saturation amplitude and curvature parameters and, when present, the media multiplier. It contains no adstock transformation. Adstock and other parameters can still move indirectly because the joint posterior couples them through the MMM likelihood; that does not make this an experimental adstock likelihood.

The default observation distribution is the following magnitude model:

In shape/rate notation, for $|m_i|>0$ and $s_i>0$ ,

Thus $\mathbb{E}(|z_i|\mid\theta)=|m_i(\theta)|$ and $\operatorname{Var}(|z_i|\mid\theta)=s_i^2$ . Here $b_i$ is the Gamma rate, not the saturation amplitude $\beta_j$ . The supplied sigma is the measurement-distribution standard deviation for this row. It is neither a new parameter-prior scale nor automatically the MMM residual standard deviation. The method does not estimate it or derive it from a confidence interval.

Rows contribute separate likelihood factors. No covariance between repeated, overlapping or otherwise dependent lift measurements is represented.

Sign handling and support

The pre-check is $\delta x_i\,\delta y_i\geq0$ . This admits a spend increase with a positive measured outcome change and a spend reduction with a negative change. It rejects an increase paired with a negative point estimate even when that estimate has substantial uncertainty.

After this check, both the predicted and observed differences are replaced by their absolute values. This is not a general signed experimental observation model. Zero-valued rows can pass the sign check, but a zero predicted mean gives an invalid mean/SD Gamma parameterisation. Zero observations can also produce non-finite boundary log densities. Passing the sign check is insufficient to establish a usable likelihood.

Python permits a different distribution via dist=..., provided that it accepts the supplied name, mu, sigma and observed arguments. The same sign pre-check and absolute-value transformations still apply. In particular, dist=pm.Normal does not turn this older method into the signed GeoLift interface. There is no separate distribution-keyword mapping in this method. Use name=... to change the default lift_measurements node name.

Why a GeoLift total is usually a different quantity

An experiment measures a contrast over units and dates. The existing lift method evaluates two input levels, without a time path. It does not:

• transform the supplied media through adstock or propagate uncertainty in the experimental carryover response;
• reconstruct treatment delivery or counterfactual media;
• sum daily responses over the outcome window, including a measured cooldown;
• infer whether a result is an average ATT, cumulative outcome or percentage;
• check reuse of MMM outcomes in the experimental estimator;
• establish that a local campaign effect transports to another geography.

For static media with no adstock, $T$ periods of a constant input change produce a cumulative contrast of $T c_y[S(u+h)-S(u)]$ , whereas one lift row predicts only $c_y[S(u+h)-S(u)]$ . With varying inputs, nonlinear saturation also means that saturating summed spend is generally different from summing saturated responses. Dividing an experimental total by the number of periods does not resolve carryover or changing spend levels.

Use this method only when the two-point saturation contrast is a justified representation of the evidence. For complete GeoLift intervention histories, use add_experiment_evidence and its explicit signed uncertainty contract.

Lifecycle and retained models

Use a new model instance, then build, attach the lift measurements and fit. The older method does not reject an already populated posterior. Attaching a likelihood after fitting does not recalibrate the existing draws.

The older lift dataframe and calibration action are not reconstructed by PanelMMM.load. A saved posterior can retain results from the calibrated fit, but that does not establish that the loaded graph contains its lift likelihood. Keep the original specification and evidence and rebuild the calibrated graph for subsequent fitting. The complete experimental-evidence interface has a separate, checked study-reconstruction contract.

Cost-per-target calibration

Use add_cost_per_target_calibration(...) when you want soft penalties on channel cost-per-target values.

This implementation averages period-level ratios of spend to original-scale channel contribution, with the contribution denominator clipped below at 1e-12. It adds a quadratic penalty around the supplied target with the supplied sigma. A mean of period-level ratios is generally different from total spend divided by total contribution. This operation does not import an experimental outcome likelihood or preserve a GeoLift study design.

mmm.build_model(X, y)
mmm.add_original_scale_contribution_variable(var=["channel_contribution"])

calibration_data = pd.DataFrame(
    {
        "geo": ["UK", "US"],
        "channel": ["tv", "search"],
        "cost_per_target": [30.0, 45.0],
        "sigma": [2.0, 3.0],
    }
)

mmm.add_cost_per_target_calibration(
    data=X,
    calibration_data=calibration_data,
    name_prefix="cpt_calibration",
)

Required prerequisites

Before you add cost-per-target calibration:

1. build the model
2. add channel_contribution_original_scale

The second step is required because cost-per-target calibration operates against original-scale channel contribution.

Required columns for calibration_data

calibration_data must include:

• channel
• cost_per_target
• sigma
• every configured entry in dims

Requirements for data

The data argument is the spend dataset used to compute calibrated cost per target.

After ammm reshapes it into xarray form, its coordinates must match the built model’s:

• same shape
• same coordinate labels
• same channel list

If the reshaped spend data does not match the model coordinates, ammm raises a validation error instead of silently reordering it.

YAML calibration

The YAML builder supports calibration through a top-level calibration: list. Each step must provide an explicit method plus a params mapping.

Supported YAML calibration methods:

• add_lift_test_measurements
• add_cost_per_target_calibration
• add_experiment_evidence

Example:

original_scale_vars:
  - channel_contribution

calibration:
  - method: add_lift_test_measurements
    params:
      df_lift_test:
        dataframe:
          data:
            channel: ["channel_1", "channel_2"]
            x: [100.0, 80.0]
            delta_x: [20.0, 10.0]
            delta_y: [15.0, 6.0]
            sigma: [3.0, 2.0]

Important YAML constraints:

• calibration steps run after build_model(...)
• original_scale_vars is applied before calibration
• only the supported calibration methods above are available in YAML
• dist is not supported in YAML yet for add_lift_test_measurements
• other calibration actions should be applied through the Python API until they have explicit YAML support

For a GeoLift study, use the separate experiment step:

calibration:
  - method: add_experiment_evidence
    params:
      study:
        geolift_export: studies/paid_social.json

The export path resolves relative to the model YAML. See Experimental evidence for the required manifest, uncertainty and independence declarations. compare_experiment is a Python reporting method, not a YAML calibration step.

Choose the right calibration path

Use the older lift method for a justified two-point saturation measurement with compatible aggregation, signs and uncertainty.

Use experimental evidence for a represented GeoLift intervention with complete media histories. A study with only an original interval can be compared; it cannot calibrate the model without a justified sampling-error model.

Use cost-per-target calibration when you want the fitted channel contribution to stay consistent with observed cost efficiency.

You can use either or both of these older methods, provided the model has been built first. The new add_experiment_evidence(...) path accepts one study and must not be combined with those methods.

Common pitfalls

• Adding calibration before build_model(...)
• Forgetting to add channel_contribution_original_scale before cost-per-target calibration
• Omitting required dims columns from calibration data
• Assuming YAML supports every Python calibration argument; dist does not currently round-trip through YAML
• Passing cumulative experimental lift to a single saturation contrast without establishing the same time and aggregation basis
• Treating a frequentist confidence interval as a parameter distribution, or using the same experiment both to construct a prior and as a likelihood
• Assuming the older lift action is reconstructed from a saved posterior

Next steps

• Read Model Fitting for the fit workflow once the model has been fully specified.
• Read Save and Load if you plan to keep calibrated models on disk.
• Read Experimental evidence for the signed GeoLift likelihood, complete Python workflow, report interpretation and demo.

Implementation references

The equations above describe ammm 3.3.2’s implementation. The relevant sources are the public methods, scaling wrapper, likelihood construction and coordinate/sign checks. They describe software behaviour; they do not establish the causal validity of a supplied measurement.