v0.41.1

Turing v0.41.1

Diff since v0.41.0

The ModeResult struct returned by maximum_a_posteriori and maximum_likelihood can now be wrapped in InitFromParams().
This makes it easier to use the parameters in downstream code, e.g. when specifying initial parameters for MCMC sampling.
For example:

@model function f()
    # ...
end
model = f()
opt_result = maximum_a_posteriori(model)
sample(model, NUTS(), 1000; initial_params=InitFromParams(opt_result))

If you need to access the dictionary of parameters, it is stored in opt_result.params but note that this field may change in future breaking releases as that Turing's optimisation interface is slated for overhaul in the near future.

Merged pull requests:

• CompatHelper: add new compat entry for DynamicPPL at version 0.38 for package test, (keep existing compat) (#2701) (@github-actions[bot])
• Move external sampler interface to AbstractMCMC (#2704) (@penelopeysm)
• Skip Mooncake on 1.12 (#2705) (@penelopeysm)
• Test on 1.12 (#2707) (@penelopeysm)
• Include parameter dictionary in optimisation return value (#2710) (@penelopeysm)

Closed issues:

• Better support for AbstractSampler (#2011)
• "failed to find valid initial parameters" without the use of truncated (#2476)
• Unify src/mcmc/Inference.jl methods (#2631)
• Could not run even the sample code for Gaussian Mixture Models or Infinite Mixture Models (#2690)
• type unstable code in Gibbs fails with Enzyme (#2706)

v0.41.0

Turing v0.41.0

Diff since v0.40.5

DynamicPPL 0.38

Turing.jl v0.41 brings with it all the underlying changes in DynamicPPL 0.38.
Please see the DynamicPPL changelog for full details: in this section we only describe the changes that will directly affect end-users of Turing.jl.

Performance

A number of functions such as returned and predict will have substantially better performance in this release.

ProductNamedTupleDistribution

Distributions.ProductNamedTupleDistribution can now be used on the right-hand side of ~ in Turing models.

Initial parameters

Initial parameters for MCMC sampling must now be specified in a different form.
You still need to use the initial_params keyword argument to sample, but the allowed values are different.
For almost all samplers in Turing.jl (except Emcee) this should now be a DynamicPPL.AbstractInitStrategy.

There are three kinds of initialisation strategies provided out of the box with Turing.jl (they are exported so you can use these directly with using Turing):

• InitFromPrior(): Sample from the prior distribution. This is the default for most samplers in Turing.jl (if you don't specify initial_params).

• InitFromUniform(a, b): Sample uniformly from [a, b] in linked space. This is the default for Hamiltonian samplers. If a and b are not specified it defaults to [-2, 2], which preserves the behaviour in previous versions (and mimics that of Stan).

• InitFromParams(p): Explicitly provide a set of initial parameters. Note: p must be either a NamedTuple or an AbstractDict{<:VarName}; it can no longer be a Vector. Parameters must be provided in unlinked space, even if the sampler later performs linking.

  • For this release of Turing.jl, you can also provide a NamedTuple or AbstractDict{<:VarName} and this will be automatically wrapped in InitFromParams for you. This is an intermediate measure for backwards compatibility, and will eventually be removed.

This change is made because Vectors are semantically ambiguous.
It is not clear which element of the vector corresponds to which variable in the model, nor is it clear whether the parameters are in linked or unlinked space.
Previously, both of these would depend on the internal structure of the VarInfo, which is an implementation detail.
In contrast, the behaviour of AbstractDicts and NamedTuples is invariant to the ordering of variables and it is also easier for readers to understand which variable is being set to which value.

If you were previously using varinfo[:] to extract a vector of initial parameters, you can now use Dict(k => varinfo[k] for k in keys(varinfo) to extract a Dict of initial parameters.

For more details about initialisation you can also refer to the main TuringLang docs, and/or the DynamicPPL API docs.

resume_from and loadstate

The resume_from keyword argument to sample is now removed.
Instead of sample(...; resume_from=chain) you can use sample(...; initial_state=loadstate(chain)) which is entirely equivalent.
loadstate is exported from Turing now instead of in DynamicPPL.

Note that loadstate only works for MCMCChains.Chains.
For FlexiChains users please consult the FlexiChains docs directly where this functionality is described in detail.

pointwise_logdensities

pointwise_logdensities(model, chn), pointwise_loglikelihoods(...), and pointwise_prior_logdensities(...) now return an MCMCChains.Chains object if chn is itself an MCMCChains.Chains object.
The old behaviour of returning an OrderedDict is still available: you just need to pass OrderedDict as the third argument, i.e., pointwise_logdensities(model, chn, OrderedDict).

Initial step in MCMC sampling

HMC and NUTS samplers no longer take an extra single step before starting the chain.
This means that if you do not discard any samples at the start, the first sample will be the initial parameters (which may be user-provided).

Note that if the initial sample is included, the corresponding sampler statistics will be missing.
Due to a technical limitation of MCMCChains.jl, this causes all indexing into MCMCChains to return Union{Float64, Missing} or similar.
If you want the old behaviour, you can discard the first sample (e.g. using discard_initial=1).

Merged pull requests:

• [breaking] v0.41 (#2667) (@penelopeysm)
• Compatibility with DynamicPPL 0.38 + InitContext (#2676) (@penelopeysm)
• Remove Sampler, remove InferenceAlgorithm, transfer initialstep, init_strategy, and other functions from DynamicPPL to Turing (#2689) (@penelopeysm)

Closed issues:

• Do we need resume_from now that we have initial_state? (#2171)
• Introduce a docs FAQ section (#2431)
• These tests should be in DynamicPPL or removed (#2475)
• Support for Distributions.ProductNamedTupleDistribution (#2659)
• Overly aggressive concretisation in bundle_samples (#2666)
• broken docs on website (#2698)

v0.40.5

Turing v0.40.5

Diff since v0.40.4

Bump Optimization.jl compatibility to include v5.

Merged pull requests:

• Temporarily change tests to use 1.11 (#2685) (@mhauru)

v0.40.4

Turing v0.40.4

Diff since v0.40.3

Fixes a bug where initial_state was not respected for NUTS if resume_from was not also specified.

Merged pull requests:

• Do not take an initial step before starting the chain in HMC (#2674) (@penelopeysm)
• no coveralls (#2677) (@penelopeysm)
• pass initial_state through for NUTS sampling (#2680) (@penelopeysm)

Closed issues:

• Allow user to disable unnecessary model evaluations after #2202 (#2215)
• Integrating Turing and MarginalLogDensities (#2398)
• Simplify the tilde-pipeline in DynamicPPL (#2422)
• Rethinking Threaded/Multichain Callbacks (#2568)
• Gibbs / dynamic model / PG + ESS reproducibility (#2626)
• Fixing initial_params when using NUTS does not fix the initial parameters (#2673)
• initial_state is not used when resume_from is not specified (#2679)

v0.40.3

Turing v0.40.3

Diff since v0.40.2

This patch makes the resume_from keyword argument work correctly when sampling multiple chains.

In the process this also fixes a method ambiguity caused by a bugfix in DynamicPPL 0.37.2.

This patch means that if you are using RepeatSampler() to sample from a model, and you want to obtain MCMCChains.Chains from it, you need to specify sample(...; chain_type=MCMCChains.Chains).
This only applies if the sampler itself is a RepeatSampler; it doesn't apply if you are using RepeatSampler within another sampler like Gibbs.

Merged pull requests:

• Fix typos in comments and variable names (#2665) (@Copilot)
• Fix multiple-chain method ambiguity (#2670) (@penelopeysm)

Closed issues:

• DynamicPPL documentation is incorrect (#2661)
• Poisson with Dirichlet prior samples negative values (#2663)
• RepeatSampler doesn't use MCMCChains.Chains (#2669)

v0.40.2

Turing v0.40.2

Diff since v0.40.1

sample(model, NUTS(), N; verbose=false) now suppresses the 'initial step size' message.

Merged pull requests:

• Improve error message for initialization failures with troubleshootin… (#2637) (@AoifeHughes)
• Suppress info message with verbose=false (#2657) (@penelopeysm)

Closed issues:

• Option to suppress "Warning" and "Info" statements (#1398)

v0.40.1

Turing v0.40.1

Diff since v0.40.0

Extra release to trigger Documenter.jl build (when 0.40.0 was released GitHub was having an outage). There are no code changes.

Closed issues:

• Broadcasting , addprob and PPL not functionning with PG and SMC sampler (#1996)
• values of logp in chain do not include logabsdetjac term (#2617)
• Skip extra re-evaluation with Prior (#2641)

v0.40.0

Turing v0.40.0

Breaking changes

DynamicPPL 0.37

Turing.jl v0.40 updates DynamicPPL compatibility to 0.37.
The summary of the changes provided here is intended for end-users of Turing.
If you are a package developer, or would otherwise like to understand these changes in-depth, please see the DynamicPPL changelog.

• @submodel is now completely removed; please use to_submodel.

• Prior and likelihood calculations are now completely separated in Turing. Previously, the log-density used to be accumulated in a single field and thus there was no clear way to separate prior and likelihood components.

  • @addlogprob! f, where f is a float, now adds to the likelihood by default.
  • You can instead use @addlogprob! (; logprior=x, loglikelihood=y) to control which log-density component to add to.
  • This means that usage of PriorContext and LikelihoodContext is no longer needed, and these have now been removed.

• The special __context__ variable has been removed. If you still need to access the evaluation context, it is now available as __model__.context.

Log-density in chains

When sampling from a Turing model, the resulting MCMCChains.Chains object now contains not only the log-joint (accessible via chain[:lp]) but also the log-prior and log-likelihood (chain[:logprior] and chain[:loglikelihood] respectively).

These values now correspond to the log density of the sampled variables exactly as per the model definition / user parameterisation and thus will ignore any linking (transformation to unconstrained space).
For example, if the model is @model f() = x ~ LogNormal(), chain[:lp] would always contain the value of logpdf(LogNormal(), x) for each sampled value of x.
Previously these values could be incorrect if linking had occurred: some samplers would return logpdf(Normal(), log(x)) i.e. the log-density with respect to the transformed distribution.

Gibbs sampler

When using Turing's Gibbs sampler, e.g. Gibbs(:x => MH(), :y => HMC(0.1, 20)), the conditioned variables (for example y during the MH step, or x during the HMC step) are treated as true observations.
Thus the log-density associated with them is added to the likelihood.
Previously these would effectively be added to the prior (in the sense that if LikelihoodContext was used they would be ignored).
This is unlikely to affect users but we mention it here to be explicit.
This change only affects the log probabilities as the Gibbs component samplers see them; the resulting chain will include the usual log prior, likelihood, and joint, as described above.

Particle Gibbs

Previously, only 'true' observations (i.e., x ~ dist where x is a model argument or conditioned upon) would trigger resampling of particles.
Specifically, there were two cases where resampling would not be triggered:

• Calls to @addlogprob!
• Gibbs-conditioned variables: e.g. y in Gibbs(:x => PG(20), :y => MH())

Turing 0.40 changes this such that both of the above cause resampling.
(The second case follows from the changes to the Gibbs sampler, see above.)

This release also fixes a bug where, if the model ended with one of these statements, their contribution to the particle weight would be ignored, leading to incorrect results.

The changes above also mean that certain models that previously worked with PG-within-Gibbs may now error.
Specifically this is likely to happen when the dimension of the model is variable.
For example:

@model function f()
    x ~ Bernoulli()
    if x
        y1 ~ Normal()
    else
        y1 ~ Normal()
        y2 ~ Normal()
    end
    # (some likelihood term...)
end
sample(f(), Gibbs(:x => PG(20), (:y1, :y2) => MH()), 100)

This sampler now cannot be used for this model because depending on which branch is taken, the number of observations will be different.
To use PG-within-Gibbs, the number of observations that the PG component sampler sees must be constant.
Thus, for example, this will still work if x, y1, and y2 are grouped together under the PG component sampler.

If you absolutely require the old behaviour, we recommend using Turing.jl v0.39, but also thinking very carefully about what the expected behaviour of the model is, and checking that Turing is sampling from it correctly (note that the behaviour on v0.39 may in general be incorrect because of the fact that Gibbs-conditioned variables did not trigger resampling).
We would also welcome any GitHub issues highlighting such problems.
Our support for dynamic models is incomplete and is liable to undergo further changes.

Other changes

• Sampling using Prior() should now be about twice as fast because we now avoid evaluating the model twice on every iteration.
• Turing.Inference.Transition now has different fields.
  If t isa Turing.Inference.Transition, t.stat is always a NamedTuple, not nothing (if it genuinely has no information then it's an empty NamedTuple).
  Furthermore, t.lp has now been split up into t.logprior and t.loglikelihood (see also 'Log-density in chains' section above).

v0.39.10

Turing v0.39.10

Diff since v0.39.9

Added a compatibility entry for DataStructures v0.19.

Merged pull requests:

• Use accumulators to fix all logp calculations when sampling (#2630) (@penelopeysm)
• CompatHelper: bump compat for DataStructures to 0.19, (keep existing compat) (#2643) (@github-actions[bot])

v0.39.9

Turing v0.39.9

Diff since v0.39.8

Revert a bug introduced in 0.39.5 in the external sampler interface.
For Turing 0.39, external samplers should define

Turing.Inference.getparams(::DynamicPPL.Model, ::MySamplerTransition)

rather than

AbstractMCMC.getparams(::DynamicPPL.Model, ::MySamplerState)

to obtain a vector of parameters from the model.

Note that this may change in future breaking releases.

Merged pull requests:

• DPPL 0.37 compat for particle MCMC (#2625) (@mhauru)
• "Fixes" for PG-in-Gibbs (#2629) (@penelopeysm)
• Fix externalsampler interface (#2640) (@penelopeysm)