bijx.reverse_dkl¶

bijx.reverse_dkl(target_ld, sample_ld)[source]¶

Estimate reverse Kullback-Leibler divergence.

Computes an importance sampling estimate of the reverse KL divergence \(D_{\text{KL}}(q \| p) = \int q(x) \log \frac{q(x)}{p(x)} dx\).

Here, \(q\) is sample_ld and \(p\) is target_ld.

When samples are drawn from q, this gives the reverse KL. If samples are drawn from p, this gives the negative forward KL divergence.

Parameters:

target_ld (Array) – Log likelihood of distribution p (up to constant shift).
sample_ld (Array) – Log likelihood of distribution q (up to constant shift).

Return type:

Array

Returns:

Estimated reverse KL divergence as a scalar.

Note

Input arrays must correspond to the same sample set. The sampling distribution determines which KL direction is being estimated.