Irreversible samplers from jump and continuous Markov processes

Abstract

In this paper, we propose irreversible versions of the Metropolis–Hastings (MH) and Metropolis-adjusted Langevin algorithm (MALA) with a main focus on the latter. For the former, we show how one can simply switch between different proposal and acceptance distributions upon rejection to obtain an irreversible jump sampler (I-Jump). The resulting algorithm has a simple implementation akin to MH, but with the demonstrated benefits of irreversibility. We then show how the previously proposed MALA method can also be extended to exploit irreversible stochastic dynamics as proposal distributions in the I-Jump sampler. Our experiments explore how irreversibility can increase the efficiency of the samplers in different situations.

Appendices

Samplers from continuous Markov processes

1.1 Proof of Theorem 1 (existence of $\mathbf {Q}(\mathbf {z})$ in (4))

Proof of Theorem 1 is comprised of two sets of ideas appearing in different fields. Here, we write the proof in two steps accordingly.

• 1. Plug the stationary solution $\pi (\mathbf {z})$ into (2), and observe that

  $$\begin{aligned} \sum _i \dfrac{\partial }{\partial \mathbf {z}_i} \left\{ \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big ) - \mathbf {f}_i\pi (\mathbf {z}) \right\} = 0. \end{aligned}$$

• 2. Constructively prove that if entries of

  $$\begin{aligned} \left\{ \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big ) - \mathbf {f}_i\pi (\mathbf {z}) \right\} \end{aligned}$$

  belong to $L^1(\mathbb {R}^d)$, then there exists a matrix $\mathbf {Q}(\mathbf {z})$ with entries in $W^{1,1}(\pi )$, such that $\sum _{j} \dfrac{\partial }{\partial \mathbf {z}_j} \Big ( \mathbf {Q}_{ij}(\mathbf {z}) \pi (\mathbf {z}) \Big ) = \mathbf {f}_i\pi (\mathbf {z}) - \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big )$.

Here we denote $L^1(\mathbb {R}^d)$ as the space of Lebesgue integrable functions on $\mathbb {R}^d$, and $W^{1,1}(\pi )$ as the Sobolev space of functions with weak derivatives and function values integrable with respect to the density $\pi$ times the Lebesgue measure. Step 1 can be found in the literature of continuous Markov processes (Qian et al. 2002; Dembo and Deuschel 2010; Qian 2013; Deuschel and Stroock 2001; Villani 2009; Pavliotis 2014). Step 2 has been found in earlier works on stochastic models in fluid dynamics and homogenization (Komorowski et al. 2012).

Step 1 is accomplished by noting that the un-normalized density function $\pi (\mathbf {z})\propto p^s(\mathbf {z})$ is also a stationary solution of (2). Hence,

$$\begin{aligned} \sum _i \dfrac{\partial }{\partial \mathbf {z}_i} \left\{ \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big ) - \mathbf {f}_i\pi (\mathbf {z}) \right\} = 0. \end{aligned}$$

(39)

Step 2 provides a possible form of $\mathbf {Q}$ in terms of $\mathbf {D}$, $\mathbf {f}$ and $\pi$. First, compare the right-hand sides of (2) and (4) and observe that they are equivalent if and only if there exists $\mathbf {Q}(\mathbf {z})$ such that:

$$\begin{aligned} \sum _{j} \dfrac{\partial }{\partial \mathbf {z}_j} \mathbf {Q}_{ij}(\mathbf {z}) \pi (\mathbf {z}) = \mathbf {f}_i\pi (\mathbf {z}) - \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big ). \end{aligned}$$

(40)

Since entries of $\left\{ \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big ) - \mathbf {f}_i\pi (\mathbf {z}) \right\}$ belong to $L^1(\mathbb {R}^d)$, Fourier transform of it exists:

$$\begin{aligned} \hat{{\mathbf {F}}}_i({\mathbf {k}}) =&\int _{\mathbb {R}^{d}} \text {d}\mathbf {z}\left( {\mathbf {f}}_i(\mathbf {z}) \pi (\mathbf {z}) - \sum \limits _{j} \dfrac{\partial }{\partial \mathbf {z}_j} \Big ({\mathbf {D}}_{ij}(\mathbf {z}) \pi (\mathbf {z})\Big )\right) \\&\quad e^{-2\pi \mathrm{i}\ {\mathbf {k}}^T \mathbf {z}}. \end{aligned}$$

Therefore, we can take $\mathbf {Q}(\mathbf {z})$ such that

$$\begin{aligned} {\mathbf {Q}}_{ij}(\mathbf {z}) \pi (\mathbf {z}) = {\displaystyle \int _{\mathbb {R}^{d}} \dfrac{{\mathbf {k}}_j\hat{{\mathbf {F}}}_i({\mathbf {k}})-{\mathbf {k}}_i\hat{{\mathbf {F}}}_j({\mathbf {k}})}{(2\pi \mathrm{i})\cdot \sum \limits _l {\mathbf {k}}_l^2} e^{2\pi \mathrm{i} \sum \limits _l {\mathbf {k}}_l \mathbf {x}_l} \text {d}{\mathbf {k}} }, \end{aligned}$$

(41)

where entries of $\mathbf {Q}(\mathbf {z})$ belong to $W^{1,1}(\pi )$.

Remark 1

It is worth noting that the condition of

$$\begin{aligned} \left\{ \sum _j\dfrac{\partial }{\partial \mathbf {z}_j}\Big (\mathbf {D}_{ij}(\mathbf {z})\pi (\mathbf {z}) \Big ) - \mathbf {f}_i\pi (\mathbf {z}) \right\} \in L^1(\mathbb {R}^d) \end{aligned}$$

can be rewritten for the vector field related to the SDE as: $\left\{ \mathbf {f}(\mathbf {z}) + \mathbf {D}(\mathbf {z})\nabla H(\mathbf {z}) - \varGamma ^{\mathbf {D}}(\mathbf {z}) \right\} \in L^1(\pi )$, where $\varGamma ^{\mathbf {D}}_i(\mathbf {z}) = \sum _{j=1}^d \dfrac{\partial }{\partial \mathbf {z}_j} {\mathbf {D}}_{ij}(\mathbf {z})$. We see that the condition is relatively mild for the purpose of constructing samplers.

1.2 Reversible and irreversible continuous dynamics for sampling

As has been previously noted (Qian et al. 2002; Dembo and Deuschel 2010; Qian 2013; Deuschel and Stroock 2001; Villani 2009; Pavliotis 2014), the stochastic dynamics of Eq. (4) and the corresponding (3) can be decomposed into a reversible Markov process and an irreversible process. Formally, this can be elucidated by the infinitesimal generator $\mathcal {G}[\cdot ]$ of the stochastic process (3) (see Pavliotis 2014; Komorowski et al. 2012 for more background). For ease of derivation, we work with the Hilbert space $L^2(\pi )$ of square integrable functions with respect to $\pi (\mathbf {z})$, equipped with inner product $<\phi , \varphi >_\pi = \int _{\mathbb {R}^{d}} \bar{\phi }(\mathbf {z}) \varphi (\mathbf {z}) \pi (\mathbf {z}) \text {d}\mathbf {z}$. For $\phi (\mathbf {z})\in W^{2,2}(\pi )$ (i.e., $\phi$ and its second-order weak derivatives belong to $L^2(\pi )$),

$$\begin{aligned} \mathcal {G} [\phi (\mathbf {z})] =&\nabla ^T \Big ( \big (\mathbf {D}(\mathbf {z}) - \mathbf {Q}(\mathbf {z})\big ) \nabla \phi (\mathbf {z}) \Big ) \nonumber \\&- \nabla H(\mathbf {z})^T \big (\mathbf {D}(\mathbf {z}) - \mathbf {Q}(\mathbf {z})\big ) \nabla \phi (\mathbf {z}). \end{aligned}$$

(42)

Then adjoint of $\mathcal {G}$ in $L^2(\pi )$ is:

$$\begin{aligned} \mathcal {G}^* [\phi (\mathbf {z})]&= \nabla ^T \Big ( \big (\mathbf {D}(\mathbf {z}) + \mathbf {Q}(\mathbf {z})\big ) \nabla \phi (\mathbf {z}) \Big ) \nonumber \\&\quad - \nabla H(\mathbf {z})^T \big (\mathbf {D}(\mathbf {z}) + \mathbf {Q}(\mathbf {z})\big ) \nabla \phi (\mathbf {z}). \end{aligned}$$

(43)

Therefore, $\mathcal {G}$ decomposes into a self-adjoint part $\mathcal {G}^S [\phi (\mathbf {z})] = \nabla ^T \Big ( \mathbf {D}(\mathbf {z}) \nabla \phi (\mathbf {z}) \Big ) - \nabla H(\mathbf {z})^T \mathbf {D}(\mathbf {z}) \nabla \phi (\mathbf {z})$ and an anti-self-adjoint part $\mathcal {G}^A [\phi (\mathbf {z})] = \nabla ^T \Big ( \mathbf {Q}(\mathbf {z}) \nabla \phi (\mathbf {z}) \Big ) - \nabla H(\mathbf {z})^T \mathbf {Q}(\mathbf {z}) \nabla \phi (\mathbf {z})$. The self-adjoint operator $\mathcal {G}^S$ corresponds to the reversible Markov process, while the anti-self-adjoint operator $\mathcal {G}^A$ corresponds to the irreversible process.

It can be seen that the reversible process is determined solely by the diffusion matrix $\mathbf {D}$, where evolution of its probability density function is:

$$\begin{aligned}&\dfrac{\partial }{\partial t} p(\mathbf {z};t)\nonumber \\&\quad = \nabla ^T \cdot \bigg ( \mathbf {D}(\mathbf {z}) \left[ p(\mathbf {z};t) \nabla H(\mathbf {z}) + \nabla p(\mathbf {z};t) \right] \bigg ) \nonumber \\&\quad = \sum _{i,j=1}^d \dfrac{\partial ^2}{\partial \mathbf {z}_i \partial \mathbf {z}_j} \bigg (\mathbf {D}_{ij}(\mathbf {z}) p(\mathbf {z};t)\bigg ) \nonumber \\&\qquad + \sum _{i=1}^d \dfrac{\partial }{\partial \mathbf {z}_i} \left( \left[ \sum _j\mathbf {D}_{ij}(\mathbf {z}) \dfrac{\partial H(\mathbf {z})}{\partial \mathbf {z}_j} - \varGamma ^{\mathbf {D}}_i(\mathbf {z})\right] p(\mathbf {z};t) \right) . \end{aligned}$$

(44)

Here, $\varGamma ^{\mathbf {D}}_i(\mathbf {z}) = \sum _{j=1}^d \dfrac{\partial }{\partial \mathbf {z}_j} {\mathbf {D}}_{ij}(\mathbf {z})$. According to Itô’s convention, (44) corresponds to reversible Brownian motion in a potential force field on a Riemannian manifold specified by the diffusion matrix $\mathbf {D}(\mathbf {z})$: $\text {d}\mathbf {z}= \big [ - \mathbf {D}(\mathbf {z}) \nabla H(\mathbf {z}) + \varGamma ^{\mathbf {D}}(\mathbf {z})\big ] \text {d}t + \sqrt{2 \mathbf {D}(\mathbf {z})} \text {d}\mathbf {W}(t)$. This is referred to as Riemannian Langevin dynamicsGirolami and Calderhead 2011; Xifara et al. 2014. When $\mathbf {D}(\mathbf {z})$ is positive definite, the reversible Markov dynamics have nice statistical regularity and will drive the system to converge to the stationary distribution.

The irreversible process is determined solely by $\mathbf {Q}$, with its probability density function evolving according to:

$$\begin{aligned} \dfrac{\partial }{\partial t} p(\mathbf {z};t) =&\nabla ^T \cdot \bigg ( \mathbf {Q}(\mathbf {z}) \left[ p(\mathbf {z};t) \nabla H(\mathbf {z}) + \nabla p(\mathbf {z};t) \right] \bigg ) \nonumber \\ =&\nabla ^T \cdot \Bigg ( \Big [\mathbf {Q}(\mathbf {z}) \nabla H(\mathbf {z}) - \varGamma ^{\mathbf {Q}}(\mathbf {z}) \Big ] {p(\mathbf {z};t)} \Bigg ). \end{aligned}$$

(45)

Here, $\varGamma ^{\mathbf {Q}}_i(\mathbf {z}) = \sum _{j=1}^d \dfrac{\partial }{\partial \mathbf {z}_j} {\mathbf {Q}}_{ij}(\mathbf {z})$. The last line of (45) is a Liouville’s equation, which describes the density evolution of ${p(\mathbf {z};t)}$ according to conserved, deterministic dynamics: $\text {d}\mathbf {z}/\text {d}t = -\mathbf {Q}(\mathbf {z}) \nabla H(\mathbf {z}) + \varGamma ^{\mathbf {Q}}(\mathbf {z})$, with $\pi (\mathbf {z})$ its invariant measure.

Combining the dynamics of (44) and (45) leads to a general SDE, (3), with stationary distribution $\pi (\mathbf {z})$. Previous methods (Roberts and Stramer 2002; Xifara et al. 2014; Welling and Teh 2011; Patterson and Teh 2013; Neal 2010) have primarily focused on solely reversible or irreversible processes, respectively, as we make explicit in Sect. 2.3.

Irreversible Jump processes for MCMC

1.1 Equivalence of (10) and (11)

We introduce a symmetric bivariate function $S(\mathbf {x},\mathbf {z}) = S(\mathbf {z},\mathbf {x}) = \dfrac{1}{2}\Big (W(\mathbf {z}|\mathbf {x})\pi (\mathbf {x}) + W(\mathbf {x}|\mathbf {z})\pi (\mathbf {z})\Big )$, and an antisymmetric bivariate function $A(\mathbf {x},\mathbf {z}) = -A(\mathbf {z},\mathbf {x}) = \dfrac{1}{2}\Big (W(\mathbf {z}|\mathbf {x})\pi (\mathbf {x}) - W(\mathbf {x}|\mathbf {z})\pi (\mathbf {z})\Big )$ for (10). A different form of the jump process (10) can be written according to $S(\mathbf {x},\mathbf {z})$ and $A(\mathbf {x},\mathbf {z})$ as

$$\begin{aligned} \dfrac{\partial p(\mathbf {z}|\mathbf {y};t)}{\partial t} =&\int _{\mathbb {R}^d} \text {d}\mathbf {x}\bigg [ S(\mathbf {x},\mathbf {z}) \dfrac{p(\mathbf {x}|\mathbf {y};t)}{\pi (\mathbf {x})} - S(\mathbf {x},\mathbf {z}) \dfrac{p(\mathbf {z}|\mathbf {y};t)}{\pi (\mathbf {z})} \\&+ A(\mathbf {x},\mathbf {z}) \dfrac{p(\mathbf {x}|\mathbf {y};t)}{\pi (\mathbf {x})}\bigg ]. \end{aligned}$$

Plugging $p(\mathbf {z}|\mathbf {y};t) = \pi (\mathbf {z})$ into the above equation, we find that as long as $\int _{\mathbb {R}^d} A(\mathbf {x},\mathbf {z}) \text {d}\mathbf {x}= 0$, $\pi (\mathbf {z})$ is a stationary solution to the equation. Since $\dfrac{S(\mathbf {x},\mathbf {z}) + A(\mathbf {x},\mathbf {z})}{\pi (\mathbf {x})}$ denotes a transition probability density, ${S(\mathbf {x},\mathbf {z}) + A(\mathbf {x},\mathbf {z})}>0$ for any $\mathbf {x}$ and $\mathbf {z}$. The restriction that $S(\mathbf {x},\mathbf {z}) \pi ^{-1}(\mathbf {x})$ and $A(\mathbf {x},\mathbf {z}) \pi ^{-1}(\mathbf {x})$ are bounded is imposed for the practical purpose of proposing samples in (12). We thereby notice that the requirement that $\pi (\mathbf {z})$ is a stationary distribution of the jump process is translated into simpler constraints.

1.2 Verifying condition 3.1 on $A(\mathbf {y},\mathbf {y}^p,\mathbf {z},\mathbf {z}^p)$ in Section 3.3

The antisymmetric function $A(\mathbf {y},\mathbf {y}^p,\mathbf {z},\mathbf {z}^p)$ (expressed in (21)) of (20) can be written as:

$$\begin{aligned}&A(\mathbf {y}, \mathbf {y}^p,\mathbf {z},\mathbf {z}^p) \\&\quad = \dfrac{1}{2 \varDelta t} \Big ( \pi (\mathbf {y})\pi (\mathbf {y}^p) p(\mathbf {z},\mathbf {z}^p|\mathbf {y},\mathbf {y}^p;\varDelta t) \\&\qquad - \pi (\mathbf {z})\pi (\mathbf {z}^p) p(\mathbf {y},\mathbf {y}^p|\mathbf {z},\mathbf {z}^p;\varDelta t) \Big ) \\&\quad = \dfrac{1}{2} \delta (\mathbf {z}^p -\mathbf {y}^p) \Big (\mathfrak {F}(\mathbf {y},\mathbf {y}^p,\mathbf {z},\mathbf {z}^p) - \mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {y},\mathbf {y}^p)\Big ) \\&\qquad - \dfrac{1}{2} \delta ( \mathbf {z}^p+\mathbf {y}^p)\delta (\mathbf {z}-\mathbf {y}) \\&\qquad \cdot \int _{\mathbb {R}^d} \Big (\mathfrak {F}(\mathbf {y},\mathbf {y}^p,\mathbf {x},-\mathbf {z}^p) - \mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {x},-\mathbf {y}^p)\Big )\text {d}\mathbf {x}. \end{aligned}$$

Below we prove that, as required,

$$\begin{aligned} \int _{\mathbb {R}^{d+d^p}} A(\mathbf {y},\mathbf {y}^p,\mathbf {z},\mathbf {z}^p) \ \text {d}\mathbf {y}\ \text {d}\mathbf {y}^p = 0. \end{aligned}$$

Proof

$$\begin{aligned}&\int _{\mathbb {R}^{d+d^p}} A(\mathbf {x},\mathbf {x}^p,\mathbf {z},\mathbf {z}^p) \ \text {d}\mathbf {x}\ \text {d}\mathbf {x}^p \\&\quad = \dfrac{1}{2} \int _{\mathbb {R}^{d}} \Big (\mathfrak {F}(\mathbf {y},\mathbf {z}^p,\mathbf {z},\mathbf {z}^p) - \mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {y},\mathbf {z}^p)\Big ) \ \text {d}\mathbf {y}\\&\qquad - \dfrac{1}{2} \int _{\mathbb {R}^{d}} \Big (\mathfrak {F}(\mathbf {z},-\mathbf {z}^p,\mathbf {x},-\mathbf {z}^p) - \mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {x},\mathbf {z}^p)\Big )\text {d}\mathbf {x}. \end{aligned}$$

One can check that in (23) and (19),

$$\begin{aligned} \widetilde{f}(\mathbf {z},\cdot \ |\mathbf {y}, -\mathbf {y}^p) = \widetilde{g}(\mathbf {z},\cdot \ |\mathbf {y},\mathbf {y}^p). \end{aligned}$$

Hence,

$$\begin{aligned} \mathfrak {F}(\mathbf {y},-\mathbf {y}^p,\mathbf {z},-\mathbf {z}^p)=\mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {y},\mathbf {y}^p). \end{aligned}$$

Therefore,

$$\begin{aligned}&\int _{\mathbb {R}^{d+d^p}} A(\mathbf {x},\mathbf {x}^p,\mathbf {z},\mathbf {z}^p) \ \text {d}\mathbf {x}\ \text {d}\mathbf {x}^p \nonumber \\&\quad = \dfrac{1}{2} \int _{\mathbb {R}^{d}} \Big (\mathfrak {F}(\mathbf {z},-\mathbf {z}^p,\mathbf {y},-\mathbf {z}^p) - \mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {y},\mathbf {z}^p)\Big ) \ \text {d}\mathbf {y}\nonumber \\&\qquad - \dfrac{1}{2} \int _{\mathbb {R}^{d}} \Big (\mathfrak {F}(\mathbf {z},-\mathbf {z}^p,\mathbf {x},-\mathbf {z}^p) - \mathfrak {F}(\mathbf {z},\mathbf {z}^p,\mathbf {x},\mathbf {z}^p)\Big )\text {d}\mathbf {x}= 0. \end{aligned}$$

(46)

$\square$

Proof of Theorem 2 (relation between forward process and adjoint process)

We first prove that for the infinitesimal generators, the backward transition probability density following the adjoint process and the forward transition probability density are related as: $\pi (\mathbf {y}) P\big (\mathbf {z}| \mathbf {y}; \text {d}t \big ) = \pi (\mathbf {z}) P^\dag \big (\mathbf {y}| \mathbf {z}; \text {d}t \big )$. Taking path integrals with respect to the infinitesimal generators leads to the conclusion.

As is standard, we use two arbitrary smooth test functions $\psi (\mathbf {y})$ and $\phi (\mathbf {z})$. Then we use the definition of the infinitesimal generator of the process P and $P^\dag$: $\mathcal {G} [\phi (\mathbf {y})] \text {d}t = \int P\big (\mathbf {z}| \mathbf {y}; \text {d}t \big ) \phi (\mathbf {z}) \text {d}\mathbf {z}- \phi (\mathbf {y})$ and obtain

$$\begin{aligned}&\int \int \text {d}\mathbf {y}\text {d}\mathbf {z}\ {P\big (\mathbf {z}| \mathbf {y}; \text {d}t\big )} {\pi (\mathbf {y})} \ \psi (\mathbf {y}) \phi (\mathbf {z}) \\&\quad = <\psi , (I+ \text {d}t \ \mathcal {G})[\phi ]>_\pi , \end{aligned}$$

and

$$\begin{aligned}&\int \int \text {d}\mathbf {y}\text {d}\mathbf {z}\ {P^\dag \big (\mathbf {y}| \mathbf {z}; \text {d}t\big )} {\pi (\mathbf {z})} \ \psi (\mathbf {y}) \phi (\mathbf {z}) \\&\quad = <(I+ \text {d}t \ \mathcal {G}^*)[\psi ], \phi >_\pi . \end{aligned}$$

Since $\mathcal {G}$ and $\mathcal {G}^*$ are adjoint in $L^2(\pi )$: $<\psi , \mathcal {G}[\phi ]>_\pi = <\mathcal {G}^*[\psi ], \phi >_\pi$,

$$\begin{aligned} \pi (\mathbf {y}) P\big (\mathbf {z}| \mathbf {y}; \text {d}t \big ) = \pi (\mathbf {z}) P^\dag \big (\mathbf {y}| \mathbf {z}; \text {d}t \big ). \end{aligned}$$

Then we take path integrals over the forward path and the backward one. Using the Markov properties,

$$\begin{aligned}&P(\mathbf {z}_N,t_N|\mathbf {z}_0,t_0)\\&\quad = \int \cdots \int \prod _{i=1}^{N-1}\text {d}\mathbf {z}_i\prod _{i=0}^{N-1} P(\mathbf {z}_{i+1},t_{i+1}|\mathbf {z}_i,t_i); \end{aligned}$$

and

$$\begin{aligned}&P^\dag (\mathbf {z}_0,t_N|\mathbf {z}_N,t_0) \\&\quad = \int \cdots \int \prod _{i=1}^{N-1}\text {d}\mathbf {z}_i\prod _{i=0}^{N-1} P^\dag (\mathbf {z}_{i},t_{i+1}|\mathbf {z}_{i+1},t_i)\\&\quad = \int \cdots \int \prod _{i=1}^{N-1}\text {d}\mathbf {z}_i\prod _{i=0}^{N-1} \dfrac{\pi (\mathbf {z}_i)}{\pi (\mathbf {z}_{i+1})} P(\mathbf {z}_{i+1},t_{i+1}|\mathbf {z}_{i},t_i)\\&\quad = \int \cdots \int \prod _{i=1}^{N-1}\text {d}\mathbf {z}_i\prod _{i=0}^{N-1} \dfrac{\pi (\mathbf {z}_i)}{\pi (\mathbf {z}_{i+1})} P(\mathbf {z}_{i+1},t_{i+1}|\mathbf {z}_{i},t_i)\\&\quad = \dfrac{\pi (\mathbf {z}_0)}{\pi (\mathbf {z}_{N})} P(\mathbf {z}_N,t_N|\mathbf {z}_0,t_0). \end{aligned}$$

Taking the time interval between $t_i$ and $t_{i+1}$ to be infinitesimal, we obtain that $\dfrac{P(\mathbf {z}^{(T)},T|\mathbf {z}^{(t)},t)}{P^\dag (\mathbf {z}^{(t)},T|\mathbf {z}^{(T)},t)} = \dfrac{\pi (\mathbf {z}^{(T)})}{\pi (\mathbf {z}^{(t)})}$. The same conclusion can be reached by proving that the semigroups $e^{t\mathcal {G}}$ and $e^{t\mathcal {G}^*}$ generated by $\mathcal {G}$ and $\mathcal {G}^*$ are also adjoint with each other (Jansen and Kurt 2014; Pazy 1983; Poncet 2017).

Experiments

1.1 Parameter settings for the irreversible jump sampler

In the 1D experiments, we use $\beta = 1.2$ for the normal distribution case (where the length of the region of definition is 10) and $\beta = 0.8$ for the log-normal distribution (where the length of the region of definition is 5). The acceptance rate is around $50\%$ in these cases. Due to the irreversibility of the sampler, a high acceptance rate can be maintained while reducing the autocorrelation time.

In the visual comparison of samplers of Fig. 2, we use $\beta =0.15$ (where the length of the region of definition is $2 \times 2$). In the 2D bimodal experiments, we use $\beta =0.4$ (where the length of the region of definition is $6 \times 3$). In the 2D multimodal experiments, we take $\beta =1.5$ (where the length of the region of definition is $14 \times 14$). For the 2D correlated distribution, we take $\beta =0.25$ (where the length of the region of definition is $5 \times 2$).

1.2 Effect of dimensionality on I-Jump versus MH

In this appendix, we explore how the relative improvement of I-Jump over MH scales with dimensionality. We consider a standard normal distribution and double the dimension from one comparison to the next. For the I-Jump sampler, we use the vanilla half-space Gaussian proposal. It can be observed from Table 4 that the potential benefits of irreversibility in the I-Jump sampler slowly diminish with increasing dimensionality.

Table 4 Comparison of $\widehat{\mathrm{ESS}}$ per second of runtime for I-Jump versus MH with different dimensions of Gaussian target distributions