Deconvolution of the Functional Ultrasound Response in the Mouse Visual Pathway Using Block-Term Decomposition

Functional ultrasound (fUS) is a relatively new neuroimaging technique that that captures changes in local blood dynamics. More precisely, fUS makes use of plane waves transmitted at an ultrafast frame rate that are backscattered by the moving red blood cells in the imaged area. Thus, the fUS signal amplitude fluctuates over time in proportion to alterations in local blood volume (Deffieux et al., 2021). The hemodynamic activity as detected by fUS can be used as an indirect report of changing neural activity through the phenomenon of neurovascular coupling (Aydin et al., 2020). In particular, local variations in neural activity induce a delayed response in blood flow and volume known as functional hyperemia, which can be modelled by a hemodynamic response function of time. The sensitivity of fUS in measuring subtle variations of blood dynamics has led to a variety of fUS-based animal and clinical studies in the past decade, ranging from detection of responses to sensory stimuli to complex brain states and behavior (Nunez-Elizalde et al., 2022). These include studies on small rodents (Macé et al., 2011; Gesnik et al., 2017; Macé et al., 2018; Koekkoek et al., 2018), nonhuman primates (Blaize et al., 2020; Norman et al., 2021), birds (Rau et al., 2018) and humans (Imbault et al., 2017; Baranger et al., 2021; Soloukey et al., 2020).

Understanding the underlying mechanisms of hemodynamic response has been an important challenge not only for fUS (Nunez-Elizalde et al., 2022), but also for several other established functional neuroimaging modalities, including functional magnetic resonance imaging (fMRI) (Schrouff et al., 2013; Roels et al., 2015) and functional near-infrared spectroscopy (fNIRS) (Shah & Seghouane, 2014)). A common way to model the measured time-series is via a function representing the impulse response of the neurovascular system, known as the hemodynamic response function (HRF) (Seghouane & Shah, 2012). In this model, the HRF gets convolved with an input signal representing the experimental paradigm (EP), which is expressed as a binary vector that shows the on- and off- times of a given stimulus. However, not all stimuli can be predefined, i.e. under the experimenters’ control (Karahanoglu et al., 2013). For instance, brain reaction to mental imagery is shown to be almost as strong as the activity evoked by real perception in certain brain regions under a variety of experimental designs, such as visual (Ganis et al., 2004) or auditory (Bunzeck et al., 2005). Therefore, the input signals that represent such tasks or events that evoke brain activity should be generalized beyond merely the preset paradigms. This issue has been addressed by (Karahanoglu et al., 2013; Karahanoglu & Van De Ville, 2015; Uruñuela et al., 2021), where the authors have defined the term activity-inducing signal, which, as the name suggests, comprises any input signal that induces hemodynamic activity. We will refer to activity-inducing signals as source signals in the rest of this paper, which steers the reader to broader terminology not only used in biomedical signal processing, but also in acoustics and telecommunications (Naik & Wang, 2014), and emphasizes that recorded output data are sourced by such signals.

An accurate estimation of the HRF is crucial to correctly interpret both the hemodynamic activity itself and the underlying source signals. Furthermore, the HRF has shown potential as a biomarker for healthy aging (West et al., 2019) or pathological brain functioning; examples of which include obsessive-compulsive disorder (Rangaprakash et al., 2021), mild traumatic brain injury (Mayer et al., 2014), Alzheimer’s disease (Asemani et al., 2017), epilepsy (Van Eyndhoven et al., 2021) and severe psychosocial stress (Elbau et al., 2018). While HRFs can as well be defined in nonlinear and dynamic frameworks with the help of Volterra kernels (Friston, 2002), linear models have particularly gained popularity due to the combination of their remarkable performance and simplicity. Several approaches have been proposed in the literature which employ linear modelling for estimating the HRF. The strictest approach assumes a constant a priori shape of the HRF, i.e. a mathematical function with fixed parameters, and is only concerned with finding its scaling (the activation level). The shape used in this approach is usually given by the canonical HRF model (Friston et al., 1998). As such, this approach does not incorporate HRF variability, yet the HRF is known to change significantly across subjects, brain regions and triggering events (Handwerker et al., 2004; Aguirre et al., 1998; Fransson et al., 1999). A second approach is to estimate the parameters of the chosen shape function, which leads to a more flexible solution (Aydin et al., 2020). Alternatively, HRF estimation can be reformulated as a regression problem by expressing the HRF as a linear combination of several basis functions (which are often chosen to be the canonical HRF and its derivatives). This approach is known as the general linear model (GLM) (Lindquist et al., 2009). Finally, it is also possible to apply no shape constraints on the HRF, and predict the value of the HRF distinctly at each time point. This approach suffers from high computational complexity and variance of the estimated HRFs, which might be of arbitrary or physiologically meaningless forms (Glover, 1999).

Note that the majority of studies which tackle HRF estimation presume that the source signal is known and equal to the EP, leaving only one unknown in the convolution: the HRF (Hütel et al., 2021). However, as mentioned earlier, a functional brain response can be triggered by more sources than the EP alone. These sources can be extrinsic, i.e., related to environmental events, such as unintended background stimulation or noise artefacts. They might also be intrinsic sources, such as mental imagery. Under such complex and multi-causal circumstances, recovering the rather ’hidden’ source signal(s) can be of interest. Moreover, even the EP itself can be much more complex than what a simple binary pattern allows for. Indeed, the hemodynamic response to, for instance, a visual stimulus, can vary greatly depending on its parameters, such as its contrast (Gesnik et al., 2017), demanding a continuous variable to represent the “on” times of the stimulus. In contrast to the aforementioned methods, where the goal was to estimate HRFs from a known source signal, there have also been attempts to predict the sources by assuming a known and fixed HRF (Caballero et al., 2011) (Karahanoglu et al., 2013). However, these methods fall short of depicting the HRF variability.

To sum up, neither the sources nor the HRF are straightforward to model, and as such, when either is assumed to be fixed, it can easily lead to misspecification of the other. Therefore, we consider the problem of jointly estimating the source signals and HRFs from multivariate fUS time-series. This problem has been addressed by (Sreenivasan et al., 2015), (Wu et al., 2013), (Cherkaoui et al., 2019) and (Cherkaoui et al., 2021). In (Sreenivasan et al., 2015), it is assumed that the source signal (here considered as neural activity) lies in a high frequency band compared to the HRF, and can thus be recovered using homomorphic filtering. On the other hand, (Wu et al., 2013) first estimates a spike-like source signal by thresholding the fMRI data and selecting the time points where the response begins, and subsequently fits a GLM using the estimated source signal to find the HRF. Both of the aforementioned methods are univariate: although they analyze multiple regions and/or subjects, the analysis is performed separately on each time series, thereby ignoring any mutual information shared amongst biologically relevant ROIs.

Recently, a multivariate deconvolution of fMRI time series has been proposed in (Cherkaoui et al., 2021). The authors proposed an fMRI signal model, where neural activation is represented as a low-rank matrix - constructed by a certain (low) number of temporal activation patterns and corresponding spatial maps encoding functional networks - and the neural activation is linked with the observed fMRI signals via region-specific HRFs. The main advantage of this approach is that it allows whole-brain estimation of HRF and neural activation. However, all HRFs are defined via the dilation of a presumed shape, which may not be enough to capture all possible variations of the HRF, as the width and peak latency of the HRF are coupled into a single parameter. Moreover, the estimated HRFs are region-specific, but not source-specific. Therefore, the model cannot account for variations in the HRF due to varying stimulus properties. Yet, the length and intensity of stimuli appear to have a significant effect on HRF shape even within the same region, as observed in recent fast fMRI studies (Chen et al., 2021).

In order to account for the possible variations of the HRF for both different sources and regions, we model the fUS signal in the framework of convolutive mixtures, where multiple input signals (sources) are related to multiple observations (measurements from a brain region) via convolutive mixing filters. In the context of fUS, the convolutive mixing filters stand for the HRFs, which are unique for each possible combination of sources and regions, allowing variability across different brain areas and triggering events. In order to improve identifiability, we make certain assumptions, namely that the shape of the HRFs can be parametrized and that the source signals are uncorrelated. Considering the flexibility of tensor-based formulations for the purpose of representing such structures and constraints that exist in different modes or factors of data (Sorber et al., 2015), we solve the deconvolution by applying block-term decomposition (BTD) on the tensor of lagged measurement autocorrelation matrices.

While in our previous work (Erol et al., 2020) we had considered a similar BTD-based deconvolution, this paper presents several novel contributions. First, we improve the robustness of the algorithm via additional constraints and a more sophisticated selection procedure for the final solution from multiple optimization runs. We also present a more detailed simulation study considering a large range of possible HRF shapes. Finally, we demonstrate the capabilities of our method on two experimental datasets recorded from mice using fUS during visual stimulation. In the first dataset we track the visual information pathway by investigating the peak latency of the HRF in key anatomical structures involved within the mouse brain’s colliculo-cortical, image-forming visual pathway: the lateral geniculate nucleus (LGN), the superior colliculus (SC) and the primary visual cortex (V1). We show that the ordering of the peak latencies agrees with prior works (Gesnik et al., 2017), confirming with fUS that visual information first travels through the subcortical targets SC and LGN, before being relayed to V1. In the second experiment we repeatedly display the visual stimuli at 5 distinct locations. We show that our technique is able to extract 5 underlying sources, and the time course of each of these sources have a one-to-one correspondence with the timing of the 5 distinct stimulus locations.

The rest of this paper is organized as follows. First, we describe our data model and the proposed tensor-based solution for deconvolution. Next, we describe the experimental setup and data acquisition steps used for fUS imaging of a mouse subject. This is followed by the deconvolution results, which are presented in two-folds: (i) Numerical simulations, and (ii) Results on real fUS data. Next, under discussion, we review the highlights of our modelling and results, and elaborate on the neuroscientific relevance of our findings. Finally, we state several future extensions and conclude our paper.

Naturally, fUS images contain far more pixels than the number of anatomical or functional regions. We therefore expect certain groups of pixels to show similar signal fluctuations along time, and we consider the fUS images as parcellated in space into several regions. Consequently, we represent the overall fUS data as an \(M \times N\) matrix, where each of the M rows contain the average pixel time-series within a region-of-interest (ROI), and N is the number of time samples.

Assuming a single source signal, a single ROI time-series y(t) can be written as the convolution between the HRF h(t) and the input source signal s(t) as:where \(L+1\) gives the HRF filter length. However, a single ROI time-series may be affected by a number of (R) different source signals. Each source signal \(s_r(t)\) may elicit a different HRF, \(h_r(t)\). Therefore, the observed time-series is the summation of the effect of all underlying sources:Finally, extending our model to multiple ROIs, where each ROI may have a different HRF, we arrive to the multivariate convolutive mixture formulation:where \(h_{mr}(l)\) is the convolutive mixing filter, belonging to the ROI m and source r (Mitianoudis & Davies, 2003). Note that, in this work we consider that the ROIs are known (for instance, via anatomical labelling (Wu et al., 2013)), or can be estimated from the data as a pre-processing step prior to deconvolution. We follow the latter approach in this paper, and apply independent component analysis (ICA) on the fUS data, regarding which more details will be explained in “Experimental Setup and Data Acquisition”.

In the context of fUS, the sources that lead to the time-series can be task-related (T), such as the EP, or artifact-related (A). The task-related sources are convolved with an HRF, whereas the artifact-related sources are directly additive on the measured time-series (Marrelec et al., 2003). Artifact sources in general are used to represent fUS signal variation of non-neural origin. Under this definition, we consider physiological noise, e.g. movement of the subject causing signal fluctuations in the entire field-of-view. Moreover, a recent study (Nunez-Elizalde et al., 2022) found out that only the low-frequency content of the fUS signal reflects neural activity. Artifact sources can as well incorporate instrumentation noise, such as thermal or electronic noise (commonly modeled as additive, (Demené et al., 2015a; Huang et al., 2019)) introduced by the ultrasound acquisition system, which can be spatially varying, becoming more prominent at deeper areas of the brain. As such, the strength of the effect that an artifact source exerts on a region should depend on the artifact type and the brain region. Hence, each \(h_{mr}(l)\) with \(r \in A\) should correspond to a scaled (by \(a_{mr}\)) unit impulse function (ensuring additivity). Finally, we rewrite Eq. 3 as:We aim at solving this deconvolution problem to recover the sources and HRFs of interest separately at each ROI m.

In this section, we will present the steps of the proposed tensor-based deconvolution method. We will first introduce how deconvolution of the observations modeled as in Eq. 4 can be expressed as a BTD. Due to the fact that this problem is highly non-convex, we will subsequently explain our approach to identifying a final solution for the decomposition. Finally, we will describe source signal estimation using the HRFs predicted by BTD.

We used two mice (C57BL/6J in the single-stimulus, and B6CBAF1/JRj in the multiple-stimuli experiment; both 7 months-old and male) for the in vivo fUS experiments. The experimental setup depicted in Fig. 4. The mice were housed with food and water ad libitum, and maintained under standard conditions (12/12 h light-darkness cycle, 22 \(^\circ\)C). Preparation of each mice involved surgical pedestal placement and craniotomy. First, an in-house developed titanium pedestal (8 mm in width) was placed on the exposed skull using an initial layer of bonding agent (OptiBond™) and dental cement (Charisma^®). Subsequently, a surgical craniotomy was performed to expose the cortex from Bregma -1 mm to -7 mm. After skull bone removal and subsequent habituation, the surgically prepared, awake mouse was head-fixed and placed on a movable wheel in front of two stimulation screens (Dell 23,8” S2417DG, 1280 x 720 pixels, 60 Hz) in landscape orientation, positioned at a 45° angle with respect to the antero-posterior axis of the mouse, as well as 20 cm away from the mouse’s eye, similar to (Macé et al., 2018). All experimental procedures were approved a priori by an independent animal ethical committee (DEC-Consult, Soest, the Netherlands), and were performed in accordance with the ethical guidelines as required by Dutch law and legislation on animal experimentation, as well as the relevant institutional regulations of Erasmus University Medical Center.

In the first experiment, the visual stimulus consisted of a rectangular patch of randomly generated, high-contrast images - white “speckles” against a black background - which succeeded each other with 25 frames per second, inspired by (Niranjan et al., 2016; Gesnik et al., 2017; Ito et al., 2017). The rectangular patch spanned across both stimulation screens such that it was centralized in front of the mouse, whereas the screens were kept entirely black during the rest (i.e., non-stimulus) periods. The visual stimulus was presented to the mouse in 20 blocks of 4 seconds in duration. Each repetition of the stimulus was followed by a random rest period between 10 to 15 seconds.

In the second experiment, the visual stimulus was a sinusodial grating presented randomly at one of 5 predefined locations, determined by dividing the mouse horizontal field-of-view (\(140^{\circ }\) (Marshel et al., 2011)) as projected on the screens into 5 equal parts. These locations are displayed in Fig. 3. The grating was drifted at 4 degrees per cycle, and at a temporal frequency (cycles per second) of 8.3 Hz. The stimulus and rest duration was kept fixed at 10 and 15 seconds respectively.

Before experimental acquisition, a high-resolution anatomical registration scan was made of the exposed brain’s microvasculature so as to locate the most ideal imaging location for capturing the ROIs aided by the Allen Mouse Brain Atlas (Allen Institute for Brain Science, 2015). For data acquisition, 20 tilted plane waves were continuously transmitted from an ultrasonic transducer (Vermon L\(22-14\)v, 15 MHz) at 800 Hz, which was coupled to the the mouse’s cranial window with ultrasound transmission gel (Aquasonic). A compound image was obtained by Fourier-domain beamforming and angular compounding, and non-overlapping ensembles were formed by concatenating 200 consecutive compound images. Next, we applied SVD-filtering to denoise the images and separate the blood signal from stationary and slow-changing ultrasound signals arising from other brain tissue. More specifically, SVD-filtering was performed on each ensemble by setting the first (i.e., largest) 30% and the last (i.e., smallest) 1% of the singular values to 0, the former rejecting tissue clutter (Demené et al., 2015b) whereas the latter removing noise (Song et al., 2017). Afterwards, the vascular signal of interest was reconstructed back from the remaining singular components (Demené et al., 2015a). Images were upsampled in the spatial frequency domain to an isotropic resolution of \(25\mu\)m, matching to that of the Allen Reference Atlas. Lastly, a Power-Doppler Image (PDI) was obtained by computing for every pixel the power of the SVD-filtered signal over the frames within the ensemble, providing a final sampling rate of 4 Hz for the PDIs. The time-series of a pixel (Eq. 4) corresponds to the variation of its power across the PDI stream.

For the selection of ROIs, the experimental data was first parcellated using spatial ICA with 10 components (Sala-Llonch et al., 2019). The components of interest were thresholded to reveal a spatial binary mask standing for an anatomical ROI. To obtain a representative time-series for each ROI, we averaged the time-series of pixels which are captured within the boundaries of the corresponding spatial mask. Finally, the ROI time-series were normalized to zero-mean and unit-variance before proceeding with the BTD.

In this study, we considered the problem of deconvolving multivariate fUS time-series by assuming that the HRFs are parametric and source signals are uncorrelated. We formulated this problem as a block-term decomposition, which jointly estimates the underlying source signals and region-specific HRFs. In other words, the proposed method for deconvolution of the hemodynamic response has the advantage of not requiring the source signal(s) nor the HRFs to be specified. As such, it can take into account regional differences of the HRF, as well as recover numerous sources besides the EP, that are unrelated to the intended task and/or outside of the experimenters’ control. We investigated the fUS response in several regions of the mouse brain, namely the SC, LGN and the visual cortex, which together compose significant pathways between the eye and the brain.

We tested our method in two in vivo fUS experiments. In the first experiment, we assumed one common task-source for the ROIs (SC, LGN and V1) and inspected the estimated source signal and HRFs. It is important to mention that the responses of the selected ROIs are shown to be modulated by the same stimulus in prior works (Ahmadlou et al., 2018; Gesnik et al., 2017; Lewis et al., 2018), as can also be seen via the stimulus-correlation maps of our experiment (Fig. 9) which reveal significant values in these ROIs. In the estimated source signal, we observed unforeseen amplitude variations along time. To better understand this behavior, we investigated the hemodynamic responses in the selected ROIs across repetitions. We noticed that the response variability in the visual system increases from the subcortical to the cortical level. Consistent with our findings, electrophysiological studies such as (Kara et al., 2000) report an increase in trial-by-trial variability from subcortex to cortex, doubling from retina to LGN and again from LGN to visual cortex. Variability in responses could be related to external stimulation other than the EP, such as unintended auditory stimulation from experimental surroundings (Ito et al., 2021). In addition, literature points to eye movements as a source of high response variability in V1, a behavior which can be found in head-fixated, but awake mice following attempted head rotation (Meyer et al., 2020), which can extraordinarily alter stimulus-evoked responses (Gur & Snodderly, 1997).

Based on the estimated HRFs, we noted that SC has the fastest reaction to stimuli, followed respectively by LGN and V1. As V1 does not receive direct input from the retina, but via LGN and SC, its delayed HRF is consistent with the underlying subcortical-cortical connections of the visual processing pathway, as has also been reported by (Brunner et al., 2021; Lau et al., 2011; Lewis et al., 2018). What’s more, the SC’s particular aptness to swiftly respond to the visual stimulus aligns with its biological function to indicate potential threats (such as flashing, moving or looming spots (Gale & Murphy, 2016; Wang et al., 2010; Inayat et al., 2015)).

In the second experiment, we explored the source-detection capabilities of our method further in a more complex setting, with five different stimulus conditions determined by five different locations that the stimulus was presented. The proposed method was able to successfully detect the timings of these locations and differentiate them from one another, particularly those that elicited a stronger and a more discriminative response in the mouse brain.

For both experiments, we considered the number of sources to be known, which is usually not the case in practice. In order to investigate the sensitivity of our approach to this choice, we tried selecting less or more sources in the second fUS experiment. Selecting less number of sources (3 sources) resulted in similar stimuli being grouped together under one source signal. Particularly, one of the estimated sources revealed the timings of both the leftmost and slight-left stimuli, whereas another estimated source reflected both the rightmost and slight-right stimuli. On the other hand, when the number of sources was overestimated, we observed that the estimated sources were more interleaved with one another, resulting in slightly smaller correlation values with the true stimulus times (at least while using the same set of parameters). Overall, we conclude that a sub-optimal choice for the number of sources still leads to reasonable results. Nevertheless, a possible future extension of this work would be to explore methods for estimating the number of sources, such as information criterion-based approaches (Bai & He, 2006).

Compared to our previous BTD-based deconvolution, we have made several improvements in this work. To start with, the current method exploits all the structures in the decomposition scheme. In particular, previously the core tensor representing the lagged source autocorrelations was structured to be having Toeplitz slices, yet, these slices were not enforced to be lagged versions of each other. Incorporating such theoretically-supported structures significantly reduced the computation time of BTD by lowering the number of parameters to be estimated. In addition, we increased the robustness of our algorithm by applying a clustering-based selection of the final HRF estimates amongst multiple randomly-initialized BTD runs. Nevertheless, the formulated optimization problem is highly non-convex with many local minima, and the simulations show that there is still room for improvement. For instance, different clustering criteria can be applied for choosing the “best” solution (Himberg et al., 2004). Furthermore, there are several hyperparameters - namely the HRF filter length, number of time lags in the autocorrelation tensor, and the number of BTD repetitions - which affect the algorithm’s performance as well as convergence speed. For instance, although the run-time of one BTD is not very long, repeating it 20 times to converge to a better solution, as done in our current setting, considerably increases the overall time. Instead, it is possible to search for different initialization strategies and run fewer BTDs. Note that the computational complexity of BTD is linear in the number of sources, in the order of the tensor and in the product of the sizes of each mode (Sorber et al., 2013b). The autocorrelation tensor in our solution is of size \(ML'\times ML'\times K\), which means that the computational complexity increases quadratically with the number of ROIs (M) and the number of variables used for calculating the autocorrelations (\(L'\)). For a higher number of M, the complexity level can be preserved by lowering \(L'\), which could result in a loss of accuracy. In that case, the trade-off between complexity and accuracy should be thoroughly analyzed. All in all, our purpose in this study was to show that the convolutive mixtures modelling and the BTD formulation together offer a deconvolution framework that is able to accommodate and unveil many unknowns regarding hemodynamic activity, yet there is more to discover by conducting more real-life experiments for fully acknowledging the potential of the proposed method.

In this paper, we deconvolved the fUS-based hemodynamic response in several regions of interest along the mouse visual pathway. We started with a multivariate model of fUS time-series using convolutive mixtures, which allowed us to define region-specific HRFs and multiple underlying source signals. By assuming that the source signals are uncorrelated, we formulated the blind deconvolution problem into a block-term decomposition of the lagged autocorrelation tensor of fUS measurements. The HRFs estimated in SC, LGN and V1 are consistent with the literature and align with the commonly accepted neuroanatomical functions and interconnections of said areas. In the meantime, the estimated source signals, whether a single task or multiple tasks were employed throughout the experiment, can be identified successfully in terms of the timings they were presented. Overall, our results show that convolutive mixtures with the accompanying tensor-based solution provides a flexible framework for deconvolution by revealing an elaborate characterization of hemodynamic responses in functional neuroimaging data.

The data and MATLAB scripts that support the findings of this study are publicly available at https://​github.​com/​ayerol/​btd_​deconv.