Multimodal Neuroimaging in Schizophrenia: Description and Dissemination

This project acquired fMRI data from 4 task activation paradigms and MEG data from 1 task activation paradigm ranging from pre-attentive processes to multisensory cognitive control. Multiple lines of research have consistently documented auditory sensory processing deficits in SP. One such deficit which has been previously examined in SP is poor sensory gating; i.e., there is a failure to inhibit the electrophysiological response for the second of two rapidly presented stimuli in SP compared to HC (Adler et al. 1982, Freedman et al. 1987). This effect is typically reported in terms of a gating ratio comparing the amplitude of the response for the first (S1) and second (S2) stimulus (S2/S1*100). Poor sensory gating has been characterized as not only a deficit in selective attention and/or in the formation of memory traces, but also as a useful bio-marker of the cognitive and social dysfunction that is typically observed in SP (Cullum et al. 1993, Lijffijt et al. 2009). Electroencephalographic (EEG) and MEG studies of sensory gating have implicated the temporal lobes, including the superior temporal gyrus, as the most likely neuronal generator of the sensory gating deficit (Hanlon et al. 2005a, Huang et al. 2003, Thoma et al. 2003, 2005). Although other invasive neuroimaging techniques suggest a role for the hippocampus and prefrontal cortex in sensory gating (Boutros et al. 2005, Freedman et al. 1996, Grunwald et al. 2003, Korzyukov et al. 2007), neither EEG nor MEG studies have revealed hippocampal activation. While this negative result may be reflective of a true lack of hippocampal activation in gating, it may also be secondary to technical limitations with EEG and MEG, such as limits in spatial resolution, failure to examine appropriate time epochs, and/or difficulties localizing sources in deep structures under certain conditions. In contrast, fMRI is not restricted by these limitations, suggesting that it may be an alternative modality for studying the entire neuronal network underlying the gating deficit, including mesial temporal and prefrontal sources. In this protocol, SP and HC were presented with identical clicks, identical tones, and non-identical tones while undergoing functional imaging. It was predicted that a double dissociation of functioning would be observed with SP exhibiting higher gating ratios for the identical tones and clicks conditions and lower gating ratios for the non-identical tones condition, whereas HC would exhibit the opposite pattern. The functional data from these conditions were directly compared and correlated with behavioral and neuropsychological measures to further quantify the relationship between brain function and the observed clinical pathology that characterizes this basic inhibitory failure. Finally, since impaired sensory gating is thought to be an endophenotypical marker for schizophrenia, the genetic contribution to our measures of electrophysiological and hemodynamic functioning in both HC and SP were investigated as well. Data recorded from other tasks within this project included an auditory cueing paradigm that measured both exogenous orienting and inhibition of return and multisensory cognitive control and detection. A multisensory cognitive control paradigm was used to determine if deficits in SP result from dysfunction within the cognitive control network (CCN; top-down) and/or unisensory cortex (bottom-up), using congruent or incongruent multisensory (auditory and visual) numeric stimuli.

The goal of this project was to study the neural mechanisms underlying multisensory integration of auditory and visual stimuli in SP and HC using MEG and fMRI combined with structural MRI (sMRI). Integration of multisensory information is critical to understanding our environment and provides an important bridge between simple sensory perception and more complex cognitive tasks. That is, the integrity of a larger, distributed neural circuit can be evaluated without engaging complex cognitive functions, where issues of strategy, for example, can confound the results. It has been established that multisensory stimulation generally improves reaction times and performance (based on percent correct) relative to unisensory stimulation at levels that are difficult to perceive (Stein et al. 1993), and more recent results indicate that multisensory facilitation can be observed across the perception spectrum (Stanford and Stein 2007). Some reports have shown that SP have slower reaction times to multisensory stimuli than HC [e.g., (Williams et al. 2010)]. Since the association areas of the temporal lobe are necessary for integration of auditory and visual information (Schroeder and Foxe 2002, Schroeder et al. 2001), altered multisensory processing is consistent with the findings that SP show functional and anatomical differences in the temporal lobes [e.g., (Patterson et al. 2008)]. In this project, auditory and visual multisensory integration was studied using an ecologically valid paradigm that simulates the sight and sound of a soccer ball bouncing either near or far from the subject. The stimuli were presented as auditory alone, visual alone, and synchronous auditory/visual conditions, to which subjects responded with a button press. Based on known deficits in the temporal lobe, it was hypothesized that the responses to combined auditory/visual stimuli would be impaired in SP relative to HC. In addition, due to a reported deficit in the visual dorsal stream in SP (Butler et al. 2007, Butler et al. 2005, Doniger et al. 2002, Foxe et al. 2001, 2005, Kim et al. 2006, 2005, O'Donnell et al. 1996, Schechter et al. 2006, 2005), a neural circuit sensitive to the spatial location of objects, it was hypothesized that there would be a difference in visual responses to the near versus far stimuli in SP, relative to HC. These deficits may be related to unisensory deficits and/or to difficulties in synchronizing activity across widespread regions connected within the circuit which will likely be seen differently in MEG versus fMRI measures, given the complementary nature of these measurement techniques. The long term goal was to identify the local cortical deficits, as well as the deficits in cortical networks that lead to the abnormalities observed in SP. Knowledge about these deficits should permit the evaluation of relationships with neurochemical abnormalities, ultimately leading to more targeted pharmacological interventions.

Prefrontal cortex (PFC) and hippocampal structures play a central role in working memory and relational memory impairments exhibited in SP (Goldman-Rakic 1994, Hanlon et al. 2005a, 2006, Honey and Fletcher 2006, Wolf et al. 2009). These PFC and hippocampal functional deficits have traditionally been attributed to properties of the cortical structures themselves. However, an alternative view is that the deficits are due to the disconnection between these structures in SP (Friston 1999, 2002, Friston and Frith 1995, Johnson 2006). Project 3 tested this disconnection hypothesis; that is, there is a functional disconnection between frontal and temporal cortices due to an abnormal anatomical connection of the underlying white matter tract, uncinate fasciculus, that links the two. The most striking evidence supporting this abnormal fronto-temporal connectivity in SP is found in neuroanatomical studies that describe abnormalities in the uncinate fasciculus (Burns et al. 2003, Kubicki et al. 2002, Park et al. 2004). Despite this finding of abnormal anatomical connectivity, the relationship between functional and anatomical connectivity of this PFC-hippocampal network has not been sufficiently assessed. The focus of this project was on examining the functional and anatomical connectivity in SP and HC using MEG and fMRI during a working-relational memory task, transverse patterning or TP (Burns et al. 2003, Hanlon et al. 2012, 2011, 2003, 2005b, Kubicki et al. 2002, Park et al. 2004). TP is very similar to the childhood-game “Rock, Paper, Scissors.” In TP, subjects chose between stimuli presented in pairs, with the correct choice being a function of the specific pairing. To complete the task the subject must discover, encode, and maintain the distinct relationships among the stimuli, thus requiring working memory and relational memory integration. Fronto-temporal functional connectivity was examined via assessing the temporal correlation, or coherence between the PFC (BA 9 and BA 10) and hippocampus during TP performance. In addition, anatomical connectivity using fractional anisotropy (FA) measures derived from DTI of the uncinate fasciculus was also evaluated. Finally, the functional/structural relationships between PFC and hippocampus, memory function, overall functioning, and clinical symptomatology were examined. Understanding these relationships could potentially lead to treatments or therapies aimed at improving memory, thereby improving overall functioning and clinical symptoms.

This project used 3 T MRI to investigate whether general cognitive functioning in SP is related to circuit level white matter (WM), metabolic, and volumetric changes in subcortical gray matter (GM) and WM regions suggestive of fronto-subcortical disconnection. The available neuroscientific literature in HC, including functional (i.e., fMRI, positron emission tomography) and biochemical/structural (i.e., ¹H–MRS, DTI, voxel-based morphometry or VBM) neuroimaging paradigms, show a striking consensus suggesting that variations in a distributed network predict individual differences found on intelligence and reasoning tasks in HC (Jung and Haier 2007). This parieto-frontal integration theory, or P-FIT model includes the dorsolateral PFC (BAs 6, 9, 10, 45, 46, 47), the inferior (BAs 39, 40) and superior (BA 7) parietal lobule, the anterior cingulate (BA 32), and regions within the temporal (BAs 21, 37) and occipital (BAs 18, 19) lobes. The main structural brain abnormality established in SP research is modest lateral ventricle enlargement indicative of atrophy (Chua and McKenna 1995), a finding established in the very first report utilizing computerized tomography (Johnstone et al. 1976), and suggestive of WM atrophic changes. Since then various brain abnormalities have been identified in SP, both cortically and subcortically, including gross morphological changes, reduced metabolism, and cellular pathology. This diffuse pathology has led some to hypothesize that SP is “a disease of neuronal connectivity,” arising from abnormalities at the level of the neuron and myelin (Andreasen 2000). Evidence for WM abnormality in SP includes structural oligodendrocyte abnormalities (Hof et al. 2002, 2003, Orlovskaya et al. 1997, 1999, Uranova et al. 2001), myelin protein alterations (Chambers and Perrone-Bizzozero 2004), myelin gene polymorphisms (Liu et al. 2005, Novak et al. 2002, Peirce et al. 2006, Qin et al. 2005, Tan et al. 2005, Wan et al. 2005), and altered expression of genes involved in the formation and maintenance of myelin sheaths as well as oligodendrocyte survival and proliferation revealed by DNA microarray methods (Bunney and Bunney 2000, Goldman-Rakic and Selemon 1997, Hakak et al. 2001). The simultaneous utilization of DTI, ¹H–MRS, and sMRI allowed us to relate WM abnormalities to both cortical and subcortical metabolic and morphological changes that, in turn, may underlie ongoing neurocognitive decline in SP. Since SP was hypothesized to be a disorder that involves the integration of information among distributed brain circuits, Project 4 investigated: 1) whether fronto-subcortical network abnormalities are present in SP; 2) whether cognitive functioning in SP is related to circuit level dysfunction; and 3) whether regions beyond the identified brain circuits contribute significantly to broad cognitive functioning deficits characteristic of SP. A hierarchical approach was explored, in which cognitive functioning was predicted by cortical thinning, WM microstructure changes, and metabolic changes within discrete fiber tracts in SP. Also tested was the hypothesis that ongoing neurocognitive impairment in SP is related to chronic disconnection of fronto-subcortical circuits, reflected in morphological contraction of discrete frontal lobe regions critical to higher cognitive functioning. Finally, we explored whether specific genes associated with the formation, development, and functioning of myelin are associated with circuit level dysfunction in SP. No research to date has simultaneously explored diffusivity, metabolic, and morphometric abnormalities within these critical brain circuits as they relate to neurocognitive dysfunction in SP. We see this as a critical line of investigation that will help us understand and potentially intervene in the ongoing cognitive dysfunction that currently limits the ability of treated patients to return to more normal occupational and social functioning.

We analyzed fMRI data from 53 HC and 42 SP obtained during rest and selected tasks from each project [for further information see (Cetin et al. 2014)]. Data for each participant were gathered across multiple fMRI scanning sessions over the course of up to 2 months (1–2 months) with prospective randomization of task presentation and close monitoring of SPs to ensure clinical stability. Functional network connectivity (FNC) based on group ICA results computed from all fMRI tasks plus rest fMRI was assessed with different cognitive demands: resting state, pre-attentive sensory processing and multisnesory processing. We used a novel analysis for assessing across paradigm connectivity patterns to identify stable (static effects) and state-based (dynamic effects) differences between HC and SP groups. A group by task full factorial ANOVA model was applied to the group average network connectivity values. Figure 6 (left panel) shows 6 of the 45 non-artifactual brain networks identified using a group independent component analysis (GICA). The right portion of the figure shows between-group differences in static FNC (across all tasks; A) and dynamic FNC (only for some tasks; B); increased static FNC in SPs was observed in a network that included the thalamus in addition to auditory, motor and visual cortices (see cluster of red boxes in lower left portion) consistent with a trait deficit in thalamic filtering. However, reductions in static FNC between the default mode network and frontal, motor and visaul cortices were also present in SP relative to HC. Finally, dynamic FNC differed in more complex ways; for example, SPs showed significantly higher FNC than HCs in resting state in the thalamic/frontal network pair, while HC FNC was higher during performance of the task-related paradigms. These results suggest that SP network connectivity deficits change depending on whether the participant is resting or actively engaged in a task, and furthermore these connectivity patterns change across task hierarchy. This approach provides a clear advantage in recognizing the complex variability in brain reactions to simple sensory stimuli, which are conditioned by experimental circumstances, in SP and HC populations. Similar integration of data across tasks and/or other imaging modalities will be supported for future external users.

The 10th annual Machine Learning for Signal Processing (MLSP) competition utlized a portion of the COBRE dataset of HCs and SPs with the goal of identifying the best objective classification of schizophrenia patients based on multimodal features derived from MRI scans. Each investigator (291 investigators and 2087 entries) developed a classifier, optional feature selection, that combined structural and functional MRI features, hence providing an example of multimodal data fusion (Calhoun and Sui 2016). The test sets were divided into Public (30 subjects) vs. Private (28 subjects) sets; multiple entries were permitted with feedback provided for Public test sets only (area under the curves or AUCs). Because there was variability between the Public and Private AUCs (Fig. 7 top), most likely due to the small sizes of the data sets, the overall distribution of overall AUCs of the entire test set was used instead. Figure 7 (bottom) shows the overall AUC distribution for all 2087 entries; many entries were able to attain an overall AUC of 0.8 or higher and several different strategies produced good results. For example, the team winning first place estimated class probabilities for test examples by means of a Gaussian process classifier with prior distribution scaled by a probit transformation. For more information on the winning strategies and the competition itself please see (Silva et al. 2014).

Network structure and network dynamics (functional network connectivity or FNC) were examined using resting state fMRI and MEG data obtained from 46 SP and 45 HC (Houck et al. 2017). Our resting fMRI FNC findings are consistent with work indicating hypoactivation in prefrontal cortex (Kuhn and Gallinat 2013). In contrast, prefrontal FNC was enhanced (i.e., hyperconnectivity) in SP for MEG (see bottom panel of Fig. 8--blue), suggesting abnormally high synchronous firing from neuronal populations in prefrontal and temporal networks. It is not possible to determine with certainty from these results whether these hyper-synchronous networks underlie core deficits of the illness, or whether they represent compensation to overcome primary functional defects. However, our MEG results suggesting hyper-synchronous prefrontal and temporal networks are consistent with the dysconnectivity model of schizophrenia (Stephan et al. 2006). Both similar and contrasting patterns of connectivity were identified in fMRI vs. MEG indicating that these modalities which measure brain function at different time scales using fundamentally different measures (hemodynamic vs. electromagnetic activity) provide access to complementary information. This point is reiterated in a separate analysis using the resting state connectivity data to determine classification accuracy with fMRI alone, MEG alone, and combined fMRI and MEG data (Cetin et al. 2016). By using both static and dynamic connectivity measures, the correct classification rate was highest using the combined fMRI and MEG connectivity measures.

These datasets also provide a unique opportunity for fusing data of various types, besides combining data across modalities. The first study mentioned in this section (Cetin et al. 2014) can be considered as multi-task data fusion and the Silva et al. (2014) competition challenge paper requires data fusion across sMRI/fMRI. Stephen et al. (2013) and Sui et al. (2015) provide more examples of the richness of these datasets where MEG data is fused with DTI data, and where MATRICS responses are fused with fMRI/dMRI/sMRI.