Spatiotemporal EEG Dynamics of Prospective Memory in Ageing and Mild Cognitive Impairment
verfasst von:
Mark Crook-Rumsey, Christina J. Howard, Zohreh Doborjeh, Maryam Doborjeh, Josafath Israel Espinosa Ramos, Nikola Kasabov, Alexander Sumich
Prospective memory (PM, the memory of future intentions) is one of the first complaints of those that develop dementia-related disease. Little is known about the neurophysiology of PM in ageing and those with mild cognitive impairment (MCI). By using a novel artificial neural network to investigate the spatial and temporal features of PM related brain activity, new insights can be uncovered. Young adults (n = 30), healthy older adults (n = 39) and older adults with MCI (n = 27) completed a working memory and two PM (perceptual, conceptual) tasks. Time-locked electroencephalographic potentials (ERPs) from 128-electrodes were analysed using a brain-inspired spiking neural network (SNN) architecture. Local and global connectivity from the SNNs was then evaluated. SNNs outperformed other machine learning methods in classification of brain activity between younger, older and older adults with MCI. SNNs trained using PM related brain activity had better classification accuracy than working memory related brain activity. In general, younger adults exhibited greater local cluster connectivity compared to both older adult groups. Older adults with MCI demonstrated decreased global connectivity in response to working memory and perceptual PM tasks but increased connectivity in the conceptual PM models relative to younger and healthy older adults. SNNs can provide a useful method for differentiating between those with and without MCI. Using brain activity related to PM in combination with SNNs may provide a sensitive biomarker for detecting cognitive decline. Cognitively demanding tasks may increase the amount connectivity in older adults with MCI as a means of compensation.
Hinweise
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Introduction
Mild cognitive impairment (MCI) represents an intermediary stage between typical ageing and dementia [1]. Individuals meeting the criteria for MCI are ten times more likely to develop Alzheimer’s disease than those without MCI [2]. While there are currently no cures for dementia-related diseases, understanding early neurophysiological differences between those experiencing cognitive decline and healthy ageing may offer useful ways to evaluate the efficacy of interventions. Cognitive domains such as executive function and attention have been extensively investigated in MCI [3, 4]. However, prospective memory (PM) is posited as a sensitive early biomarker for those at risk of dementia-related disease [5].
Prospective memory is the ability to remember to perform an action at a future point in time (e.g. remembering to take a medication at the correct time) [6]. PM constitutes a large part of everyday memory [7] and everyday memory failures [8]. The ubiquity of PM underpinning many activities of daily living is, therefore, an essential feature of autonomy. Often, PM is one of the first patient-reported complaints to family members or health professionals [9] and is subsequently of clinical relevance.
Anzeige
Little is known about the neurophysiology of prospective memory (PM) in healthy older adults and older adults experiencing mild cognitive impairment (MCI). Neurocognitive research has particularly implicated the anterior prefrontal cortex (aPFC) in PM function, demonstrating the importance of the aPFC in PM encoding, maintenance, and retrieval [10, 11]. Concerning PM intention retrieval, cortical areas such as the insula, posterior cingulate cortex (PCC), and medial temporal lobe (MTL) are of particular importance [12]. These areas, and connections between them, are found to be impaired in older adults with MCI [13, 14].
Electroencephalography (EEG) signals that are time-locked to the presentation of a stimulus or behaviour are known as event-related potentials (ERPs). ERPs provide high temporal resolution of underlying cortical activity [15]. It is understood that there are co-occurrences of the N300 ERP (300–500 ms) over the posterior scalp regions and the frontal positivity in anterior cortices [16], related to cue detection. A late parietal positivity complex (600–1200 ms), known as the prospective positivity, is related to the retrieval of the PM intention from memory [17]. ERP differences between young and older adults are routinely reported in PM [11, 17, 18]; however, only one previous study has evaluated neurophysiological differences in older adults with MCI suggesting poorer PM performance is due to early processing deficits in stimulus familiarity and the ability to reorient attention [19].
Past studies have provided important insights into understanding PM in ageing and cognitive decline. However, most extant analytical techniques create models by separately processing spatial and temporal information. For ERP data, this is problematic as the temporal component may display complex interactions that dynamically alter over time [20]. Recent developments in artificial intelligence allow us to overcome this limitation through models that can learn changes in temporal information over time while preserving the spatial relationship of the data. The third generation of artificial neural networks, known as spiking neural networks (SNN), can model spatiotemporal interactions with spatiotemporal brain data (STBD) in a biologically plausible way [21]. The application of SNNs to dimensionally high STBD has proved to be an effective way of modelling and extracting knowledge from a variety of data sets that possess time and space qualities [21]. Previous studies have proven the efficacy of SNN modelling in fMRI [22], EEG resting-state [23] and ERP data [20, 24], enabling multidimensional learning, which can be interpreted through 3D visualisation, pattern recognition and classification.
Aims and Hypotheses
The current study aims to build on the SNN methodologies for modelling the spatiotemporal dynamics of ERP data. The study proposes new approaches for modelling, learning, visualising and extracting knowledge from ERP data related to working memory, PM and cognitive decline. The study aims to explore and further understand the spatiotemporal and functional differences between younger, older and older adults with MCI. Through the machine learning functionally of SNNs, this study will evaluate the efficacy of using PM as an early indicator of cognitive decline in older adults. For the most part, classification studies of MCI have used resting-state EEG data [25]. However, little research has evaluated the ability to classify brain activity of individuals with and without MCI when performing those tasks most relevant to their diagnosis, namely memory. Evaluating cognitive aspects of memory may be more effective than using resting-state or structural MRI data for classification of individuals with MCI [26]. By comparing the neurocognitive functioning of individuals performing working memory and PM tasks, the current study will determine which of these aspects of cognition is more effective in differentiating between the groups. It would be expected that given PM is one of the first cognitive complaints of those who go on to develop MCI [9], one would expect to find greater classification accuracy for PM stimuli. The current study is broken down into two experiments: firstly, ERP responses to working memory and PM will be modelled and classification accuracies evaluated; secondly, using statistical methods, as demonstrated in previous research [27], new knowledge of local (intra-region) and global (inter-region) connectivity will be extracted from the SNN models.
Anzeige
To this end, the current study hypothesises that (1) there will be differences in visualised SNNs between young adults, older adults and older adults with MCI. (2) SNNs will provide better classification accuracy between the groups when modelling responses to PM stimuli relative to working memory stimuli, and (3) SNNs will have superior classification accuracy compared to traditional machine learning methods. (4) There will be differences in local and global connectivity between young and older adults, and older adults with MCI will have decreased levels of connectivity at the local and global level.
Methods
Participants
Thirty young adults (YA; 13 females, mean age = 24.7 years, SD = 3.43), thirty-nine typically ageing older adults (OA; 24 females, mean age = 72.87, SD = 4.18) and twenty-seven older adults with a diagnosis of MCI (MCI, 12 females; mean age = 77.54, SD = 6.49) were recruited for the study. All participants were right-handed and had no history of dyslexia, no history of drug or alcohol abuse and fluent in English. Older adults with a diagnosis of MCI were referred via memory assessment clinics or through Join Dementia Research. All participants completed the Hopkins verbal Learning Test-Revised (HVLT-R) [28] to ensure that individuals were not likely to have Alzheimer’s disease or experiencing MCI if part of the healthy ageing group. Participants also completed the geriatric depression scale [29] to ensure symptoms of depression did not confound the results. The study was approved by the Health Research Authority, UK (REC reference: 17/EM/1010).
Procedure
Participants were sat approximately 60 cm from an LCD monitor. EEG equipment was attached before impedance checks were made. Participants firstly completed an ongoing-only working memory task before completing two PM tasks. The two PM tasks built upon the ongoing task with the PM cues embedded within the stream of ongoing task stimuli (Fig. 1). Participants, therefore, completed the PM task and the ongoing task simultaneously. PM instructions were given at the start of the task and were followed by a delay before the first PM cue appeared. In line with past research [30], only 10% of all stimuli were PM cues allowing participants to re-engage with the ongoing stimuli and better simulate real-life PM events.
Fig. 1
Experimental prospective memory (PM) paradigm. a The 1-back (ongoing-only) word categorisation task. Arrow direction indicates the evaluation of stimulus in relation to the previous word. Related words are those which are from the same category as the previous word and require a response. Unrelated words are those that did not belong to the same category as the previous word and did not require a response. b The embedded prospective memory tasks. Examples of the perceptual and conceptual prospective memory cues are highlighted with grey bars
×
Ongoing Task
A 1-back word categorisation task was used as the ongoing task (Fig. 1a). Participants made continuous decisions of whether the word presented is of the same category as a preceding word. Participants were instructed to press a button on a response box with the right index finger if the word was semantically related to the previous word. The ongoing task compromised 300 stimuli from the category norm data base [31] with a 25% chance of a word belong to the same semantic category as the previous category. Each word was presented for 500 ms with a 2-s stimulus onset asynchrony between words. Stimuli requiring a response during the ongoing task will be referred to as a 1-backtarget.
Prospective Memory Task
The PM paradigm (Fig. 1b) incorporates two PM conditions: perceptual and conceptual, where perceptual relates to visual features of the cue which prompts a PM response and conceptual relates to a semantic cue that is less salient than perceptual cues and therefore requires greater attention to remember. Participants were instructed to press a button with their right index finger when a word was of the same semantic category (ongoing task) and to press another button with their left index finger if there was a PM cue. For the perceptual PM condition (PMpercept), participants were told to respond to words appearing in capitals (e.g. ‘SWORD’). For the conceptual PM condition (PMconcept), participants were told to respond to the word of four-footed animals (e.g. ‘cat’). Each PM task contained 600 stimuli 10% of which were pseudo-randomly presented PM stimuli.
Electrophysiological Recording and Processing
Electroencephalographic activity was recorded with a 128-channel Active Two Acquisition system (BioSemi, Amsterdam, Netherlands) sampling at 2048 Hz. Data were collected using ActiView V6.05 (National Instruments, TX, USA). EEG preprocessing was performed in MATLAB R2019a (The Mathwords, Inc) using custom scripts and EEGLAB. Data was referenced to linked mastoids and downsampled to 256 Hz. A high-pass finite impulse filter (FIR) was applied at 0.01 Hz and a low-pass FIR filter at 35 Hz. Line noise was removed using CleanLine. Independent components (runica analysis) were visually inspected and artefacts were rejected. ERP epochs were 1000 ms from stimulus onset with a baseline correction.
Spiking Neural Network Computational Architecture for Modelling and Visualising Working Memory and Prospective Memory Activity
The proposed SNN architecture is an evolving spatiotemporal data machine (eSTDM) modelled on neuromorphic, brain-inspired SNN processing concepts [32]. It is designed to map brain data into a 3D brain space of spiking artificial neurons (AN) while preserving the topological information of the recorded brain activity. Principally, this architecture draws its inspiration from the biological rules (e.g. small-world (SW) connectivity and leaky-integrate and fire models (LIFM)), which govern memory learning dynamics of neurons exhibited within the brain.
Each AN within the SNN behaves as an information-processing unit. It learns from the temporal data that is propagated through it, adapting and memorising the patterns of activity by influencing the interconnected neurons within the network. Akin to the brain, SNNs incorporate time into their computation and thus are superior in biological plausibility compared to previous neural networks that do not account for temporal dynamics. The architecture to be employed processes several different modules based on evolving SNN framework [33]. As illustrated in Fig. 2, the modules consist of an input-encoder module (Fig. 2d), where data is encoded into spike-trains and the spatiotemporal variables are mapped into input neurons that transfer the spike-trains to the SNN model; a 3D SNN module (Fig. 2e), where the characteristics of space and time are recorded and learnt in an unsupervised mode; a visualisation module (Fig. 2f), where captured spatiotemporal connectivity of the brain can be visualised; a SNN classification module, where the spatiotemporal patterns from the 3D SNN module are classified or used to predict an output (Fig. 2g); an optimisation module (Fig. 2h), to fine-tune the parameters of the system; a pruner module (Fig. 2i) where inactive ANs are removed and only functional ANs (ANs that emitted spikes during the unsupervised learning) and neural connections are retained for further analysis. The following steps detail the methods applied in this study:
1.
The temporal data are encoded into sequences of spikes using the threshold-based representation algorithm [34].
2.
A 3D SNN model of LIFM ANs is created, where the spatial mapping of the ANs is defined using the Talairach brain template [35].
3.
The EEG channels are mapped as input ANs to their corresponding location in the Talairach template.
4.
The mapped SNN model is initialised, where ANs are connected using the SW connectivity proposed in Braitenberg and Schüz [36].
5.
The initialised SNN model is trained with the encoded spike sequences from ERP data, entering via the input ANs. The learning rule is the unsupervised spike-time-dependent plasticity (STDP) [37] that changes the weight of the connection between every pair of connected ANs. During this process, the SNN model learns from the temporal information and forms pathways that can be interpreted, which the SNN will use to classify new information.
6.
The spike sequences of the EEG data are again propagated through the SNN for supervised learning related to the classification tasks. Output ANs are created for each sample (i.e. one output AN for each participant). Each output AN is connected to all ANs of the 3D SNN model.
7.
The deSNN algorithm is applied for supervised learning [38] and adapts the connections between the 3D SNN model and the output ANs.
8.
For the classification of new temporal data, steps 6 and 7 are repeated. Then, the data are classified by applying the K-nearest neighbours algorithm using the K nearest (similar) output ANs created during step 6 to the new output AN.
Fig. 2
Proposed Spiking Neural Network Architecture for prospective memory ERP analysis. ERP data collection across the three participants groups: a younger, healthy older and older adults with mild cognitive impairment. b recording of the data during the experimental working and prospective memory tasks. c ERP data is extracted and cleaned. d cleaned ERP data is transformed into spike-trains. e spike-trains for each EEG channel are propagated into a 3D space of artificial neurons via the input neurons which are depicted as green artificial neurons. f trained SNN can be visualised. g output neurons are created and represent the final classification of the data. Each of the artificial neuron connects to the output neurons. h a grid search method is used to find optimal parameter settings for classifying between the participant groups i pruner module removes all connections which did not change for each group to create sparse models for each participant group and are visualised
×
Input ERP Data Encoding in the SNN Model
The pre-processed and baseline corrected ERP data were firstly ordered into a temporal sequence of real-value vectors. These vectors were then encoded into a series of discrete spike-trains using a threshold-based representation method (TBR), demonstrated to be able to construct large scale networks with arbitrary, configurable synaptic connectivity [39]. This algorithm is employed to identify relevant changes in the ERP signal thus reducing noise. If the value of change in the signal surpasses a predefined threshold value, then a spike is encoded. Upward and downward changes in the ERP data are interpreted as positive or negative (1 or − 1) spikes. Bi-directional algorithms like this are well suited to EEG data due to their sensitivity to significant changes within gradient signal changes.
SNN Initialisation, Mapping, Learning, Classification and Visualisation
Following the encoding of the ERP signal to spike-trains (Formula 1, Appendix 1), a 3D SNN structure was created that can map the functional and structural characteristics of the data from which it is recorded. To this end, Talairach coordinates [35] were used to map the 128 EEG channels to the 3D SNN model (visualised as the green ANs in Fig. 2e. These coordinates define the position of the spiking ANs in a brain-like SNN model and the position of the EEG electrodes [40] as the input ANs. Each neuron in the network then represents one cm3 of the human brain and the entire network consists of 1471 ANs [41].
A LIF architecture was used to model the ANs [42] (Formula 2, Appendix 1). The SNN was initialised according to a biologically plausible model of SW connectivity [43], where neurons that are topographically closer possess stronger interconnectedness and therefore capture patterns of interest from the model. Following completion of the unsupervised learning, a deSNN algorithm [38] was used to train an output classifier in a supervised learning method. A RO learning rule [44] was applied to initialise the connection weights, and then the STDP rule [37] was used to adjust these weights according to the spikes that follow the initial spikes to the postsynaptic AN (Formula 3, Appendix 1). The STDP accounts for the timing of pre and postsynaptic action potentials causing automatic adjustments to be made to the synaptic strengths and sensitivity of the postsynaptic ANs and consequently captures the spatiotemporal dynamics of the input data. Two other important variables for the classifier module are mod and drift (Formulas 4 and 5, respectively). Each training sample provided to the model is associated with an output AN, which is connected to all the other ANs in the 3D SNN and the connection weights are initially set to zero. The weights of these output ANs change as a function of the RO learning rule, which itself is calculated by the order of incoming spikes (mod) from different connections. The earlier a spike arrives in the output AN from 3D SNN, the greater its importance in increasing the corresponding connection weights. In terms of ERP data, it is useful to think of this as where the greatest amount of emphasis is placed within the ERP (i.e. toward to start or the end of the ERP). These newly formed connection weights will then increase or decrease according to the following number of spikes (drift) when the next spikes arrive at the AN over time.
Experimental Framework
To extract the learnt patterns of activity of the SNN models, computational experiments are performed for each of the memory tasks and their class (i.e. participant group). Each class contains n samples which are used to train the SNN model and are validated through a tenfold cross-validation to assess the accuracy of the model. Once the best model is found, then it is possible to extract the individual contributions of each of the network classes over epochs of interest (200–400 ms, 400–800 ms, 0–1000 ms). After training, those ANs that did not emit a spike were identified and removed (pruned) along with their connections.
Knowledge Extraction from SNN Models
To further extract knowledge from the SNN models from the SNN models. Knowledge extraction was performed in two ways:
1.
ANOVA was applied to test for differences in local connection weights in each of the SNN models as a function of group and topography (scalp region).
2.
Network analysis was applied to uncover the global neurocognitive interactions between the different topographical areas for each of the SNN models.
The weights of the input ANs (i.e. EEG electrodes) were averaged according to the outlined topographical clusters depicted in Fig. 3 (Appendix 2), which were informed through previous PM ERP research [16, 30, 45, 46]. Averaging the connection weights of the input neurons offers a way of understanding differences in local connection weight changes between the groups within a specific area. Average clusters were created as a means of controlling the number of comparisons [47, 48] and to capture the mean of the AN weight changes within an area connected to input AN. Analyses were performed in JASP 0.10.2. A series of mixed measures ANOVAs was performed for each of the created SNN models. Therefore, each stimulus type at each time point (i.e. 200–400, 400–800 and 0–1000) was analysed separately using a 4 (Cluster: frontal, central, parietal, occipital) × 3 (Group: YA, OA, MCI) ANOVA to evaluate group differences at midline clusters. For lateral clusters, a 7 (Cluster: frontal, frontocentral, central, frontotemporal, parietal, inferior parietal, occipital) × 2 (Hemisphere: left, right) × 3 (Group: YA, OA, MCI) ANOVA was used to analyse group differences in the networks. Post-hoc (linear contrasts) tests were used to further explore group differences, and Bonferroni corrections were applied to account for multiple comparisons. Partial eta squared was reported for each Group effect as an indicator of effect size [49]. Previous research has demonstrated the effectiveness of using ANOVAs for understanding local intra-region wiring in young adults with and without autism spectrum disorder [27, 50].
Fig. 3
Topographical clusters used for ANOVAs and network analyses. Purple, frontal; green, frontocentral; yellow, central; red, frontotemporal; blue, parietal; pink, occipital; grey, inferior parietal
×
The averaged connection weights were then used for the network analysis to understand global connectivity between areas across the cortex at each time frame. The network used describes a graphical representation of the correlations between each of the clustered weights. In these networks, clusters are represented as nodes and the correlations as edges connecting/wiring nodes together. The line thickness and transparency of the network graph represent the strength of the correlation, where thicker edges represent stronger correlations. The generated network displays are fixed in line with the cluster layout of the scalp map in (e.g. Fig. 10) for ease of comparison between groups.
A network model that analyses all possible correlations within the network requires the estimation of many parameters, including n threshold parameters for the nodes and n*(n-1)/2 for pairwise correlations between nodes. Estimations in the current study are equal to 153 parameters. One available solution to this problem is to apply the ‘least absolute shrinkage and selection operator’ (LASSO) technique [51]. This method enables some edges to shrink to zero and be omitted from the model. This is achieved by LASSO through the continuous shrinking of coefficients toward 0 as λ increases. A benefit of applying LASSO is the ability to handle more variables than observations [52, 53]. However, given the high expected correlations between topographically close variables, i.e. averaged cortical clusters, the irrepresentable condition assumption would be violated [53]. This assumption requires that variables relevant to the model may not be highly correlated with irrelevant variables. An alternative method proposed by Zou [54], known as ‘adaptive least absolute shrinkage and selection operator’ (adaLASSO, Formula 6, Appendix 1), can be employed to adjust for this violation. While variables are all equally penalised with the LASSO method, variables are assigned different weights in adaLASSO and can subvert the irrepresentable condition assumption.
The adaLASSO produces a sparse, more conservative network model with only a small number of edges enabling a more interpretable model of the relationship between node weights for each participant group. Prior to analysis, a tuning parameter is required to control the level to which the omission of small correlations is applied. This tuning parameter was selected through bootstrapping and was validated using cross-validation. The importance of each node in the network was then evaluated through betweenness and degree indices. Betweenness provides a measure of the number of shortest paths passing through a specific node. A node with higher betweenness is said to have more network control as more information is passing through that node [55]. Degree is the total amount of connections each node has, therefore indicating the strength of links to other areas (reported as ‘node strength’). A node with a higher degree can be thought of as having a greater influence on connecting nodes [56].
Graphical models of brain data have proven their efficacy in a variety of imaging modalities, such as fMRI [57], EEG [58, 59], magnetoencephalography (MEG) [60] and DTI [61]. Of these, sparse graphical models are distinctly efficient at determining connectivity between in highly interconnected brain data and at offering a robust and interpretable model of the most significant interactions between cortical areas [62].
Results
ERP Data Modelling Using the SNN Architecture
Figure 4 provides an overview of the steps of the current study. To explore the differences in cognition between YA, OA and MCI, an SNN architecture used to model, learn, classify and visualise the EEG data related to the different memory tasks (1-backtarget, PMpercept and PMconcept). A SW connectivity radius of 2.5 units (distance between two consecutive neurons) was used for the SNN model, which has previously demonstrated its effectiveness for ERP modelling using SNN [24]. The SW connectivity rule allows the network the potential to form neuronal connections two ANs away in each of the x, y, z directions of the coordinate space. Small random weights are applied to each neuron (−0.1, +0.1). In previous studies modelling EEG data with SNNs, an 80/20 positive–negative initial connection weight ratio has been applied [23, 63]. This ratio of inhibitory neurons is reflective of the 20–30% of inhibitory, GABAergic neurons found in the mammalian brain and is demonstrated as an optimal percentage for maximising the learning of a neural network [64, 65]. However, initial results demonstrated that this ratio was not optimal for modelling the current data. Figure 5 illustrates the trained network for the 80/20 positive–negative ratio (a) and the 50/50 positive–negative ratio (b). The results show a greater amount of model learning for the 50/50 ratio as evidenced by the greater amount of connection changes from the initial connections. Therefore, a model containing 50% inhibitory connections demonstrated a better level of discrimination between the classes and was subsequently used for modelling the ERP data.
Fig. 4
Flow-chart of the proposed work for understanding spatiotemporal dynamics of prospective memory between younger, older and older adults with mild cognitive impairment
Fig. 5
The number of connections and the connection weights in the SNN models for the 80/20 positive to negative neuron ratio model a and the 50/50 positive to negative neuron ratio model b
×
×
Similar to a biological neuron, when the simulated LIF AN receives spikes over time, its membrane potential increases until it reaches a pre-defined threshold. When the AN fires and emits an output spike, it cannot produce a new spike within a refractory period and its membrane potential is said to leak. The membrane potential can have certain leakage between spikes, which is defined by a leak parameter. The training of the SNN model requires EEG signals to be transformed into a spike-train of binary positive and negative spikes (−1 or 1; Fig. 6). These spikes reflect the changes in amplitude of the EEG signal and are created based on an encoding algorithm. A bi-directional TBR [34] was applied to all the EEG channel signal’s gradient relative to the time series. The neural connections in the initialised SNN model were later modified during an unsupervised learning process with the input spikes steaming to the SNN model via input ANs.
Fig. 6
Example of how an event-related potential encoded into a spike-train. a Event-related potential taken from one participant (electrode Cz) in response to a conceptual prospective memory stimulus. b Event-related potential encoded into a binary spike-train using the TBR algorithm
×
The model was then trained using these spikes-trains in an unsupervised mode employing a STDP learning rule [37]. The application of this algorithm allows spiking ANs to learn successive temporal relationships between data points from the data across and within EEG channels. These connections in the model architecture can be analysed and used to draw new understanding of the data. Figure 7 shows the final SNN following the creation of neuronal connections created during STDP learning, which reflect the dynamic patterns of connectivity.
Fig. 7
The learnt patterns of activity from the initialised SNN models. The top SNN models show the initialisation of random connections. The bottom SNN models show the learnt patterns of spatiotemporal activity for each of the different stimuli
×
When the supervised learning process is completed, the connection weights between the output ANs and the 3D SNN model are established. Then in the validation phase, the new ERP samples which were excluded from the learning phases are used to test the model. For every new testing ERP sample, an output testing AN is evolved and connected to the already trained SNN model, and its connections are modified while the ERP sample is passed to the SNN model. Then for classification of this testing AN, a K-nearest neighbour (KNN) algorithm was used, where the newly formed testing AN connection vector is compared with the existing output ANs’ connections, and the top k similar output ANs (referring to the top similar ERP samples) identify the class label of this testing AN (ERP sample). This procedure is repeated for all the testing samples, one by one, through creating different output testing samples and classifying them.
A grid search method was used in the current study for fine-tuning a combination of parameters and reducing the classification error. Each parameter was searched within a range, specified by a minimum and maximum, through 5 iterations. A tenfold cross-validation was used to validate the results. Therefore, for every model creation, 78,125 iterations of training (using 9 folds of samples except the holdout fold) and testing (using the holdout fold) were performed with different combinations of these parameters. The parameters that resulted in the best accuracy have been reported as the optimal parameters:
The threshold for firing was set to 0.5, the refractory time was set to 5 and the LIF neuron model was set to 0.005.
The STDP rate of the unsupervised learning algorithm was set to 0.002 for positive synaptic modifications and 0.003 for negative connections.
The mod parameter was set to 0.4 and the positive and negative drift was set to 0.002, 0.004, respectively.
The KNN was set to 13 nearest neighbours.
The optimisation procedure finds the best performing model for each of the SNN models. Table 1 presents the final test-fold of the best performing model (i.e. the held-out fold). Each of the SNN models was compared against other machine learning methods. Model accuracies was calculated using confusion matrices. At each memory stimulus type, the SNN model outperformed the other methods. Additionally, the results show that the PMpercept and PMconcept SNN models were better at classifying brain activity of the groups (83.33% and 80%, respectively) compared to the 1-backtarget models (73.94%). It is also seen that with the use of ERP data, YA are well discriminated from the other two groups, while the ERPs of OA and MCI overlap to a certain degree. From the trained 3D SNN networks, we can begin to see the patterns of connectivity emerge.
Table 1
Classification results of the ERP samples across the three class groups
ERP data classes
YA
OA
MCI
Accuracy (%)
Total accuracy (%)
SNN-based methodology (1-backtarget)
YA
8
0
0
100
73.94
OA
2
9
0
81.82
MCI
0
3
2
40
Traditional machine learning methods
Methods
Accuracy (%)
MLP
49.94
SVM
50.29
MLR
46.98
SNN-based methodology (PMpercept)
YA
8
0
0
100
83.33
OA
0
9
1
90
MCI
0
2
3
60
Traditional machine learning methods
Methods
Accuracy (%)
MLP
62.07
SVM
47.28
MLR
50.29
SNN-based methodology (PMconcept)
YA
8
0
0
100
80
OA
1
8
1
80
MCI
0
2
3
60
Traditional machine learning methods
Methods
Accuracy (%)
MLP
50.8
SVM
44.52
MLR
51.08
Table 2
Skewness and kurtosis and normality test p-value from a Kolmogorov–Smirnov test of the SNNs following pruning
Group
Skewness
Kurtosis
p-value
1-backtarget normality test
YA
−0.70
4.80
< 0.001
OA
−0.70
4.79
< 0.001
MCI
−0.67
4.71
< 0.001
PMpercept normality test
YA
−0.81
4.50
< 0.001
OA
−0.78
4.44
< 0.001
MCI
−0.75
4.35
< 0.001
PMconcept normality test
YA
−0.80
5.03
< 0.001
OA
−0.84
5.10
< 0.001
MCI
−0.78
4.93
< 0.001
To understand within-group differences, the training samples were separated by propagating only the information for that class (i.e. YA, OA or MCI) through newly created networks that kept the same initialised network and parameter settings attained during the supervised learning stage. By using the same initialised connections and only allowing the EEG data from one group to make changes to the network, different patterns of connectivity for each group can be learnt. Thus, three separated SNN models were trained with each of the classes. The initialised SNN models were modified during the STDP learning that adapted the spatiotemporal connections. After the training, those neural connections that had not changed for each model were considered inactive and were pruned from the network.
The removal of inactive ANs enables the creation of a fine-tuned, sparse networks [66] showing only the most important connections for the pre-trained class, enabling better visualisation of the differences between the groups. This step was performed across three different time periods to reveal the neural connections across time. The first time period was selected as 200–400ms to capture the early cognitive processes associated with cue detection and monitoring [67]. The second time epoch was 400–800ms, which encapsulates the later processing of stimuli and is related to deeper contextual and memory processes [16]. Finally, the full epoch was propagated through the network for each class to understand the learnt connections across the entire data range. The pruned networks can be visualised for each stimulus type in Fig. 8 for the 1-backtarget, Fig. 9 for PMpercept and Fig. 10 for PMconcept.
Fig. 8
1-backtarget SNN following pruning at 200–400 ms, 400–800 ms and 0–1000 ms for each of the three classes: younger adults (YA), healthy older adults (OA) and older adults with MCI (MCI). Positive connections are displayed in blue, and inhibitory connections are displayed in red. The amount of positive–negative connections is shown under each pruned model
Fig. 9
PMpercept SNN following pruning at 200–400 ms, 400–800 ms and 0–1000 ms for each of the three classes: younger adults (YA), healthy older adults (OA) and older adults with MCI (MCI). Positive connections are displayed in blue, and inhibitory connections are displayed in red. The amount of positive–negative connections is shown under each pruned model
Fig. 10
PMconcept SNN following pruning at 200–400 ms, 400–800 ms and 0–1000 ms for each of the three classes: younger adults (YA), healthy older adults (OA) and older adults with MCI (MCI). Positive connections are displayed in blue, and inhibitory connections are displayed in red. The amount of positive–negative connections is shown under each pruned model
×
×
×
From Fig. 8, it is apparent that there are similar patterns of activity occurring for each of the defined time periods across the groups. However, it is difficult to see how the network is differentiating between the classes at both the 200–400ms and 400–800ms time range. Differences appear to be more prominent when the whole 0–1000ms epoch is pruned for each of the classes. This may suggest that information outside of the 200–800ms epoch is important in understanding ageing and cognitive decline. In particular, differences between OA and MCI appears to be substantial in the 0–1000ms PMconcept model. There are fewer positive and negative connections for the MCI group. Interestingly, the pruning method appears to have removed many ANs in the left frontotemporal region for the 0–1000ms model in the MCI group in each of the models, implying relatively little spatiotemporal activity for the 1-backtarget during the working memory task.
Additionally, for PM stimuli across the wider epoch (0–1000ms), MCI demonstrated fewer positive and inhibitory connections relative to OA and YA. In the PMpercept stimuli, these inhibitory connections are spread more globally across the 3D SNN for healthy groups (YA, OA), but within the MCI group; the inhibitory connections are restricted to occipitoparietal, frontocentral and frontal regions. It appears inhibitory connections are spread similarly across the network, albeit fewer overall inhibitory connections for the MCI group. However, there appeared to be more positive connections for the healthy groups relative to MCI.
To validate the visualised changes from the pruned SNN models, histograms were plotted showing the pruned SNN connections weights (Fig. 11a). Compared to the initialised weights of each SNN, each model placed more emphasis on negative connection weights through training. Moreover, it is apparent that the weights now follow a somewhat Laplacian distribution, characterised by the heavy tails as demonstrated in the QQ-plots (Fig. 11b) and the high Kurtosis values (Table 2), with the addition of the failure to reject the distribution being from a normal distribution (Table 2). This distribution type has been shown to respond well to variable selection features, such as different applied methods of the adaLASSO [68].
Fig. 11
Validation of changes in SNN models. a Connection weights of each of the SNN models following pruning. Left to right: 1-backtarget, PMpercept, PMconcept. b QQ-plots of distributions
Table 3
Summary of significant effects 1-backtarget SNN ANOVAModels
Lower
F-value
DF
p-value
ηp2
Post-hoc tests
200–400 ms lateral SNN weights
Group
9.57
2,87
< 0.001
0.19
Cluster*group
2.06
10.27, 446.07
0.025
0.05
F
5.71
2,87
0.005
0.12
YA > OA = MCI
FC
4.79
2,87
0.011
0.11
YA > OA
C
9.81
2,87
< 0.001
0.20
YA > OA > MCI
FT
5.69
2,87
0.005
0.12
YA > OA = MCI
P
3.48
2,87
0.035
0.08
YA > MCI
IP
4.47
2,87
0.014
0.10
YA > OA = MCI
YA
15.93
6,162
< 0.001
0.37
C > FC = F = FT = IP > P = OC
OA
18.58
5.36, 198.39
< 0.001
0.33
C > FC = F = FT = IP = OC > P
MCI
8.31
6,75
< 0.001
0.33
C > FC = F = FT = IP = OC > P
400–800 ms lateral SNN weights
Group
12.67
2,87
< 0.001
0.24
YA > OA = MCI
Clusters: F frontal, FC frontocentral, C central, FT frontotemporal, P parietal, IP inferior parietal, YA young adults, OA healthy older adults, MCI older adults with mild cognitive impairment
N.B. ‘ > ’ represents greater connections weights in this table
×
Knowledge Extraction from Spiking Neural Networks
Due to the non-normal distribution of the SNN weights, data connection weights were firstly transformed using a natural logarithm. The adaLASSO regularisation was performed to discover the most important connections for each pruned epoch for the early (200–400ms) and later (400–800ms) pruned SNN epochs, along with the complete (0–1000ms) epoch. A 10-fold cross-validation was performed for each of the variables. To increase the robustness of the results, each process was repeated 1000 times. Network plots were mapped to a scalp array dependent on their topographical features. For example, Fig. 12 graphically illustrates the most important edges within the working memory (1-backtarget) network as a result of non-essential connections being forced to zero. The thickness and colour intensity of the lines are proportional to the wiring connection strength given as edge weight (EW). Positive wiring connections are displayed in blue and represent strength of a positive correlation between two regions and red line show the strength of a negative correlation of two regions negative connections are displayed in red. From these networks, the differences and similarities between groups are revealed.
Fig. 12
Working memory features extracted from the trained SNN using adaLASSO network analysis across the 200–400 ms, 400–800 ms and across the entire 0–1000 ms trained epoch. YA, young adults; OA, healthy older adults; MCI, older adults with mild cognitive impairment. Blue wiring connections show positive associations between clusters and red wiring connections show negative associations between clusters
×
Ongoing Working Memory (1-backtarget)
A summary of all significant local connectivity Group effects and interactions for the 1-backtarget stimuli is presented in Table 3. No Group differences were found at midline clusters or in the 0–1000 ms SNN model. Graphical networks are presented in Fig. 12.
Table 4
Summary of significant group effects and interactions for the perceptual PM SNN ANOVA models
Lower
F-value
DF
p-value
ηp2
Post-hoc tests
Lateral 200–400 SNN weights
Cluster*group
1.87
12,510
0.036
0.05
F
5.30
1,87
0.007
0.12
OA > MCI = YA
YA
10.74
6,168
< 0.001
0.29
F = FC = C = P = OC > FT = IP
OA
16.40
6,198
< 0.001
0.33
F = C | F > FC = C = FT = P = OC | F, FC, C, P, OC > IP
MCI
6.76
6,132
< 0.001
0.26
C, F, OC, P > IP | C > FC > P
Lateral 400–800 ms SNN weights
Cluster*hemisphere*group
1.90
12,510
0.046
0.05
Cluster*hemisphere
YA
15.21
6,168
< 0.001
0.36
F, FC, FT, P, IP, OC: L > R | L: FT & C > FC | P > OC | R: FT = C = IP > FC = IP = OC > F
OA
14.41
6,198
< 0.001
0.30
F, FT, OC: L > R | L: FT = C = P = C = IP = OC > FC | R: FT > C, F, FC, OC | P > F, FC, OC
MCI
13.31
6,132
< 0.001
0.45
F, FC, C, FT, IP, OC: L > R | P: R > L | L: FT > P = IP = OC = FC = F | R: FT > all clusters | C = IP > P = OC = FC = F | IP > F
Lateral 400–800 ms SNN weights
Cluster*group
R
1.90
12,510
0.033
0.05
C: YA > OA = MCI
Hemisphere*group
F
5.54
2,87
0.006
0.12
R: OA > YA | YA, OA, MCI: L > R
P
5.56
2,87
0.005
0.12
L: YA > MCI | YA: L > R
Lateral 0–1000 ms SNN weights
Cluster*group
3.29
3.07,130.50
0.022
0.08
FT
4.12
2,87
0.020
0.09
YA > MCI | YA > OAa
YA
116.09
1.62,45.37
< 0.001
0.81
F > FC, IP, P, OC | C > FC | all clusters > FT
OA
89.17
1.54,56.91
< 0.001
0.73
F > IP, P, OC | C > IP | IP > OC | all clusters > FT
MCI
33.60
1.43,31.45
< 0.001
0.65
F, FC, P, OC > IP | all clusters > FT
Clusters: F frontal, FC = frontocentral, C central, FT frontotemporal, P parietal, IP inferior parietal, OC occipital, L left hemisphere, R right hemisphere, | separator between post hoc-tests, YA young adults, OA healthy older adults, MCI older adults with mild cognitive impairment
N.B. > in this table represents greater connection weights
a trending toward significance
Ongoing: 200–400 ms ANOVA Connection Weights
At bilateral clusters, there was a significant Cluster × Group interaction due to greater connection weights for YA relative to OA (frontal, p = 0.019; frontocentral, p = 0.009; central, p < 0.001; frontotemporal, p = 0.005; inferior parietal, p = 0.029) and YA relative to MCI (parietal, p = 0.011). OA had greater connection weights at bilateral central clusters relative to MCI (p = 0.008). The interaction was also explained by an effect of Cluster for all participants, where bilateral central clusters had larger connection weights than all other clusters (ps < 0.001). In YA, bilateral inferior parietal cluster connection weights were larger than in parietal clusters (p < 0.001).
Ongoing: 400–800 ms ANOVA Connection Weights
At lateral clusters there was a significant effect of Group, where YA had significantly greater connection weights across all bilateral clusters than OA (p < 0.001) and MCI (p < 0.001). No differences were found between the OA and MCI groups (p > 0.05). There were no other significant Group effects for 1-backtarget stimuli.
Ongoing Task: Network Analysis 200–400 ms
For the 200–400ms network analysis of SNN models of the working memory ongoing task, the number of edges reduced to zero was approximately 95%, 97% and 98% for YA, OA and MCI, respectively. The centrality indices reveal a common area of importance across all participants groups over the right parietal cluster (YA node strength = 1.49; OA node strength = 1.73; MCI node strength = 1.18). The cluster interactions that are formed with the right parietal are as follows: the right inferior parietal (EW = 0.64) for the YA; the left inferior parietal in the OA (EW = 0.53) and the right frontocentral in the MCI group (EW = 0.61). For both the YA and OA networks, there are connections from the frontal clusters to the frontocentral clusters (YA EW = 0.70; OA EW = 0.32), albeit on the opposite hemispheres. The hemispheric symmetry between OA and YA is also found over frontocentral clusters, whereby the mid central cluster wire with left frontocentral clusters in the YA (EW = 0.51) but is instead shared with the right frontocentral cluster for OA (EW = 0.17). The same wiring patterns are not found within the MCI group.
Network Analysis 400–800 ms
Edges were reduced by approximately 92% for young adults, 98% for OA and MCI networks over the 400–800ms epoch. In the OA and MCI networks at 400–800ms, importance is placed occipitally such that most connections are retained within the occipital clusters relative to the rest of the cortex. In the OA group, the mid occipital cluster was wired to right occipital (EW = 0.46), which in turn was connected to the right central (EW = 0.46) and the left occipital cluster (EW = 0.26). In the MCI group, the mid occipital cluster had a positive connection with the left occipital cluster (EW = 0.63) and a negative connection with the right frontocentral cluster (EW = −0.43). Both the OA and MCI groups retain network relevance for the left occipital cluster (OA node strength = 0.46; MCI node strength = 2.64), but the right occipital cluster is also important in the OA group (node strength = 3.45). A different connection pattern is found for the YA group, where connectivity is localised over the right frontotemporal clusters with connections to right inferior parietal, right parietal, right temporal and right central (average EW = 0.32) with the right inferior parietal cluster demonstrating the greatest network control (betweenness = 2.62).
Network Analysis 0–1000 ms
For the entire epoch, edges were reduced to approximately 95% for YA, 98% for OA and MCI groups. Across all groups, the mid central cluster demonstrates the largest node strength (YA = 0.89; OA = 2.03; MCI = 2.40) along with connections being retained between mid central–mid parietal clusters (YA EW = 0.55; OA EW = 0.73; MCI EW = 0.78). The network analysis shows significant node strength for the left frontocentral cluster for all groups; the YA demonstrate the lowest left frontocentral node strength (0.04) compared to the older adult groups who exhibit comparable strength indices (OA = 1.25; MCI = 1.03). However, network-wiring differences are apparent between the older adult groups. From the left frontocentral cluster, OA had positive wiring with the mid parietal (EW = 0.17) and mid frontal (EW = 0.15) clusters. The MCI group demonstrated a negative connection from the left frontocentral cluster with the right inferior parietal cluster (EW = −0.42). Moreover, positive connections were found in anterior scalp regions in the YA. In the YA group, the right frontocentral cluster had the largest node strength (2.15). The right frontocentral was connected to the left frontotemporal cluster (EW = 0.47) and the left frontal cluster (EW = 0.62), which in turn shared connection with the left frontocentral cluster (EW = 0.31).
A summary of all statistically significant Group effects from the ANOVAs is presented in Table 4. No significant Group differences were found over midline clusters. Graphical networks are presented in Fig. 13.
Table 5
Summary of significant effects for 200–400 Conceptual PM SNN ANOVA Models
Lower
F-value
DF
p-value
ηp2
Post-hoc tests
Lateral 200–400 SNN weights
Cluster*groups
2.73
8.65,376.22
0.005
0.06
C
9.59
2,87
< 0.001
0.19
YA > OA, MCI
IP
3.53
2,87
0.034
0.08
YA > OAa
OC
4.14
2,87
0.019
0.09
YA > OA
YA
16.19
3.82,106.92
< 0.001
0.37
OC > F, FC, C, FT, P | IP > FC, FT |P > FC
OA
30.17
3.62,133.82
< 0.001
0.45
C < all clusters | OC > all clusters | IP > F, FT
MCI
25.58
3.72,81.88
< 0.001
0.59
OC > all clusters | F, IP, P > C | IP, P > FT
Lateral 400–800 ms SNN weights
Cluster*hemisphere*group
1.96
10.30,448.12
0.035
0.05
Cluster*hemisphere
YA
16.44
4.01,112.39
< 0.001
0.37
F, FT, P, IP, OC: L > R | R: F, C, P > FT = OC | IP > OC | L: FT < all clusters | C > F = IP | OC: R > L
Lateral 400–800 ms SNN weights
Cluster*group
OA
14.58
4.54,167.95
< 0.001
0.28
F, FT, IP, P: L > R | | R: F, FC, C, OC > FT = P | OC, C > IP| L: F = P = IP > FT | P > FC = OC | OC: R > L
MCI
15.57
4.65,102.27
< 0.001
0.46
F, FT, IP, P: L > R | OC: L > R | L: IP > FC | P > OC = C| R: C = OC > FT = P | C > F > FT
Left
2.33
8.29,360.50
0.018
0.05
F: YA > OA | FC: YA > OA = MCI | C: YA > MCI = OA| P: YA > OA
Hemisphere*group
Right
3.50
10.66,463.87
< 0.001
0.08
F: YA > MCI | C: YA > OA = MCI | FT: OA > YA | P: YA > MCI
FT
3.51
2,87
0.035
0.08
R: OA > YA
Lateral 0–1000 ms SNN weights
Hemisphere*group
3.89
2,87
0.025
0.09
R
4.23
2,87
0.015
0.10
MCI > YA
OA
25.18
1,37
< 0.001
0.41
R > L
MCI
9.46
1,22
0.007
0.35
R > L
Clusters: F frontal, FC frontocentral, C central, FT frontotemporal, P parietal, IP inferior parietal, OC occipital, L left hemisphere, R right hemisphere, | separator between post hoc-tests, YA young adults, OA healthy older adults, MCI older adults with mild cognitive impairment
N.B. > represents greater connections weights in this table
atrending toward significance
Fig. 13
Perceptual prospective memory (PMpercept) features extracted from the trained SNN using network analysis (adaLASSO) across the 200–400 ms, 400–800 ms and across the entire 0–1000 ms trained epoch. YA, young adults; OA, healthy older adults; MCI, older adults with mild cognitive impairment. Blue wiring connections show positive associations between clusters and red wiring connections show negative associations between clusters
×
Perceptual PM 200–400 ms ANOVA Connection Weights
At lateral clusters, there was a significant interaction effect of Cluster × Group. This interaction effect can be explained by a significant effect of Group at frontal clusters (OA > MCI, p = 0.030; OA > YA, p = 0.018). Additionally, the interaction can be explained by an effect of Cluster for YA (inferior parietal < all other clusters, ps < 0.05), OA (inferior parietal < all other clusters, ps < 0.001; frontal > all other clusters, ps < 0.001 but not including central, p > 0.05) and MCI (inferior parietal < central, frontal, occipital, parietal, ps < 0.05; central > frontocentral, p = 0.020; frontocentral > parietal, p = 0.028).
Perceptual PM 400–800 ms ANOVA Connection Weights
At lateral clusters, there was a significant Cluster × Hemisphere × Group interaction. This in part can be explained by a Cluster × Group interaction in the right hemisphere (central: YA > OA, p = 0.031; YA > MCI, p = 0.032). Additionally, it can be explained by a Hemisphere × Group interaction at frontal clusters (all groups: right > left, ps < 0.05; right hemisphere: OA > YA, p = 0.027) and in parietal clusters (YA: left > right, p = 0.032; left: YA > MCI, p = 0.034). The three-way interaction can also be explained by a Cluster × Hemisphere for YA (left > right in frontal, frontocentral, frontotemporal occipital, inferior parietal, ps < 0.05; right > left in parietal, p < 0.001; left hemisphere: frontotemporal, central > frontocentral, ps < 0.05; parietal > occipital, p = 0.007; right hemisphere: central, frontotemporal, parietal > all other clusters, ps < 0.01; frontocentral, inferior parietal, occipital > frontal, ps < 0.01), OA (left > right in frontal, frontotemporal, occipital; left hemisphere: central, frontotemporal, parietal > frontocentral, ps < 0.05; right hemisphere: frontotemporal > central, frontal, frontocentral, occipital, ps < 0.01; parietal > frontal, frontocentral, occipital, ps < 0.05) and MCI (left > right in frontal, frontocentral, central, frontotemporal, occipital, inferior parietal; right > left in parietal; left hemisphere: frontotemporal > all other clusters except central, ps < 0.05; right hemisphere: frontotemporal > all other clusters, ps < 0.05).
Perceptual PM 0–1000 ms ANOVA connection weights
At lateral clusters, there was a significant interaction of Cluster × Group, due to a Group effect at frontotemporal clusters (YA > MCI, p = 0.029; YA > OA¸ p = 0.062 (trend)). Additionally, the interaction is explained by a significant effect of Cluster for YA (frontotemporal < all other clusters, p < 0.001; frontal > inferior, parietal, occipital, ps < 0.05; central > frontocentral, p = 0.003) OA (frontotemporal < all other clusters, p < 0.001; frontal > inferior, parietal, occipital, ps < 0.05; central > inferior parietal, p = 0.045; inferior parietal > occipital, p = 0.006) and MCI (frontotemporal < all other clusters, ps < 0.001; inferior parietal < all clusters except frontotemporal, ps < 0.05).
Perceptual PM Network Analysis: 200–400 ms
Network models for 200–400ms induced a sparsity level of 97% for OA and 98% for YA and MCI groups. As seen in Fig. 13, the right frontal cluster has strong associations with neighbouring clusters across all participant groups. In the YA, the network reveals a strong positive connection between the right frontal and the right frontocentral cluster (EW = 0.33), which also exhibits significant network control (betweenness = 4.01) through a strong positive relationship with the right parietal cluster (EW = 0.27). A similar level of betweenness is exhibited in the OA group; however, network control is instead demonstrated by the right frontal cluster with connections to the left frontal cluster (EW = 0.38) and right frontotemporal cluster (EW = 0.49), which is not apparent for the MCI group. Additionally, for both the OA and MCI groups, positive connections and increased strength were found in the left central cluster (OA node strength = 0.80; MCI node strength = 1.58). The OA group demonstrates a positive left central to left frontocentral regions (EW = 0.43), whereas MCI shows a left central to mid occipital cluster relationship (EW = 0.35).
Perceptual PM Network Analysis: 400–800 ms
For the network models of 400–800ms, approximately 91%, 96% and 98% of all edges were set to zero for the YA, OA and MCI groups, respectively. The models demonstrate the importance of the right frontocentral cluster over this epoch as all participant groups show similar levels of degree strength (YA node strength = 1.55; OA node strength = 1.91; MCI node strength = 1.08), although only significant levels of betweenness for YA (2.70) and OA (3.56) were found implying a possible loss of network control in MCI. Similarly, the mid frontal cluster also shows strong network strength and network control but only for YA (node strength = 1.62; betweenness = 1.79) and OA (node strength = 2.10; betweenness = 1.62) but not in the MCI group. A comparable pattern is also seen in the left frontocentral cluster, where network strength is found for both YA (node strength = 1.50) and OA (node strength = 1.09), but not for MCI groups. The MCI groups instead show strong betweenness and network strength for the right inferior parietal (node strength = 2.43; betweenness = 4.01) cluster and node strength in the left parietal cluster (2.31).
Perceptual PM Network Analysis: 0–1000 ms
The approximate percentage of edges set to zero for the entire epoch are 94%, 97% and 99% for the YA, OA and MCI groups, respectively. The node strength indices highlight two areas of particular importance for PMpercept stimuli for all groups: the right frontocentral cluster (YA = 1.24; OA = 1.63; MCI = 1.36) and the left frontal cluster (YA = 2.60; OA = 1.85; MCI = 2.20). These clusters exhibit the greatest strength within the network, although differences were found in the measures of betweenness. The left frontal cluster exhibits significant network control in the YA (2.82) and OA groups (1.08) but not in the MCI group. In the YA, there were strong positive connections from the left frontal cluster with the right frontal (EW = 0.38), right frontocentral (EW = 0.49) and with the left frontocentral cluster (EW = 0.42). A similar pattern is observed between the left frontal cluster in OA with the right frontal (EW = 0.41) and the left frontocentral cluster (EW = 0.29). This connectivity pattern is not found in the MCI group. The YA group exhibits positive bilateral frontocentral connections with the mid occipital cluster (left frontocentral EW = 0.25; right frontocentral EW = 0.16) and a negative right frontal – left occipital connection (EW = -0.31). The OA group exhibits a local network feature over the right frontotemporal scalp region, characterised by the strong positive right central–right temporal (EW = 0.30), right temporal–right frontocentral (EW = 0.36) and mid central–right frontocentral (EW = 0.25) connections, where network control is ascribed to the right frontocentral cluster (betweenness = 2.57). The same local network is not found in the MCI group, but a negative connection is found for right frontocentral–left parietal clusters (EW = −0.38).
All significant Group effects ANOVA results are presented in in Table 5. There were no significant Group effects over midline clusters. Graphical networks are presented Fig. 14.
Fig.14
Conceptual prospective memory (PMconcept) features extracted from the trained SNN using network analyses (adaLASSO) from the 200–400 ms, 400–800 ms and across the entire 0–1000 ms trained epoch. YA, young adults; OA, healthy older adults; MCI, older adults with mild cognitive impairment. Blue wiring connections show positive associations between clusters and red wiring connections show negative associations between clusters
×
Conceptual PM 200–400 ms Connection Weights ANOVA
At lateral clusters, there was a significant Cluster × Group interaction, due to an effect of Group at occipital (YA > OA, p = 0.025), inferior parietal (YA > OA, p = 0.056 (trend)) and central clusters (YA > OA, p < 0.001; YA > MCI, p = 0.006). The interaction effect was also due to an effect of Cluster in YA (occipital > all other clusters, ps < 0.001; inferior parietal > frontocentral, p < 0.001; inferior parietal > frontotemporal, p = 0.045; parietal > frontocentral, p = 0.001), OA (occipital > all other clusters, ps < 0.001; central < all other clusters, ps < 0.05; inferior parietal > frontal, p = 0.023; inferior parietal > frontotemporal, p = 0.005) and MCI (occipital > all other clusters, ps < 0.001; central < frontal, p = 0.002; central < parietal, inferior parietal, ps < 0.001; frontotemporal > inferior parietal, parietal, ps < 0.05).
Conceptual PM 400–800 ms Connection Weights ANOVA
At lateral clusters, there was a significant three-way Cluster × Hemisphere × Group interaction. This can be explained by a significant Cluster × Group in the left hemisphere, such that YA had significantly larger connection weights than OA in frontal (p = 0.005), frontocentral (p = 0.004), central (p = 0.002) and parietal clusters (p = 0.021) and larger connections weight than MCI in frontocentral (p = 0.015) and central clusters (p = 0.003). The three-way interaction can also be explained by a Hemisphere × Group effect at frontotemporal clusters (F2,87 = 3.51, p = 0.035, ηp2 = 0.08), whereby in the right hemisphere, OA had larger connection weights than in YA (p = 0.015). The three-way interaction can also be explained by a Cluster × Hemisphere for YA (left > right in frontal, frontotemporal, parietal, inferior parietal, ps < 0.05; right hemisphere: frontal, central, parietal > frontotemporal, occipital, ps < 0.005; inferior parietal > occipital, p = 0.001; left hemisphere: frontotemporal < all other clusters, ps < 0.001; central > frontal, inferior parietal, ps < 0.01), OA (left > right in frontal, frontotemporal, parietal, inferior parietal, ps < 0.05; right hemisphere: frontal, frontocentral, central, occipital > frontotemporal, parietal, ps < 0.005; central, occipital > inferior parietal, ps < 0.001; left hemisphere: frontal, parietal, inferior parietal > frontotemporal, ps < 0.05; parietal > frontocentral, occipital, ps < 0.01) and MCI (left > right in frontal, frontotemporal, parietal, inferior parietal, ps < 0.05; right hemisphere: central, occipital > frontotemporal, parietal, ps < 0.05; central > frontal, p = 0.008; frontal > frontotemporal, p = 0.002; left hemisphere: inferior parietal > frontotemporal, p = 0.033; parietal > central, occipital, ps < 0.05). All groups demonstrated greater connection weights in the right relative to the left hemisphere in occipital clusters (ps < 0.05).
Conceptual PM 0–1000 ms ANOVA Connection Weights
At lateral clusters, a significant Hemisphere × Group interaction Group is explained by a significant effect of Group in the right hemisphere (MCI > YA, p = 0.024) and a significant effect of Hemisphere in OA and MCI, such that the right hemisphere connection weights were significantly greater than the left hemisphere (OA: p < 0.001; MCI: p = 0.007).
Conceptual PM Network Analysis: 200–400 ms
Over 200–400ms, approximately 97% of edges for YA and OA and 95% of edges for the MCI group were set to zero. A common cluster across all groups for this earlier epoch is the right occipital cluster, where the MCI group exhibits the greatest node strength (1.66), relative to YA (0.60) and OA (0.25). From the right occipital cluster, connections to mid parietal clusters were found for both the OA (EW = 0.33) and MCI groups (EW = 0.34), along with a right parietal–right central relationship (OA EW = 0.49; MCI EW = 0.23). Additionally, the MCI group shares similar network strength in the left frontal cluster (node strength = 1.78) as the YA group (node strength = 1.25). Both the MCI and YA groups demonstrate the greatest amount of network control originating from the left frontal cluster (YA betweenness = 3.36, MCI betweenness = 1.31). However, while the YA group demonstrates connections from the left frontal cluster to the right frontal (EW = 0.32), left temporal (EW = -0.31) and left frontocentral (EW = 0.44), the model shows longer connections toward posterior regions for the MCI group with the left frontal cluster connecting to left occipital (EW = 0.70), right occipital (EW = 0.20) and the left central clusters (EW = 0.42).
Conceptual PM Network Analysis: 400–800 ms
Model sparsity was approximately 90% for YA, 96% for OA and 94% for MCI. There are several network similarities between the groups at this period. All groups demonstrate considerable degree strength at left frontal clusters (YA = 0.12; OA = 0.93; MCI = 1.43). The network highlights the importance, as indicated by a measure of betweenness, of mid parietal and left parietal for YA (mid parietal = 0.05; left parietal = 0.48), OA (mid parietal = 1.20; left parietal = 1.53) and MCI (mid parietal = 1.76; left parietal = 0.27) groups. Differences are apparent in the connections between groups. In the frontal clusters, the YA demonstrate a left frontal–left parietal connection (EW = 0.46); OA demonstrate a similar left frontal–mid parietal connection (EW = 0.25), but MCI exhibits a shorter left frontal–left frontocentral connection (EW = 0.58). The MCI group displays a strong interhemispheric connection between left temporal and right temporal clusters (EW = 0.56).
Conceptual PM Network Analysis: 0–1000 ms
The approximate number of edges reduced to zero for the entire PMconcept epoch was 95% for YA and OA, and 92% for MCI. The left frontocentral cluster contains the most amount of network control (YA betweenness = 1.66; OA betweenness = 4.01; MCI betweenness = 2.98). Additionally, YA and OA both show left frontocentral connections with the left central cluster (YA EW = 0.32; OA EW = -0.44). Conversely, where a connection for the YA is found between the left frontocentral cluster and right frontocentral cluster (EW = 0.32), a negative connection is found between the two clusters for the MCI group (EW = −0.60). Figure 12 shows network connections between left frontal–right frontal (EW = 0.19) and left frontocentral–right frontocentral (EW = 0.18) in YA. MCI show a negative connection from the right frontocentral cluster to the left frontocentral cluster (EW = −0.18). The YA model favours a left frontocentral connectivity pattern, whereas the older adult groups incorporate more temporal-posterior regions.
Discussion
The current study aimed to improve the modelling and understanding of the neurocognitive dynamics that underpin PM across young adults, older adults and older adults experiencing MCI using a brain-inspired computational framework. Through the classification functionality of the SNN architecture, the current experiment sought to determine whether PM stimuli would enable better classification accuracy of brain activity between the groups compared to working memory stimuli. Two conclusions can be drawn from the results: firstly, greater accuracy was achieved by using a SNN methodology relative to the traditional ML methods (SVM, MLP, MLR); secondly, greater SNN classification accuracy was achieved with brain responses to PM stimuli compared to the ongoing working memory task at classifying brain activity between young adults, older adults and older adults with MCI. This study also sought to uncover new insights of spatiotemporal connectivity between the groups through a series of ANOVAs and network analyses on each of the SNN models. In general, younger adults exhibited greater local cluster connectivity compared to healthy older adults and older adults with MCI indicated through increased wiring across the scalp. Network analyses largely show that older adults with MCI had decreased wiring across the cortex in response to working memory and perceptual PM tasks. However, network activity in response to conceptual PM revealed greater cortical wiring in those with MCI relative to young and healthy older adults.
Prospective memory may have enabled greater classification accuracy relative to working memory due to the higher cognitive demands required to complete the PM tasks [69]. Coping with additional cognitive demand is problematic for older adults with MCI [70, 71], which would be reflected in neural signalling and subsequently in the SNNs allowing for easier discrimination of neurophysiological activity. Thus, the earlier detection of dementia-related diseases may be achieved through the application of ML methods in cognitively demanding memory tasks as opposed to simple cognitive tasks. Although, this would need to be confirmed, for example, within a longitudinal design. PM tasks may be particularly suitable for discerning neurocognitive differences in those with MCI as it is one of the first reported cognitive complaints of older adults who go on to develop MCI and dementia [72].
The current study demonstrates that 3D SNN model interpretability can be enhanced by propagating ERP data (as spike-trains) from a specific group and pruning network connections that did not change. The removal of these connections produced sparse SNN models which better reflect patterns of learnt activity. Therefore, these sparse networks offer a novel method to interpret models of task-based neurophysiological activity to gain a deeper understanding of connectivity through spatial and temporal dimensions. Additionally, pruning non-important connections enabled the connection weights to follow distributions that allow for statistical analyses to be performed on the models.
Analyses demonstrated the importance of anterior clusters for all participant groups in response to perceptual PM stimuli. Both younger and older adults had significantly greater levels of wired regions across the cortex relative to older adults with MCI, particularly in frontal and frontocentral clusters. The aPFC plays an important role in the balance of attention between the ongoing task and the intermetal representation of PM stimuli [73]. FMRI evidence implicates functional deterioration in attention networks (dorsal and ventral) in older adults with amnestic MCI and Alzheimer’s disease [74‐76]. The decrease of frontocortical connectivity in older adults with MCI possibly represents a deterioration in the functional networks responsible for performing the attentional balance between working memory and PM intention maintenance.
Contrary to the perceptual PM models, older adults with MCI showed greater cluster-to-cluster wiring across the cortex than younger and older adults in the conceptual PM. This concurs with evidence of hyperconnectivity and faster memory decline [77]. Some researchers report decreased functional connectivity in those with MCI and early Alzheimer’s disease [78‐81], while others report increases in cortical connectivity [82‐86]. In a comparison of resting and task-based connectivity, Jiang and Zheng [87] demonstrated that while functional connectivity in an MCI group decreased relative to healthy controls at rest, in the working memory task, the MCI group exhibited group exhibited greater inter- and intra-hemispheric connectivity. Jiang and Zheng suggest that levels of cortical connectivity increase as a means of supporting atrophied cortical regions during memory tasks. Although the current study did not compare the results with resting-state activity, similar compensatory mechanisms may be being utilised throughout the cortex in the MCI group during the more difficult conceptual PM tasks relative to the highly salient perceptual PM task and the relatively simple working memory task. Indeed, evidence shows greater inter-hemispheric EEG connectivity in those with MCI when memory task demands increase [88], which was not found in healthy controls. Increased functional connectivity between parahippocampus and the middle frontal gyrus is associated with decreased episodic memory performance in those with MCI [86]. Similarly, frequent rewiring is seen in ASD individuals as the individual adapts to increased cognitive demands [27]. Taken together with the current findings, decreases in connectivity may be apparent in resting-state and simple memory tasks in those with MCI. Potentially, however, as the difficulty of a task increases, so might the connectivity for individuals with MCI possibly reflecting compensatory mechanisms.
The current study used a 50/50 positive to negative connection ratio to initialise the SNN models. While this provided a better rate of learning within the current study than the more commonly used 80/20 ratio, other configurations were not tried. Potentially, there are better positivity to negative ratios for modelling ERP data in a SNN architecture. Therefore, future researchers are encouraged to incorporate the experimentation of different initial connection weight ratios during the optimisation stage in SNNs.
The proposed approach has shown an excellent ability to discriminate functional brain connectivity networks for young adults, older adults and older adults with MCI. As such, this SNN method may be further utilised to measure the effect of treatments applied over time in older adults with MCI in their PM ability. In addition, turning the identified spatio-temporal patterns of functional connectivity into linguistic knowledge (rules) as suggested in [21] can help to better understand PM decline and MCI.
The current study proposes the utility of comparing cortical connectivity in those with MCI between simple cognitive tasks and more cognitively demanding tasks. Future research should explore other cognitive domains that can be varied in their difficulty to confirm the differences in hypo- and hyper-connectivity in MCI. It would be expected that as cognitive decline progresses, the more challenging cognitive tasks will also be associated with decreases of functional connectivity in line with the declines found in AD [89].
Conclusion
The current study shows that the spatiotemporal connectivity in working memory and PM tasks can be modelled and visualised using SNNs to gain an increased understanding of the effects of ageing and cognitive decline. The SNNs demonstrated that classification accuracy of brain activity related to working memory and PM is better than conventional ML methods. Moreover, STBD in response to PM stimuli provides better classification accuracy than working memory in SNNs. Visualisation of the task-based memory activity can be improved through pruning of inactive ANs and neural connections in the SNN models. Analyses of the SNN models revealed different spatiotemporal connectivity between the groups at a local and global level. In general, local cluster connectivity was greater for younger adults but older adults had increased connectivity in frontal clusters during the perceptual PM models. The older adults with MCI had decreased global connectivity relative to healthy older adults and younger adults in the working memory and perceptual PM, but in the conceptual PM models, MCI patients had increased cortical connectivity potentially reflecting compensatory mechanisms during a more cognitively demanding PM task.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The formula for encoding positive and negative spikes is given as:
$$spike\left(t\right)=\{1 then V\left(t\right) \leftarrow V\left(t-1\right)+\theta ; if S\left(t\right)\ge V\left(t-1\right)+ \theta$$
The variability in signal amplitude over time is denoted by V(t) for a signal S(t) over time t = 1, 2, …, n where at baseline, V(1) = S(1). If the upcoming signal amplitude S(t) is greater than V(t –1) + θ (where θ is a defined threshold) at the next point t, then a positive spike is produced, while a negative spike is created for a decreased signal.
Formula 2
The membrane potential leakage between spikes can be defined by a parameter τ. The LIFM is defined via:
where τm represents the membrane time constant, vrest reflects the resting potential, R is the resistance and I is the input current.
Formula 3
$$F\left(\Delta t\right)= \left\{\begin{array}{c}{A}_{+}exp\left(\frac{\Delta t}{{\tau }_{+}}\right) if \Delta t<0 \\ -{A}_{-}exp\left(-\frac{\Delta t}{{\tau }_{-}}\right) if \Delta \ge 0\end{array}\right.$$
F(∆t) describes the adjustment of synaptic plasticity with respect to the pre-synaptic and post-synaptic spiking time in the interval of ∆t = tpre-tpost. The parameters A + and A- are the maximum amounts for synaptic adjustment, which apply if ∆t is close to zero. The parameters \(\tau\) + and \(\tau\)—control the interval of pre- to post-synaptic spikes during which the weakening and the strengthening of the synaptic connection occur.
Formula 4
At time t the postsynaptic potential (PSP) of the artificial neuron j is calculated as follows:
where mod is a modulation factor (a parameter between 0 and 1) and order(i) represents the order of the spikes in time between artificial neurons i and j.
Formula 5
The drift parameter modifies connection weight Wi,j by accounting for the following spikes at artificial neuron j at time t, denoted as spike(t).
$${W}_{i,j}\left(t\right)= \left\{\begin{array}{c}{W}_{i,j}\left(t-1\right)+drift if {spike}_{j}\left(t\right)=1\\ {W}_{i,j}\left(t-1\right)-drift if {spike}_{j}\left(t\right)=0\end{array}\right.$$
where \({\widehat{w}}_{j}\) = 1/\({\left|{\widehat{\beta }}_{j}^{*}\right|}^{2}\), γ > 0 and \({\widehat{\beta }}_{j}^{*}\) is an initial parameter estimate. As the sample size grows, the weights diverge (to infinity) for zero coefficients, whereas, for the non-zero coefficients, the weights converge to a finite constant.
Appendix 2
Cluster
Electrode locations
Left frontal
AF7
AFF5
AFF5h
FP1
AF3
AFF3h
Right frontal
AFF4h
AF4
FP2
AFF6h
AFF6
AF8
Mid frontal
AFFz
Fz
FFCz
FCz
Left frontocentral
FC5h
FC3
FC3h
FFC5h
F3
FFC3h
FC1
F1
Right frontocentral
FC4h
FC4
FC6h
FFC4h
F4
FFC6h
FC2
F2
Left central
C3h
C3
C5h
C5
T7h
Mid central
Cz
CCPz
CPz
C2h
CCP2h
C1h
CCP1h
Right central
C4h
C4
C6h
C6
T8h
Left frontotemporal
T7
FT7
F7
Right frontotemporal
F8
FT8
T8
Left parietal
CPP3h
P3
CCP1
CP5
CP5h
CP3
CPP5h
Mid parietal
CPPz
Pz
PPOz
POz
Right parietal
CPP3h
P4
CPP6h
CP6
CP6h
CP4
CCP2
Left inferior parietal
TP7
TP7h
P5
P7
P9
Right inferior parietal
P10
P8
P6
TP8
TP8h
Left occipital
PPO5
PO7
PO9
PO11
O1
POO5
PO3h
Mid occipital
POOz
Oz
OIz
Iz
Right occipital
O2
POO6
PO4h
PPO6
PO8
PO10
PO12
Electrode locations are given in relation to the 10–5 system
