Computational insights on asymmetrical \(D_{1}\) and \(D_{2}\) receptor-mediated chunking: implications for OCD and Schizophrenia

The basal ganglia and striatum are critically involved in sensorimotor chunking, constructing performance units of sequence representations that once learned, can be treated as separate entities (Graybiel 1998; Solopchuk et al. 2016; Jin and Costa 2010; Jin et al. 2014). What are the computational advantages of the two parallel dopaminergic receptor systems (\(D_{1}\) and \(D_{2}\) receptor families) in chunk learning and recall? The classical view is that stimulation of striatal dopamine \(D_{1}\) receptors of GABA-ergic principal medium spiny neurons (MSN) facilitates the direct pathway activity and movement, while dopamine \(D_{2}\) receptors of MSNs influences the indirect pathway and inhibits movement or competing actions (Gerfen and Surmeier 2011; Cruz et al. 2022; Cui et al. 2013). Imbalances in the activity of the \(D_{1}\)-expressing direct pathway and \(D_{2}\)-expressing indirect pathway MSNs can contribute to basal ganglia disorders (Krajeski et al. 2019).

Recurrent collateral connections among MSNs are not symmetrical. \(D_{2}\) MSNs have additional and stronger inhibitory connections on \(D_{1}\) MSNs than vice versa (Planert et al. 2010; Taverna et al. 2008). Thus, the dominating interaction between these two projection systems relies mostly on collateral projections from \(D_{2}\) MSNs to \(D_{1}\) MSNs.

Phasic dopamine release primarily increases \(D_{1}\) occupancy, whereas \(D_{2}\) occupancy is less affected (Dreyer et al. 2010). Low-level baseline tonic dopamine release is sufficient for altering \(D_{2}\) receptors (Schultz 2007; Durstewitz and Seamans 2008). Dopamine can bias action selection by modulating striatal direct and indirect pathways (Howard et al. 2017).

Detection and marking of chunk boundaries over time and adjusting the boundary detection threshold is a dynamic and crucial process for adaptive habit formation (Ramkumar et al. 2016; Barnes et al. 2005). Sequence segmentation can rely on detecting boundaries between events by transients (Asabuki et al. 2018). It is likely that different mechanisms with increasing degrees of abstraction representing sequence knowledge can operate in parallel to each other and both chunking as well as transitioning might benefit from recognizing start / stop signals (Dehaene et al. 2015).

To improve understanding of how \(D_{1}\) and \(D_{2}\) asymmetries may contribute to the dynamics of sequence processing and disorders, eight computational experiments were performed with modifications to the reservoir computing model described by Asabuki et al. (2018). The first experiment investigated the effects of varying MSN response time constants. The second set of simulations explored the self-feedback to each MSN population. The third experiment studied the effects of varying connection probabilities. The fourth simulation set examined the effect of varying weights within the \(D_{1}\) and \(D_{2}\) receptor mediated MSN populations. In the fifth experiment, we investigated the effects of assymetric feedback between the \(D_{1}\) and \(D_{2}\) receptor dominated MSN populations. The sixth simulation set examined the effects of an increased input chunk size. In the seventh experiment, a combination of 3 separate parameters were altered. Those parameters were chosen from values which contributed separately to earlier chunk recognition. Finally, in the eighth simulation set we studied the effects of variations in learning rate and training times on chunk recognition. Throughout the computational experiments, time offset of the peak activity (TOPA) was calculated, which can represent the recognition of the chunk. The first five experiments are illustrated in Fig. 1.

We adapted the modeling framework published and specified previously by (Asabuki et al. 2018). The following equation was employed for the two reservoirs:where each reservoir is composed of N_G neurons with i=1,2,...N_G. The dynamics of each reservoir neuron is represented by x. \(I_{\mu }(t)\) represents the activity of the inputs and N_I is the number of input neurons. \(\xi _{i}(t)\) denotes the random (Wiener) process and \(\sigma\) is the standard deviation. \(J^{GG}\), \(J^{GZ}\) and \(J^{GI}\) describe the recurrent internal connection matrix of the reservoir, the self-feedback matrix and the matrix between input and the reservoir neurons, respectively. The recurrent internal and self-feedback connections are non-plastic. The parameter \(g_{G}\) scales the recurrent internal weight matrices. The parameter h_G  scales the self-feedback weight matrices. The activation function is defined as: The instantaneous output is given by z(t), where w is the readout weight vector:The readout unit is connected with n reservoir neurons by the weight vector w, which is trained using the FORCE learning algorithm (Sussillo et al. 2009). Weights w undergo learning with a teaching signal given by the output of the partner network. The teaching signal is described according to:where \({\widehat{z}}_{j}(t)\) is the normalized output of a readout unit (see in methods in Asabuki et al.), the threshold linear function \([x]_{+}\) returns 0 if x \(\leqq\) 0, and \([x]_{+} = x\) if \(x>0\). Note that in simulation set 5 this constraint was removed, such that function f can go negative. The constant \(\beta\) = 3 was set as in the original work (Asabuki et al. 2018). The error signal was defined as:The parameter \(\kappa\) was 1 in all experiments except in simulation set 5, where it was varied between \(-1\) and 1. For initial values of weight matrices, see methods in (Asabuki et al. 2018). The connection probability p of reservoir neurons is 1 in all experiments except 3, 4 and 5. In experiments 3 and 4, the parameter p3 indicates a common connection probability in matrices J^GG, J^GZ and J^GI. Parameter values are summarized in Table 2 for each experiment.

Simulations were performed on a compute cluster at the UPPMAX supercomputer center at Uppsala University. Simulations used python 3.8 and had a timestep of 1 ms. Simulations were averaged over 10 different seeds for each parameter regime.

In simulation experiment 1, we examined the effects of varying time constants (\(\tau\)) in the two parallel MSN networks (Fig. 1.A., Fig. 2., Fig.3.). The time constants of each population were independently varied between 2 and 30 ms.

We found that neural activity often peaked earlier during the chunk when the time constants of the reservoir were shorter, perhaps similar to the response of Start cells (Figs. 2.A., B, 3.A.). With increased time constant values, the peak activity shifted more often towards the end of the chunk, perhaps similar to Stop cells (Fig. 2.C., 3.B.). This modeling result implies that asymmetric time constants of the two parallel \(D_{1}\) and \(D_{2}\) dominated MSNs may contribute to differential signaling at the start and stop positions in chunks. The distributions of peak activity (TOPA) are represented as histograms in Fig 2. With short time constants, TOPA may occur during leading chunk segments, as can be seen with four clusters in the histogram, likely corresponding to presented chunk characters (Fig. 2.B). With longer time constants, TOPA occurs most often at the end of chunk presentation (Fig. 2.C).

In computational experiment 2, we altered the self-feedback gain (\(h_{G}\)) within the reservoir networks (Fig. 1.B.). A monotonic increase was observed as the gain \(h_{G}\) increased. With lower self-feedback gain (or weaker feedback coupling), the peak activity more often signals the beginning of the chunk. As self-feedback gain increased, peak activity shifted towards the end of the chunk (Fig. 4.A.). Histograms show that with low feedback gain, TOPAs can occur earlier in chunks (Fig. 4.B.), while with stronger feedback, TOPAs accumulate at the end of chunks (Fig. 4.C.). This implies that TOPAs are sensitive to the contributions of previous states, depending on time constants. Note that in Fig. 4.A. TOPA was averaged across both populations, but treated as separate samples in Fig. 4.B and C.

In simulation experiment 3, the effects of varying combined connection probabilities (p3) were examined (Fig. 5.A.). The connection probability was varied between 0.1 and 1 on input projections to reservoir units, internal recurrent connections within the reservoirs themselves and the weights in the self-feedback matrix. When p3 is low, TOPAs during chunk presentation occured with lower z-scores, indicating a weaker signal strength. As p3 increases, TOPA z-scores gradually increase, indicating higher statistical significance with denser connectivity (Fig. 5.A.). The z-score represents the number of standard deviations away from mean activity of the reservoirs.

In simulation experiment 4, the effects of scaling the intrinsic weight matrix was analyzed (\(g_{G}\)) (Fig. 1.D.). This simulation used p3 = 0.3 for connection probabilities. A non-monotonicity was observed as the gain of the intrinsic weight matrix was varied (Fig. 5.B.), with higher values the network is more likely to signal the end of the chunk. The highest TOPA was observed at g_G = 1.5 for both populations, which also had the highest z-score (not shown). These results suggest that weaker D₁ \(MSN\) \(\rightarrow\) D₁ \(MSN\) (Planert et al. 2010; Taverna et al. 2008) may result in network activity which more often signals the beginning of chunk while stronger D₂ \(MSN\) \(\rightarrow\) D₂ \(MSN\) connections more often contribute to signaling the end of chunks (Fig. 5.B.).

In simulation experiment 5, the reservoir teaching signals were investigated, which originate from the opposing population. During training, the differences in activity levels are used as error signals to adjust the readout weights. This experiment included coefficients \(\kappa\) for teaching signals from the opposite reservoir and independently varied them from -1 to 1. On activation peaks during chunk recognition, a z-score was computed. In the baseline case \(\kappa\) is 1 on both teaching signals, which result in the reservoirs seeking to achieve a consensus. When \(\kappa\) is -1 on both teaching signals, the reservoirs compete, perhaps as go and no-go pathways do. Fig. 6.A. shows the positive z-scores when activation peaks are positive and Fig. 6.B. shows the negative z-scores when activation peaks are negative. Low z-scores approach the mean population activity levels. When the teaching signals have opposing signs, the reservoirs do not learn properly, and the z-scores are near the mean. The corners where the signs are the same show either collaborative (\(\kappa\) = 1) or competitive (\(\kappa\) = -1) activations. Fig. 6.C. is a single trial that shows one population in red with \(\kappa\) = 1 and the other in blue with \(\kappa\) = 0.5. Fig. 6.D. is one emergent variation with \(\kappa\) = -1 and competitive dynamics. Fig. 6.E. is another emergent variation with \(\kappa\) = -1 that shows one population in red peaking at the beginning of the chunks and the other population in blue peaking at the ending of the chunks. These dynamics may indicate that some level of competition is necessary between populations to achieve start/stop chunk signaling.

In simulation experiment 6, the size of the chunk was increased from 4 to 6 characters. As in experiment 1, the \(\tau _{1}\) and \(\tau _{2}\) parameter space was explored. Small \(\tau \) values again support early peak activity on partial chunk recognition (Fig. 7.A.), which disappear with longer time constants (Fig. 7.B., C.). The distribution histogam in Fig. 7.B. at \(\tau = 2\) in both populations now shows 6 clusters rather than the original 4, likely corresponding with the additional presented chunk characters. The histogram in Fig. 7.C. has \(\tau =30\) in both populations, showing a distribution gradient peaking at the presentation of the last chunk character. These results indicate that time constants (τ) may contribute to start/stop signaling at least partially independent of chunk size.

In simulation experiment 7, the combined parameter space results in recognition of the earlier part of the chunk while the basic structure of the previously observed tendency is preserved, such that shorter time constants result in earlier chunk recognition (Fig. 8.A.). The nonmonotonicity in the 2-dimensional surface reflects the chunk segments, having 4 characters.

Simulation experiment 8 examined the learning properties of the model with \(\tau =10\) in both populations, by varying the training times from 1 to 10 s (baseline) and the learning rate \(\alpha\) from 20 to 160 (100 in baseline). Fig. 8.B. shows that training converged to a maximum learning capability at a learning rate of α = 100 and training time of 10 s.

We propose that D₁ and D₂ receptor-dominated striatal MSN dynamics can contribute to start and stop signaling cues and thus enable a chunking strategy which improves performance in sequence processing (Solopchuk et al. 2016). In OCD and schizophrenia, this chunking strategy is altered, which can give rise to some of the overlapping symptomatology. Chunking during sequence learning is a dopamine-dependent process (Tremblay et al. 2010; Taylor 2010). In a rodent study, the D₁ and D₂ MSNs showed different patterns of lick sequence-related activity and different phases of oscillation time-locked to the lick cycle, both at coarse and fine timescales (Chen et al. 2021). A D₂ receptor antagonist has been shown to have deleterious effects on the chunking of separate movements into integrated motor sequences in monkeys (Tremblay et al. 2009). Perseverative errors in schizophrenia result in continued re-selection of previously activated outputs (Yogev et al. 2003; Szalisznyó et al. 2019). Over-switching is a counterpart of perseveration, and both can be characteristic of schizophrenia and OCD. The underlying D₁ and D₂ receptor pathology could contribute to parsing and concatenation error in these conditions (Fig. 9.) (Yogev et al. 2003). Compared with normal performance of the same motor tasks, OCD rituals are longer in duration and comprise a greater repertoire of idiosyncratic (unnecessary) acts (Eilam 2017). A human study showed that baseline striatal D₂/D₃ receptor binding is decreased in OCD, supporting the hypothesis of chronically increased endogenous dopaminergic activity (Denys et al. 2013).

D₁ receptors have been demonstrated to have a greater sensitivity for phasic dopamine transmission (Dreyer et al. 2010). On the other hand, D₂ activation might facilitate switching between conflicting mental representations (Bensmann et al. 2020; Agnoli et al. 2013). The literature is controversial on whether the D₁ and D₂ MSNs signal more initiation or termination of sequences. D₁ neurons are more relevant to cue perception and initiation of specific motor action, whereas D₂ neurons are more involved in post-movement events (Sheng et al. 2019). One rodent study (Jin et al. 2014) implicated that the initiation vs. execution of actions involve different subsets of D₁ and D₂ MSNs. When the MSN neurons were subdivided, a similar percentage of D₁ MSNs signaled the sequence start vs. stop, while the majority of D₂ MSNs preferentially displayed activity related to the start rather than the end of the sequence (Jin et al. 2014). Another rodent study demonstrated that the major role of D₂ MSNs is in action initiation (Augustin et al. 2020). In the following sections, related literature is discussed in the context of our modeling results.

We used an abstract and mechanistic model to represent the \(D_{1}\) and \(D_{2}\) MSN population dynamics. We did not take into account the certain overlap of these two neuronal pools. Further, the feedback from the striatum to the cortex via the cortico-striato-thalamo-cortical neural pathways is not represented in the current study. We did not take into account the fact that \(D_{1}\) receptor activation of working memory in the prefrontal cortex occurs at lower concentrations than needed for the modulation of motor function, thus the phasic or tonic distinction is an oversimplification. The temporal dynamics of the dopaminergic input is also excluded which carries an important dynamical modulation and opens several other avenues for future work. A recent modeling study challenged the view that differences in receptor affinity introduce asymmetries in \(D_{1}\) and \(D_{2}\) signaling and proposed that both pathways respond to the whole range of dopamine signals and integrate the dopamine signal over longer time scales (Hunger et al. 2020). Further computational studies should investigate the boundary signaling in light of these simulations. The teaching signals between reservoirs in the utilized model drove the populations towards consensus (similar TOPAs) when κ = 1 for both populations, rather than driving the populations towards opposite ends of the chunks. Further work can explore additional mechanisms on separating TOPAs for start/stop signaling. Our analysis was based on network and single cell dynamical constraints from animal (mostly rodent) data since human results are still limited, even though animal properties are not always directly translatable to human physiology. Both OCD and schizophrenia are heterogenous diseases. We sought to dissect the possible striatal \(D_{1}\) and \(D_{2}\) MSN and small network effects on sequence processing and did not consider possible larger circuit pathologies.

To summarize, this study provides a mechanistic computational framework for some aspects of the \(D_{1}\) and \(D_{2}\) receptor-mediated chunk learning. In our model, the dopamine related modulation can contribute to chunk boundary signaling. Our modeling results imply that dynamical differences between the two segregated dopaminergic striatal populations may be advantageous, providing complementary functions for sequence start/stop recognition and execution. Some aspects of the functional dichotomy can likely be better explained by larger network modulations. We related our computational predictions to possible underlying neuropathophysiologies of OCD and schizophrenia symptoms (Szalisznyó and Silverstein 2019). Such biologically grounded approaches and diagnostics will contribute to the development of better informed dimensional taxonomies and classification systems of psychopathology (Sharma and Acharya 2021; Szalisznyó and Silverstein 2021).