Predicting ionizing radiation exposure using biochemically-inspired genomic machine learning

Datasets

Murine gene expression datasets²³ were obtained from peripheral blood (PB) mononuclear cell samples of ten-week old C57B16 mice that either received total body radiation at 50 cGy, 200 cGy, or 1000 cGy or were not exposed. Post-exposure, total RNA was isolated after 6 hours and expression was determined by microarray analysis using Operon Mouse V3.0.1 (Gene Expression Omnibus (GEO): GPL4783 from GSE10640[GPL4783])²⁴ and Operon Mouse V4.0 arrays (GEO: GPL6524 from GSE10640[GPL6524])²⁴. Similar analyses were performed with human expression microarrays¹⁸, including datasets GEO: GSE6874[GPL4782]⁹, GSE10640[GPL6522]²⁴, GSE1725²⁵, and GSE701²⁶. GSE6874 and GSE10640 consist of PB samples collected 6 hours post-exposure from healthy donors and patients undergoing total body irradiation at 150–200 cGy analyzed with Operon Human V3.0.2 (GEO: GPL4782) and Operon Human V4.0 (GEO: GPL6522) microarrays. GSE10640[GPL6522] consists of 32 patients treated with alkylator-based chemotherapy without radiation. GSE1725 contains lymphoblastoid cell line samples derived from 57 subjects treated with 500 cGy. RNA was extracted 4 hours after exposure. Expression was measured using Affymetrix Human Genome U95 Version 2 Array (GEO: GPL8300). GSE701 contains lymphoblastoid cell lines from Fondation Jean Dausset-CEPH which were irradiated at 300 cGy or 1000 cGy and extracted 1–24 hours after exposure. Expression was measured using the Affymetrix Human Genome U95A Array (GEO: GPL91). The GSE77254 dataset²⁷ was also used to validate our human signatures. This dataset consisted of blood samples collected from baboons that were either total body or partial body irradiated with Cobalt 60 at either 2.5 or 5 Gy. Expression for each subject was measured 1 to 2 days after exposure and was related to their hematologic acute radiation syndrome (HARS) scores.

Preprocessing (Figure 1, panel i)

Rows and columns of microarray data that are less than 95% complete were removed and any remaining missing values were imputed using the nearest-neighbor algorithm. Only genes that are common across all datasets have been retained. Expression values of each probe were transformed to z-scores and the mean expression value of probes for the same gene have been assigned as the expression of each gene. Human and murine signatures were derived separately.

Figure 1. Flow chart of the biochemically inspired machine learning pipeline used to derive gene signatures.

In panel (v), k-fold validation splits data into k sections, where each section acts as a test set in turn while the remaining sections act as a training set. Panel (v) depicts k-fold validation for k = 3. Coloured circles represent the samples in a dataset where different colours represent different radiation doses. In panel (vi), quantile normalization forces data into the same distribution. To demonstrate this, thirty random genes were chosen to form a signature. The histograms on the left represent the distributions of expression levels of these genes in the pre-normalized datasets GSE1725 and GSE6874[GPL4782]. The histograms on the right represent the distribution of expression levels of the same genes post-normalization.

Biochemically-inspired gene selection^18–20 (Figure 1, panel ii)

A literature search has been conducted to identify genes implicated in radiation response using the search queries “radiation genes,” “radiation response genes,” and “radiation signatures” on PubMed. Cited genes comprise those differentially expressed after radiation exposure, genes present in DNA repair databases and other radiation signatures, and evolutionarily conserved genes that were highly expressed in radio-resistant species. A list of 998 genes was compiled^28–41, Supplementary Table X) for deriving signatures.

Minimum Redundancy Maximum Relevance (mRMR) gene ranking^11,12 (Figure 1, panel iii)

Rank is assigned by incremental selection of genes based on the mutual information difference (MID) criterion^16,17. Highly ranked genes have expression information that shares mutual information with radiation exposure and shares little information with expression of other genes. The MID criterion used to select the next ranked gene is ${\text{max}}_{\text{i}\in \Omega }\left[I(i,h)-\frac{1}{|S|}{\displaystyle {\sum }_{j\in S}I(i,j)}\right],$ where i is a gene selected from Ω, the total gene space, S is the set of genes selected before i, |S| is the number of genes selected before i, I(i, h) is the mutual information between expression of gene i and radiation dose (h), and I(i, j) is the mutual information between expression of gene i and expression of gene j.

Support Vector Machine (SVM) Learning

SVM models are classifiers that use hyperplane boundaries to separate samples into exposure classes by maximizing the distance between the separating hyperplanes and samples of each class. The fitcecoc function of MATLAB 2017a’s Statistics and Machine Learning Toolbox⁴² with a SVM template was used to fit SVM models to training data. The fitcecoc function was used because it allows the fitting of multiclass models, which was required for analysis of murine samples that were irradiated at four different exposure levels. The SVM models use the Gaussian radial basis function kernel and a range of selected box-constraint and kernel-scale parameters. The box-constraint, denoted by the variable C, determines how severely misclassifications are penalized during training. The kernel-scale, denoted σ, represents the width of the Gaussian radial basis function. These parameters collectively control the tradeoff between underfitting and overfitting⁴³. After feature selection, a grid search is performed to determine the optimal (C,σ) combination for values of C and σ between 1 and 100000 (inclusive) by powers of 10 such that C ≥ σ.

Feature selection (FS) (Figure 1, panel iv)²¹

Greedy feature selection was used to derive signatures. Complete sequential feature selection (CSFS) sequentially adds genes to an initially empty base set. The added gene is the highest mRMR-ranked gene that is not already included. This is repeated until all genes have been evaluated and the best performing subset of genes is identified. Forward sequential feature selection (FSFS) sequentially adds genes from the top 50 mRMR ranked genes to an initially empty base set. The added gene is the one whose addition improves the model by the greatest margin. Backward sequential feature selection (BSFS) sequentially removes genes from the top 30 mRMR ranked genes. The gene removed is the one whose removal causes the greatest improvement in the model. For BSFS and FSFS, we measure model improvement using misclassification or log loss during k-fold validation (see Performance metrics section below). Genes are added or removed until model performance plateaus. During feature selection, C and σ parameters need to be chosen for SVM learning (see SVM Learning section above). Thus, each signature is characterized by the feature selection algorithm used, the dataset used to derive it, and the C-σ combination used for its SVM models during feature selection. This leads to a large number of possible signatures (see Supplementary Files Y1–Y7). Supplementary Files Y1–Y3 and Supplementary Files Y6–Y7 contain k-fold validation results from which the top 20 signatures (evaluated using average validation log loss), in particular, were analyzed (Figure 2, Figure 3, Figure 6, Figure 7).

Figure 2. Heat map depicting the gene compositions of the top 20 murine signatures derived from different datasets: GSE6874[GPL4783] and GSE10640[GPL6524].

Gene frequency values are first scaled within datasets and then scaled across datasets to ensure values between 0 and 1.

Figure 3. Scatter plot depicting the mutual information each gene’s expression shares with radiation exposure (averaged over GSE6874[GPL4783] and GSE10640[GPL6524]).

The size of each circle is proportional to the frequency at which the gene appears in the top 20 murine signatures ranked by log loss averaged over GSE6874[GPL4783] and GSE10640[GPL6524]. The genes presented match those of the Figure 2.

Validating signatures (Figure 1, panel v)

Stratified k-fold validation was used to validate signatures. Samples of the validation dataset were partitioned into k sets, comprised of an approximately equal distribution of radiation levels. For validation, each set was used to test a model trained on the remaining sets, resulting in predictions for all samples in the dataset. Advantages of this approach are that variation between datasets is not pertinent and that signatures can be validated on differently labeled datasets (with samples irradiated at different levels).

Validating models (Figure 1, panel vi)

Model validation requires separate training and test datasets (the training set is often used for FS). Genes from the signature are extracted from the training and test sets and their expression values are quantile normalized by sample. An important distinction between our approach and a previous study⁷ is that quantile normalization is applied immediately before validation, so expression of only the genes present in the signature being validated have been normalized. By contrast, previous approaches perform quantile normalization over entire datasets; while this reduces variability in expression values within datasets, it also suppresses the dynamic range, with potential consequential effects on the prognostic value of expression data. After normalization, an SVM model was fit to training datasets and used to generate predictions from the test dataset.

Performance metrics (Figure 1, panel vii)

Performance was determined by comparing predicted radiation doses with actual radiation exposures of each sample. Metrics included misclassification error rate, goodness-of-fit, and multi-class log loss. Misclassification is the percentage of samples that were incorrectly classified, goodness-of-fit is the average absolute value difference between predicted radiation exposure and actual radiation exposure, and multi-class log loss is $-\frac{1}{N}{\displaystyle {\sum }_{i=1}^N{\displaystyle {\sum }_{j=1}^M{y}_{ij}}}\text{ln}{p}_{ij}$ where N is the number of samples, M is the number of class labels, p_ij is the predicted probability that observation i is in class j, and y_ij is an indicator variable equal to 1 if sample i is in class j and 0 otherwise.

Murine gene signatures

Table 1 displays the murine signatures derived using our pipeline which had the best performance metrics during k-fold validation on an independent dataset. In addition to the signature information, we report the feature selection algorithm (FS Algorithm) used to discover the signature, the internal validation performance metrics (FS Misclassification fraction and FS Log Loss function). Validation performance metrics on external dataset(s) are indicated by the Validation Misclassification fraction, Validation Log Loss function, and Validation goodness of fit or (GoF). In the FS Misclass. and FS Log Loss columns, one value is always N/A because signatures are derived by optimizing either misclassification or log loss, but never both. The remaining murine signatures are presented in Supplementary Files Y6 and Supplementary Files Y7.

A list of the most consistently appearing genes in the best performing signatures were obtained by pooling the top 20 murine signatures (assessed by validation log loss) from GSE6874[GPL4783] and GSE10640[GPL6524], and respectively collating the top 17 and 19 most frequent genes. The union of these two sets comprises 33 genes displayed in a heat map based on the frequencies of each gene (Figure 2). Surprisingly, the compositions of signatures derived from both datasets are not as similar as one may expect. The genes that appear more frequently in signatures derived from one dataset infrequently appear in the other even though both datasets consisted of the same types of samples irradiated at the same exposure levels.

The shared mutual information of these expressed genes with radiation dose (Figure 3) indicates whether only high mutual information genes appear in the best signatures or whether some lower mutual information genes may also be selected by our feature selection algorithms. The frequency of each gene among these signatures (represented by diameter of the circle) correlates with the mutual information between expression and radiation dose (ρ = 0.8016). However, it would be an oversimplification to create signatures based solely upon mutual information, since some genes in lower performing signatures exhibit higher mutual information content. Development of accurate signatures requires more than a collection of gene features whose individual expression values share information with radiation dose, since many of these genes may reveal similar information, and redundant machine learning model features. For instance, Bax and Blnk are both common among the best murine signatures, even though Blnk shares much more mutual information with radiation dose than Bax expression. Since Blnk and Bax are involved in completely different pathways – Bax is an inducer of apoptosis⁴⁴ whereas Blnk is involved in a B-cell antigen receptor signaling pathway required for optimal B-cell development⁴⁵, they provide different types of information to the overall model. Conversely, we also observe that genes with high information content, such as Ms4a1, may appear less frequently than genes with lower information content, such as Eif2ak4 or Ccng1.

Although mRMR prioritizes genes with non-redundant, complementary contributions, subsequent wrapper steps of forward and backward sequential feature selection occur independently of the mRMR ranking. mRMR reduces the list of features considered by these algorithms, but it is possible for only high mutual information genes to be selected for the final signature. Thus, the inclusion of lower mutual information genes, such as Ube2v1 and Urod, reinforces the effectiveness of the mRMR method.

The cellular roles of these protein products (Figure 2 and Figure 3) demonstrate a variety of pathways and functions (Figure 4), some of which have previously discussed⁴⁶. These include DNA repair genes (Polk²⁹ and Pold1³²), inducers of apoptosis (Ei24³⁶, Bax³⁶, and Phlda3³⁶), chaperonins (Cct3²⁸ and Cct7²⁸), cell cycle regulators (Ccng1³³ and Cdkn1a³⁶), B-cell development genes (Cd79b²⁴ and Blnk²⁴), B-cell antigens (Cd72⁹ and Ms4a1²⁴), and a stress-response kinase that inhibits protein synthesis globally (Eif2ak4³¹).

Figure 4. Depiction of the major cellular functions of most frequently appearing genes of the best murine signatures (same genes presented in Figure 1 and Figure 2).

One of the best murine signatures derived from GSE10640[GPL4783]: Phlda3, Blnk, Bax, Cdkn1a, Cct3, Pold1, Cd79b, Ei24, Eif2ak4, Ccng1, Glipr2, Hexb, Pou2af1, Swap70, Apex1, Ptpn1, Mdm2, Tpst1, Ly6e, Sdcbp consistently achieved <10% misclassification error with SVM parameters C = 10, σ = 10. However, for samples that are incorrectly classified according to this signature, the misclassification percentage does not reveal the actual deviation from the correct dose. The confusion matrix visualizes the prediction accuracy of this signature on GEO: GSE10640[GPL6524] (Figure 5). Indeed, the performance of the matrix shows that the predicted errors for a small fraction of samples deviate from the actual exposures by no more than a single adjacent exposure level. Although the predictions presented in the confusion matrix come from a single iteration of k-fold validation, the standard error associated with misclassification for this signature is extremely low (0.0013) so this confusion matrix is representative of nearly all possible iterations of k-fold validation.

Figure 5. Confusion matrix for murine signature: Phlda3, Bax, Cdkn1a, Cct3, Tfam, Pold1, Cd72, Cd79b, Ei24, Galt, Eif2ak4, Ms4a1, Ccng1, Glipr2, Gga2, Sh3bp5, Hexb, Gcdh, Pou2af1, Swap70, Apex1, Ptpn1, Mdm2, Tpst1, Ly6e, Sdcbp, Lcn2, Suclg2.

Numerators represent the number of samples in each category while denominators represent the total number of samples that were irradiated at a given exposure level (i.e. is the sum of the number of samples in each row).

Human gene signatures

The best performing signatures obtained from each human dataset, assessed by k-fold validation, are presented in Table 2. Although four human radiation datasets were available, GSE701 contained only 10 samples, which was insufficient for derivation of a unique gene signature. While k-fold validation removes the requirement for inter-dataset normalization, it assesses the ability of signatures (genes) to predict radiation exposure without tying the signatures to corresponding models. Each signature is characterized by the feature selection algorithm and its validation statistics, which have been averaged over the 3 independent datasets that were excluded from the original data used to derive the signature.

Since traditional validation typically requires separate training and test sets that feature samples irradiated at the same exposure levels, only signatures derived from GEO: GSE6874[GPL4782] and GEO: GSE10640[GPL6522] could be analyzed. Table 3 presents the best human signatures according to this validation approach. This type of external validation is the most challenging due to the variability associated with different microarray experiments and batch effects of different platforms. This potentially explains the lower performance obtained by traditional validation (Table 3) compared with k-fold validation on the same datasets (Table 2). The remaining human signatures are described in Supplementary Files Y1–Y5.

To determine which human genes are most consistently selected, the most frequently appearing genes (11 or 12 depending on number of equally prevalent genes in different signatures) were compiled from the top 20 human signatures (assessed by lowest average log loss during k-fold validation) from GSE10640[GPL6522], GSE6874[GPL4782], and GSE1725. The union of these three lists indicates the relative frequencies of each gene (Figure 6). Figure 7 visualizes the mutual information of gene expression (Figure 6) shared with radiation dose.

Figure 6. Heat map depicting the gene compositions of the top 20 human signatures derived at different radiation doses: 150-200 cGy (GSE10640[GPL6522], GSE6874[GPL4782]) and 500 cGy (GSE1725).

Frequencies are first scaled within and then between datasets to ensure values between 0 and 1.

Figure 7. Scatter plot depicting the mutual information each gene’s expression shares with radiation exposure (averaged over GSE10640[GPL6522], GSE6874[GPL4782], and GSE1725).

The size of each circle is proportional to the frequency at which the gene appears in the top 20 human signatures ranked by average validation log loss from GSE10640[GPL6522], GSE6874[GPL4782], and GSE1725. The genes shown are also are the same as those indicated in Figure 6.

While most genes have similar representation in signatures derived from different datasets, GADD45A and DDB2, in particular, are significantly more frequent in those derived from GSE1725 and GSE10640[GPL6522]. GADD45A and DDB2 are present in signatures derived from samples irradiated at different exposures (GADD45A – 500 cGy, DDB2 – 150-200 cGy). This raises questions as to whether these genes have a larger influence on the accuracy of individual signatures and whether their expression is calibrated to radiation exposure levels. Removal of these gene features was performed to address their impact. Genes of interest have been removed from each of the top 20 human signatures derived from various datasets and then the signatures were revalidated excluding these features (Table 4). The difference between the validation metrics preceding and following removal of a gene represents the weight of the gene within a signature. ΔMC, ΔLL, and ΔGoF represent the changes in misclassification, log loss, and goodness of fit, respectively.

GADD45A appears in 14 of the top 20 signatures derived from GSE1725. Of the 14 signatures, 10 were single gene signatures, as GADD45A alone was expected to sufficiently distinguish irradiated from unirradiated samples. In these cases, it was assumed that a null signature would perform as well as a predictor that randomly draws predictions from a uniform distribution of doses. Removal of GADD45A from these 14 signatures, results in an average increase in misclassification, log loss, and goodness of fit by 0.319, 0.368, and 109 cGy, respectively (see Table 4a). In contrast, elimination of BAX, which only appears in 2 of the top 20 signatures derived from GSE1725 and results in an average increase in misclassification, log loss, and goodness of fit by 0.018, 0.147, and 2.95 cGy respectively (Table 4e). Comparing the effects of removing DDB2 (Table 4c) and PRKAB1 (Table 4f) from the top 20 GSE10640[GPL6522] signatures confirms the impact of genes that frequently occur within the most accurate gene signatures.

However, the diagnostic contributions of GADD45A and DDB2 expression to the radiation levels at which samples were exposed (500 cGy and 150-200 cGy respectively) are confounding. The effects on model performance resulting from removal of GADD45A from the GSE10640[GPL6522] signatures (Table 4b) versus the GSE1725 signatures (Table 4a) are discordant. ΔMC is higher when GADD45A is removed from GSE1725, but ΔLL is higher when GADD45A is removed from GSE10640[GPL6522]. ΔLL is large when GADD45A is removed from both datasets, which is consistent with the importance of GADD45A at both radiation doses. Indeed, GADD45A expression has been demonstrated to be rapidly induced by radiation levels as low as 2 Gy⁴⁷. Similar discordance was observed in the feature removal experiments of DDB2 (Table 4c, 4d).

As was the case with murine signatures, genes appearing in the best human signatures do not necessarily share high mutual information with radiation dose. However, the compositions of the human signatures are dominated by four genes, DDB2, GADD45A, PCNA, and PPM1D, which all share a lot of information with radiation dose (DDB2: 0.55, GADD45A: 0.39, PCNA: 0.51, PPM1D: 0.46). The functions associated with these and less frequently appearing genes are depicted in Figure 8⁴⁶. The pathways and functions represented include keratinocyte differentiation (PRKCH⁹), induction of apoptosis (BCL2L³⁷ and BAX³⁶), DNA repair (TP53BP1²⁹, RAD17³⁰, DDB2²⁴, PRKDC²⁹, and PCNA³³), actin nucleation (ARPC1B²⁸), and regulation of JNK-p38 (MAPK14) signalling (GADD45A³³ and PPMD1³³). The four common genes belong to the DNA repair and regulating JNK-p38 (MAPK14) pathways, which may imply particular significance to these functions in human response to radiation exposure. Interestingly, GADD45A and PPMD1 are antagonistic, that is, GADD45A activates while PPMD1 inhibits p38.

Figure 8. Depiction of major cellular functions of most frequently appearing genes of the best human signatures (see Figure 5 and Figure 6).

Validating gene signatures on partial body irradiated samples

We also evaluated the total body irradiation human signatures with expression data from baboons (GSE77254) that were exposed to partial body irradiation. All signatures derived from human samples (see Supplementary Files Y4 and Y5) were completely contained in this dataset and so were eligible for validation. The signatures chosen contained all datapoints, circumventing the need to perform nearest neighbour imputation. Paralogous baboon genes were cross-referenced with those that were used to derive human signatures and expression values of multiple probes within the same gene were averaged.

Signatures were used to differentiate between various label combinations: (1) unirradiated vs. 1 day post-irradiation, (2) unirradiated vs. 2 day post-irradiation, (3) 1 vs. 2 day post-irradiation, (4) unirradiated vs. 1° and 2° HARS, (5) unirradiated vs. 2° and 3° HARS, and (6) 1° and 2° HARS vs. 2° and 3° HARS. Supplementary Files Z1 and Z2 contain validation results based on baboon expression data with human signatures.

Multiple Y4 signatures achieved 0% misclassification in distinguishing unirradiated samples from radiated samples (above label combinations 1, 2, 4, and 5) and multiple Y5 signatures achieved 0% misclassification in label combinations 1, 2, and 5. However, the best performing signatures on this dataset were not the best performing signatures obtained during validation on GSE6874 (Y4) and GSE10640 (Y5). We speculate that technical factors involved in the study design explain why signatures performed differently. For example, the human signatures were derived from blood samples that were collected 6–24 hours after exposure whereas the baboon blood samples were obtained 24–48 hours after exposure. Also, a different microarray platform was used to obtain expression values for the baboon samples.

We also investigated total body radiation signatures on predicting exposures with different sources of partial body irradiation expression data: GSE66372⁴⁸ and GSE84898⁴⁹. These murine and baboon datasets lacked several genes present in the signatures we derived. None of the Y4 and Y5 signatures were completely contained in GSE66372; the PSMD9 single gene signature was the only human signature that was completely contained in GSE84898. However, the PSMD9 signature has poor performance among Y5 signatures based on its log loss metric on GSE6874.