Security authentication is one of the most important levels of information security. Nowadays, human biometric techniques are the most secure methods for authentication purposes that cover the problems of older types of authentication like passwords and pins. There are many advantages of recent biometrics in terms of security; however, they still have some disadvantages. Progresses in technology made some specific devices, which make it possible to copy and make a fake human biometric because they are all visible and touchable. According to this matter, there is a need for a new biometric to cover the issues of other types. Brainwave is human data, which uses them as a new type of security authentication that has engaged many researchers. There are some research and experiments, which are investigating and testing EEG signals to find the uniqueness of human brainwave. Some researchers achieved high accuracy rates in this area by applying different signal acquisition techniques, feature extraction and classifications using Brain–Computer Interface (BCI). One of the important parts of any BCI processes is the way that brainwaves could be acquired and recorded. A new Signal Acquisition Strategy is presented in this paper for the process of authorization and authentication of brain signals specifically. This is to predict image memorability from the user’s brain to use mental imagery as a visualization pattern for security authentication. Therefore, users can authenticate themselves with visualizing a specific picture in their minds. In conclusion, we can see that brainwaves can be different according to the mental tasks, which it would make it harder using them for authentication process. There are many signal acquisition strategies and signal processing for brain-based authentication that by using the right methods, a higher level of accuracy rate could be achieved which is suitable for using brain signal as another biometric security authentication.
Notes
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
1 Introduction
For many years, authentication technology plays a crucial role in terms of data security. The authentication process is to determine whether someone or something is, who or what it declares itself to be. Biometric authentications are the most secure methods that we are using nowadays. Besides all of the great advantages of the recent biometric methods, they still have some weaknesses and issues [1] which brainwave authentication can cover those.
Brain-based authentication is getting very popular in researcher’s works recently. Brainwave can be the most secure biometric authentication, as it does not have some issues and disadvantages, which other biometric techniques have. In terms of security, brainwaves can have a higher level of security as it is not visible to duplicate and the ID pattern would be changeable.
Advertisement
There are many research experiments, which used electroencephalography (EEG) to test brainwaves for authentication purposes using Brain–Computer Interface (BCI). Every BCI process has three important parts including signal acquisition, feature extraction and classification. This process would be different regards to the main goal of using human brainwaves. For example, using BCI for authentication purposes needs specific ways of the main three parts of the BCI process. Every part can have a big effect on the results.
There are many researches, which have used different methods of signal acquisition, extracting the specific features and classification techniques. According to the weakness of the human brains, which are a type of electromagnetic waves, and the devices that we are using to record them at the time that this paper is written, we need to use special techniques for signal recording for any specific purposes to achieve the best results.
This paper is presenting a new brainwave acquisition strategy that is specifically for brain-based authorization and authentication by predicting picture memorability, which through that, users can register their brain-ID to the database, authorize themselves and pass the authentication security by using a unique visualization pattern in their mind. Many different patterns have been used by researchers including relaxation [2], muscle movements [3], visualizing [4], breathing and sport activity [5], Rapid Serial Visual Representation (RSVP) [6], pass-thoughts [7] and many others. Mental imagery and picturizing pattern using Signal Acquisition Strategy, which is presented in this paper, can be the most secure brain-ID pattern, in which this research is concentrated on it specifically.
The remainder of this work is organized as follows: (1) Related existing brain-based authentication methods using different brain-based paradigms. (2) The proposed signal acquisition strategy. (3) The process of preprocessing, feature extraction and classification. (4) The experiment’s results and evaluation. (5) Finally, conclusion the experimentation according to the presented approach.
Advertisement
1.1 Literature background and related works
Current research in this area can be categorized as Biometrics, Brain–Computer Interface and Brain-Based Authentication. The majority of the recent research in these years shows that brain wave is a biometric characteristic that is unique and inherent in every person, but there are still some limitations which need to be covered by more research to have better results and the highest accuracy rates to use brain signal as a new biometric authentication.
1.2 Biometrics
There are two different concepts in biometrics that we should concentrate on them, which are Behavioral/Physical Biometrics and Authentication/Identification. Behavioral Biometrics focuses on studying the non-biological or non-physiological structures of human body. It considers the uniqueness of human behavioral characteristics like voice, signature, keystrokes and gait. Physical Biometric is focusing on analyzing the biological and physiological characteristics of human body [8]. In terms of security and usability, biometric authentication is one of the most well-known approaches because of the uniqueness of some specific features. These features are so hard to duplicate and accurately produce. However, for security purposes, physiological traits are more practical. The most commonly used physiological biometrics are face detection, fingerprint identification and iris/retina scan. Table 1 shows some important advantages and disadvantages of these methods.
Table 1
Typical biometric methods advantages and disadvantages [1]
Biometric advantages and disadvantages
Advantages
Disadvantages
Security: Biometric technology brings different types of solutions, which are nearly impossible to hack unlike passwords
Physical traits are not changeable: We can reset a password, but we never can change our fingerprints or retina; these are fixed
Accuracy: Biometric works with individual’s physical traits such as fingerprints, face, retina among others that will always serve you accurately anywhere, anytime
Error rate: Usually, biometric devices make two types of errors, False Acceptance Rate (FAR) and False Rejection Rate (FRR). When the device accepts an unauthorized person, it is known as FAR, and when it rejects an authorized person, it is known as FRR
Convenient: Your credentials are with you forever, so it does not require you to memorize or note down anything
Delay: Some biometric devices take more than the accepted time and a long queue of workers form waiting to be enrolled in large companies
Flexibility: You have your own security credentials with you so you do not need to bother memorizing awkward alphabets, numbers and symbols required for creating a complex password
Unhygienic: In contact-based biometric techniques, a biometric device is used a lot times by enormous amount of people. Everyone is actually sharing his or her germs with each other via the device
Trustable: Reports claim that the young generations trust biometric solutions more than other solutions
Physical disability: Some people are not fortunate enough to be able to participate in the enrollment process. They might have lost or damaged body parts such as fingers or eyes
Scalability: Unlike other solutions, biometrics are highly scalable solutions for all types of projects. It is possible for any kinds of projects because of the scalability of its solutions
Environment and usage matters: Environment and usage can affect the overall measurements taken
Save time: Biometric solutions are highly time conserving
Save money: With a little money, any company can track their employees and reduce the extra costs they are paying for years
On the other hand, the potential of using brainwaves as a new biometric identification has risen recently. However, it needs more time and doing more experiments before becoming a method of security.
Neural oscillations, or brainwaves, are an essential mechanism to enable the synchronization of neural activity inside and around brain areas and helps the accurate temporal organization of neural processes underlying memory, cognition, behavior and perception [9].
Electroencephalography (EEG) is an electrophysiological observing method to acquire electrical activities generated by human brain using electrodes placed on the surface of the scalp [10]. It is an excellent tool for studying the processes of neurocognitive underlying person behavior because of some reasons, for example, (1) EEG has very high time resolution and captures cognitive processes in the time frame in which cognition occurs. (2) EEG directly measures neural activity. (3) EEG is inexpensive, lightweight and portable. (4) EEG monitors cognitive-affective processing in the absence of behavioral responses [11]. In terms of frequency, we all have five types of brainwave (Gamma, Beta, Alpha, Theta, Delta) and “each frequency is measured in cycles per second (Hz) and has its own set of characteristics representing a specific level of brain activity and a unique state of consciousness” [12]. This is represented in Table 2.
Table 2
Types of brainwaves and their frequency rates and mental state situation [1]
Wave
Frequency
Mental state
Gamma
Above 40 Hz
Thinking, integrated thought
Beta
13–40 Hz.
Alertness, focused, integrated, thinking, agitation, aware of self and surroundings
Alpha
8–12 Hz.
Relaxed, non-agitated, conscious state of mind
Theta
4–7 Hz
Intuitive, creative, recall, fantasy, dreamlike, drowsy and knowing
Delta
0.1–4 Hz
Deep, dreamless sleep, trance and unconscious
1.3 Brain–Computer Interface (BCI)
BCI technology is a communication process for both users and systems. To complete a communication, no external devices or muscle involvement is needed to issue instructions [13]. The BCI can be separated into invasive, partially invasive and non-invasive types [14]. In invasive BCI, recording the signals occurs when electrodes enter brain tissue by surgery. In partially invasive BCI, electrodes are placed inside of the skull on the gray matter over the surface and it needs surgery. In a non-invasive BCI, no need to do any surgery. Instead, the electrodes are placed over the head like a hat or a headset.
A BCI process has five sequential stages: signal acquisition, preprocessing or signal enhancement, feature extraction, classification and the application interface. In this process, the signal acquisition stage is a considerable challenge. According to the user strategy, signals will be acquired from the user’s brain. In general, there would be some artifacts and noises in the recorded brain such as blinks, eye movements and heartbeat.
For this matter, a couple of different methods are presented by different studies for artifact removal. These techniques have specific purposes that could match each objective of experiments conducted [15]. After preprocessing and filtering, the EEG signals will pass through the feature extraction process and select particular features by some feature selection methods. In most existing BCI, the identification process relies on a classification algorithm. These procedures are used to identify “patterns” of brain activity [16]. Classification algorithms are divided into five different categories: linear classifiers, neural networks, nonlinear Bayesian classifiers, nearest neighbor classifiers and combinations of classifiers [17].
1.4 Brain-based authentication
Currently, biometrics, such as voice, fingerprint, iris, face, have been widely studied, especially in real-life situations. But, there are some weaknesses in these biometrics [18]. For example, fingerprints can be replicated through latex milk, plastic mold and wood glue [19]; a fingerprint made by a 2D picture from a normal printer [20]; high-resolution photography [21]. Face, fingerprints, retina and iris are all non-cancellable. This means that they cannot be replaced and a new eye or finger cannot be grown again or face volitionally cannot be changed. We need a biometric more secure than any of these, which would be more difficult to replicate, and it would be cancellable. Brain electrical activity may meet these criteria.
There are a couple of unique advantages in EEG compared to other biometrics methods. First, to record EEG, the person must be alive [22]. Fingerprint and face can be maintained even from a dead human body, or the iris, after death, is still valid for recognition for a few hours after death [23]. According to this matter, the user has to be alive and in a conscious state to produce EEG data. Second, brain signal voltage will fall off dramatically with distance from the brain.
Despite the many advantages of brain-based biometric, it is still not extensively adopted because considerable research still must be done. There are seven factors to evaluate the reasonability of using a biometric for security purposes including universality, uniqueness, collectability, permanence, performance, acceptability and circumvention [24].
Table 3 compares five biometrics including brainwaves in terms of the seven factors mentioned [18].
Table 3
Comparison of some biometrics with brainwave biometrics
Biometric identifier
Universality
Uniqueness
Permanence
Collectability
Performance
Accessibility
Circumvention
DNA
H
H
H
L
H
L
L
Face
H
L
M
H
L
H
H
Fingerprint
M
H
H
M
H
M
M
Iris
H
H
H
M
H
L
L
Brainwave
H
H
H
M
H
M
H
Recently, scientists and researchers have been doing many attempts to observing the pattern uniqueness of the brain signal. Several different methods have been used to analyze EEG signals.
In regard to the recent progression of EEG signal acquisition devices, the capability of providing better results is going higher and these processes are getting simpler. Some studies in this research area were reported good results with higher accuracy rates in their experiences [25]. These studies used different methods of BCI process for signal acquisition, feature extraction and classification on brainwaves to use them for authentication purposes.
An authentication system proposed by Riera et al. [26] was using a combination of ECG and EEG signals for discriminating against the situation. The achieved accuracy for this study was higher than others even for relaxation task. This is due to the added ECG channel. A different approach proposed by Jayarathne et al. [27] tested the possibility of person authentication by thinking about a specific number. Yeom et al. [28] presented a method to extract unique signals using “self- and non-self” face pictures. In response to self-face, each subject has its own characteristics, so EEG patterns must be unique in regard to this subject.
Chen et al. [6], proposed an authentication system using Rapid Serial Visual Presentation (RSVP) stimulus. Linear Discriminant Analysis (LDA) was used to classify EEG signals obtained by a brain amplifier. The important features are calculated using a particular association constant. According to the author’s notation, a password can be hidden effectively in certain compulsive situations.
Chuang et al. [29] presented a new approach which used the MindWave to obtain data. Seven tasks were executed, including sports activity, breathing, audio listing, simulation of finger movement, color, reciting and identifying music with singing, and pass-thoughts. The classification process is done with the k-nearest neighbor (k- NN) algorithm. The most accurate strategies were for color, audio and sport. The most difficult one was for the pass-thought task according to the results of the questionnaire that determined for user-friendliness with different tasks. Breathing, audio and color were the straightforward tasks.
La Rocca et al. [2] presented an approach centered around connectivity within EEG spectral coherence. In this method, data samples were gathered from 108 participants during open resting and closed eyes positions. EEG data were captured using a system consisting of 64 different channels with a rate of 160 Hz. Data were filtered to 50 Hz via a low-pass anti-aliasing filter. Spectral coherence (COH) and power spectral density (PSD) analysis techniques were used to extract mental features. To calculate uniqueness, two different algorithms were used separately in this process which was Mahalanobis classifiers that were based on distance and match score fusion system. This technique is strong and very accurate for user identification. The performance of classification has the possibility of not functioning properly if this classification was used for a larger group of users on traditional hardware and it is less than 100%.
Ruiz-Blondet et al. [30] presented a protocol known as CEREBRE with a band-pass filtering between 1 and 55 Hz, and based on normalized cross-correlation, a simple discriminant function was used for classification. The nominal (4 categories, 3 channels) classifier showed the highest accuracy when all the patterns were used, but both maximum and minimum classifiers showed 100% accuracy. The results presented that the most accuracy was for the stimulus oddball and food. The resting pattern had reduced performance in terms of classification. Authentication centered on a memory-evoking task (also known as “pass-thoughts” in other studies) [31] also showed weak results too; this is due to the inconstant time that was consumed to allow thinking.
1.5 Mental visualization
There are three groups of protocols used for recording EEG in general: mental tasks, resting states and tasks with an external stimulus. Choosing each protocol can influence the procedure of authentication and the accuracy. For instance, for mental tasks or the resting states an EEG recording device is required, while external stimuli tasks need devices to make the appropriate stimulation. Then again, resting states tasks can be effortlessly influenced by artifacts and noisy environment, while a higher “signal-to-noise ratio” (SNR) can be seen in tasks followed by external stimuli and mental tasks. This can be accomplished by recognizing “Event-Related Potential” (ERP) [32]. For example, to observe the ERP reactions to the visual stimuli, the visual cortex, occipital lobe and central region are engaging.
According to this and with some compromising, the number electrodes that have been utilized in this study to record brain signals have been reduced. Many different mental tasks have been tested and good results achieved. However, most of them are very complex and time-consuming for authentication process. Tasks like: imagine some physical body movements [33], counting numbers in mind, sing a song, and focus on a desired thought [29], imagining the movement of a given geometric shape around an axis [34] and some other sets of mental tasks that individuals are asked to perform.
It is shown that better results have been achieved from imaginary tasks and protocols in comparison with the physical ones. On the other hand, a popular theory called “dual coding” [35] showed that graphical substances such as images, shapes or pictures are easier to remember in comparison with the number, words and sequences. Basically, this theory explains that these types of objects are determined (memorized) with 2 particular codes (verbal and pictorial), whereas a number sequence or a word is determined by a single verbal code. It has been recommended in a study that despite the fact that photographs and words share indistinguishable semantic meaning, pictures are easier to remember and more memorable because they have more particular codes than words [36]. Many studies concluded in their results that words are more memorable than numbers, and graphical patterns, shapes and images are more memorable than words. These progresses reached out to the long-term memories as appropriate as the short-term memories. Mental imagery or colloquially “visualizing,” “seeing in the mind’s eye,” “hearing in the head,” “imagining the feel of” resembles imaginary experience, but happens in the lack of the real external stimuli. Mental images are always pictures of something or other which appears in mind and by this means functioning as a form of mental picture [37]. Some research has shown that visual mental imagery is in control of frontal–parietal regions and can rely on occipital–temporal regions of the brain.
In a memory test experience, participants responded “remember,” “know” or “new.” “In the imagery test, participants responded “high vividness,” “moderate vividness,” or “low vividness.” Visual memory (old-remember) and visual imagery (old-high vividness) were commonly associated with activity in frontal–parietal control regions and occipital–temporal sensory regions. In addition, “visual memory produced greater activity than visual imagery in parietal and occipital–temporal regions” [38].
According to the literature, pictures and shapes can be easier to memorize and choosing a good authorization paradigm could be very important for authentication purposes. Therefore, we decided to concentrate on mental imagery and picturizing pattern using a picture as a brain-ID, which in this study 2 different types of pictures (2D geometric shapes and 3D real pictures) have been tested by the proposed method to predict image memorability.
A new algorithm is designed to acquire the brain signal. Therefore, the simple types of images that can be remembered and recalled could be an appropriate brain-based paradigm for authorization and authentication processes.
2 SaS-BCI
SaS is for Signal Acquisition Strategy, which is for a user to register the unique brain signal on an application or device and use it as an authentication method.
SaS-BCI is divided into 4 stages as follows:
×
The proposed method is divided into three stages: The first stage is to record EEG signals obtained from human participants (User ID). In the next stage, these signals will be preprocessed using the notch filter method, band-pass filter in the range of 0.5–7 Hz, noise removal for then eye blink and normalization. The last stage will be classifying the data and then achieving the accuracy of the method for the authentication purposes (Fig. 1).
Fig. 1
New Signal Acquisition Strategy using Brain–Computer Interface
×
In this study, the acquired data are divided into three portions: training data, cross-validation data and testing. The training data are utilized to ensure the machine perceives image paradigms in the data, and the cross-validation data are utilized to guarantee the effectiveness of the used algorithm and a better accuracy for the trained model. At the end, the test data are utilized to perceive how well the images can be predicted based on its training.
This process has been done on three mental tasks in particular; for looking at a white wall, looking at pictures and imagining (visualizing) the pictures in mind. A dataset is provided that contains historical data from which to learn patterns. It needs the outcomes to determine the features that best predict the outcomes. However, the main purpose of this study is testing the visualizing paradigm and predicting the pictures by recalling them.
We experimented to investigate if brain signal can be used to predict image recall. EEG data were collected from users by seeing and picturizing different pictures. Users were then asked to imagine the pictures from the two categories alternatively; they saw and kept in their memories. They were then asked to reimagine the pictures they memorized.
2.1 Step 1: data acquisition
To acquiring the signals, we used a special subject strategy, a non-invasive BCI device (14 channels Emotiv Epoc +) [39], a Laptop and Emotiv SDK Pro software to record the signals, digitize and make them ready for the other processes.
The EEG data were acquired from twenty healthy participants between 15 and 45 years old. Each participant sat in front of a white wall for 15–30 min.
The EEG data of each participant were recorded in two sessions and three different situations: First, watching at the white wall. Second, watching at a specific picture (printed on an A4 size paper). And the last one, visualizing or imagining the same picture in their mind while they were watching the white wall with open eyes and naming the pictures they could remember. Twenty different pictures were showed to each participants, ten easy to remember 2D single color objects with white background and ten hard to remember real object photos with more details. In the first session, the data were recorded from each position of the participants for 10s per picture. The second session had just one position, which was recalling the memorized pictures from the previous session (visualizing or imagining). It was in the same place, and the same situation and participants have been asked to do the imagination position when they were watching at the white wall and recalling the pictures by naming them. The data were recorded similar to the first session for each participants. Measurements were taken by Emotiv device from the 14 channels that were placed on the participant’s scalp. All channels were sampled at 250 Hz. You can see three different positions in two sessions have been exerted as user strategy for signal acquisition part of the project (Fig. 2).
Fig. 2
User strategies and signal acquisition from the participants in two sessions
×
2.2 Step 2: preprocessing (enhancement)
EEGLAB and BCILAB MATLAB toolboxes were used to process the data. These toolboxes are for the prototyping, designing, experimentation, testing and evaluation of Brain–Computer Interfaces; the raw EEG data need to be processed to obtain specific features and use it to train the model.
Signal filtering can significantly improve the visibility of a defect signal. Figure 3 shows the EEG data, which are recorded from one participant. In the mentioned figure, you can see that each colored bit signifies the spectrum of the activity of one data channel. For example, the leftmost scalp map shows the scalp distribution of power at 6 Hz. The other scalp maps indicate the distribution of power at 10, 16, 22 and 60 Hz. Plotting channel spectra and mapping computed the specific time windows in the data. Each channel has a colored line, which shows their signal characteristics. The characteristics and the distances between each of the lines show that these data need to be filtered and reject artifact to be analyzed in the further works.
Fig. 3
Plotting channel spectra and map
×
Filtering the continuous EEG data reduces the outline of filtering artifacts at epoch boundaries. A high-pass filter is often necessary to remove linear trends of the data. The reason for using the high-pass filter is removing slow and possibly large amplitude drifts in the signals. In this case, a basic Finite Impulse Response (FIR) filtering technique [40] is used for high-pass filtering. For the lower edge of the frequency passband and the higher edge of the frequency, passband 0.5 Hz and 50 Hz exerted, respectively.
The amount of 0.5 Hz for lower edge band-pass means everything below that frequency will be removed which is needed for Independent Component Analysis (ICA) analysis that will be exerted in the next step. This analysis algorithm is very sensitive to the lower shift frequencies. Figure 4 shows the filter response after filtering for the same channel. You can see that it filtered everything lower than 0.5 Hz passband and everything above 50 Hz passband and it removed sharp line noises. This filtering method is doing the both forward and backward phase shifts on the signals.
Fig. 4
Left: The original activity power spectrum in frequency. Right: The filtered data with high-pass filter technique
×
The EEG signals contain artifacts in preparation. There are two different kinds of artifacts: external and internal which are unwanted noises in a signal. Outer actions are external artifacts, and the actions made by the subject itself are internal artifacts. On the other hand, sometimes some EEG channels are not working properly for several reasons. In EEG, sometimes to record anything interesting, the quality of the connection between the electrode and the scalp is too low. It is important to identify the sensors with weak signal quality because the artifact removal will be more efficient. Removing cardiac and blink artifacts with some bad channels may not work fine, and worse; it will spread the bad signals to all the channels [41]. All channels checked and the bad channels rejected from the data to make it ready for ICA process to reject large artifacts.
The amplitude of artifacts is often larger than the amplitude of the brain data, which potentially decrease the signal/noise ratio, bias data analysis and potential results.
Many types of the artifact are visible and easy to find with eyes in EEG data such as eye movements, muscle movements, high noises, linear trends and discontinuity. All of the 14 channels of EEG data checked and the bad channels identified and removed. “Independent component analysis (ICA) aims to solve the problem of signals separation from their linear mixture. ICA is a special case of blind source separation when separation is performed without the aid of information (or with very little information) about the source signals or the process of signal mixing” [42]. ICA is excellent for identifying and removing blink artifacts because they are large in amplitude, have a discrete source and are extremely reliable from blink to blink. ICA algorithm applied on the EEG data in EEGLab tool. According to the volume of EEG data, the ICA process took a significant time to complete.
2.3 Step 3: feature extraction
This study extracted feature arrays from the EEG signals using PSD method. This was done to surge the effectiveness due to a larger number of data points and potential connections between channels across the whole brain. Therefore, it is a significant process in interpreting an input signal to use it as an authentication key.
Power Spectral Density (PSD) displays that different mental tasks have different frequency ranges with different powers. It defines the power distribution of a signal over frequency [43]. Fourier Transform (FT) [44] was exerted to convert the EEG signal from the time domain into the frequency domain.
PSD obtained over squaring of the total value of Fourier transformed data per segment. Based on that, the non-dominant region of the power spectrum [45] and the concavity of spectral distribution [46] variance of spectral power were considered as EEG features for recognition purpose.
After spotting the maximum level of the power spectrum, its tenth part was calculated and implemented as a measure. You can see the definition of this process in Fig. 5.
Fig. 5
Illustration of the concavity of spectral definition
×
The frequencies values of power spectral that were under the calculated measure were squared and then summed (1) where \(f_{j}^{u} \left( { j = 1,2,,3, \ldots ,N} \right)\) is frequency values under the measure. \(F_{u}\) is stated as a feature from the concavity of power spectral distribution. In the alpha band, a feature has been adopted from a power spectral variance that expects the spectrum which signifies the increase in spectral distribution (2)
where power spectral values are \(p_{k} \left( {k = 1,2,3, \ldots ,L} \right)\) and the mean value the alpha band is \(\bar{p}\). To distinguish individuals, the convexity of power spectral can be another key feature, which is defined as follows:
In the alpha band, the spectral values have been ranked, and afterward, the frequencies of the top three values were averaged, respectively. This procedure is shown as (I) in Fig. 6. Here, the top three power spectral values and the frequencies are \(p_{1} ,p_{2} ,p_{3}\) and \(f_{1} ,f_{2} ,f_{3}\), respectively, which their mean values are given by (3)(4)
Fig. 6
Process of the convexity of power spectral for frequencies and power spectral values
$$P_{m} = \frac{{p_{1} + p_{2} + p_{3} }}{3}$$
(3)
$$F_{m} = \frac{{f_{1} + f_{2} + f_{3} }}{3}$$
(4)
×
As shown in Fig. 6(II), the power spectral values that are bigger than the mean are
\(f_{i}^{g} \left( {i = 1,2,3, \ldots ,M} \right)\) and their summation is the following equation:
In this study, \(F_{m} ,P_{m} ,F_{g}\) from the convexity in power spectral distribution are extracted as features. The spectral analysis showed the frequency bands in which amplitude was different between the tasks and normal situation and specially the differences between each tasks and participants. Based on the classification method using MATLAB software, we used the extracted feature obtained using this technique.
2.4 Step 4: classification
Classification process was performed based on whether the mental imagery was successfully recalled or not using Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM) classifiers separately. At this stage, a choice of good discriminative features is very important for getting a better classification result that influences the project’s intentions. Because of different environments and different experiment settings, there is no particular method to compare the results between different methods.
LDA and SVM techniques are fast and less complex which makes them to be a good option specifically for authentication processes. LDA is often considered good for real-time BCIs because of its simplicity, speed of execution and low computational cost.
SVM works generally well depending on a clear margin of distinguishing two classes. It will be more efficient in spaces with high dimensions. It is fairly memory efficient. The main classification technique for this study is SVM method. LDA has been used separately to check the possible differences in results and accuracy rates.
Recently, SVM and Neural Networks (NN) methods [47], [48] are the most commonly used techniques in EEG authentication studies. However, in a simple way, SVM without kernel is a single neuron inside neural networks, nevertheless with diverse cost function. By adding a kernel function, it will be comparable with two layer NNs. Actually, in terms of the model performance, SVMs are sometimes equivalent to a shallow neural network architecture [49, 50]. Generally, a NN will outperform an SVM when there is a large amount of data.
However, many factors such as preprocessing, feature extraction and classification stage are the most important part of any BCI applications processes, which has highly influences on accuracy rate. Therefore, LDA and SVM classification methods have been used to see the differences and compare the results. But the main classifier for the aim of the project is SVM.
LDA is one of the most well-known data reduction methods. By utilizing this procedure, the hyperplanes are employed to distinguish the data from various classes. In this process, LDA expects normal distribution of the data, with equivalent covariance framework for the two classes.
The distinguishing hyperplane was gotten by looking for the estimate that maximizes the separation between the means of the two classes and limiting the interclass variance. For an N-class issue (\(N > 2\)), a number of hyperplanes are utilized in the process. Here, the following equation [51] can define it, which is expanded over every linear projections, w:
In Eq. (6), S represents a variance, m represents the mean, and the subscripts signify the two classes [52]. LDA technique is used by MATLAB software to find a linear transformation that discriminates between different classes.
Support Vector Machine (SVM) is a classification technique that creates a hyperplane that separates the dataset into classes dependent on kernel functions on a feature space with two dimensions [53] as shown in Fig. 7.
Fig. 7
Fundamental idea of linear SVM characterized by the optimal hyperplane
×
(\(x_{i}\)), which is the training data belonged to two separated classes (\(y_{i}\)), (7) represented with the optimal hyperplane (8)
The \(\varphi\)() function maps the input space into a greater dimensional space. Support vectors are the closest training data samples to the hyperplane.
SVM classification technique was performed using tenfold cross-validation to find out if the EEG signals formed with a picture presenting can be used to predict that image’s comprehended memorability. This classification process has been performed based on two labels of least memorable images and most memorable images according to correctly and incorrectly recalled.
3 Results
The total recorded EEg data are divided into three categories for each user position. Three PSD feature vectors are created and classified by SVM and LDA classifiers. Table 4 shows the average acquired accuracy rate for the pictures recalled from 20 participants. These percentages are by image category per user.
Table 4
Image recall success percentage for both 2D and real object pictures
User ID
2D object (%)
Real object (%)
1
100
65
2
100
63
3
98
54
4
98
30
5
98
65
6
98
23
7
92
58
8
98
30
9
96
11
10
98
45
11
98
65
12
100
54
13
94
18
14
100
21
15
98
45
16
98
33
17
100
61
18
94
15
19
100
18
20
100
38
The results showed that the visualization of the image could be considered as secure brain-based paradigm or Brain-ID for authentication.
According to the results, there might be many reasons that are producing the EEG signals to be higher for the image that was successfully recalled later. For instance, as identified by the BCI, the increase in mental activities could be credited to the user trying to mentally pronounce the image name.
2D object images with higher EEG amplitudes had a higher chance of being recalled real object images with lower amplitudes.
It is significant here that there may be various reasons that are causing the EEG signals to be higher for the images that were successfully recalled later. For instance, as identified by the BCI the ascent in mental activities could be recognized to the user to name the image mentally. Another explanation behind this ascent could be that the image is causing a mental imagery.
It shows that real object pictures are harder to remember and recall. The 2D object pictures are having a much higher successful recall (98%) than the real object pictures (41%). The recall differences between individuals can be noticed when it comes to real object pictures. About 60% success rate achieved for a quarter of the users for recalling the real object images. SVM classifier has been used to predict image recall. The data used in this experiment are for both successfully and unsuccessfully recalled images averaged over the 14 channels per user.
These data were used for feature extraction and selection in the domains deliberated earlier. The performance of the classifiers based on the extracted feature sets is shown in Table 5.
Table 5
Performance of image recalled vs not recalled by two different classifiers including SVM & LDA
Classifier
Feature set
Precision
Recall
F-Score
Accuracy
AUC
LDA
PSD
0.89
0.83
0.85
0.86
0.92
SVM
PSD
0.89
0.86
0.86
0.88
0.93
The PSD resulted in an average rate of 88% accuracy with 0.93 of AUC by SVM classifier and an average rate of 88% accuracy with 0.92 of AUC by LDA classifier. It is important to mention that the image recall’s prediction achieved in this experiment is only from the short-term memory, and the results could be different for long-term memory. This study specifically concentrated on the pictures that users were asked to memorize in their minds. It provides an understanding into how the brain observes images with different characteristics and helps provide a significant improvement for mental imagery security while recognizing the human unique feature in the system.
Most of the researchers reported that a higher accuracy rates can be achieved with more complex tasks [3, 6, 30], but complex tasks make the system less practical, because both processes of system training and authentication would be very time-consuming. In regard to this matter, the presented authentication method with the SaS brain-based paradigm is simpler and faster in comparison with the other methods in different studies. Table 6 shows a comparison between this research and two other studies for brain-based authentication. The accuracy of the presented method is 88% which is lower than Gui et al. [54] with 90% accuracy.
Table 6
Comparison of three different experiments with different strategies and classifiers
Experiments
Signal acquisition strategy
The number of subjects
Channels
Features
Classifier
Ave accuracy
Gui et al.
Read the words silently
32
6
Wavelet packet decomposition
Neural network
90%
Yeom et al.
Visual evoked potentials
10
18
Dynamic feature
SVM
86%
Proposers
SaS
20
14
PSD
SVM
88%
Although it is less than Gui et al. results, the authentication method that used in the proposed study is simpler and less complex than Gui et al. Higher accuracy can be achieved by combining several extracted features, but the classification takes a long time to process and it makes the experiment more complex. Task complexity makes the computational cost higher, and it is more time-consuming which makes the authentication process slow and less practical.
Using more subjects rises the accuracy rate of classification methods. It is confirmed by Chen et al. [6] with execution multi- and single-trial classification. Using fewer subjects delivers a better accuracy rate than exerting more subjects; therefore, the similar ones can be found in a bigger set [2]. According to this matter, approaches, which demonstrate high accuracies with small numbers of subjects, might not represent a strong result for a big number of populations in terms of security, because the risk of hacking the system would be higher.
According to this matter, the average accuracy rate for the “SaS” strategy is in the second place by using 20 subjects in comparison with Yeom et al. [55] with fewer subjects. This means that the presented authentication system achieved a higher accuracy with more subjects using the SaS strategy. On the other hand, brain patterns for a specific mental task can change over time. Specifically, accuracy of frameworks trained by preference-based tasks [30] can corrupt as user tastes change.
It is important to note that the experiment performed in this study can be used for the both short-term and long-term memories. It means that for the long-term memory issues, the brain can produce the same signal amplitude by re-seeing the same picture that have been seen (memorized) as a Brain-ID.
EEG brain signals are distinctive in general; however, the processes of feature extraction to normalize differences in time are a reason for some loss of uniqueness, which is why the obtained results in some parts are not without errors. There are many different ways and methods for signal processing such as different types of signal filtering and noise removing, feature extraction algorithms and many other classification methods and classifiers that could achieve different results but not significantly.
According to the literature, for this type of authentication method the alpha and beta frequencies are the most appropriate bands because the main stimulus is thinking and imagining a picture (visualizing) which is active. However, in some cases, the combination of Beta and Alpha bands (8–40 Hz) could acquire better accuracy than the separate frequency bands. The differences between the accuracy rates for each user with the same user strategy show that capturing the raw signal can affect the results. It means for some users the signal strength of the electrodes was not steady in the time of recording because some of the participants had long hair and the hair had influence on the quality of the signals, and on the other hand, the situation of electrodes in Emotiv device was not suitable for the shape of the some user’s head.
What this experiment adds is explicit to images in that users are required to memorize; then, participants have been asked to remember the introduced images. This study gives a knowledge into how the brain sees pictures with various attributes, and what that can educate us concerning these images recall, which may help improving security process for brain-based authentication methods.
4 Conclusion
Selecting each of the general protocols for recording EEG including physical tasks and imaginary tasks can affect the process and the accuracy of brain-based authentication process.
Different mental tasks or brain-based paradigm will have different outcomes. According to the literature, it is shown that better results have been achieved from imaginary tasks and protocols in comparison with the physical ones. Many different mental tasks have been tested and good results achieved. However, most of them are very complex and time-consuming for authentication process.
For a secure and fast brain-based authentication process, we need an imaginary brain paradigm that can be used as a brain-ID which in terms of security it is much better than physical brain paradigms. Regarding to the literature, pictures and shapes can be easier to memorize and choosing a good authorization paradigm could be very important for authentication purposes. Therefore, in this study a brain picturizing pattern has been tested by the proposed method to predict image memorability and see which types of images are easier to remember and are more appropriate to use as brain-IDs.
This result shows that this strategy could be a very useful way of authorization from each human’s brain-ID. For this process, the user can look at a specific picture, memorize it and register the brain-ID with imagining the picture in the mind. Therefore, in terms of security, the combination of these two could be a new hybrid strategy to get better results for using the brain signals as an authentication technique.
Nowadays, biometric authentications are the most useful techniques for smart devices and applications as a result of the ease of use, security level. However, there are a few detriments for certain techniques. Brainwave is another human biometric, which can cover the limitations of other types and could be the most secure biometric authentication.
There were many studies utilizing brain signals as an authentication technique, which in certain strategies high accuracy achieved. However, they are only for some particular mental tasks.
Mental imagery and using visualizing pattern in the mind could be a more secure brain pattern for using as a brain-based authentication. Memorizing a picture in the mind as a user ID would be a very useful way in terms of security, because no one can see it in user’s mind and user can change the pattern whenever is necessary.
In this paper, we investigated the use of an EEG brain signal to predict image memorability and the possibility of using mental imaginary pattern as a biometric authentication method. A new strategy (SaS) was used which in comparison with other studies in terms of security could be one of the most useful brain-based authentication patterns. It should be noted also that this project by excreting all the methods which were chosen for the preprocessing and feature extraction and classification resulted an average of 88% accuracy rate, and the results could be slightly different by using different methods of signal processing and classification.
The big challenge for all brain-based authentication studies is that the brain patterns can change depending on the brain situation. For instance, EEG data of a person can change in situations like being sick, being drunk, being addicted to drugs and getting mental issues like anxiety, high stress, deep sadness and many other situations that any human can experience. Therefore, it needs doing more experiments in this area to improve the results, which can be useful on all different mental situations of the human’s brain, not just for a specific tasks in a specific situation. In future work, more experiments would be done to test different brain patterns in different brain situations to find the uniqueness of human brain, which can be static in all different mental situations of human brains.
Compliance with ethical standards
Conflict of interest
The authors confirm that there is no conflict of interest with this submission and this is an original work.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
