1 Review
1.1 Introduction
1.2 Cognitive radio
1.2.1 1.2.1 The cognitive cycle
-
Sensing the environment: In cognitive radio networks, the primary network has the priority to use the spectrum than the secondary network. The secondary network may use the available spectrum but without causing harmful interference to the primary network. Therefore, it needs to primarily quantify and sense its surrounding environment parameters such as (1) channel characteristics between base station and users; (2) availability of spectrum and power; (3) availability of spectrum holes points in frequency, time, and space; (4) user and application requirements; (5) power consumption; and (6) local policies and other limitations [10].××
-
Analyzing the environment parameters: The sensed environment parameters will be used as inputs for resource management in all dimensions such as time, frequency, and space. The main resource allocation objectives in CR include but are also not limited to (1) minimizing the bit error rate, (2) minimizing the power consumption, (3) minimizing the interference, (4) maximizing the throughput, (5) improving the quality of service, (6) maximizing the spectrum efficiency, and (7) maximizing the user quality of experience. In practice, cognitive radio aims at satisfying multiple objectives; however, the combination of some objectives may create conflicting solutions such as minimizing the power consumption and bit error rate simultaneously. Therefore, tradeoff solutions are needed to guarantee a balance between the objective functions [11].
-
Making decisions for different decision variables: In order to achieve the objectives mentioned before, the CR network needs to make decisions concerning the following important variables: (1) power control, (2) frequency band allocation, (3) time slot allocation, (4) adaptive modulation and coding, (5) frame size, (6) symbol rate, (5) rate control, (6) antenna selection and parameters, (7) scheduling, (8) handover, (9) admission control, (10) congestion control, (11) load control, (12) routing plan, and (13) base station deployment [12]. The decision-making can be based on optimization algorithms; however, in order to reduce the complexity and achieve efficient real-time resource allocation, cognitive radio networks use machine learning and artificial intelligence.
1.2.2 1.2.2 Cognitive radio tasks and corresponding challenges
1.3 Learning in cognitive radio networks
1.3.1 1.3.1 Introduction to artificial intelligence and machine learning
1.3.2 1.3.2 Applying artificial intelligence techniques to CRs
1.4 Discussion: challenges and evaluations
1.4.1 1.4.1 Learning techniques evaluation: strengths, limitations, and challenges
Learning technique | Spectrum sensing (SS) | Decision-making | Strengths | Limitations and challenges |
---|---|---|---|---|
Adaptation ability to minor changes | Require training data labels | |||
Neural networks | × | × | Construction using few examples, | Poor generalization |
thus reducing the complexity | Overfitting | |||
Support | Generalization ability | Requires training data labels | ||
vector | × | × | Robustness against noise and outliers | and previous knowledge of the system |
machine | Complex with large problems | |||
Multi-objective optimization | Require prior knowledge of the system | |||
Genetic algorithms | × | Dynamically configure the CR | Suitable fitness function | |
based on environment changes | High complexity with large problems | |||
Game theory | Related to the capabilities of the | × | Reduces the complexity of adaptation | Requires prior knowledge of the system |
spectrum-sensing technique used | Solutions for multi-agent systems | and labeled training data | ||
Reinforcement | × | × | Learning autonomously using feedback | Needs learning phase of the policies |
learning | Self-adaptation progressively in real time | |||
Fuzzy logic | Related to the capabilities of the | × | Simplicity, decisions are | Needs rules derivation |
spectrum-sensing technique used | directly inferred from rules | Accuracy is based on these rules | ||
Entropy approach | × | × | Statistical model | Requires prior knowledge of the system |
Related to the capabilities of the | Simplicity | Requires prior knowledge of the system | ||
Decision tree | spectrum-sensing technique used | × | Decision using tree branches | May suffer overfitting |
Requires labeled training data | ||||
Artificial | × | Parallel search for solutions | Requires prior knowledge of the system | |
bee colony | Requires a fitness function | |||
Bayesian | × | × | Probabilistic models | Requires prior knowledge of the system |
May face computational complexity | ||||
Markov model | × | × | Statistical models | Requires prior knowledge of the system |
Scalable | May face computational complexity | |||
Case- | Find acceptable solution based | Complex search in large databases | ||
based | × | on the existing case found | Requires predefined and relevant cases | |
reasoning | in the case database | Mistakes propagation |
-
Neural networks Inspired from the biological nervous system, the artificial neural network is used to accomplish the learning process, identify new patterns, perform classifications, and improve the decision-making process. ANNs are based on empirical risk minimization. They provide an adaptation ability to minor changes of the environment and confidence information about the decision made. ANNs have the ability to describe a multitude of functions and are conceptually scalable. They are not sequential or deterministic; however, they process in parallel which permits solutions to problems where multiple constraints must be satisfied simultaneously. In addition, the neural network will have the ability to work on forward propagation mode as an analytical tool on other data once the network is trained to a satisfactory level. The output of a forward propagation run is the predicted model for the data which can then be used for further analysis and interpretation. Neural networks can be constructed by only a few samples, thus reducing the complexity of the solution. ANNs can be combined with CBR and GA in the training phase.ANNs are categorized under supervised learning which requires prior knowledge of the environment and training data. Neural networks may face slow training depending on the network size. In addition, it is possible to overtrain a neural network, which means that the network has been trained exactly to respond to only one type of input. The network will be then memorizing all input situations instead of learning. ANN will then face overfitting when the developed model is not general. ANNs may also face difficulty with infinite recursion and structured representations. Neural networks provide multiple solutions associated with local minima and for this reason may not be robust over different samples.Since artificial neural networks require training data labels, the outcome accuracy and algorithm performance are highly related to the data used for training the model. Therefore, the main challenge is to collect the training data and make sure to use clean and task-relevant data in the training phase. The second challenge is to deal with the data and the choice of initial parameters and attributes that can be nonlinear, complex, and numerous which may lead to slow training performance.
-
Support vector machines are supervised learning models used for classification, regression analysis, and outlier detection. They provide high performance in many real-world problems due to their generalization ability and robustness against noise and outliers. The basic idea of SVMs is detecting the soft boundary of a given set of samples so as to classify new points as belonging to that set or not. A good separation is achieved by the hyperplane that has the largest distance to the nearest training data points of any class, since in general the larger the margin the lower the generalization error of the classifier. SVMs map the input features into a high-dimensional feature space in which they become separable. This mapping from the input vector space to the feature space is a nonlinear mapping achieved by using kernel functions. SVM can accommodate high dimensional spaces and can still be effective in cases where the number of dimensions is greater than the number of samples. SVMs produce very efficient classifiers with high prediction accuracy and less overfitting even if training examples contain errors. SVMs deliver a unique solution, since the optimality problem is convex, which is considered an advantage compared to Neural Networks, which provide multiple solutions associated with local minima. SVMs are versatile, different Kernel functions can be specified for the decision function. Common kernels are provided, but it is also possible to specify custom kernels.SVM may provide high performance in small problems, however, its complexity increases in large problems. If the number of features is much greater than the number of samples, the method is likely to give poor performance. Support vector machines are powerful tools, but their computation and storage requirements increase rapidly with the number of training vectors. The core of an SVM is a quadratic programming problem, separating support vectors from the rest of the training data. They do not directly provide probability estimates, these are calculated using an expensive five-fold cross-validation. Therefore, SVMs are considered computationally expensive, they run slow with long training time.By introducing the kernel, SVMs gain flexibility in the choice of the functional margins form, however, the kernel function can be any of the following: linear, polynomial, sigmoid, Gaussian, radial basis function or custom kernels as a python function or by pre-computing the Gram matrix. Choosing the appropriate and convenient Kernel function may be challenging since it would affect the algorithm performance, efficiency and accuracy. In addition, the size of the kernel cache has a strong impact on run times for larger problems. Support vector machine algorithms require labeled training data and previous knowledge of the system which may be also challenging due to the diversity of the features affecting the decisions. SVMs are not scale invariant, so it is highly recommended to scale the data for efficient system performance.
-
Genetics algorithms are a randomized heuristic search strategy where the population is composed of candidate solutions obtained and emerged via mutation and crossover. GA solve multi-objective optimization problem and dynamically configure the CR in response to the changing wireless environment. They aim to optimize nonlinear systems with a large number of variables by providing multiple solutions having fitness scores measuring how close a candidate is to being a solution. Genetic algorithms search in parallel from a population. Therefore, it has the ability to avoid being trapped in a local optimal solution like traditional methods, which search from a single point. GAs are then faster and consume lower memory than searching a very large space. In addition, GA are simple and easy since the values of the fitness objective function are used for optimization purposes.Optimization based on fitness function cannot assure constant optimization response times which limits the genetic algorithm’s use in real-time applications. In addition, GA may not converge to a global optimum specially when the populations have a lot of subjects and performance metrics. The algorithm may get stuck on local maxima and may not provide optimal and complete solutions. The algorithm performance highly depends on the fitness function which may generate bad chromosome blocks in spite of the fact that only good chromosome blocks cross over. GAs are considered slow since evaluation of the fitness is computationally intensive. The main challenge is that GAs require prior knowledge-based learning and derived fitness functions; thus, new rules are formulated on the basis of training examples and/or patterns observed in previous iterations of the search. Setting up the problem and selecting the parameters to express the fitness function are critical to bias the next generation towards better genes. Determining chromosomal representation of parameters, domain, and range are also challenging since they are dependent on the studied problem.
-
Game theory reduces the complexity of adaptation algorithms in large cognitive networks. It provides solutions for decentralized multi-agent systems, similar to the players in a game, under partial observability assumptions. Games can be either cooperative or competitive. In cooperative games, all players are concerned about all the overall benefits, and they are not very worried about their own personal benefit. However, in competitive games, every user is mainly concerned about his personal payoff, and therefore, all its decisions are made competitively. Game theory primary applications are in linear programming, statistical decision-making, and operations research.One of the most basic limitations of game theory is that each player must know the cost functions of the other players. Game theory can use Bayesian analysis to reason about the cost based on observations of actions chosen by the player. In addition, in the case of nonzero-sum games, multiple Nash equilibria may exist. Game theory is also limited by the number of players in the game, since the analysis of the gaming strategies turns out to be increasingly complicated.Therefore, game theory requires prior knowledge of the system as well as labeled training data. The players require knowledge of different parameters such as SINR, power, and price from base stations, which is impractical in many situations.
-
Reinforcement learning can be used by agents to learn autonomously using the feedback of the actions. RL is used with multi-agent systems to solve classification and decision-making tasks. Reinforcement learning uses rewards to learn a successful agent function. The system can achieve self-adaptation progressively in real time. RL agents learn to interact with an environment and have the goal to optimize the cumulative reward.The main limitation of RL is that the actions of the agents and reward function should be defined based on the system and task requirements. A feedback is needed to make decisions.Using RL, the system engine needs to perform a learning phase to acquire the optimal and suboptimal policy. In dynamic environments, it would be challenging, sometimes even impossible, to provide the agents with the correct actions associated with the current situation.
-
Fuzzy logic is flexible, provides intuitive knowledge-base design with easy computation, and simple implementation and interpretation. Fuzzy logic is used as an application to systems that are difficult to model with unclear quality boundaries. Instead of using complicated mathematical formulations, fuzzy logic uses human-understandable fuzzy sets and inference rules to obtain the solution that satisfies the desired system objectives. Therefore, the main advantage of fuzzy logic is its low complexity and its suitability for real-time applications such as cognitive radio applications. Using fuzzy logic, solutions can be obtained given imprecise, noisy, and incomplete input information.There may be some disadvantages to fuzzy logic; for example, stability, accuracy, and optimality of the system are not guaranteed. In addition, increasing the dimensionality may result in inefficient and memory intensive settings for most functions. Fuzzy logic highly depends on the fuzzy inference rules which may be tuned manually, and settings can be made by trial and errors.The derivation of rules is challenging. These rules need to be relevant and rely on relevant features. Moreover, they should be updated when needed. Therefore, the accuracy of the fuzzy logic system depends on the completeness and accuracy of the rules. The system may return inappropriate decisions if the rules are not convenient and accurate.
-
Entropy is a measure of the uncertainty in a random variable which quantifies the expected value of the information contained in a message. The information entropy can therefore be seen as a numerical measure which describes how uninformative a particular probability distribution is. Independently of any Bayesian considerations, the problem considered as a constrained optimization problem has the entropy function as the objective function. Entropy can be combined with reinforcement learning and the Markov model.Although entropy is often used as a characterization of the information content of a data source, this information content is not absolute: it depends crucially on the probabilistic model. In addition, it requires prior knowledge of the system (mainly prior knowledge of the model-describing parameters to be learned).
-
Decision tree learning uses a decision tree to create a model that predicts the value of a target variable based on several input variables. The decision tree is simple to understand and to interpret by visualization. The decision can be directly made based on the tree branches. The cost of using the tree for prediction and decision-making is logarithmic in the number of data points used to train the tree. Decision tree learning can handle different types of data such as numerical and categorical data and multi-output problems. Each path of the decision tree contains the elements of the paths and the assigned risk factor which is the estimated likelihood of occurrence of the terminal event in the path. Decision trees depict future decision points and possible chance events. They add to the confidence and accuracy of the decisions.A decision tree needs labeled training data and does not support missing values. The decision tree may face overfitting which will make the system model loose its learning aspect. The model is highly related to the input training data which may make the system unstable because small variations in the data might result in a completely different tree being generated. In addition, the problem of learning an optimal decision tree may be nondeterministic-polynomial (NP) time NP-complete which may require heuristic algorithms to obtain solutions. Such algorithms may not guarantee global optimal solutions for decision trees.The main challenge is that the decision tree requires labeled training data and prior knowledge of the system. The selection of training data is critical. On one hand, decision trees may suffer overfitting when all cases are taken into consideration for building the tree, on the other hand, decision trees may be biased if some classes dominate in the training data set. Therefore, a balanced labeled data set is required prior to fitting with the decision tree without any missing values.
-
Artificial bee colony algorithm finds the optimal position of a food source representing a possible solution to the optimization problem. ABC searches in parallel over several constructive computational threads based on local problem data and a dynamic memory structure containing information on the quality of the previously obtained results. The ABC algorithm provides uni-modal and multi-modal numerical optimization.However, the artificial bee colony is considered as a NP-hard problem with high dimensionality. ABC is very effective in solving small- to medium-sized generalized assignment problems. Its efficiency depends on the size of the solution space, number of variables and constraints used in the problem modeling, and the structure of the solution space such as convexity. In addition, ABC needs prior knowledge of the system.
-
Bayesian approach is based on probabilistic learning. It provides exact inferences which do not rely on large sample approximations with simple interpretations. Bayesian inference estimates a full probability model and allows prior knowledge and results to be used in the current model. Bayesian inference has a statistical decision to facilitate decision-making, it includes uncertainty in the probability model, yielding more realistic predictions. The Bayesian approach does not face overfitting since it uses observed data only.The Bayesian approach is only useful when prior knowledge is reliable and distribution for all parameters is known. It may also face computational complexity since it involves high-dimensional integrals.
-
Markov model chain generates sequences of observation symbols by making transitions from state to state in time or space with fixed probabilities. The Markov chain contains a finite number of states and corresponding transition probabilities. The current state of the system depends on previous events and their successive structure to determine the process. For instance, a first-order Markov chain takes account of only a single step in the process, but it can be extended so that the probabilities associated with each transition depend on multiple events earlier than the immediately preceding one. Markov models are relatively easy to derive and infer from successive data. The Markov model is scalable and can model complex statistical models. It can be used for classification and prediction based on experiences. It does not require deep insight into the mechanisms of dynamic change. The basic transition matrix summarizes the essential parameters of dynamic change.However, the Markov model is computationally complex. It is expensive, both in terms of memory and time. The Markov model requires good training sequence. In some cases, the data available will be insufficient to estimate reliable transition probabilities, especially for rare transitions. In addition, the Markov model is a successional model where the performance efficiency depends on predictions of system behavior over time.The construction of Markov models requires accurate data information to determine the transfer probabilities and state conditions.
-
Case-based reasoning is capable of solving problems based on predefined cases. CBR is used to determine an acceptable solution based on the existing case found in the case database. CBR allows learning without knowledge on how rules and cases are created. The system will learn from previous and new situations and update its database accordingly which makes development and maintenance easier. CBR can be used when a domain model is difficult to generate, complex and has multiple-variable situations.CBR faces many limitations. It can only be used when records of initial and previously successful solutions exist. It can take a large storage space for all the cases and may face a large processing time to find similar cases in the database. CBR relies on previous cases which might include irrelevant patterns. Therefore, CBR needs appropriate training before deployment since incorrectly solved cases may lead to mistake propagation. In addition, populating and searching a large database can be time consuming and difficult which highlight the need of fast and efficient case-selection algorithms. Removing irrelevant patterns may reduce the case retrieval time and lead to higher efficiency.
-
Multi-agent systems are suitable for multi-player decision problems. They provide negotiation, learning, and cooperative-based approaches between agents which can be in our case cognitive radio users. MAS are quick, reliable, and flexible. They can accommodate multiple user networks; however, large networks may cause high cost and complexity. MAS can be combined with RL and game theory that also provide solutions using interactions between users for enhancing the performance of the system.