Intelligent Techniques for Data Science

In this digital era, data is proliferating at an unprecedented rate. Data sources such as historical customer information, customer’s online clickstreams, channel data, credit card usage, customer relationship management (CRM) data, and huge amounts of social media data are available. In today’s world, the basic challenge is in managing the complexity in data sources, types and the velocity with which it is growing. Obviously, data-intensive computing is coming into the world that aims to provide the tools we need to handle the large-scale data problems. The recent big data revolution is not in the volume explosion of data, but in the capability of actually doing something with the data; making more sense out of it. In order to build a capability that can achieve beneficial data targets, enterprises need to understand the data lifecycle and challenges at different stages.

Set is defined as a collection of entities that share common characteristics. From the formal definition of the set, it can be easily determined whether an entity can be member of the set or not. Classically, when an entity satisfies the definition of the set completely, then the entity is a member of the set. Such membership is certain in nature and it is very clear that an entity either belongs to the set or not. There is no intermediate situation. Thus, the classical sets handle bi-state situations and sets membership results in either ‘true’ or ‘false’ status only. These types of sets are also known as crisp sets. In other words, a crisp set always has a pre-defined boundary associated with it. A member must fall within the boundary to become a valid member of the set. An example of such classical set is the number of students in a class, ‘Student’. Students who have enrolled themselves for the class by paying fees and following rules are the valid members of the class ‘Student’. The class ‘Student’ is crisp, finite and non-negative. Here are some types of crisp sets.

Intelligence is a key resource in acting effectively on problems and audiences. Whatever the business is, if it is done with added intelligence and insight, it can provide high rewards and increases in quality of product, services and decisions. Intelligence can be defined as an ability to acquire knowledge as well as having wisdom to apply knowledge and skills in the right way. It is also defined as an ability to respond quickly, flexibly and by identifying similarities in dissimilar solutions and dissimilarity in similar situations. Some mundane actions such as balancing, language understanding and perception are considered as highly intelligent activities; these actions are difficult for machines. Some complex actions by animals, on other hand, are considered as non-intelligent activities. An interesting experiment has been carried out on the wasp (an insect that is neither bee nor ant, but similar to these two), which behaves in very complicated way while searching and preserving food. The experiment is described on a website presenting reference articles to Alan Turing (http://​www.​alanturing.​net/​). According this source, a female wasp collects food, puts it near its burrow, and goes inside the burrow to check for intruders. If everything is safe, the wasp comes out and puts the food into the burrow. During the experiment, the food is moved a few inches from its original place. Instead of finding the food just a few inches away, the wasp goes in search of new food, again puts the food near the burrow and repeats the procedure. This behaviour is complex, however, non-intelligent. Besides the aforementioned mundane tasks, there are expert problem solving and scientific tasks such as theorem proving, fault finding and game playing which also come under the intelligent category.

Evolutionary algorithms are inspired from the Nature’s ability to evolve. Evolutionary algorithms are a component of evolutionary computing in the field of Artificial Intelligence. They are inspired from the biological evolution of random population by employing various modifying operations on the basic pattern of the candidates of the population. In nature, evolution through such modification happens in such a way that the next population will consist of members that are comparatively more fit to survive in the given situation. In a case where the modification results in poorer candidates, they cannot survive, and hence they will be automatically deselected from the population. Only the fit candidates will survive in the population. That is why such an approach is called the survival of the fittest approach. A broad outline of the approach is given as follows.

This chapter considers some of the effective metaheuristics and classification techniques that have been applicable in intelligent data analytics. Firstly, metaheuristics approaches such as adaptive memory procedures and swarm intelligence are discussed, and then classification approaches such as case-based reasoning and rough sets are presented.

In this chapter, we outline some advanced tools and technologies, including the Apache Hadoop ecosystem, real-time data streams, scaling up machine learning algorithms, and fundamental issues such as data privacy and security. We cover much of the basic theory underlying the field of big data analytics in this chapter, but of course, we have only scratched the surface. Keep in mind that to apply the concepts contained in this overview of big data analytics, a much deeper understanding of the topics discussed herein is necessary.

This chapter offers a concise overview of the fundamental functionality of the R language.