2. Emergence of Statistical Methodologies with the Rise of BIG Data

Due to the acceleration of electronic computation and the generation of the “BIG”datasets at an unprecedented pace in many fields, there have been great advancements in the development of statistical/machine learning methodologies as we enter the twenty-first century. It is very important to be able to analyze such complex and high dimensional datasets yielding valuable information that deepens understanding, improves decision making, and enhances the performance of predictive models. For instance, the current problems encountered in manufacturing industry, such as quality improvement initiatives, determination of user expectations for a new product, and manufacturing cost estimation, become more difficult to solve as the high dimensional and complex structured data have become available. In order to overcome some of the today’s challenges of a complex manufacturing system, statistical/machine learning techniques have been utilized and found that these have been remarkably helpful to handle the problems arising in this field. We have three objectives in this chapter. The first is to highlight the developments in new statistical/machine learning algorithms, including—but not limited to—deep learning, random forests, support vector machine, dimension reduction, and sparse modeling. The second is to present how these new enormously ambitious data-driven algorithms play important role in analyzing datasets, generated continuously every second from various scientific disciplines such as engineering, biology, neuroscience, chemistry. As the complexity and size of generated data increased for the past two decades, the invention of data-driven algorithms has flourished and became more important than their inferential justifications which are an important aspect of a statistical analysis. Therefore, the third objective of the chapter is to explain how inferential analyses, the theories by which statisticians choose among competing methods, evolve in the twenty-first century as the invention of these new algorithms continues at a fast pace.