Big Data in Finance

In the financial sector, the importance of big data in decision-making continues to increase. This growth is facilitated by ongoing improvements and refinements in artificial intelligence (AI), where model performance in prediction tasks has continued to improve with recent innovations such as deep learning. The ability for financial models to predict more accurately, in conjunction with big data, has led to many innovations in areas such as automated lending, portfolio construction and management (robo-advising), risk management, fraud detection, quantitative and high-frequency trading, as well as customer support. However, while AI-based analyses of big data have led to many positive developments in finance, critical questions remain, including how data is collected (consumer privacy and ethical concerns), stored (environmental impacts), secured, as well as analyzed and used. This edited book critically analyzes new developments at the intersection of big data and finance and provides different perspectives on their impact on the financial sector and the way it operates. This introductory chapter summarizes the use of big data in finance and provides an overview of the eleven contributions featured in the collection.

Financial data plays an integral role in the decision-making process in finance. Economic actors increasingly rely on non-financial data, broadly classified as alternative data. Using data sources that are not widely available can provide great value; however, collecting, validating, and maintaining datasets is known to be very costly. The situation is changing with recent advances in big data technologies that allow for new data sources to be collected, transferred, and processed efficiently at a lower cost. Furthermore, a large universe of alternative datasets is now available for purchase from an ever-increasing pool of vendors. Currently, alternative data stems from a multitude of sources. A large segment is occupied by data generated from internet activity, such as text data, images, and web traffic logs from a variety of sources such as news sites, chat boards, e-commerce websites, job search websites, and social media platforms. Other common types of alternative data include retail transaction data, credit card data, data accessed through the Freedom of Information Act, and satellite imagery. Unlike traditional financial data, alternative data points are not easily quantifiable and require a great deal of context for analysis. This chapter presents an overview of the contemporary alternative data landscape, detailing the unique risks, challenges, and opportunities associated with its use. We show that as the cost of acquiring and processing data continues to decline, we can expect the adoption of alternative data to keep accelerating into the future.

This chapter proposes an algorithmic trading (AT) strategy based on a newly developed investment indicator called the “Balanced Investment Indicator” (BII), which has been shown to be able to balance risk and profitability accurately. This indicator is crucial for developing an AT strategy that allows algorithmic traders to use big data to analyze portfolios and seek the BII algorithm's highest value. The chapter reviews and analyzes current AT strategies and compares them with the proposed strategy of the chapter. The results of this comparison show that the indicator performs strongly, as its investment recommendations coincide in some cases with relevant institutions, such as the Bank of America. For investors, this chapter provides decision-making tools for selecting different portfolios that balance profitability with default risk.

This chapter investigates the impact of high-frequency trading (HFT) on market efficiency in the Moroccan Stock Exchange. The endorsement of market efficiency suggests that investors cannot beat the market, while a rejection of market efficiency implies that investors can use HFT to realize higher returns than those of the market. We obtain data at the precision level of milliseconds from the Casablanca Stock Exchange and cover the period from August 1, 2016 (the beginning of high-frequency trading in the Moroccan stock market) to July 31, 2021. The chapter finds from conducting statistical analyses using this data that the market efficiency hypothesis is rejected at very high frequencies: one millisecond, one second, and 30 second. At lower time frequencies such as 1, 2, 5, 10, and 15 minutes, the market efficiency hypothesis is not rejected, which lends evidence that the market is efficient at these frequencies. Based on these results obtained, the main conclusion from this chapter is that investors with a faster market connection (at the millisecond and second level) and an efficient algorithm can use privileged information to realize returns higher than those of the Moroccan Stock Exchange market. Further studies should be completed in other markets to determine if these results are consistent across exchanges.

Trustable models exploit the diversity of model forms developed using symbolic regression via genetic programming to define ensemble models. These models have been shown empirically to have a strong predictive performance and the ability to extrapolate into regions of unknown parameter space or detect changes in the underlying system. This chapter demonstrates how the same technique for quantifying uncertainty is helpful in risk management workflows for alternative investing, especially when applying behavioral science principles. The use cases cover assets such as publicly traded private equities, specifically when the optimization objectives include financial and environmental, social, and governance (ESG) criteria, and ESG ETFs. This chapter provides an overview of these asset classes and a critical review of the issues with how current ESG ratings are formulated by rating agencies. Additionally, explicit uncertainty ranges are obtained, using an ensemble modeling approach, at a sufficiently high accuracy level to trust the uncertainty measurement. Future research is necessary to refine the approach presented as more data becomes available.

The credit assessment process has traditionally been based on a relatively stable set of financial indicators such as overall indebtedness, cash flow stability, and borrower history. While this approach ensures that borrowers with established credit records have ongoing access to funds, it can lead to the exclusion of those with thin credit files or atypical histories. The disruption in financial services is starting to change this scenario by drawing on alternative sources of data and innovative computational techniques to develop new methods of consumer credit assessments. This chapter explores the motivations behind these changes, discusses the significant evolution in data and analytics that enable the development of new credit quality indicators, and highlights the privacy and ethical challenges of using new and emerging credit assessments in the age of big data.

At the frontier of personal finance and FinTech, robo-advisors aim to provide customized portfolio strategies without human intervention. These new investment technologies typically propose passive strategies that match investor objectives and risk profiles at a low cost. This chapter explores how digital advisors lack precision in capturing clients’ attitudes towards risk and exposure. In this context, leveraging big data and artificial intelligence techniques can improve the principal strength of robo-advisors, i.e., their ability to provide automated, personalized investment solutions. Text data from dialogue systems, such as chatbots, can be employed to improve the client profile, while recommendation systems can use big data from financial social networks to recommend targeted investment strategies. Analysis of big data through machine learning methods can also improve the performance of the optimization algorithms employed by digital advisors. This chapter explores the vast potential for exploiting big data and artificial intelligence in automated asset management.

Is Bitcoin the payment system of the future? This chapter argues that Bitcoin is neither currency nor gold; instead, it is a tradeable asset and an alternative form of investment. Moreover, Bitcoin also exhibits some investment features similar to collectibles. This chapter shows that the true value of Bitcoin lies not in its speculative nature but in its embedded technology, blockchain, which has the potential to revolutionize traditional finance. Blockchain technology can provide solutions to big data challenges and provide an off-ramp during times of political uncertainty. Lastly, this chapter discusses how Bitcoin’s long-term survivability and viability as an asset will largely depend on its diversification, institutional adoption, tax treatment, and regulations.

This chapter examines the influence of culture on the use of digital assets. Based on relevant data from the World Bank database, we examine the relationship between financial institutional development and the use of digital assets. Using four ways to measure the use of digital assets: digital payments, mobile payments, using the internet to pay bills, and online shopping, we find that financial institution development is positively related to each measure of the use of digital assets. We then investigate how national culture may play a role in digital asset use. Using Hofstede’s definitions of cultural dimensions and the World Value Survey results, we identify that the traits of “individualism” and “trust” are positively related to the increased use of digital assets. Overall, these results suggest that the use of digital assets in a country relates to the national development of financial institutions as well as national cultural traits.

This chapter presents a case study of establishing a credit union by a Toronto Muslim community based on the principles of Islamic Finance. One of the biggest obstacles to establishing a new credit union in Ontario, Canada, is receiving regulatory approval from the provincial regulator, the Financial Services and Regulator Authority (FSRA). A core component of the application process is the collection of in-depth financial and market data from several thousand prospective members. This chapter examines the power of big data tools employed by the proposed Islamic Credit Union for Community (ICUC) to collect the massive amount of data required to receive regulatory approval. Such tools include state-of-the-art modeling techniques such as recurrent neural networks (RNNs), deep reinforcement learning, and attention mechanisms using transformers for time-series modeling. These tools are extremely useful in building dynamic and stochastic banking models along with other predictive analytics. This chapter illustrates both the methodology and practical steps for determining the feasibility of a new financial institution in a heavily regulated financial sector of a G8 country. More specifically, it shows how big data tools apply to serve the needs of a financial institution in a specialty market.

Due to its massive energy consumption and large carbon footprint, Bitcoin's energy hunger has triggered a heated debate in academic literature. Unfortunately, it is difficult to measure Bitcoin's actual electricity consumption, and literature on the topic produces inconsistent estimates that lead to different assessments of the network's carbon footprint. The objective of this chapter is to provide a reasonable economic approximation and a meaningful forecast of the carbon footprint of cryptoassets. This chapter examines the relationship between the trading volumes of cryptocurrencies and Bitcoin's energy consumption using a vector autoregression (VAR) framework. Using causality tests, we find evidence of one-way directional causality from Bitcoin's trading volume to the network's electricity consumption. By making reasonable assumptions about network energy sources, we find that bitcoin mining was responsible for approximately 43 million metric tons of carbon in 2020 (the equivalent to 0.14% of the global total yearly carbon emissions in that year). Using an impulse-response analysis, we also find that the impact of one standard deviation shock in Bitcoin's trading volume on the network's energy consumption is persistent and amounts to 8.8% on average per month (or 63% annually) over a period of twelve months. This growth rate implies that the total electricity consumption of Bitcoin is expected to generate 492 million metric tons of carbon emissions by the end of 2026, translating to approximately 1.6% of current total carbon emissions worldwide. These alarming statistics demand attention and highlight the negative environmental impact of digital currencies.

The onset of COVID-19, coupled with substantially heightened financial pressures on households, has led to a renewed interest in the topic of financial literacy. While it is difficult to arrive at a single and precise definition of financial literacy, it has been popularly used to refer to a combination of financial knowledge, attitude, abilities, and behaviors. This chapter reviews the current literature on financial literacy, focusing specifically on how the concept of financial literacy has evolved over the decades. The advent of large-scale and fine-grained data on the efficacy and correlates of financial literacy training has now made it possible to develop data-informed and technology-enabled frameworks for financial literacy enhancement. We highlight some key learnings from implementing national-level financial education programs in Singapore. Early insights reveal the growing importance of financial confidence as a driver for behavioral outcomes among learners, as well as population-level heterogeneities involved in financial knowledge, confidence, and behavior. We draw on these insights to recommend to policymakers and educators how to design and derive value from similar financial literacy enhancement programs worldwide.