The Role of Vertical Elastic Namenode in Handling Big Data in Small Files

Hadoop is a distributed system used exclusively for BigData analysis and processing that is Hadoop distinguished itself through its high performance and a solid availability. Hadoop cluster is available for use at any time and this is one of Hadoop’s solid attributes that make it popularly known in data analysis and sciences. However, there are a several factors impacting Hadoop cluster, causing it to be inaccessible. One of these factors is BigData in small files whereby Hadoop’s availability shortage accrued when a massive amount of small files dataset is pushed toward a Hadoop cluster. This will harm the cluster’s performance, making it unavailable for access and use. This negative factor affects the Namenode itself as the Namenode is a single point of failure. Hence, once it crashes, the whole cluster will be out of service and need to jump again manually. This paper will introduce the elastic Namenode in lieu of the current traditional one. The elastic Namenode has an ability to adapt to the frequent negative factors that are affecting the whole cluster, causing it to become unavailable. The elastic Namenode will adopt the vertical elasticity manner, this type of elasticity will add more memory resources to the Namenode based on a direction from a script that traces the Namenode memory itself. The result will be a cloud elastic Namenode that can be expanded and shrunk upon request, which allows Hadoop cluster to treat BigData in small files without any negative factor or issue.