Intelligent and independent processes for overcoming big graphs

Poor locality as a natural property of graph data structures causes enormous amount of network traffic in large-scale distributed graph processing systems. Moreover, data transmission through the network is one of the most expensive operations in a distributed system. Therefore, reduction of network usage is highly required by a new graph computational model. In this paper, increasing the degree of machine independency has been considered a key factor of network traffic reduction. The proposed system benefits from a three-layered computational model to perfectly leverage the power of local information as much as possible. Moreover, this model simultaneously takes the advantages of both message-based and shared-state communication paradigms. Vertices can read and update values of others in the lowest layer directly, while they must send messages in other layers. By the use of memorization techniques, the proposed model introduces a new kind of intelligence that has encouraging effects on removing useless communications. Distinctive results of our experiments confirm significant improvements of the proposed model in relation to the previous systems like Pregel, GPS, and Blogel, as well as ExPregel. The results also show that the overhead of making processes independent along with intelligent is negligible in comparison with the cost of additional network communications.