Inaccuracy in Private BitTorrent Measurements

Recently, BitTorrent communities are rapidly evolving into private torrent sites (PT). PT sites have employed several incentive rules to improve the performance and availability of the system. Many studies have been dedicated for measuring and modeling the PT systems in order to better understand the new rules and their impact on the users’ behavior in order to improve the usability of the system. These studies have been performed on different PT sites that differ in their implementation of the system and in their system’s user incentive rules. Therefore, current measurement findings cannot reflect accurate results and, more importantly, the current conclusions may be biased. In this paper, we investigate the accuracy of previous measurement studies on PT sites, while emphasizing the incentive rules employed and the interplay between these rules and corresponding objective factors. We evaluate the behavior regulation policies of the front-end website and the tracker and examine the semantics of provided data. Using this information we have designed a new crawling methodology and conducted a large-scale measurement study across four representative PT sites over a year. Interestingly, we find that most reported measurements have neither considered design goals, nor thought through the incentive policies and their interplay. This lack of awareness in turn may lead to inaccurate conclusions of system properties. For example, the Seeder to Leecher Ratio (SLR), which is reported in most of the available measurements, is routinely at least 16–45 % less than the real SLR because the sites ignored the “partial seeders” in their calculation. This study aims to offer fundamental insights into designing an accurate methodology when conducting measurement studies on PT sites.