Performance Evaluation of Passenger Information Systems

Providing expected time of arrival (ETA) information to passengers/operators with the help of the latest technologies is reported as a way to improve the effectiveness of the public transport system. There are several solutions to generate such information using automatic vehicle location data. Along with developing solutions, it is also equally important to ensure that delivered information is accurate and would be trusted by the passengers. However, most of the transport agencies which implemented such solutions fail because of the lack of proper evaluation methodology for the solution provided to them. The present study addresses this challenge and reports the development of a systematic evaluation methodology for passenger information systems. A prediction algorithm that was developed using the concepts of classification schemes and filtering techniques was considered as a sample solution. Performance evaluation of algorithms was carried out at various levels such as routes, trips, and bus stops in terms of prediction error (difference between actual arrival time and predicted arrival time to a given bus stop). Prediction accuracy was checked at all bus stops from 30 min ahead onwards. The prediction error thresholds are organized into different bins depending on the prediction horizon. Finally, prediction errors are visualized in a novel way using a parallel coordinate plot that is developed in Java interface. A unique feature of this algorithm is that one can visualize errors at multiple levels and can add multiple attributes at a time. The developed methodology can help operators to test how improvements to the prediction algorithm are impacting the predictions that are seen by passengers.