Consumer Monitoring of Infrastructure Performance in a Public Cloud

Many web information systems and applications are now run as cloud-hosted systems. The organization that owns the information system or application is thus a consumer of cloud services, and often relies on the cloud provider to monitor the virtual infrastructure and alert them of any disruption of the offered services. For example, Amazon Web Services’ cloud disruptions are announced by the cloud provider on a dedicated RSS feed so that the consumers can watch and act quickly. In this paper, we report on a long-running experiment for the monitoring and continuous benchmarking of a number of cloud resources on Amazon Cloud from a consumer’s perspective, aiming to check whether the service disruptions announced by the cloud provider are consistent with what we observe. We evaluate the performance of cloud resources over several months. We find that the performance of the cloud can vary significantly over time which leads to unpredictable application performance. Our analysis shows also that continuous benchmarking data can help detect failures before any announcement is made by the provider, as well as significant degradation of performance that is not always connected with Amazon service disruption announcements.