Reliability prediction is an important task in software reliability engineering, which has been widely studied in the last few decades. However, existing studies have mostly focused on reliability analysis of traditional software, but little on modelling and predicting user-perceived reliability of emerging software services, which remains an open research problem. Unlike traditional software components, software services, such as Web services and Web APIs, generally provide black-box functionalities to users over the Internet. Traditional reliability models become inapplicable due to a lack of internal information for reliability analysis. Furthermore, from a user's perspective, reliability not only depends on a service itself, but also is highly influenced by the context of service invocations (e.g., service load and network conditions). In this work, we propose CARP, a context-aware reliability prediction approach for black-box services. CARP models user-perceived reliability as a function jointly determined by the (user, service, context) tuple of a service invocation, and further constructs the model by a novel formulation of context-specific matrix factorization. To guarantee computational efficiency, CARP comprises an offline step to train the context-aware reliability model from historical invocation data, and another online step to support on-demand reliability predictions for ongoing service invocations. Through context-aware reliability modelling, CARP is able to alleviate the data sparsity problem that heavily limits prediction accuracy of other existing approaches. The evaluation results from a real service usage dataset collected on Amazon EC2 show that CARP can better capture the characteristics inherent in reliability of black-box services, and therefore yields a significant improvement in prediction accuracy (up to 41% in MAE and 38% in RMSE) over the state-of-the-art reliability prediction models.
Read more information from our paper:
Jieming Zhu, Pinjia He, Zibin Zheng, and Michael R. Lyu, "CARP: Context-Aware Reliability Prediction of Black-Box Web Services," in Proc. of the 24th International Conference on Web Services (ICWS), 2017.
The data used in our work is a publicly-released reliability dataset. It provides the measured reliability data that was collected from distributed agents on the Amazon EC2 cloud. If you would like to use the dataset, please contact the data owner Dr. Marin Silic from University of Zagreb.
The source code of our implementations on CARP has been publicly released. You can fork it on our GitHub repository. We have also implemented 30+ other existing QoS prediction approaches for Web service recommendation in the WS-DREAM project. The code is well structured and can be easily extended to new implementations. Please feel free to contact us if you have any comments or questions regarding the code. We also appreciate any contributions from you.