×

Alibi Explain: algorithms for explaining machine learning models. (English) Zbl 07415124

Summary: We introduce Alibi Explain, an open-source Python library for explaining predictions of machine learning models (https://github.com/SeldonIO/alibi). The library features state-of-the-art explainability algorithms for classification and regression models. The algorithms cover both the model-agnostic (black-box) and model-specific (white-box) setting, cater for multiple data types (tabular, text, images) and explanation scope (local and global explanations). The library exposes a unified API enabling users to work with explanations in a consistent way. Alibi adheres to best development practices featuring extensive testing of code correctness and algorithm convergence in a continuous integration environment. The library comes with extensive documentation of both usage and theoretical background of methods, and a suite of worked end-to-end use cases. Alibi aims to be a production-ready toolkit with integrations into machine learning deployment platforms such as Seldon Core and KFServing, and distributed explanation capabilities using Ray.

MSC:

68T05 Learning and adaptive systems in artificial intelligence

References:

[1] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng. Tensorflow: A system for large-scale machine learning. In12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 265-283, Savannah, GA, Nov. 2016. USENIX Association.ISBN 978-1-931971-33-1.URLhttps: //www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi.
[2] M. Alber, S. Lapuschkin, P. Seegerer, M. H¨agele, K. T. Sch¨utt, G. Montavon, W. Samek, K.-R. M¨uller, S. D¨ahne, and P.-J. Kindermans. innvestigate neural networks!Journal of Machine Learning Research, 20(93):1-8, 2019. URLhttp://jmlr.org/papers/v20/18540.html.
[3] D. W. Apley and J. Zhu. Visualizing the effects of predictor variables in black box supervised learning models.arXiv preprint arXiv:1612.08468, 2016.
[4] V. Arya, R. K. E. Bellamy, P.-Y. Chen, A. Dhurandhar, M. Hind, S. C. Hoffman, S. Houde, Q. V. Liao, R. Luss, A. Mojsilovic, S. Mourad, P. Pedemonte, R. Raghavendra, J. T. Richards, P. Sattigeri, K. Shanmugam, M. Singh, K. R. Varshney, D. Wei, and Y. Zhang. Ai explainability 360: An extensible toolkit for understanding data and machine learning models.Journal of Machine Learning Research, 21(130):1-6, 2020. URLhttp://jmlr. org/papers/v21/19-1035.html.
[5] U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y. Jia, J. Ghosh, R. Puri, J. M. F. Moura, and P. Eckersley. Explainable machine learning in deployment. InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* ’20, page 648-657, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450369367. doi: 10.1145/3351095.3375624. URLhttps://doi.org/10.1145/3351095.3375624.
[6] A.Coca.Distributedblack-boxmodelexplanationwithRay,2020.URL https://ray2020.sched.com/event/aWFq/distributed-black-box-modelexplanation-with-ray-alexandru-coca-seldon.
[7] C. Cox, G. Sunner, A. Saucedo, R. Dawson, A. Gonzalez, and R. Skolasinski. Seldon Core: A framework to deploy, manage and scale your production machine learning to thousands of models., 2018. URLhttps://github.com/SeldonIO/seldon-core.
[8] A. Dhurandhar, P.-Y. Chen, R. Luss, C.-C. Tu, P. Ting, K. Shanmugam, and P. Das.Explanations based on the missing:Towards contrastive explanationswithpertinentnegatives.InS.Bengio,H.Wallach,H.Larochelle, K.Grauman,N.Cesa-Bianchi,andR.Garnett,editors,Advances in Neural Information Processing Systems 31, pages 592-603. Curran Associates, Inc.,
[9] J. Heo, S. Joo, and T. Moon. Fooling neural network interpretations via adversarial model manipulation. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alch´e-Buc, E. Fox, and R. Garnett, editors,Advances in Neural Information Processing Systems, volume 32, pages 2925-2936. Curran Associates, Inc., 2019. URLhttps://proceedings.neurips. cc/paper/2019/file/7fea637fd6d02b8f0adf6f7dc36aed93-Paper.pdf.
[10] ICO. Project explain interim report, 2019. URLhttps://ico.org.uk/about-the-ico/ research-and-reports/project-explain-interim-report/.
[11] KFServing. KFServing: Serverless inferencing on kubernetes, 2019. URLhttps://github. com/kubeflow/kfserving.
[12] J. Klaise, A. Van Looveren, C. Cox, G. Vacanti, and A. Coca. Monitoring and explainability of models in production.arXiv preprint arXiv:2007.06299, 2020.
[13] N. Kokhlikyan, V. Miglani, M. Martin, E. Wang, B. Alsallakh, J. Reynolds, A. Melnikov, N. Kliushkina, C. Araya, S. Yan, and O. Reblitz-Richardson. Captum: A unified and generic model interpretability library for pytorch, 2020.
[14] S. M. Lundberg and S.-I. Lee.A unified approach to interpreting model predictions.InAdvances in Neural Information Processing Systems 30, pages 4765- 4774. 2017.URLhttps://papers.nips.cc/paper/7062-a-unified-approach-tointerpreting-model-predictions.
[15] S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, and S.-I. Lee. From local explanations to global understanding with explainable ai for trees.Nature Machine Intelligence, 2(1):56-67, Jan 2020. ISSN 25225839. doi: 10.1038/s42256-019-0138-9. URLhttps://doi.org/10.1038/s42256-0190138-9.
[16] C. Molnar.Interpretable Machine Learning.2019.https://christophm.github.io/ interpretable-ml-book/.
[17] P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, et al. Ray: A distributed framework for emerging AI applications. In13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 561-577, 2018.
[18] H. Nori, S. Jenkins, P. Koch, and R. Caruana. Interpretml: A unified framework for machine learning interpretability.arXiv preprint arXiv:1909.09223, 2019.
[19] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala.Pytorch: An imperative style, high-performance deep learning library.In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alch´e-Buc, E. Fox, and R. Garnett, editors,Advances
[20] M. T. Ribeiro, S. Singh, and C. Guestrin. Anchors: High-precision model-agnostic explanations. InAAAI Conference on Artificial Intelligence, 2018. URLhttps://www.aaai. org/ocs/index.php/AAAI/AAAI18/paper/view/16982.
[21] M. Sundararajan, A. Taly, and Q. Yan.Axiomatic attribution for deep networks.In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 3319-3328. JMLR.org, 2017.
[22] A. Van Looveren and J. Klaise. Interpretable counterfactual explanations guided by prototypes.arXiv preprint arXiv:1907.02584, 2019. URLhttps://arxiv.org/abs/1907. 02584.
[23] S. Wachter, B. Mittelstadt, and C. Russell. Counterfactual explanations without opening the black box: Automated decisions and the GDPR.Harvard journal of law & technology, 31:841-887, 04 2018. URLhttps://arxiv.org/abs/1711.00399.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.