Operating Enterprise AI as a Service

This paper discusses the challenges in providing AI functionality “as a Service” (AIaaS) in enterprise contexts, and proposes solutions to some of these challenges. The solutions are based on our experience in designing, deploying, and testing AI services with a number of customers of ServiceNow, an Application Platform as a Service that enables digital workflows and simplifies the complexity of work in a single cloud platform. Some of the underlying ideas were developed when many of the authors were part of DxContinuum inc, a machine learning (ML) startup that ServiceNow bought in 2017 with the express purpose of embedding ML in the ServiceNow platform. The widespread adoption of ServiceNow by the majority of large corporations has given us the opportunity to interact with customers in different markets and to appreciate the needs, fears and barriers towards adopting AIaaS and to design solutions that respond to such barriers. In this paper we share the lessons we learned from these interactions and present the resulting framework and architecture we adopted, which aims at addressing fundamental concerns that are sometimes conflicting with each other, from automation to security, performance, effectiveness, ease of adoption, and efficient use of resources. Finally, we discuss the research challenges that lie ahead in this space.