WaterBot: Arizona State University

The Arizona State University Artificial Intelligence Cloud Innovation Center, powered by Amazon Web Services (AWS), collaborated with Dr. Stephen Carradini at Arizona State University (ASU) to reconfigure and launch an interactive, generative AI chat assistant. The purpose of this chat assistant is to leverage verified and approved knowledge bases to answer the public's targeted questions about Arizona’s water management and conservation practices.

This solution overview outlines the successful build and enhancement of Arizona Water Chatbot’s serverless architecture deployment on AWS and the resulting cost reduction, improved performance, and scalability.

Water is a precious resource, and Arizona residents regularly seek accurate information about the State’s water position. Easy access to reliable information about water usage, management, and conservation is crucial for the public to  make informed water choices. The Arizona Water Chatbot (or WaterBot) project, led by Assistant Professor Stephen Carradini at ASU, aims to meet this need by leveraging large language model (LLM) capabilities and retrieval augmented generation architecture.

Dr. Carradini received grant funding from the State of Arizona as part of the Arizona Water Innovation Initiative, a $40 million investment the state of AZ made to ASU.  After Dr. Carradini and his research assistant Briana Rajan built the original solution framework for Waterbot, they realized the solution lacked scalability. The initial framework also would result in increasing costs that would reduce the uptime of the bot in the short term and limit the sustainability of the bot in the long term.

Dr. Carradini sought out the assistance of the AI CIC to provide an alternate approach to creating the back-end architecture of the WaterBot. He sought a solution that could meet the needs of the community long term with an infrastructure that can be scaled up and down based on demand. 

To enhance WaterBot's deployment and address existing challenges, the AI CIC transitioned the platform that was built on Heroku, and that leveraged OpenAI APIs, to a serverless architecture built entirely on AWS. We used essential services such as Amazon ECR for secure Docker image storage, Amazon ECS for scalable container orchestration, and Application Load Balancer (ALB) for efficient traffic management. The migration involved a systematic approach: first, we defined application dependencies, built Docker images, and stored them securely in ECR. Next, we deployed these images to ECS by creating Task Definitions and ECS Services to handle and scale the tasks effectively. For the network infrastructure, we established a Virtual Private Cloud (VPC) with public subnets across multiple Availability Zones, attached an Internet Gateway, and configured an ALB to distribute traffic efficiently. Integrating ECS with ALB ensured load balancing and fault tolerance, resulting in a robust and highly available deployment environment for WaterBot on AWS. Through the use of Amazon Bedrock and the toggleable ability to employ the state of the art Claude Sonnet LLMs, the WaterBot solution is built end-to-end using AWS technology.

Industry Impact and Problem Solving

The optimized deployment of WaterBot on AWS has significantly improved its performance and operational efficiency. Hosting costs were reduced by 80%, demonstrating substantial cost savings. Performance has seen a marked improvement with enhanced responsiveness, thanks to efficient load balancing via the Application Load Balancer (ALB). Scalability has been vastly improved, enabling WaterBot to seamlessly handle fluctuating traffic demands, ensuring reliable access for users. This successful transformation underscores how leveraging modern cloud technologies can revolutionize government and educational tools, making vital information more accessible and cost-effective to deliver.

"I learned about the Cloud Innovation Center after hearing Arun speak at an AI in Education summit. After hearing him explain the problem-solving nature of the CIC, I sought him out to see if he and his team could help optimize the bot and bring its running costs down. His team did both! His team built out a version of the bot in AWS that produced an estimated 80% decrease in running cost compared to our previous managed hosting solution. Even though the costs came down, the performance did not: they made the bot run quickly. The flexibility of the setup will allow us to easily handle high levels of traffic or low levels of traffic equally easily. Being inside the AWS ecosystem provides an array of opportunities for further feature development above what we were expecting to be able to do on our previous solution, as well. And best of all, the whole process took only a couple of weeks! We got great help fast for free. Who else can boast that? I highly recommend the CIC."

- Dr. Stephen Carradini, ASU Assistant Professor, Technical Communication Program

Potential for Wider Application

WaterBot's model can be adapted to various other public services. Imagine utility companies using chatbots to provide instant information on electricity usage or billing, without the need to search through extensive documents. Dr. Carradini is also reaching out to municipalities across Arizona to demonstrate how effectively created chatbots can help with public communication on many topics of civic importance. Given the many sustainability areas in Arizona, Dr. Carradini has a goal of developing bots for concerns such as heat, sustainable energy, and more. This approach could streamline information access and enhance user satisfaction across multiple sectors.