Skip to main content

Self-hosted Deployment

k8s-architecture.png

Waii's self-hosted K8s deployment operates within the customer's own Kubernetes cluster. This setup includes:

Waii Service Pod, which includes following components:

  1. UI/API service: Hosts UI and API endpoints for Waii.
  2. Metadata Management service: Retrieve database schema information from customer databases (e.g., Postgres, Snowflake) and store it in an internal PostgreSQL database.
  3. Query Generator / Compiler service: Utilizes the stored information to produce SQL queries. These queries are generated by the LLM model using the stored information and subsequently refined by Waii's query compiler/optimizer to produce the final query.

Multiple instance of Waii Service Pods can be deployed to handle HA, it will handle leader selection and failover automatically.

Network Access Requirements​

The service operates entirely within the customer's Virtual Private Cloud (VPC), ensuring that no data is transmitted to Waii-hosted services. The deployment relies on the following external dependencies:

  1. LLM Services: Such as OpenAI, Anthropic, etc., to assist in query generation. Depends on where the LLM model is hosted, it can be either Internet (for example OpenAI) or within the same VPC (for example Bedrock or Azure OpenAI).
  2. Postgres Database: To store metadata and query history, which can be either self-hosted or managed (e.g., RDS).

Resource Requirements​

The self-hosted K8s deployment requires the following resources:

  • For Waii Service Pods: at least 2 vcores and 8GB RAM. We recommend allocate 4 vcores and 16GB RAM for better performance.
  • For Postgres Database:
    • 2 vcores / 4 GB RAM for small deployments (100s of tables, 1000s of columns, <10 concurrent users). Similar to t3.medium RDS instance.
    • 4 vcores / 8 GB RAM for medium deployments (1000s of tables, 10000s of columns, <100 concurrent users). Similar to t3.large RDS instance.
    • If you have a large number of semantic context (10-100k semantic statements), we recommend 4-8 vcores and 16GB RAM. Similar to t3.xlarge RDS instance.

K8s version requirements​

Our service doesn't use new/advanced features of K8s, so it should work with most K8s versions. We recommend using K8s version 1.23 or later. You can use managed service like EKS, GKE, AKS, or self-hosted K8s.

We haven't certified Waii K8s deployment on AWS Fargate yet, if you are using Fargate, please let us know so we can provide more guidance.

Deploying Waii service to your own Kubernetes cluster​

The following guide will walk you through the process of deploying a self-hosted Waii service and connecting it to your own database infrastructure.

Setup Postgres database​

Please refer to the Setup Postgres database for self-hosted Waii documentation for detailed instructions on how to setup a Postgres database for Waii self-hosted.

Install Chart​

Refer https://github.com/waii-ai/waii-helm-chart/blob/main/waii/README.md for detailed documentation on chart deployment.

Refer https://github.com/waii-ai/waii-helm-chart/blob/main/waii/values.yaml for all customisable parameters of the chart.

Adding Database Connections​

Please refer to the Adding a Database Connection documentation for detailed instructions on how to add a new database connection.