AWS Deployment

Prerequisites

• AWS Account with appropriate permissions
• AWS CLI configured with your credentials
• kubectl configured for your EKS cluster
• Access to Admin Console at admin.predictionguard.com

Deployment Process

1. Create EKS Cluster

Create an Amazon EKS cluster in your AWS account:

$
# Set your cluster name that reflects the AI system on the Admin Console
$
export CLUSTER_NAME=<your-ai-system-name>
$
$
eksctl create cluster \
>
    --name $CLUSTER_NAME \
>
    --region us-west-2 \
>
    --nodegroup-name workers \
>
    --node-type t3.large \
>
    --nodes 3 \
>
    --nodes-min 1 \
>
    --nodes-max 5

2. Configure kubectl

$
# Update kubeconfig
$
aws eks update-kubeconfig --region us-west-2 --name $CLUSTER_NAME
$
$
# Verify connection
$
kubectl get nodes

3. Set AWS-Specific Configuration

• Node Groups: Configure your EKS node groups
• Storage Classes: Use EBS CSI driver for persistent volumes
• Load Balancer: Configure ALB or NLB for ingress
• VPC: Specify your VPC and subnet configuration

4. Create an AI System in the Admin Console

If you have not already created your AI system in the Admin Console, follow the Quick Start or the Custom System guide to create your system and generate the installation command.

5. Get the Deployment Command

Navigate to your system in the Admin Console and click the Deploy button in the top-right corner of the system management page.

This opens the Deploy Command modal. Select kubectl as the deployment method, then click Copy to copy the generated installation command.

6. Execute the Installation on Your Cluster

Paste and run the copied command on a machine with kubectl access to your EKS cluster. The command authenticates with your Prediction Guard instance and bootstraps all services into the predictionguard namespace.

After a few minutes, verify the installation:

$
kubectl get pods -n predictionguard

You should see running pods including pg-inside, indicating the system has been successfully installed. The system will also show as Healthy in the Admin Console.

Configuring Ingress and Reverse Proxy

Prediction Guard comes preconfigured for NGINX and a default Ingress which can be enabled on the system within the Edit section of the Systems page. Here you can configure the desired domain names and have NGINX deploy into the predictionguard namespace with preconfigured settings for the Prediction Guard API. Then, simply ensure that your DNS entry is routable to the ingress IP on your Kubernetes cluster or load balancer in AWS.

Post-Deployment

Once deployed, your system is fully manageable from the Admin Console dashboard.

From here you can:

• API Keys: Manage API keys for secure access to your system endpoints
• Models: Deploy private, managed, or external models and their configurations
• MCP Servers: Configure Model Context Protocol servers and their connections
• Advanced Settings: Configure system settings, resource limits, networking, and cluster-specific options

AWS Integration

Your deployment automatically integrates with:

• EBS: Persistent storage for models and data
• ALB/NLB: Load balancing for high availability
• CloudWatch: Monitoring and logging
• IAM: Service account and role management

Need help? Contact our support team for assistance with your AWS deployment.