Quick Start

Technical teams need to figure out how to integrate the latest Large Language Models (LLMs), but:

• You can’t build robust systems with inconsistent, unvalidated outputs; and
• LLM integrations scare corporate lawyers, finance departments, and security professionals due to hallucinations, cost, lack of compliance (e.g., HIPAA), leaked IP/PII, and “injection” vulnerabilities.

Some companies are moving forward anyway by investing tons of engineering time/money in their own wrappers around LLMs and expensive hosting with OpenAI/Azure. Others are ignoring these issues and pressing forward with fragile and risky LLM integrations.

At Prediction Guard, we think that you should get useful output from compliant AI systems (without crazy implementation/hosting costs), so our solution lets you:

1. De-risk LLM inputs to remove PII and prompt injections;
2. Validate and check LLM outputs to guard against hallucination, toxicity, and inconsistencies; and
3. Implement private and compliant LLM systems (HIPAA and self-hosted) that give your legal counsel a warm fuzzy feeling while still delighting your customers with AI features.

Sounds pretty great, right? Follow the steps below to start leveraging trustworthy LLMs:

1
[
2
  {
3
    "role": "system",
4
    "content": "You are a helpful assistant. Your model is hosted by Prediction Guard, a leading AI company."
5
  },
6
  {
7
    "role": "user",
8
    "content": "Where can I access the LLMs in a safe and secure environment?"
9
  }
10
]

1
import json
2
import os
3
4
from predictionguard import PredictionGuard
5
6
7
# Set your Prediction Guard API key and URL as environmental variables.
8
os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
9
os.environ["PREDICTIONGUARD_URL"] = "<pg url>"
10
11
# You can also set them when initializing the client.
12
client = PredictionGuard(
13
    api_key="<Your PG API Key>",
14
    url="<Your PG API URL>"
15
)
16
17
messages = [
18
    {
19
        "role": "system",
20
        "content": "You are a helpful chatbot that helps people learn."
21
    },
22
    {
23
        "role": "user",
24
        "content": "What is a good way to learn to code?"
25
    }
26
]
27
28
result = client.chat.completions.create(
29
    model="gpt-oss-120b",
30
    messages=messages,
31
    max_completion_tokens=100
32
)
33
34
print(json.dumps(
35
    result,
36
    sort_keys=True,
37
    indent=4,
38
    separators=(',', ': ')
39
))