Technical teams need to figure out how to integrate the latest Large Language Models (LLMs), but:

• You can’t build robust systems with inconsistent, unvalidated outputs; and
• LLM integrations scare corporate lawyers, finance departments, and security professionals due to hallucinations, cost, lack of compliance (e.g., HIPAA), leaked IP/PII, and “injection” vulnerabilities.

Some companies are moving forward anyway by investing tons of engineering time/money in their own wrappers around LLMs and expensive hosting with OpenAI/Azure. Others are ignoring these issues and pressing forward with fragile and risky LLM integrations.

At Prediction Guard, we think that you should get useful output from compliant AI systems (without crazy implementation/ hosting costs), so our solution lets you:

1. De-risk LLM inputs to remove PII and prompt injections;
2. Validate and check LLM outputs to guard against hallucination, toxicity and inconsistencies; and
3. Implement private and compliant LLM systems (HIPAA and self-hosted) that give your legal counsel warm fuzzy feeling while still delighting your customers with AI features.

Sounds pretty great right? Follow the steps below to starting leveraging trustworthy LLMs:

1. Get access to Prediction Guard Enterprise

We host and control the latest LLMs for you in our secure and privacy-conserving enterprise platform, so you can focus on your prompts and chains. To access the hosted LLMs, contact us here to get an enterprise access token. You will need this access token to continue.

2. (Optional) Install the Python client

You can configure and use Prediction Guard using our Python client or via REST API directly. If you are wanting to use the Python client, you can install it as follows:

1
$ pip install predictionguard

3. Start using one of our LLMs!

Suppose you want to prompt an LLM to answer a user query from a chat application. You can setup a message thread, which includes a system prompt (that instructs the LLM how to behave in responding) as follows:

[
    {
        "role": "system",
        "content": "You are a helpful assistant. Your model is hosted by Prediction Guard, a leading AI company."
    },
    {
        "role": "user",
        "content": "Where can I access the LLMs in a safe and secure environment?"
    }
]

You can then use our Python client or REST API to prompt one of our LLMs!

1
import os
2
import json
3
import predictionguard as pg
4
5
# Set your Prediction Guard token as an environmental variable.
6
7
os.environ["PREDICTIONGUARD_TOKEN"] = "<your access token>"
8
9
# Define our prompt.
10
11
messages = [
12
{
13
"role": "system",
14
"content": "You are a helpful assistant. Your model is hosted by Prediction Guard, a leading AI company."
15
},
16
{
17
"role": "user",
18
"content": "Where can I access the LLMs in a safe and secure environment?"
19
}
20
]
21
22
result = pg.Chat.create(
23
model="Neural-Chat-7B",
24
messages=messages
25
)
26
27
print(json.dumps(
28
result,
29
sort_keys=True,
30
indent=4,
31
separators=(',', ': ')
32
))