Getting Started

Getting Started

Technical teams need to figure out how to integrate the latest Large Language Models (LLMs), but:

  • You can’t build robust systems with inconsistent, unvalidated outputs; and
  • LLM integrations scare corporate lawyers, finance departments, and security professionals due to hallucinations, cost, lack of compliance (e.g., HIPAA), leaked IP/PII, and “injection” vulnerabilities.

Some companies are moving forward anyway by investing tons of engineering time/money in their own wrappers around LLMs and expensive hosting with OpenAI/Azure. Others are ignoring these issues and pressing forward with fragile and risky LLM integrations.

At Prediction Guard, we think that you should get useful output from compliant AI systems (without crazy implementation/ hosting costs), so our solution lets you:

  1. De-risk LLM inputs to remove PII and prompt injections;
  2. Validate and check LLM outputs to guard against hallucination, toxicity and inconsistencies; and
  3. Implement private and compliant LLM systems (HIPAA and self-hosted) that give your legal counsel warm fuzzy feeling while still delighting your customers with AI features.

Sounds pretty great right? Follow the steps below to starting leveraging trustworthy LLMs:

1. Get access to Prediction Guard Enterprise

We host and control the latest LLMs for you in our secure and privacy-conserving enterprise platform, so you can focus on your prompts and chains. To access the hosted LLMs, contact us here (opens in a new tab) to get an enterprise access token. You will need this access token to continue.

2. (Optional) Install the Python client

You can configure and use Prediction Guard using our Python client or via REST API directly. If you are wanting to use the Python client, you can install it as follows:

$ pip install predictionguard

3. Start using one of our LLMs!

Suppose you want to prompt an LLM to answer a user query from a chat application. You can setup a message thread, which includes a system prompt (that instructs the LLM how to behave in responding) as follows:

        "role": "system",
        "content": "You are a helpful assistant. Your model is hosted by Prediction Guard, a leading AI company."
        "role": "user",
        "content": "Where can I access the LLMs in a safe and secure environment?"

You can then use our Python client or REST API to prompt one of our LLMs!


Note, you will need to replace <your access token> in the above examples with your actual access token.

This should result in something similar to the following output. The choices[0].message.content field contains the raw LLM output.

    "id": "chat-mLf2CfeudF7UHkaF4tmzGteM2liac",
    "object": "chat_completion",
    "created": 1701881134,
    "model": "Neural-Chat-7B",
    "choices": [
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "As a helpful assistant, I can guide you to access the LLMs (Language Language Models) in a safe and secure environment through Prediction Guard's AI platform. Prediction Guard is a leading AI company that provides a secure and compliant platform for hosting and deploying AI models, including LLMs. Here are the steps to access the LLMs on Prediction Guard's platform:\n\n1. Sign up for a Prediction Guard account: Visit the Prediction Guard website and",
                "output": null

4. Explore other models, and guides

This is only the beginning of what your can do with Prediction Guard (not to mention what is on our roadmap). Now that you have a working example, consider exploring: