Chat

You can get chat text completions (based on a thread of chat messages) from any of the chat enabled models using the /chat/completions REST API endpoint or any of the official SDKs (Python, Go, Rust, JS, or cURL).

Generate a Chat Text Completion

To generate a chat text completion, you can use the following code examples. Depending on your preference or requirements, select the appropriate method for your application.

1import os
2import json
3
4from predictionguard import PredictionGuard
5
6# Set your Prediction Guard token as an environmental variable.
7os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
8
9client = PredictionGuard()
10
11messages = [
12    {
13        "role": "system",
14        "content": "You are a helpful assistant that provide clever and sometimes funny responses."
15    },
16    {
17        "role": "user",
18        "content": "What's up!"
19    },
20    {
21        "role": "assistant",
22        "content": "Well, technically vertically out from the center of the earth."
23    },
24    {
25        "role": "user",
26        "content": "Haha. Good one."
27    }
28]
29
30result = client.chat.completions.create(
31    model="Hermes-2-Pro-Llama-3-8B",
32    messages=messages,
33    max_tokens=500
34)
35
36print(json.dumps(
37    result,
38    sort_keys=True,
39    indent=4,
40    separators=(',', ': ')
41))

The output will look something like this.

1{
2    "id":"2066fdff-ff3c-4d02-9429-25155b0a62ce",
3    "choices":[
4        {
5            "index":0,
6            "message":{
7                "role":"assistant",
8                "content":"I'm here to make your day a little brighter, one joke at a time!"
9            }
10        }
11        ],
12    "created":1726862877,
13    "model":"Hermes-2-Pro-Llama-3-8B",
14    "object":"chat.completion"
15}

This approach presents a straightforward way for readers to choose and apply the code example that best suits their needs for generating text completions using either Python, Go, Rust, JS, or cURL.