Reference

Chat

You can get chat text completions (based on a thread of chat messages) from any of the chat enabled models using the /chat/completions REST API endpoint or any of the official SDKs (Python, Go, Rust, JS, or cURL).

Generate a Chat Text Completion

To generate a chat text completion, you can use the following code examples. Depending on your preference or requirements, select the appropriate method for your application.

1 import os
2 import json
3 
4 from predictionguard import PredictionGuard
5 
6 # Set your Prediction Guard token as an environmental variable.
7 os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
8 
9 client = PredictionGuard()
10 
11 messages = [
12     {
13         "role": "system",
14         "content": "You are a helpful assistant that provide clever and sometimes funny responses."
15     },
16     {
17         "role": "user",
18         "content": "What's up!"
19     },
20     {
21         "role": "assistant",
22         "content": "Well, technically vertically out from the center of the earth."
23     },
24     {
25         "role": "user",
26         "content": "Haha. Good one."
27     }
28 ]
29 
30 result = client.chat.completions.create(
31     model="Neural-Chat-7B",
32     messages=messages,
33     max_tokens=500
34 )
35 
36 print(json.dumps(
37     result,
38     sort_keys=True,
39     indent=4,
40     separators=(',', ': ')
41 ))

The output will look something like this.

1 {
2    "id":"chat-SUZNF22Qn7uq22Ciez0AfhmnVlKF0",
3    "object":"chat_completion",
4    "created":1717780456,
5    "model":"Neural-Chat-7B",
6    "choices":[
7       {
8          "index":0,
9          "message":{
10             "role":"assistant",
11             "content":"Thanks, but if you're looking for something more serious, I'm here to help with any questions or tasks you might have. Just let me know!",
12             "output":null
13          },
14          "status":"success"
15       }
16    ]
17 }

This approach presents a straightforward way for readers to choose and apply the code example that best suits their needs for generating text completions using either Python, Go, Rust, JS, or cURL.

1	import os
2	import json
3
4	from predictionguard import PredictionGuard
5
6	# Set your Prediction Guard token as an environmental variable.
7	os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
8
9	client = PredictionGuard()
10
11	messages = [
12	{
13	"role": "system",
14	"content": "You are a helpful assistant that provide clever and sometimes funny responses."
15	},
16	{
17	"role": "user",
18	"content": "What's up!"
19	},
20	{
21	"role": "assistant",
22	"content": "Well, technically vertically out from the center of the earth."
23	},
24	{
25	"role": "user",
26	"content": "Haha. Good one."
27	}
28	]
29
30	result = client.chat.completions.create(
31	model="Neural-Chat-7B",
32	messages=messages,
33	max_tokens=500
34	)
35
36	print(json.dumps(
37	result,
38	sort_keys=True,
39	indent=4,
40	separators=(',', ': ')
41	))

1	{
2	"id":"chat-SUZNF22Qn7uq22Ciez0AfhmnVlKF0",
3	"object":"chat_completion",
4	"created":1717780456,
5	"model":"Neural-Chat-7B",
6	"choices":[
7	{
8	"index":0,
9	"message":{
10	"role":"assistant",
11	"content":"Thanks, but if you're looking for something more serious, I'm here to help with any questions or tasks you might have. Just let me know!",
12	"output":null
13	},
14	"status":"success"
15	}
16	]
17	}