Chat

You can get chat text completions (based on a thread of chat messages) from any of the chat enabled models using the /chat/completions REST API endpoint or Chat Python client class.

Generate a chat text completion

To generate a chat text completion, you can use the following code examples. Depending on your preference or requirements, select the appropriate method for your application.

1 import os
2 import json
3 
4 import predictionguard as pg
5 
6 # Set your Prediction Guard token as an environmental variable.
7 os.environ["PREDICTIONGUARD_TOKEN"] = "<your access token>"
8 
9 messages = [
10     {
11         "role": "system",
12         "content": "You are a helpful assistant that provide clever and sometimes funny responses."
13     },
14     {
15         "role": "user",
16         "content": "What's up!"
17     },
18     {
19         "role": "assistant",
20         "content": "Well, technically vertically out from the center of the earth."
21     },
22     {
23         "role": "user",
24         "content": "Haha. Good one."
25     }
26 ]
27 
28 result = pg.Chat.create(
29     model="Neural-Chat-7B",
30     messages=messages,
31     max_tokens=500
32 )
33 
34 print(json.dumps(
35     result,
36     sort_keys=True,
37     indent=4,
38     separators=(',', ': ')
39 ))

If you’d like to have tabs to show multiple CodeBlocks at a time, you can use the CodeBlocks component as demonstrated above.

Getting started

Models

Using LLMs

Process LLM Input

Validating LLM Output

Guides

Reference

Support

Generate a chat text completion

1	import os
2	import json
3
4	import predictionguard as pg
5
6	# Set your Prediction Guard token as an environmental variable.
7	os.environ["PREDICTIONGUARD_TOKEN"] = "<your access token>"
8
9	messages = [
10	{
11	"role": "system",
12	"content": "You are a helpful assistant that provide clever and sometimes funny responses."
13	},
14	{
15	"role": "user",
16	"content": "What's up!"
17	},
18	{
19	"role": "assistant",
20	"content": "Well, technically vertically out from the center of the earth."
21	},
22	{
23	"role": "user",
24	"content": "Haha. Good one."
25	}
26	]
27
28	result = pg.Chat.create(
29	model="Neural-Chat-7B",
30	messages=messages,
31	max_tokens=500
32	)
33
34	print(json.dumps(
35	result,
36	sort_keys=True,
37	indent=4,
38	separators=(',', ': ')
39	))