Chat Completions Streaming — Prediction Guard

Generate chat completions based on a conversation history with streaming support.

Request

This endpoint expects an object.

model

stringRequired

The chat model to use for generating completions.

messages

list of objectsRequired

An array of messages used for generating completions.

max_tokens

integerOptional

The maximum number of tokens in the generated completion.

temperature

doubleOptional

The temperature parameter for controlling randomness in completions.

top_p

doubleOptional

The diversity of the generated text based on nucleus sampling.

top_k

integerOptional

The diversity of the generated text based on top-k sampling.

stream

booleanOptional

Turn streaming on.

input

objectOptional

Options to affect the input of the request.

Response

This endpoint returns a list of objects

data

objectOptional

Errors

1 curl -X POST https://api.predictionguard.com/chat/completions - Streaming Version \ 2 -H "x-api-key: <apiKey>" \ 3 -H "Content-Type: application/json" \ 4 -d '{ 5 "model": "Neural-Chat-7B", 6 "messages": [ 7 { 8 "role": "user", 9 "content": "How do you feel about the world in general?" 10 } 11 ], 12 "max_tokens": 1000, 13 "temperature": 1, 14 "top_p": 1, 15 "top_k": 50, 16 "stream": true, 17 "input": { 18 "pii": "replace", 19 "pii_replace_method": "random" 20 } 21 }'

1 [ 2 { 3 "data": { 4 "id": "chat-QTBHDiLvhW6XJJKn4skhBDADZTIgd", 5 "object": "chat.completion.chunk", 6 "created": 1717678791, 7 "model": "Neural-Chat-7B", 8 "choices": [ 9 { 10 "index": 0, 11 "generated_text": "null", 12 "logprobs": 0, 13 "finish_reason": "null", 14 "delta": { 15 "content": " I" 16 } 17 } 18 ] 19 } 20 }, 21 { 22 "data": { 23 "id": "chat-BZvsPJsNvcIK0TfPOvC6UObFIsGSy", 24 "object": "chat.completion.chunk", 25 "created": 1717678792, 26 "model": "Neural-Chat-7B", 27 "choices": [ 28 { 29 "index": 0, 30 "generated_text": "I feel that the world is a complex and dynamic place. It has its fair share of challenges and difficulties, but also offers numerous opportunities for growth, learning, and connection. The world is constantly evolving, and as individuals, we must adapt and find ways to contribute positively to society. It's essential to maintain a balance between acknowledging the problems and celebrating the progress and achievements, while also remembering the importance of compassion, empathy, and understanding towards one another. Overall, I believe it's essential to stay optimistic and work together towards a better future for all.", 31 "logprobs": 0, 32 "finish_reason": "stop" 33 } 34 ] 35 } 36 } 37 ]

1	curl -X POST https://api.predictionguard.com/chat/completions - Streaming Version \
2	-H "x-api-key: <apiKey>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"model": "Neural-Chat-7B",
6	"messages": [
7	{
8	"role": "user",
9	"content": "How do you feel about the world in general?"
10	}
11	],
12	"max_tokens": 1000,
13	"temperature": 1,
14	"top_p": 1,
15	"top_k": 50,
16	"stream": true,
17	"input": {
18	"pii": "replace",
19	"pii_replace_method": "random"
20	}
21	}'

1	[
2	{
3	"data": {
4	"id": "chat-QTBHDiLvhW6XJJKn4skhBDADZTIgd",
5	"object": "chat.completion.chunk",
6	"created": 1717678791,
7	"model": "Neural-Chat-7B",
8	"choices": [
9	{
10	"index": 0,
11	"generated_text": "null",
12	"logprobs": 0,
13	"finish_reason": "null",
14	"delta": {
15	"content": " I"
16	}
17	}
18	]
19	}
20	},
21	{
22	"data": {
23	"id": "chat-BZvsPJsNvcIK0TfPOvC6UObFIsGSy",
24	"object": "chat.completion.chunk",
25	"created": 1717678792,
26	"model": "Neural-Chat-7B",
27	"choices": [
28	{
29	"index": 0,
30	"generated_text": "I feel that the world is a complex and dynamic place. It has its fair share of challenges and difficulties, but also offers numerous opportunities for growth, learning, and connection. The world is constantly evolving, and as individuals, we must adapt and find ways to contribute positively to society. It's essential to maintain a balance between acknowledging the problems and celebrating the progress and achievements, while also remembering the importance of compassion, empathy, and understanding towards one another. Overall, I believe it's essential to stay optimistic and work together towards a better future for all.",
31	"logprobs": 0,
32	"finish_reason": "stop"
33	}
34	]
35	}
36	}
37	]