Chat Completions Streaming

POST
Generate chat completions based on a conversation history with streaming support.

Request

This endpoint expects an object.
model
stringRequired
The chat model to use for generating completions.
messages
list of objectsRequired
An array of messages used for generating completions.
max_tokens
integerOptional
The maximum number of tokens in the generated completion.
temperature
doubleOptional
The temperature parameter for controlling randomness in completions.
top_p
doubleOptional
The diversity of the generated text.
stream
booleanOptional
Turn streaming on.

Response

This endpoint returns a list of objects
data
objectOptional

Errors

POST
1curl -X POST https://api.predictionguard.com/chat/completions - Streaming Version \
2 -H "x-api-key: <apiKey>" \
3 -H "Content-Type: application/json" \
4 -d '{
5 "model": "Neural-Chat-7B",
6 "messages": [
7 {
8 "role": "user",
9 "content": "How do you feel about the world in general?"
10 }
11 ],
12 "max_tokens": 1000,
13 "temperature": 1.1,
14 "top_p": 0.1,
15 "stream": true
16}'
1[
2 {
3 "data": {
4 "id": "chat-QTBHDiLvhW6XJJKn4skhBDADZTIgd",
5 "object": "chat.completion.chunk",
6 "created": 1717678791,
7 "model": "Neural-Chat-7B",
8 "choices": [
9 {
10 "index": 0,
11 "generated_text": "null",
12 "logprobs": 0,
13 "finish_reason": "null",
14 "delta": {
15 "content": " I"
16 }
17 }
18 ]
19 }
20 },
21 {
22 "data": {
23 "id": "chat-BZvsPJsNvcIK0TfPOvC6UObFIsGSy",
24 "object": "chat.completion.chunk",
25 "created": 1717678792,
26 "model": "Neural-Chat-7B",
27 "choices": [
28 {
29 "index": 0,
30 "generated_text": "I feel that the world is a complex and dynamic place. It has its fair share of challenges and difficulties, but also offers numerous opportunities for growth, learning, and connection. The world is constantly evolving, and as individuals, we must adapt and find ways to contribute positively to society. It's essential to maintain a balance between acknowledging the problems and celebrating the progress and achievements, while also remembering the importance of compassion, empathy, and understanding towards one another. Overall, I believe it's essential to stay optimistic and work together towards a better future for all.",
31 "logprobs": 0,
32 "finish_reason": "stop"
33 }
34 ]
35 }
36 }
37]