Chat Completions

Generate chat completions based on a conversation history.

Request

This endpoint expects an object.

modelstringRequired

The chat model to use for generating completions.

messagesstring or list of objectsRequired

frequency_penaltydoubleOptional

A value between -2.0 and 2.0, with positive values increasingly penalizing new tokens based on their frequency so far in order to decrease further occurrences.

logit_biasanyOptional

Modifies the likelihood of specified tokens appearing in a response.

max_completion_tokensintegerOptional

The maximum number of tokens in the generated completion.

parallel_tool_callsbooleanOptional

Whether to enable parallel function calling during tool use.

presence_penaltydoubleOptional

A value between -2.0 and 2.0, with positive values causing a flat reduction of new tokens based on their existing presence so far in order to decrease further occurrences.

stopstring or list of stringsOptional

streambooleanOptional

Whether to stream back the model response.

temperaturedoubleOptional

The temperature parameter for controlling randomness in completions.

tool_choicestring or objectOptional

toolslist of objectsOptional

The content of the tool call.

top_pdoubleOptional

The diversity of the generated text based on nucleus sampling.

top_kintegerOptional

The diversity of the generated text based on top-k sampling.

outputobjectOptional

Options to affect the output of the response.

inputobjectOptional

Options to affect the input of the request.

max_tokensintegerOptional

Deprecated. Please use max_completion_tokens.

Response

Successful response.

idstring or null

Unique ID for the chat completion.

objectstring or null

Type of object (chat completion).

createdinteger or null

Timestamp of when the chat completion was created.

modelstring or null

The chat model used for generating completions.

choiceslist of objects or null

The set of result choices.

1	curl -X POST https://api.predictionguard.com/chat/completions \
2	-H "Authorization: Bearer <token>" \
3	-H "Content-Type: application/json" \
4	-d '{
5	"model": "Hermes-3-Llama-3.1-70B",
6	"messages": [
7	{
8	"role": "user",
9	"content": "How do you feel about the world in general?"
10	}
11	],
12	"max_completion_tokens": 1000,
13	"frequency_penalty": 0.1,
14	"logit_bias": {
15	"128000": 10
16	},
17	"parallel_tool_calls": false,
18	"presence_penalty": 0.1,
19	"stop": "hello",
20	"temperature": 1,
21	"top_p": 1,
22	"top_k": 50,
23	"output": {
24	"factuality": true,
25	"toxicity": true
26	},
27	"input": {
28	"pii": "replace",
29	"pii_replace_method": "random"
30	}
31	}'

1	{
2	"id": "chat-d079eca8-1d6a-451a-a4ac-b0fea1e11a96",
3	"object": "chat.completion",
4	"created": 1727890629,
5	"model": "Hermes-3-Llama-3.1-70B",
6	"choices": [
7	{
8	"index": 0,
9	"message": {
10	"role": "assistant",
11	"content": "I feel that the world in general is a complex and ever-evolving place. It has its fair share of beauty, challenges, and opportunities. There are moments of joy, connection, and growth, as well as pain, conflict, and loss. The world is a reflection of the people who inhabit it, and it's essential to maintain a balance between appreciating its wonders and working towards making it a better place for all. It's a constant journey of learning, adapting, and striving for a more harmonious existence."
12	}
13	}
14	]
15	}

Headers

Request

Response

Errors