Completions with Streaming

1 curl -X POST https://api.predictionguard.com/completions with Streaming \ 2 -H "Authorization: Bearer <token>" \ 3 -H "Content-Type: application/json" \ 4 -d '{ 5 "model": "Hermes-3-Llama-3.1-70B", 6 "prompt": "Will I lose my hair?", 7 "max_tokens": 1000, 8 "frequency_penalty": 0.1, 9 "logit_bias": { 10 "128000": 10 11 }, 12 "presence_penalty": 0.1, 13 "stop": "hello", 14 "temperature": 1, 15 "top_p": 1, 16 "top_k": 50, 17 "output": { 18 "factuality": true, 19 "toxicity": true 20 }, 21 "input": { 22 "pii": "replace", 23 "pii_replace_method": "random" 24 } 25 }'

1 { 2 "id": "cmpl-d20e5639-ed8d-48b2-a403-08fe0fdc7add", 3 "object": "text_completion", 4 "created": 1727889988, 5 "model": "Hermes-3-Llama-3.1-70B", 6 "choices": [ 7 { 8 "index": 0, 9 "text": "Most people lose some hair every day. It’s a natural process called shedding. On average, you can lose 50-100 strands of hair a day. However, if you notice an increase in hair loss or if you are experiencing hair loss in large clumps, then it could be a sign of an underlying condition. Hair loss can be triggered by a number of factors such as stress, illness, hormone imbalance, medication, and genetics.\nIf you’re concerned about hair loss, you should consult with a trichologist, a hair and scalp specialist. They can help determine the cause of your hair loss and recommend the appropriate treatment." 10 } 11 ] 12 }

Retrieve text completions based on the provided input.

Request

This endpoint expects an object.

modelstringRequired

The chat model to use for generating completions.

promptstringRequired

The prompt to use for generating completions.

frequency_penaltydoubleOptional

A value between -2.0 and 2.0, with positive values increasingly penalizing new tokens based on their frequency so far in order to decrease further occurrences.

logit_biasanyOptional

Modifies the likelihood of specified tokens appearing in a response.

max_tokensintegerOptional

The maximum number of tokens in the generated completion.

presence_penaltydoubleOptional

A value between -2.0 and 2.0, with positive values causing a flat reduction of new tokens based on their existing presence so far in order to decrease further occurrences.

streambooleanOptional

Turn streaming on.

stopstring or list of stringsOptional

temperaturedoubleOptional

The temperature parameter for controlling randomness in completions.

top_pdoubleOptional

The diversity of the generated text based on nucleus sampling.

top_kintegerOptional

The diversity of the generated text based on top-k sampling.

outputobjectOptional

Options to affect the output of the response.

inputobjectOptional

Options to affect the input of the request.

Response

Successful response.

idstringOptional

Unique ID for the completion.

objectstringOptional

Type of object (completion).

createdintegerOptional

Timestamp of when the completion was created.

modelstringOptional

The model used for generating the result.

choiceslist of objectsOptional

The set of result choices.

Headers

Request

Response

Errors