Models

Prompt Formats

You might be used to entering simple text prompts into systems like ChatGPT. However, when you utilize certain open access LLMs directly, you might want to follow a specific prompt format. These models are fine-tuned using prompt data, and if you match your prompt formats to that training data format then you can see boosts in performance.

Check out the model details page to learn which prompt formats match certain LLMs. We’ve included some of the most important prompt formats below.

Note on Chat model prompts - For your convenience, we automatically apply the right prompt formats when you supply a messages object to our /chat/completions endpoint or via the pg.Chat.create() method in the Python client. You don’t have to add in special tokens or apply the below prompt formats as this will duplicate the formatting. However, if you want to use chat-tuned models in the /completions endpoint or via the pg.Completions.create() method, you should apply the appropriate one of the below prompt formats.

Alpaca

This format has two options. First, for prompts that are single instructions, without relevant context (e.g., retrieved context):

### Instruction:
<prompt>
### Response:
<leave a newline blank for model to respond>

For prompts where context is injected:

### Instruction:
<prompt>
### Input:
<additional context>
### Response:
<leave a newline blank for model to respond>

(Replace the portions of the prompt below in <...> with the appropriate information, and do not keep the < or > characters)

Neural Chat

(Replace the portions of the prompt below in <...> with the appropriate information, and do not keep the < or > characters)

### System:
<prompt>
### User:
<context or user message>
### Assistant:

ChatML

(Replace the portions of the prompt below in curly braces {...} with the appropriate information, and do not keep the curly braces)

<|im_start|>system
{prompt}<|im_end|>
<|im_start|>user
{context or user message}<|im_end|>
<|im_start|>assistant<|im_end|>

Deepseek

(Replace the portions of the prompt below in curly braces {...} with the appropriate information, and do not keep the curly braces)

<|begin_of_sentence|>{prompt}
### Instruction:
{context or user message}
### Response:

Llama3

(Replace the portions of the prompt below in curly braces {...} with the appropriate information, and do not keep the curly braces)

<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{context or user message}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
Built with