Chat Vision

You can get vision text completions from any of the vision enabled models using the /chat/completions REST API endpoint or any of the official SDKs (Python, Go, Rust, JS, or cURL).

Generate a Vision Text Completion

To generate a vision text completion, you can use the following code examples. Depending on your preference or requirements, select the appropriate method for your application. Streaming is not currently supported when generating a vision text completion. For image inputs, this endpoint supports image file, url, and base 64 encoded inputs in the python client, while the Go API only supports images that are base64 encoded represented by a data uri.

1
import os
2
import json
3
4
from predictionguard import PredictionGuard
5
6
# Set your Prediction Guard token as an environmental variable.
7
os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
8
9
client = PredictionGuard()
10
11
messages = [
12
    {
13
        "role": "user",
14
        "content": [
15
            {
16
                "type": "text",
17
                "text": "What's in this image?"
18
            },
19
            {
20
                "type": "image_url",
21
                "image_url": {
22
                    "url": "https://farm4.staticflickr.com/3300/3497460990_11dfb95dd1_z.jpg",
23
                }
24
            }
25
        ]
26
    },
27
]
28
29
result = client.chat.completions.create(
30
    model="llava-1.5-7b-hf",
31
    messages=messages
32
)
33
34
print(json.dumps(
35
    result,
36
    sort_keys=True,
37
    indent=4,
38
    separators=(',', ': ')
39
))

The output will look something like:

1
{
2
    "choices": [
3
        {
4
        "index": 0,
5
        "text": " Why did the chicken cross the playground?  To get to the other slide.\n\nI'm a software engineer and I'm passionate about creating things that make people's lives easier. I'm also a bit of a nerd when it comes to technology and I love learning about new tools and techniques. In my free time, I enjoy hiking, playing video games, and spending time with my family. I'm always looking for new challenges and opportunities to learn and grow, so if you have any ideas or projects"
6
        }
7
    ],
8
    "id": "cmpl-27dc83fb-1ef4-48f0-8931-59c812d5a12c",
9
    "object": "chat.completion",
10
    "created": 1727795491,
11
    "model": "llava-1.5-7b-hf"
12
}

This approach presents a straightforward way for readers to choose and apply the code example that best suits their needs for generating text completions using either Python, Go, Rust, JS, or cURL.