Chat Vision

You can get vision text completions from any of the vision enabled models using the /chat/completions REST API endpoint or any of the official SDKs (Python, Go, Rust, JS, or cURL).

Generate a Vision Text Completion

To generate a vision text completion, you can use the following code examples. Depending on your preference or requirements, select the appropriate method for your application. Streaming is not currently supported when generating a vision text completion. For image inputs, this endpoint supports image file, url, and base 64 encoded inputs in the python client, while the Go API only supports images that are base64 encoded represented by a data uri.

1
import os
2
import json
3
4
from predictionguard import PredictionGuard
5
6
# Set your Prediction Guard token as an environmental variable.
7
os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
8
9
client = PredictionGuard()
10
11
messages = [
12
    {
13
        "role": "user",
14
        "content": [
15
            {
16
                "type": "text",
17
                "text": "What's in this image?"
18
            },
19
            {
20
                "type": "image_url",
21
                "image_url": {
22
                    "url": "https://pbs.twimg.com/media/GKLN4qPXEAArqoK.png",
23
                }
24
            }
25
        ]
26
    },
27
]
28
29
result = client.chat.completions.create(
30
    model="llava-1.5-7b-hf",
31
    messages=messages
32
)
33
34
print(json.dumps(
35
    result,
36
    sort_keys=True,
37
    indent=4,
38
    separators=(',', ': ')
39
))

The output will look something like:

1
{
2
   "id":"chat-zstjj2MKpRdKfBmCMILK9wgx7I09N",
3
   "object":"chat_completion",
4
   "created":1723228682,
5
   "model":"llava-1.5-7b-hf",
6
   "choices":[
7
      {
8
         "index":0,
9
         "message":{
10
            "role":"assistant",
11
            "content":"In the image, there is a person standing on the back of a vehicle in a city. The vehicle is situated next to a taxi on the street. Another car can be seen approaching, and there is also a traffic light visible in the scene. The person standing on the vehicle could be a man drying his clothes, creating a unique and eye-catching sight. Additionally, a handbag is present near the person riding on the vehicle, likely belonging to them.",
12
            "output":null
13
         },
14
         "status":"success"
15
      }
16
   ]
17
}

This approach presents a straightforward way for readers to choose and apply the code example that best suits their needs for generating text completions using either Python, Go, Rust, JS, or cURL.