API Reference

Tokenize

Generates tokens for a string using a model's tokenizer.

Headers

AuthorizationstringRequired

Bearer authentication of the form Bearer <token>, where token is your auth token.

Request

This endpoint expects an object.
modelstringRequired
The name of the model to use for tokenization.
inputstringRequired
The text to tokenize.

Response

Successful response.
idstring or null
Unique ID for the token.
objectstring or null

Type of object (tokens).

createdinteger or null
Timestamp of when the tokens were generated.
modelstring or null
The name of the model used.
tokenslist of objects or null
An array of token data.

Errors