Factuality

You can get factuality scores (or rather factual consistency scores) from the /factuality REST API endpoint or any of the official SDKs (Python, Go, Rust, JS, or cURL).

The output will include a score that ranges from 0.0 to 1.0. The higher the score, the more factuality consistency between the text and the reference.

Generate a Factuality Score

To generate a factuality score, you can use the following code examples. Depending on your preference or requirements, select the appropriate method for your application.

1
import os
2
import json
3
4
from predictionguard import PredictionGuard
5
6
# Set your Prediction Guard token as an environmental variable.
7
os.environ["PREDICTIONGUARD_API_KEY"] = "<api key>"
8
9
client = PredictionGuard()
10
11
# Perform the factual consistency check.
12
result = client.factuality.check(
13
    		reference="The sky is blue.",
14
    		text="The sky is green."
15
)
16
17
print(json.dumps(
18
    result,
19
    sort_keys=True,
20
    indent=4,
21
    separators=(',', ': ')
22
))

The output will look something like:

1
{
2
    "checks": [
3
        {
4
            "index": 0,
5
            "score": 0.26569077372550964
6
        }
7
    ]
8
    "id": "fact-5a364943-c990-44af-9b59-712a4a23e06f",
9
    "object": "factuality.check",
10
    "created": 1727795218,
11
}

This approach presents a straightforward way for readers to choose and apply the code example that best suits their needs for generating text completions using either Python, Go, Rust, JS, or cURL.