Alternative Output Structuring with PredictionGuard

This example demonstrates how to integrate PredictionGuard with custom data structures for output parsing, utilizing the Pydantic library. The focus is on structuring outputs in a specific format, in this case, a joke, ensuring the setup ends with a question mark.

Requirements

• PredictionGuard API
• Pydantic library
• LangChain library

Implementation Steps

1. Set Environment Variable for PredictionGuard: Ensure that your PredictionGuard API token is correctly set up in your environment variables.

1
import os
2
os.environ["PREDICTIONGUARD_TOKEN"] = "<your access token>"

2. Import Necessary Libraries: Import PredictionGuard, Pydantic for data validation, and LangChain for output parsing and prompt templating.

1
import predictionguard as pg
2
from langchain.output_parsers import PydanticOutputParser
3
from langchain.prompts import PromptTemplate
4
from pydantic import BaseModel, Field, validator

3. Define Your Data Structure: Create a Pydantic model to define the structure of a joke, including setup and punchline, with a validator to ensure the setup is properly formed.

1
class Joke(BaseModel):
2
    setup: str = Field(description="question to set up a joke")
3
    punchline: str = Field(description="answer to resolve the joke")
4
5
    @validator("setup")
6
    def question_ends_with_question_mark(cls, field):
7
        if not field.endswith("?"):
8
            raise ValueError("Badly formed question!")
9
        return field

4. Set Up an Output Parser: Utilize LangChain’s PydanticOutputParser to enforce the data structure defined by the Pydantic model on the output.

1
parser = PydanticOutputParser(pydantic_object=Joke)

5. Create and Format a Prompt: Use LangChain’s PromptTemplate to structure your query, incorporating instructions for output formatting derived from the parser.

1
prompt = PromptTemplate(
2
    template="Answer the user query.\n{format_instructions}\n{query}\n",
3
    input_variables=["query"],
4
    partial_variables={"format_instructions": parser.get_format_instructions()},
5
)

6. Generate and Parse Output: Call PredictionGuard’s text completion model “Neural-Chat-7B” to generate an output based on the formatted prompt, then parse the output into the Pydantic model. Handle exceptions for parsing errors.

1
result = pg.Completion.create(
2
    model="Neural-Chat-7B",
3
    prompt=prompt.format(query="Tell me a joke."),
4
    max_tokens=200,
5
    temperature=0.1
6
)
7
8
try:
9
    joke = Joke.parse_raw(result['choices'][0]['text'])
10
    print(joke.setup)
11
    print(joke.punchline)
12
except Exception as e:
13
    print(f"Error parsing joke: {e}")

Let’s Test This Out

After running the implementation with the query “Tell me a joke.”, the structured output generated by PredictionGuard, parsed and validated by our Pydantic model, looks like this:

1
{
2
  "setup": "Why did the tomato turn red?",
3
  "punchline": "Because it saw the salad dressing."
4
}

Output Explanation:

• Setup: “Why did the tomato turn red?” - This question successfully passes the validator check, ending with a question mark as required.
• Punchline: “Because it saw the salad dressing.” - Provides a humorous answer to the setup question.

This example demonstrates the effective structuring and validation of output data, ensuring that the generated joke adheres to our defined format.

Conclusion:

This approach allows for the flexible and structured generation of outputs, leveraging PredictionGuard’s capabilities alongside Pydantic’s validation features.