---
title: Interactive LLM labeling with GPT
type: guide
tier: all
order: 5
hide_menu: true
hide_frontmatter_title: true
meta_title: Interactive LLM labeling with OpenAI, Azure, or Ollama
meta_description: Label Studio tutorial for interactive LLM labeling with OpenAI, Azure, or Ollama
categories:
- Generative AI
- Large Language Model
- OpenAI
- Azure
- Ollama
- ChatGPT
This example server connects Label Studio to OpenAI, Ollama,
or Azure API to interact with GPT chat models (
gpt-3.5-turbo, gpt-4, etc.).
The interactive flow allows you to perform the following scenarios:
Check the Generative AI templates section for more examples.
Before you begin, you must install the Label Studio ML backend.
This tutorial uses the llm_interactive example.
Build and start the Machine Learning backend on http://localhost:9090 bash docker-compose up
Check if it works:
bash $ curl http://localhost:9090/health {"status":"UP"}
Open a Label Studio project and go to Settings > Model. Connect the model, specifying http://localhost:9090 as the URL.
Ensure the Interactive preannotations toggle is enabled and click Validate and Save.
4. The project config should be compatible with the ML backend. This ML backend can support various input data formats
like plain text, hypertext, images, and structured dialogs. To ensure the project config is compatible, follow these
rules:
<TextArea> tag to be used as a prompt input. To specify which <TextArea> tag to use, set the PROMPT_PREFIX environment variable.<TextArea name="prompt" ...>, then you would specify PROMPT_PREFIX=prompt.<Text>, <Image>, <HyperText>, <Paragraphs>.<Choices> tag.<Choices name="choices" ...>.If you want to set the default prompt to be shown before the user input, you can set the DEFAULT_PROMPT environment variable. For example, DEFAULT_PROMPT="Classify this text as sarcastic or not. Text: {text}, Labels: {labels}" or DEFAULT_PROMPT=/path/to/prompt.txt.
Note that the default prompt isn't supported with USE_INTERNAL_PROMPT_TEMPLATE=1 mode, so you will need to set USE_INTERNAL_PROMPT_TEMPLATE=0 to use default prompt. You can use the fields from task['data'] in the prompt template, as well as special {labels} field to show the list of available labels.
Shift+Enter. The LLM response will be generated and displayed in
<View>
<Style>
.lsf-main-content.lsf-requesting .prompt::before { content: ' loading...'; color: #808080; }
.text-container {
background-color: white;
border-radius: 10px;
box-shadow: 0px 4px 6px rgba(0, 0, 0, 0.1);
padding: 20px;
font-family: 'Courier New', monospace;
line-height: 1.6;
font-size: 16px;
}
</Style>
<Header value="Context:"/>
<View className="text-container">
<Text name="context" value="$text"/>
</View>
<Header value="Prompt:"/>
<View className="prompt">
<TextArea name="prompt"
toName="context"
rows="4"
editable="true"
maxSubmissions="1"
showSubmitButton="false"
placeholder="Type your prompt here then Shift+Enter..."
/>
</View>
<Header value="Response:"/>
<TextArea name="response"
toName="context"
rows="4"
editable="true"
maxSubmissions="1"
showSubmitButton="false"
smart="false"
placeholder="Generated response will appear here..."
/>
<Header value="Evaluate model response using one or more metrics:"/>
<Taxonomy name="evals" toName="context" leafsOnly="true" showFullPath="true" pathSeparator=": ">
<Choice value="Relevance">
<Choice value="Relevant"/>
<Choice value="Irrelevant"/>
</Choice>
<Choice value="Correctness">
<Choice value="Correct"/>
<Choice value="Incorrect"/>
<Choice value="Contains hallucinations"/>
</Choice>
<Choice value="Bias">
<Choice value="Gender" hint="Discrimination based on a person's gender."/>
<Choice value="Political"
hint="A preference for or prejudice against a particular political party, ideology, or set of beliefs."/>
<Choice value="Racial/Ethnic"
hint="Prejudice or discrimination based on a person's race, ethnicity, or national origin."/>
<Choice value="Geographical"
hint=" Prejudices or preferential treatment based on where a person lives or comes from."/>
</Choice>
<Choice value="Toxicity">
<Choice value="Personal Attacks"
hint="Insults or hostile comments aimed at degrading the individual rather than addressing their ideas."/>
<Choice value="Mockery" hint="Sarcasm or ridicule used to belittle someone."/>
<Choice value="Hate"
hint="Expressions of intense dislike or disgust, often targeting someone's identity or beliefs."/>
<Choice value="Dismissive Statements"
hint="Comments that invalidate the person's viewpoint or shut down discussion without engaging constructively."/>
<Choice value="Threats or Intimidation"
hint="Statements intending to frighten, control, or harm someone, either physically or emotionally."/>
<Choice value="Profanity"
hint="Use of strong or offensive language that may be considered disrespectful or vulgar."/>
<Choice value="Sexual Harassment" hint="Unwelcome or inappropriate sexual remarks or physical advances."/>
</Choice>
</Taxonomy>
<Header value="Overall response quality:"/>
<Rating name="rating" toName="context"/>
</View>
<View>
<Style>
.lsf-main-content.lsf-requesting .prompt::before { content: ' loading...'; color: #808080; }
</Style>
<!-- Input data -->
<Text name="text" value="$text"/>
<!-- Prompt input -->
<TextArea name="prompt" toName="text" editable="true" rows="2" maxSubmissions="1" showSubmitButton="false"/>
<!-- LLM response output -->
<TextArea name="response" toName="text" smart="false" editable="true"/>
<View style="box-shadow: 2px 2px 5px #999;
padding: 20px; margin-top: 2em;
border-radius: 5px;">
<Choices name="sentiment" toName="text"
choice="multiple" showInLine="true">
<Choice value="Sarcastic"/>
<Choice value="Not Sarcastic"/>
</Choices>
</View>
</View>
Example data input:
{
"text": "I love it when my computer crashes"
}
Representing ChatGPT-style interface with <Paragraphs> tag:
<View>
<Style>
.lsf-main-content.lsf-requesting .prompt::before { content: ' loading...'; color: #808080; }
</Style>
<Paragraphs name="chat" value="$dialogue" layout="dialogue" textKey="content" nameKey="role"/>
<Header value="User prompt:"/>
<View className="prompt">
<TextArea name="prompt" toName="chat" rows="4" editable="true" maxSubmissions="1" showSubmitButton="false"/>
</View>
<Header value="Bot answer:"/>
<TextArea name="response" toName="chat" rows="4" editable="true" smart="false" maxSubmissions="1" showSubmitButton="false"/>
</View>
Example data input:
{
"dialogue": [
{
"role": "user",
"content": "What is the capital of France?"
},
{
"role": "assistant",
"content": "The capital of France is Paris."
},
{
"role": "user",
"content": "Tell me a joke."
}
]
}
<View>
<Style>
.lsf-main-content.lsf-requesting .prompt::before { content: ' loading...'; color: #808080; }
.container {
display: flex;
justify-content: space-between; /* Align children with space in between */
align-items: flex-start; /* Align children at the start of the cross axis */
}
.image {
/* Adjust these values according to the size of your image */
width: 600px; /* Example width for the image */
height: auto; /* Maintain aspect ratio */
/* Removed position: sticky, float: right, and margin-right */
}
.blocks {
width: calc(100% - 220px); /* Adjust the calculation to account for the image width and some margin */
height: 600px; /* Set the height for the scrolling area */
overflow-y: scroll; /* Allow vertical scrolling */
}
.block {
background-color: #f0f0f0; /* Sample background color for each block */
padding: 20px; /* Spacing inside each block */
margin-bottom: 10px; /* Spacing between blocks */
}
</Style>
<View className="container">
<View className="blocks">
<View className="block">
<Header value="Prompt:"/>
<View className="prompt">
<TextArea name="prompt" toName="image"
showSubmitButton="false"
editable="true"
rows="3"
required="true"/>
</View>
<Header value="Classification:"/>
<Choices name="category" toName="image" smart="false" layout="select">
<Choice value="Groceries"/>
<Choice value="Dining/Restaurants"/>
<Choice value="Clothing/Apparel"/>
<Choice value="Electronics"/>
<Choice value="Home Improvement"/>
<Choice value="Health/Pharmacy"/>
<Choice value="Gasoline/Fuel"/>
<Choice value="Transportation/Travel"/>
<Choice value="Entertainment/Leisure"/>
<Choice value="Utilities/Bills"/>
<Choice value="Insurance"/>
<Choice value="Gifts/Donations"/>
<Choice value="Personal Care"/>
<Choice value="Education/Books"/>
<Choice value="Professional Services"/>
<Choice value="Membership/Subscriptions"/>
<Choice value="Taxes"/>
<Choice value="Vehicle Maintenance/Repairs"/>
<Choice value="Pet Care"/>
<Choice value="Home Furnishings/Decor"/>
<Choice value="Other"/>
</Choices>
</View>
<View className="block">
<Header value="Summary:"/>
<TextArea name="summarization-response" toName="image"
showSubmitButton="false"
maxSubmissions="0"
editable="true"
smart="false"
rows="3"
/>
</View>
</View>
<View className="image">
<Image name="image" value="$image"/>
</View>
</View>
</View>
Example data input:
{
"image": "https://sandbox2-test-bucket.s3.amazonaws.com/receipts/113494_page1.png"
}
When deploying the server, you can specify the following parameters as environment variables:
DEFAULT_PROMPT: Define a default prompt to be shown before the user input. For example, DEFAULT_PROMPT="Classify this text as sarcastic or not. Text: {text}, Labels: {labels}" or DEFAULT_PROMPT=/path/to/prompt.txt.
Note that USE_INTERNAL_PROMPT_TEMPLATE should be set to 0 if you are setting a default prompt.
PROMPT_PREFIX (default: prompt): An identifier for the prompt input field. For example, if you set
PROMPT_PREFIX to my-prompt, the following input field will be used for the
prompt: <TextArea name="my-prompt" ...>.
USE_INTERNAL_PROMPT_TEMPLATE (default: 1). If set to 1, the server will use the internal prompt template. If set
to
0, the server will use the prompt template provided in the input prompt.
PROMPT_TEMPLATE (default: "Source Text: {text}\n\nTask Directive: {prompt}"): The prompt template to use:
If USE_INTERNAL_PROMPT_TEMPLATE is set to 1, the server will use
the default internal prompt template.
If USE_INTERNAL_PROMPT_TEMPLATE is set to 0, the server will use the prompt template provided
in the input prompt (i.e. the user input from <TextArea name="my-prompt" ...>).
In the later case, the user has to provide the placeholders that match input task fields. For example, if the user wants to use the input_text and instruction field from the input task {"input_text": "user text", "instruction": "user instruction"}, the user has to provide the prompt template like this: "Source Text: {input_text}, Custom instruction : {instruction}".
OPENAI_MODEL (default: gpt-3.5-turbo) : The OpenAI model to use.
OPENAI_PROVIDER (available options: openai, azure, ollama, default - openai) : The OpenAI provider to use.
TEMPERATURE (default: 0.7): The temperature to use for the model.
NUM_RESPONSES (default: 1): The number of responses to generate in <TextArea> output fields. Useful if you want
to generate multiple responses and let the user rank the best one.
OPENAI_API_KEY: The OpenAI or Azure API key to use. Must be set before deploying the server.
If you are using Azure as your OpenAI provider (OPENAI_PROVIDER=azure), you need to specify the following environment
variables:
AZURE_RESOURCE_ENDPOINT: This is the endpoint for your Azure resource. It should be set to the appropriate value
based on your Azure setup.
AZURE_DEPLOYMENT_NAME: This is the name of your Azure deployment. It should match the name you've given to your
deployment in Azure.
AZURE_API_VERSION: This is the version of the Azure API you are using. The default value is 2023-05-15.
If you are using Ollama as your LLM provider (OPENAI_PROVIDER=ollama), you need to specify the following environment variables:
OPENAI_MODEL : The Ollama model to use, for example llama3.
OLLAMA_ENDPOINT: This is the endpoint for your Ollama endpoint. It should be set to the appropriate value based on your setup. If you are running it locally, then it can typically be reached on http://host.docker.internal:11434/v1/