---
title: YOLO ML Backend for Label Studio
type: guide
tier: all
order: 50
hide_menu: true
hide_frontmatter_title: true
meta_title: YOLO ML Backend for Label Studio
meta_description: Tutorial on how to use an example ML backend for Label Studio with YOLO
categories:
- Computer Vision
- Object Detection
- Image Segmentation
- YOLO
image: "/guide/ml_tutorials/yolo.png"
---
# YOLO ML backend for Label Studio
The YOLO ML backend for Label Studio is designed to integrate advanced object detection,
segmentation, classification, and video object tracking capabilities directly into Label Studio.
This integration allows you to leverage powerful YOLOv8 models for various machine learning tasks,
making it easier to annotate large datasets and ensure high-quality predictions.
VIDEO
**Supported Features**
| YOLO Task Name | LS Control Tag | Prediction Supported | LS Import Supported | LS Export Supported |
|--------------------------------------------------------------|--------------------------------------|----------------------|---------------------|---------------------|
| Object Detection | `` | ✅ | YOLO, COCO | YOLO, COCO |
| Oriented Bounding Boxes (OBB) | `` | ✅ | YOLO | YOLO |
| Image Instance Segmentation: Polygons | `` | ✅ | COCO | YOLO, COCO |
| Image Semantic Segmentation: Masks | `` | ❌ | Native | Native |
| Image Classification | `` | ✅ | Native | Native |
| Pose Detection | `` | ✅ | Native | Native |
| Video Object Tracking | `` | ✅ | Native | Native |
| [Video Temporal Classification](https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/README_TIMELINE_LABELS.md) | `` | ✅ | Native | Native |
* **LS Control Tag**: Label Studio [control tag](https://labelstud.io/tags/) from the labeling configuration.
* **LS Import Supported**: Indicates whether Label Studio supports Import from YOLO format to Label Studio (using the LS converter).
* **LS Export Supported**: Indicates whether Label Studio supports Export from Label Studio to YOLO format (the **Export** button on the Data Manager and using the LS converter).
* **Native**: Native means that only native Label Studio JSON format is supported.
## Before you begin
Before you begin, you need to install the [Label Studio ML backend](https://github.com/HumanSignal/label-studio-ml-backend?tab=readme-ov-file#quickstart).
This tutorial uses the [YOLO example](https://github.com/HumanSignal/label-studio-ml-backend/tree/master/label_studio_ml/examples/yolo).
## Quick start
1. Add `LABEL_STUDIO_URL` and `LABEL_STUDIO_API_KEY` to the `docker-compose.yml` file.
These variables should point to your Label Studio instance and its API key, respectively.
For more information about finding your Label Studio API key, [see our documentation](https://labelstud.io/guide/user_account#Access-token).
2. Run docker compose
```bash
docker-compose up --build
```
3. Open Label Studio and create a new project with the following labeling config:
```xml
```
4. Then from the **Model** page in the project settings, [connect the model](https://labelstud.io/guide/ml#Connect-the-model-to-Label-Studio). The default URL is `http://localhost:9090`.
5. Add images or video (depending on tasks you are going to solve) to Label Studio.
6. Open any task in the Data Manager and see the predictions from the YOLO model.
## Labeling configurations
### Supported object & control tags
**Object tags**
- `` - [Image to annotate](https://labelstud.io/tags/image)
- `` - [Video to annotate](https://labelstud.io/tags/video)
**Control tags**
- `` - [Classification](https://labelstud.io/tags/choices); image classification task
- `` - [Bounding boxes](https://labelstud.io/tags/rectanglelabels); object detection task
- `` - [Polygons](https://labelstud.io/tags/polygonlables); segmentation task
- `` - [Video bounding boxes](https://labelstud.io/tags/videorectangle); object tracking task for videos
- `` - [Key points](https://labelstud.io/tags/keypointlabels); pose detection task
- `` - [Temporal labels for videos](https://labelstud.io/tags/timelinelabels); multi-label temporal classification task for videos
**How to skip the control tag?**
If you don't want to use the ML backend for some control tags,
you can force skipping by adding the `model_skip="true"` attribute to the control tag:
```xml
```
### Mixed object and control tags
You can mix different object and control tags in one project.
The YOLO model will detect all known control tags and make predictions for them.
For example:
```xml
```
In this example, both `RectangleLabels` and `Choices` will be detected and predicted by the YOLO model.
You can also use different YOLO models for the same task to compare them visually:
```xml
```
### Label and choice mapping
```mermaid
graph TD
A[Label Studio :: Labeling Config :: Labels or Choices] <--> B[ML Model :: Names]
```
If you use a common YOLO model, you have to add mapping between your labels and the ML model labels.
By default, the YOLO ML backend will use the same (or lowercased) names as you specified in the `value` attribute.
In this example the label "Jeep" will be mapped to "jeep" in the ML model:
```xml
Tip: How to find all YOLO model names?
Tip: How to map my labels to YOLO names using an LLM?
3. **Mapping Instructions:**
- Map the labels from the Label Studio config to the closest matching ML model labels as follows:
1. Use the `value` attribute from each `` tag to identify the label.
2. Find all similar and relevant labels from the ML model corresponding to each `` label.
3. Add a `predicted_values=","` attribute inside each `` tag using only labels from the ML model.
4. **Output:**
- Provide the final labeling config with the `predicted_values` attribute added, using all relevant labels from the ML model, without any explanations.
```
## Supported YOLO Versions: YOLOv5, YOLO11, and others
- **YOLOv5**: This model is supported for object detection tasks. Make sure to specify `model_path="yolov5nu.pt"` (don't forget the **`u`**) to use the YOLOv5 model.
- **YOLOv8**: These models have been successfully tested with the current ML backend. Check the full list of [v8 models here](https://docs.ultralytics.com/models/yolov8/#supported-tasks-and-modes).
- **YOLO11**: YOLO11 models have also been successfully tested with this ML backend. Check the full list of [v11 models here](https://docs.ultralytics.com/models/yolo11/#supported-tasks-and-modes).
**Warning 1**: You must upgrade the `ultralytics` package to the latest version (`pip install -U ultralytics`) or rebuild the ML backend Docker from scratch (`docker-compose build --no-cache`) if you used it before the latest Ultralytics update on **Monday, September 30, 2024**.
**Warning 2**: YOLO11 models do not use the `v` in their naming convention. For example, use **`yolo11n.pt`** instead of `yolov11n.pt`, unlike the naming convention in YOLOv8.
- For a full list of supported YOLO versions and models, refer to the Ultralytics documentation:
[Ultralytics Supported YOLO Models](https://docs.ultralytics.com/models/)
**Note**: Some YOLO models listed in the Ultralytics documentation have not been tested with this ML backend, but they might still work.
## Your own custom YOLO models
You can load your own YOLO labels using the following steps:
1. Mount your model as `/app/models/.pt` inside of your Docker.
2. Add `model_path=".pt"` to the control tag in the labeling configuration, e.g.:
```xml
```
Step by step guide : Using your own custom YOLOv8 model`, ``, ``, etc.).
**Example for Object Detection with ``:**
```xml
```
**Important Notes:**
- **`model_path`**: The **`model_path`** should match the filename of your custom model and must be located in the `/app/models` directory inside the Docker container.
- **Class names mapping**: Ensure that the `value` attributes in your `` tags correspond to the **class names** your model predicts. If they differ, use the `predicted_values` attribute to map them. Example:
```xml
## Training
The current Label Studio ML backend doesn't support training YOLO models. You have to do it manually on your side.
Or you can contribute to this repository and add training support for this ML backend.
`
YOLO provides a classification model and Label Studio supports this with the `Choices` control tag.
More info: https://docs.ultralytics.com/tasks/classify/
```
### Parameters
| Parameter | Type | Default | Description |
|-------------------------|--------|---------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#Your-own-custom-YOLO-models). |
| `choice` | string | single | Possible values: `single`, `single-radio`, `multiple`. If you use `choice="single"` (default) you can select only one label. The ML backend will return the label with the highest confidence using argmax strategy. If you use `choice="multiple"` you can select multiple labels. The ML backend will return all labels with confidence above the `model_score_threshold`. |
For example:
```xml
```
### Default model
`yolov8n-cls.pt` is the default classification model.
```
### Parameters
| Parameter | Type | Default | Description |
|-------------------------|--------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#Your-own-custom-YOLO-models). |
| `model_obb` | bool | False | Enables Oriented Bounding Boxes (OBB) mode. Typically it uses `*-obb.pt` yolo models. |
For example:
```xml
```
### Default model
`yolov8m.pt` is the default object detection model.
`yolov8n-obb.pt` is the default OBB object detection model.
### Oriented Bounding Boxes (YOLO OBB)
Oriented (rotated) bounding boxes will be generated automatically if you use an OBB model.
Specify `model_obb="true"` in the `RectangleLabels` tag to enable this mode:
```xml
```
More info: https://docs.ultralytics.com/tasks/obb/
```
### Parameters
| Parameter | Type | Default | Description |
|-------------------------|--------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#Your-own-custom-YOLO-models). |
For example:
```xml
```
### Default model
`yolov8n-seg.pt` is the default segmentation model.
```
### Parameters
| Parameter | Type | Default | Description |
|-------------------------|--------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#Your-own-custom-YOLO-models). |
| `model_score_threshold` | float | 0.5 | Sets the minimum confidence threshold for bounding box detections. Keypoints that are related to the detected bbox with a confidence below this threshold will be disregarded. |
| `model_point_threshold` | float | 0.0 | Minimum confidence threshold for keypoints. Keypoints with confidence below this value will be ignored. |
| `model_add_bboxes` | bool | True | Adds bounding boxes for detected keypoints. All keypoints will be grouped by parent bounding boxes on the region panel. See details in the tip below. |
| `model_point_size` | float | 1 | Size of the keypoints in pixels. Just a visual parameter. |
| `model_index` | int | None | Index of the keypoint in the YOLO model output starting from 0. It's used in `Label` tags only to build mapping between a Label and an output point. |
For example:
```xml
```
### Default model
`yolov8n-pose.pt` is the default keypoint detection model.
### Grouping keypoints with bounding boxes
When using keypoint detection, the ML backend groups keypoints by the bounding box (bbox) associated with each detected person or object.
You will see the grouping under the **Regions** panel on the right side of Label Studio. Note that you can drag and drop region items as necessary.
The bounding boxes are added to the prediction results by default. You can enable or disable this behavior by setting `model_add_bboxes`:
```xml
```
Tip: How to only keep keypoints and discard bounding boxes?
To enable both keypoint detection and bounding box detection in the same task, you have to
1. set `model_add_bboxes="false"` in the `KeyPointLabels` tag,
2. remove `RectangleLabels` tag.
You can use this labeling configuration to get rid of bounding boxes and keep only keypoints:
```xml
```
```
### Trackers
https://docs.ultralytics.com/modes/track/?h=track#tracker-selection
The best tracker to use with Ultralytics YOLO depends on your specific needs.
The default tracker is [BoT-SORT](https://github.com/NirAharon/BoT-SORT), which is generally well-suited for most scenarios.
However, if you're looking for an alternative with different strengths,
[ByteTrack](https://github.com/ifzhang/ByteTrack) is another good choice that you can easily configure.
ByteTrack is known for its high performance in multi-object tracking,
especially in situations with varying object appearances and reappearances.
Both trackers can be customized using YAML configuration files to fit your specific use cases.
You can specify the tracker in the control tag:
* ``
* ``
### Parameters for bounding boxes
The tracker works with the object detection model (bounding boxes).
The first step is to detect bounding boxes, the second step is to track them (find the same boxes among frames).
These parameters are related to the first step - bounding box detection.
Read more about these parameters:
https://docs.ultralytics.com/modes/track/?h=track#tracking-arguments
| Parameter | Type | Default | Description |
|-----------------|--------|-----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `model_conf` | float | 0.25 | Sets the minimum confidence threshold for detections. Objects detected with confidence below this threshold will be disregarded. Adjusting this value can help reduce false positives. |
| `model_iou` | float | 0.7 | Intersection Over Union (IoU) threshold for Non-Maximum Suppression (NMS). Lower values result in fewer detections by eliminating overlapping boxes, useful for reducing duplicates. |
| `model_tracker` | string | `botsort` | Sets the tracker to use for multi-object tracking. Options include `botsort`, `bytetrack`, or a custom YAML file. |
| `model_path` | string | None | Path to the custom YOLO model. See more in the section [Your own custom YOLO models](#Your-own-custom-YOLO-models). |
For example:
```xml
` control tag](https://labelstud.io/tags/timelinelabels) in Label Studio.
There are two modes available:
- **Simple:** In the simple mode, the model uses pre-trained YOLO classes to generate predictions without additional training.
- **Trainable:** In the [trainable mode](https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/README_TIMELINE_LABELS.md), the model can be trained on custom labels and annotations submitted in Label Studio using few-shot learning as training is performed on a small number of annotations.
VIDEO
### Labeling config
```xml
```
### Model training
For more details on using the `TimelineLabels` ML backend, including training the model
and adjusting neural network classifier parameters, please refer to
**[README_TIMELINE_LABELS.md](https://github.com/HumanSignal/label-studio-ml-backend/blob/master/label_studio_ml/examples/yolo/README_TIMELINE_LABELS.md)**.
### Default model
`yolov8n-cls.pt` is the default classification model for simple mode.