数据标注工具
编辑 | blame | 历史 | 原始文档

TimeSeries Tag – Developer Guide

This document explains how the TimeSeries component in Label Studio is built, how its synchronisation with other media works (video / audio), and how the playback cursor logic is implemented.

The goal is to give both humans and LLMs enough context to confidently extend or debug the code.

1. High-level architecture

TimeSeries (MST model + React container)
├── Channels (one per data column) – individual SVG plots
│   ├── Hover tracker (grey)          – shows XY under mouse
│   └── Playhead line   (customizable) – current playback position
├── MultiChannel (grouped visualization) – multiple channels in single plot
│   ├── Channel Legend               – interactive channel visibility controls
│   ├── TimeSeriesVisualizer         – unified D3 rendering component
│   └── Automatic color palette     – consistent channel colors
├── Overview (small plot at bottom)   – brush for zoom / pan
│   └── Playhead line   (customizable)
└── Region brushes                    – user labelled time ranges
  • Model – TimeSeriesModel (MobX-state-tree). Holds data, view state and actions. Mixin order:
    SyncableMixin → ObjectBase → PersistentStateMixin → AnnotationMixin → Model.
  • View – HtxTimeSeriesViewRTS (React) renders overview + channel children.
  • Channels – legacy rendering via ChannelD3 (being replaced by TimeSeriesVisualizer).
  • MultiChannel – MultiChannelModel + HtxMultiChannel component that groups channels together.
  • TimeSeriesVisualizer – unified D3 rendering component that replaces ChannelD3 for both single and multi-channel views.
  • ChannelLegend – interactive legend with visibility controls and hover highlighting.
  • Overview – rendered inside the same file; supplies brush that changes brushRange.

Important reactive fields

Field Type Meaning
brushRange [number, number] Visible time window (native units)
cursorTime number \| null Current playhead position (native)
seekTo number \| null One–shot instruction for overview to centre view
scale number Cached zoom factor (forces rerender)
canvasWidth number Cached width in px for correct math
isPlaying & co. … Playback loop state
cursorcolor string Hex/colour string for playhead (default: --color-neutral-inverted-surface)
suppressSync boolean Temporarily disable sync events (during overview drag)

Native units = ms when timeformat is a date, otherwise raw numeric indices/seconds.

2. MultiChannel functionality

The MultiChannel tag enables grouping multiple data channels in a single visualization with the following features:

2.1 Channel Legend

Interactive legend component that allows users to:
* Toggle channel visibility by clicking on legend items
* Highlight channels on hover for better visual focus
* Automatically assigns colors from a predefined palette

2.2 Color Palette System

  • Automatic color assignment based on channel index
  • Colors from design system: grape, mango, kale, persimmon, sand, kiwi, canteloupe, fig, plum, blueberry
  • Optimized for contrast and accessibility
  • Located in palette.js with getChannelColor(index) function

2.3 TimeSeriesVisualizer

Unified D3-based rendering component that replaces the legacy ChannelD3 approach:
* Supports both single channel and multichannel visualizations
* Handles brush interactions for region creation
* Manages playhead cursor positioning
* Provides consistent rendering logic across MultiChannel and Channel
* Eliminates code duplication between single and multi-channel rendering

2.4 Usage Example

<TimeSeries name="ts" value="$timeseries" cursorColor="#ff0000">
  <MultiChannel height="300" showAxis={true}>
    <Channel column="velocity" legend="Velocity" units="m/s"/>
    <Channel column="acceleration" legend="Acceleration" units="m/s²"/>
  </MultiChannel>
</TimeSeries>

3. Synchronisation between object tags

The SyncableMixin provides a small intra-tab message bus.

  • Each tag may specify sync="groupA". All tags with the same name join one SyncManager.
  • Supported events: play, pause, seek, speed.
  • A 100 ms SYNC_WINDOW prevents infinite feedback loops – the originator is "locked" and only its events propagate during that window.
  • TimeSeries registers handlers for seek / play / pause when FF FF_TIMESERIES_SYNC is on.
  • Outgoing events are emitted via syncSend:
  • Overview brush drag → emitSeekSync() (fires on updateTR).
  • Manual click on main plot → handled in handleMainAreaClick.
  • Note: Overview dragging temporarily sets suppressSync = true to prevent cursor jumps.
  • Incoming events call _handleSeek, _handlePlay, _handlePause which in turn
  • move cursor (cursorTime),
  • optionally restart playback loop,
  • update view by calling _updateViewForTime only if needed.

Audio tags honour mute logic so that only one sound is audible.

4. Playback cursor implementation

4.1 Data flow

  1. Source of truth – cursorTime in the model (native units).
  2. cursorTime is written by:
  • Playback loop (playbackLoop),
  • _handleSeek (remote seek),
  • _updateViewForTime (local scroll),
  • Manual click (setCursor) when inside current view.
  1. Channels and Overview subscribe through React useEffects and D3 – whenever item.cursorTime changes they reposition their SVG playhead line.

4.2 Channel playhead (TimeSeriesVisualizer / legacy ChannelD3)

this.playhead = this.main.append('line')
  .attr('stroke', parent.cursorcolor)
  .attr('x1/x2', this.x(cursorTime))

updatePlayhead(time) hides the line if the time is outside current x.domain() or null.

Note: In modern MultiChannel components, this logic is handled by TimeSeriesVisualizer which provides unified cursor management across all channels.

4.3 Overview playhead

Identical logic but uses scaled brush coordinate.

4.4 Click without recentering

  • If click time is inside brushRange we call setCursor(time) – only cursor moves.
  • If outside – _updateViewForTime recentres view and may emit sync.

4.5 Click during playback synchronization

  • Problem solved: Previously, clicking during video playback would seek the video but TimeSeries cursor would continue from its old position.
  • Solution: When isPlaying is true and user clicks, the restartPlaybackFromTime(time) action:
  • Cancels current animation frame
  • Converts clicked time to relative seconds for playback state
  • Updates playStartPosition and playStartTime to the clicked position
  • Restarts the playback loop from the new position
  • Result: Both video and TimeSeries cursor jump to clicked position and continue playing synchronously.

4.6 Shared click handling logic

  • handleTimeSeriesMainAreaClick() in helpers.js provides unified click handling for both:
  • TimeSeries main component (TimeSeries.jsx)
  • TimeSeriesVisualizer component (TimeSeriesVisualizer.jsx for MultiChannel)
  • Eliminates code duplication (~80 lines reduced to single function call)
  • Handles coordinate calculation, boundary checks, cursor updates, and playback restart logic
  • Uses proper MobX-state-tree actions to avoid protection errors

4.7 Overview dragging behavior

  • When user starts dragging the overview brush (brushstarted), suppressSync is set to true.
  • This prevents emitSeekSync() from firing during the drag, keeping cursor fixed.
  • On brushended, suppressSync is reset to false (with 0ms delay to let range settle).
  • Result: dragging overview changes viewport without moving playhead or syncing with video/audio.

5. Important actions

Action Purpose
updateTR(range) Central method to change visible window; triggers rerender + optional sync
scrollToRegion(r) Ensure a labelled region is visible (may expand brush)
setCursorAndSeek(t) Update both cursorTime & seekTo (internal only)
setCursor(t) Update only cursorTime (no brush movement)
restartPlaybackFromTime(t) Restart playback loop from specific time (handles click during playback)
_updateViewForTime(t) Convert time → pixels & adjust brush + cursor
setSuppressSync(flag) Temporarily disable sync emissions

6. Helper functions

The helpers.js file contains shared utility functions used across TimeSeries components:

6.1 Click handling

  • handleTimeSeriesMainAreaClick(event, timeSeriesItem, mainDisplayElement) – Unified click handling logic used by both:
  • TimeSeries main component
  • TimeSeriesVisualizer (for MultiChannel)
  • Handles coordinate calculation, boundary checks, view updates, and playback synchronization
  • Automatically calls appropriate actions (setCursor, _updateViewForTime, restartPlaybackFromTime)
  • Ensures MobX-state-tree compliance by using proper actions instead of direct property modification

6.2 Other utilities

  • sparseValues() – Data thinning for performance with large datasets
  • getRegionColor() – Color calculation for labeled regions
  • formatTrackerTime() – Time formatting for tracker display
  • checkD3EventLoop() – D3 event loop prevention

7. Adding new functionality

  • New attributes – extend TagAttrs with MST types.optional, then read item.<attr> in views.
  • New MultiChannel features – modify MultiChannelModel actions or extend ChannelLegend component.
  • Color customization – extend palette.js or add channel-specific color attributes.
  • Styling – prefer Tailwind utility classes or inline SVG attributes.
  • Performance – huge datasets are thinned with sparseValues(); thresholds controlled by zoomStep.
  • Visualization – extend TimeSeriesVisualizer for custom D3 rendering behaviors.

8. Glossary

Term Meaning
Native units Raw numeric time values used in dataset (ms for dates, seconds/indices otherwise)
Relative seconds Seconds offset from dataset start – format used in sync messages
Brush D3 brush in Overview controlling visible window (brushRange)
MultiChannel Component that groups multiple data channels in a single visualization
Channel Legend Interactive legend component for controlling channel visibility and highlighting
TimeSeriesVisualizer Unified D3-based rendering component that replaces legacy ChannelD3
ChannelD3 Legacy D3 rendering component (being replaced by TimeSeriesVisualizer)
Color Palette Predefined set of colors automatically assigned to channels
Playback Sync Real-time synchronization between video/audio playback and TimeSeries cursor position
Click Handling Unified logic for processing user clicks on TimeSeries visualizations