8 track converters read a tracker's output into one of
4 registered schemas, and 3 label
converters read manual annotations. A converter declares the schema it emits,
in one place, and that declaration is what validation and the tracks index
both read.
Every schema validates a standardized parquet table. Spatial columns are video
pixels and X/Y are the body center, except in the centimeter families,
which say so in their names. A physical unit is otherwise obtained downstream
by the scale-to-cm feature rather than stored in the table.
The same contract as trex_v2, in centimeters rather than pixels, with X/Y still the body center. Use it for archived data whose cm_per_pixel factor cannot be recovered -- notably TRex exports predating TRex 2.0.0, which scaled their output without recording by how much. Its own schema family: a feature run whose scope resolves both this and a pixel schema is refused rather than mixed. For pixel data, scale-to-cm converts to centimeters as a recorded step instead.
The tracker-neutral standard: per-frame, per-id tracks in video pixels. X/Y are the individual's body center -- for a pose-only tracker the mean of that frame's keypoints, for one that measures a centroid its own. Keypoints are optional: a centroid-only tracker emits none rather than copying X/Y into a fabricated poseX0, and the features that need keypoints refuse without them. Every spatial column is pixels; a physical unit is a feature, not a column. Derived quantities (velocity, speed, heading) are forbidden: they belong to features, where the method is chosen and recorded. group is required but may be empty.
Minimal T-Rex-like per-frame, per-id tracks with centroid/pose columns. group is required but may be empty (an optional namespace, not the canonical grouping -- use tags / run_feature(scope=Scope(entries=...)) for that).
mosaic_v1 plus what TRex genuinely measures, unscaled to pixels. TRex reports speed and heading itself, and a weighted centroid distinct from its blob centroid, so none of the derived set is forbidden here. X/Y carry the body center (TRex's #wcentroid), and TRex's own bare X/Y -- which are the head, and present only where posture was calculated -- are kept as X#head/Y#head.
An editor's saved states, kept as immutable revisions under
labels_raw/<series>/<key>/rev<N>/. A revision holds its payload, which is
what its identity and every consumer read, and exports generated from the
payload for other tools.
The saved state of a keypoint annotation set: every frame, finished or not,
every pose its objects use with their aliases and mirror pairs, and every
object with the box the annotator drew. JSON, compact, keys sorted.
Its COCO Keypoints export holds the finished frames, one category per pose,
and each object's alias, origin and box provenance under attributes.
Field
Type
Default
Constraints
Description
format
"mosaic-pose-annotations"
required
Always mosaic-pose-annotations.
version
integer
required
The format version the file was written in. This is version 1.
bbox_policy
BboxPolicy
required
How the annotator's tool derived a box the annotator did not draw.
poses
list of PoseRecord
required
Every pose an object uses, ascending by id.
frames
list of FrameRecord
required
Every frame of the set, finished or not, ascending by image.
AliasRecord
Field
Type
Default
Constraints
Description
id
integer
required
The authoring store's id; a rename keeps it.
name
string
required
What the alias is called when saved.
BboxPolicy
Field
Type
Default
Constraints
Description
method
"tight" | "isotropic" | "oriented"
"tight"
Which strategy derives the box from the keypoints when the source supplied none. Known values are tight, isotropic and oriented.
margin
number
0.1
The padding added around the tight hull of valid keypoints, as a fraction of the hull's own size. Used only when method is tight.
pad_frac_of_body
number
0.3
The padding added around the tight hull of valid keypoints, as a fraction of body length. Used when method is isotropic, and as the isotropic fallback for oriented when the head or tail keypoint is invalid, or when the head-tail distance is zero.
min_pad_px
number
20.0
The floor for the padding around the tight hull, for degenerate or overlapping keypoints. Used when method is isotropic, and as the isotropic fallback for oriented when the head or tail keypoint is invalid, or when the head-tail distance is zero. [px]
length_pad_frac
number
0.25
The padding extending the oriented rectangle beyond the head and tail, as a fraction of the head-tail distance. Used only when method is oriented.
side_pad_frac
number
0.35
The padding extending the oriented rectangle across the head-tail axis, as a fraction of the head-tail distance. Used only when method is oriented.
head_index
integer | None
null
The keypoint index of the head. Required when method is oriented. When given together with tail_index for isotropic, body length is the distance between them instead of the tight hull's diagonal.
tail_index
integer | None
null
The keypoint index of the tail. Required when method is oriented. When given together with head_index for isotropic, body length is the distance between them instead of the tight hull's diagonal.
FrameRecord
Field
Type
Default
Constraints
Description
image
string
required
The image, relative to the dataset the set was saved in; absolute when it lies outside it.
width
integer
required
Image width in pixels.
height
integer
required
Image height in pixels.
usable
boolean
required
The annotator's sign-off that the frame is complete. Only a usable frame is trained on or exported.
sequence
string | None
null
The recording the frame came from.
frame_index
integer | None
null
The frame's position in that recording.
objects
list of ObjectRecord
constructed
The objects, in the order the annotator's tool holds them.
ObjectRecord
Field
Type
Default
Constraints
Description
pose
integer
required
The id of the object's pose.
keypoints
list of (tuple of (number, number, 1 | 2) | None)
required
One entry per keypoint of the pose: [x, y, visibility] in image pixels, where visibility 1 is occluded and 2 visible, or null when the keypoint was not placed.
alias
integer | None
null
The id of the object's alias, or null.
origin
"human" | "model" | "heuristic" | None
null
Who placed it, or null when the source did not say.
source_ref
string | None
null
What produced it when not a person, such as a model run.
track_id
string | None
null
Identity across frames, when the source has one.
bbox
tuple of (number, number, number, number) | None
null
The box the annotator drew, [x, y, width, height] in pixels from the top-left. Null means derived from the keypoints under bbox_policy.
PoseRecord
Field
Type
Default
Constraints
Description
id
integer
required
The authoring store's id; a rename keeps it.
name
string
required
What the pose is called when saved.
keypoints
list of string
required
Keypoint names, in the order every object stores its points.
skeleton
list of tuple of (integer, integer)
constructed
Edges as pairs of 0-based keypoint positions, ascending.
symmetries
list of tuple of (integer, integer)
constructed
Left-right mirror pairs as 0-based positions, lower first. What a trainer that flips images swaps.