52 features are registered. Each one runs as
mosaic run --feature <name> and constructs directly in a notebook, and the
parameter tables below are the same JSON-Schema that
mosaic features describe <name> --json prints.
Every run is addressed by <version>-<hash of params, inputs and frame range>,
so the version column is what a bump moves: change it and every run of that
feature re-addresses.
In the tables below, required means the parameter has no default and must
be supplied. constructed means it is optional but its default is built by a
factory rather than being a literal, so there is no value to print.
Version 0.3 · mosaic.behavior.feature_library.approach_avoidance.ApproachAvoidance
Approach-avoidance events between every pair of individuals in a sequence.
Parameter
Type
Default
Constraints
Description
interpolation
InterpolationConfig
constructed
Interpolation settings applied to position and orientation gaps before computing distances and velocities.
sampling
SamplingConfig
constructed
Frame-rate configuration. Only fps_default is read, as the fallback frames-per-second used when the table has no fps column, or that column does not resolve to exactly one value.
velocity_units
"per_frame" | "per_second"
"per_frame"
The time unit a computed speed is expressed in before comparing it against a velocity threshold.
angle_units
"radians" | "degrees" | "auto"
"radians"
Unit of the orientation column, read when orientation_gate_cos is set. auto detects radians or degrees from the value range.
consecutive_frame_delta
number
1.0
The expected frame step between consecutive rows. A velocity is left unset where the actual step does not match it. [frames]
distance_threshold
number
200.0
> 0
Maximum inter-animal distance, in position units, for a frame to be AA-eligible.
approacher_velocity_threshold
number
5.0
>= 0
Minimum speed of the approaching animal, in position units per frame or per second as velocity_units selects.
avoider_velocity_threshold
number
5.0
>= 0
Minimum speed of the avoiding animal, in position units per frame or per second as velocity_units selects.
cos_approacher_threshold
number
0.8
>= -1, <= 1
Minimum cosine between the approacher's velocity and the direction toward its partner.
cos_avoider_threshold
number
0.5
>= -1, <= 1
Minimum cosine between the avoider's velocity and the direction away from its partner.
min_event_length
integer
10
>= 1
Length of the trailing window checked for qualifying rows. [frames]
min_event_count
integer
5
>= 1
Minimum count of qualifying rows within the min_event_length window required for a candidate row to count as part of an event. Values above min_event_length are silently clamped down to it. [frames]
orientation_gate_cos
number | None
0.8660254037844386
Unset, the orientation gate does not apply. Set, the approacher's body orientation must align with its velocity to at least this cosine, and a table with no orientation column raises. The default is cos(30 degrees).
smooth_window_sec
number | None
null
Unset, thresholds apply framewise. Set, average distance, speed and cosine metrics over this many seconds before thresholding. [s]
InterpolationConfig
Parameter
Type
Default
Constraints
Description
linear_interp_limit
integer
10
>= 1
Maximum run of consecutive missing values filled by linear interpolation. [frames]
edge_fill_limit
integer
3
>= 0
Maximum run of consecutive missing values forward-filled, then back-filled, after interpolation leaves them unresolved. [frames]
max_missing_fraction
number
0.1
>= 0.0, <= 1.0
Fraction of missing columns above which a row is dropped entirely.
SamplingConfig
Parameter
Type
Default
Constraints
Description
fps_default
number
30.0
> 0
Frame rate used when the data's fps column is absent or does not resolve to exactly one value. [fps]
smooth_win
integer
0
>= 0
Length of the moving-average window applied to smooth position and angle values before feature computation. A value of 1 or less disables smoothing. [frames]
Version 0.2 · mosaic.behavior.feature_library.attention_target.AttentionTarget
Per-frame attention target with group-membership labeling.
Parameter
Type
Default
Constraints
Description
id_group_map
object | None
null
Maps each individual id, as a string, to a group label. Unset, focal_group and target_group are always NA and attention_type is none (no facing target) or unknown (a target was picked but its group label is missing).
Version 0.1 · mosaic.behavior.feature_library.heading.HeadingFeature
Per-frame body heading derived from pose keypoints.
Parameter
Type
Default
Constraints
Description
method
"two_point" | "pca"
"two_point"
Which method derives the heading. two_point points from rear_idx toward front_idx, a determined direction. pca takes the first principal component of every keypoint, an axis whose sign is arbitrary.
front_idx
integer
0
The keypoint index of the forward landmark, such as the head, snout or rostrum. Used when method is two_point.
rear_idx
integer
1
The keypoint index of the rearward landmark, such as the tail base or abdomen. Used when method is two_point.
output_col
string
"ANGLE"
The column the heading is written to. Defaults to the library's own orientation column, the name a downstream feature reading ANGLE already expects.
Version 0.1 · mosaic.behavior.feature_library.keypoint_kinematics.KeypointKinematics
Per-keypoint smoothed speed and acceleration, and inter-keypoint distances.
Parameter
Type
Default
Constraints
Description
keypoint_names
list of string | None
null
A name for each keypoint, in the table's keypoint order. Used in the output column names, and accepted wherever a parameter refers to a keypoint. Its length must equal the table's keypoint count. Unset, keypoint k is labelled kp{k} and is referred to by its index.
keypoints
list of (integer | string) | None
null
The keypoints to report speed and acceleration for, by index or by name. An index is a keypoint's position in the table's keypoint order, which is its column number when the keypoints are numbered from 0 without a gap. Unset, every keypoint the table carries. An empty list reports none, for a run that wants only distances.
fps
number | None
null
The recording's frame rate, which sets the time step of every derivative. Unset, it is the reciprocal of the median time step per frame read off the time column, and a table whose time column cannot give one is refused. [fps]
window
integer
7
>= 3
The Savitzky-Golay window length: odd, and larger than polyorder. It is also the longest gap in a keypoint's track that is linearly interpolated before filtering. A longer gap ends the stretch the filter runs over. [frames]
polyorder
integer
2
>= 2
The order of the polynomial fitted in each window. At least 2, because acceleration is the fit's second derivative and a lower order would report it as zero everywhere.
distances
list of (tuple of (integer | string, integer | string))
constructed
Keypoint pairs, by index or by name, whose separation is reported on every row as dist__{a}__{b}. Euclidean over the table's two or three axes, and measured on the unsmoothed positions.
normalize_by
tuple of (integer | string, integer | string) | None
null
A keypoint pair whose median separation over each individual's rows in the sequence divides every distance, speed, vertical velocity and acceleration, making them independent of body size. Unset, they stay in the table's units. An individual on which the pair is never seen reports NaN throughout.
Version 0.1 · mosaic.behavior.feature_library.local_order_metrics.LocalOrderMetrics
Per-(frame, id) local order parameters (Tunstrom et al. 2013, Fig 7B/D).
Parameter
Type
Default
Constraints
Description
radius
number
required
> 0
The neighborhood radius, in the unit radius_units selects. mosaic tracks record no physical scale, and this field has no default.
radius_units
"position" | "body_scale"
"position"
How radius is interpreted. Known values are position and body_scale. position reads radius directly in X/Y units. body_scale multiplies it by the sequence's mean body-scale output, the median pairwise distance among an individual's finite keypoints rather than a fixed body length.
body_scale
BodyScaleResult | None
null
Reference to the upstream body-scale run. Required when radius_units is body_scale, and left unset when radius_units is position so no dependency is resolved.
heading_source
"auto" | "orientation" | "velocity"
"auto"
Which signal defines each individual's heading. Known values are auto, orientation and velocity. auto uses the ANGLE column when it contains a finite value, falling back to a velocity-derived heading otherwise. orientation requires a usable ANGLE column and raises otherwise. velocity always derives heading from motion.
n_shells
integer
6
>= 1
The number of concentric shells the group is divided into about its center of mass.
n_peripheral
integer
5
>= 1
How many of the most peripheral individuals define the group's outer radius, by their median distance to center.
min_neighbors
integer
1
>= 0
The minimum number of heading-valid neighbors within the disc for the local order metrics to be finite for that individual. Only neighbors with a usable heading count, excluding one with a valid position alone. Below the minimum, local polarization is NaN instead of a value computed over the focal individual alone.
subgroup_col
string | None
null
Confine every disc, centroid, group outer radius and shell to rows sharing this column's value. Unset, one neighborhood spans the whole frame. May not name a metadata or emitted column.
filter_expr
string | None
null
A pandas DataFrame.query expression selecting which rows remain. Applied after headings are computed. An excluded row does not change its successor's heading.
exclude_cols
list of string
constructed
Boolean columns whose truthy rows are dropped, applied after headings are computed like filter_expr. A name absent from the input is ignored.
BodyScaleResult
Parameter
Type
Default
Constraints
Description
feature
string
"body-scale"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
Version 0.1 · mosaic.behavior.feature_library.movement.filter_interp.MovementFilterInterpolate
Filter low-confidence points and interpolate gaps using movement.
Parameter
Type
Default
Constraints
Description
confidence_threshold
number
0.6
Confidence cutoff below which a point is set to NaN. Skipped when the input has no confidence columns.
interpolation_method
string
"linear"
Method used to interpolate over NaN gaps in position data. Known values are linear, nearest, zero, slinear, quadratic, cubic, quintic, polynomial, barycentric, krogh, pchip, spline, akima and makima.
max_gap
integer | None
null
The longest gap that is interpolated. Unset, gaps of any length are interpolated. [frames]
include_centroid
boolean
true
Include the body center (X, Y) as an additional keypoint named centroid, alongside the pose keypoints.
fps
number | None
null
Frame rate used for the movement dataset's time dimension. Unset, the time dimension uses frame numbers instead of seconds. [fps]
keypoint_names
list of string | None
null
Names for the pose keypoints, one per poseX/poseY column pair in column order. Unset, they are named keypoint_0, keypoint_1, and so on.
Version 0.1 · mosaic.behavior.feature_library.movement.smooth.MovementSmooth
Smooth trajectory positions using the movement library.
Parameter
Type
Default
Constraints
Description
method
"rolling" | "savgol"
"savgol"
The movement library filter used to smooth the trajectory. savgol applies a Savitzky-Golay filter, and rolling applies a rolling-window statistic.
window
integer
5
The size of the smoothing window, as the number of observations covered by each window. [frames]
statistic
"median" | "mean" | "max" | "min"
"median"
Which statistic the rolling filter computes over each window. Read only when method is rolling.
min_periods
integer | None
null
Minimum observations required within a window for the rolling filter to produce a value there rather than a gap. Unset, it equals window. Read only when method is rolling. [frames]
polyorder
integer
2
The order of the polynomial fit within each smoothing window. Must be less than window. Read only when method is savgol.
savgol_mode
string
"nearest"
The edge-padding mode for the Savitzky-Golay filter, forwarded to scipy.signal.savgol_filter. Read only when method is savgol. Known values are mirror, constant, nearest, wrap and interp. Any other value raises. interp also raises on a window with a NaN at its edge, under scipy 1.17 and later.
include_centroid
boolean
true
Include the X/Y body center as an additional keypoint named centroid, alongside any pose keypoints. False on a table with no pose columns raises.
fps
number | None
null
Frame rate used to build the time axis. Unset, the time axis uses frame numbers instead. [fps]
keypoint_names
list of string | None
null
Names for the pose keypoints, one per poseX/poseY column pair. Unset, defaults to keypoint_0, keypoint_1, and so on. A count that does not match the number of pose column pairs raises.
Version 0.3 · mosaic.behavior.feature_library.nn_delta_response.NearestNeighborDelta
Change in a focal individual's position, heading and speed, by its nearest neighbor.
Parameter
Type
Default
Constraints
Description
sampling
SamplingConfig
constructed
Frame rate settings. Only fps_default is read; smoothing is not applied.
speed_col
string
"SPEED"
The focal individual's speed column. Falls back to speed when absent.
nn_id_col
string
"nn_id"
The nearest neighbor's id column. Falls back to nn_fishID when absent.
nn_dx_ego_col
string
"nn_delta_x_ego"
The column that records the neighbor's x offset in the focal's ego frame.
nn_dy_ego_col
string
"nn_delta_y_ego"
The column that records the neighbor's y offset in the focal's ego frame.
nn_dx_world_col
string
"nn_delta_x"
The column that records the neighbor's x offset in the world frame, rotated into the ego frame when either nn_dx_ego_col or nn_dy_ego_col is absent.
nn_dy_world_col
string
"nn_delta_y"
The column that records the neighbor's y offset in the world frame, rotated into the ego frame when either nn_dx_ego_col or nn_dy_ego_col is absent.
focal_col
string
"Focal_fish"
The column flagging a focal individual. Used to look up whether each row's nearest neighbor is itself focal, published as neighbor_focal, and passed through to the output when present.
diff_numframes
integer
4
>= 1
How far ahead the response delta is measured, unless diff_numframes_col overrides it for a row. [frames]
diff_numframes_col
string | None
null
The column that records a per-row lag in frames, used instead of diff_numframes wherever it rounds to a finite whole number of frames of at least one. Unset, or naming a column the table does not have, every row uses diff_numframes. Expresses a speed-adjusted delay (tau_i = tau_ref * S_ref / S_group) that one scalar cannot vary per entry.
emit_backward
boolean
false
Also emit the backward window: dx_back and dy_back, computed as the position at t minus the position at t-tau. A row with no valid backward window gets NaN instead of being dropped.
wrap_angle
boolean
true
Wrap the heading difference to [-pi, pi]. When False, the raw difference is kept.
divide_dangle_by_frames
boolean
true
Divide the heading change by the number of frames it spans. When False, the raw windowed difference is kept.
scale_dangle_by_fps
boolean
true
Multiply the heading change by the sampling frame rate. Together with divide_dangle_by_frames this gives radians per second.
tag_cols
list of string
constructed
Extra columns passed through unchanged on the focal row, and looked up for the neighbor as neighbor_ when the column exists on the full table.
SamplingConfig
Parameter
Type
Default
Constraints
Description
fps_default
number
30.0
> 0
Frame rate used when the data's fps column is absent or does not resolve to exactly one value. [fps]
smooth_win
integer
0
>= 0
Length of the moving-average window applied to smooth position and angle values before feature computation. A value of 1 or less disables smoothing. [frames]
Version 0.2 · mosaic.behavior.feature_library.orientation_relative.OrientationRelativeFeature
Orientation-aware relative features between animal pairs, order-agnostic to pose points.
Parameter
Type
Default
Constraints
Description
scale
BodyScaleResult
constructed
Reference to the upstream body-scale run whose per-sequence mean scale normalizes distances into scale-free units. Where the dependency does not resolve, each row falls back to the focal individual's median pairwise keypoint distance in that frame.
nearest_k
integer
3
>= 1
Number of nearest pose-point distances to emit per pair.
quantiles
list of number
[0.25, 0.5, 0.75]
Quantiles of the scale-normalized distance from each of the other individual's keypoints to the focal individual's centroid. Rotating into the focal's body frame changes each point's direction but not this distance, which depends only on scale.
BodyScaleResult
Parameter
Type
Default
Constraints
Description
feature
string
"body-scale"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
Version 0.1 · mosaic.behavior.feature_library.pair_egocentric.PairEgocentricFeatures
Egocentric and kinematic features of every pair, from pose keypoints.
Parameter
Type
Default
Constraints
Description
interpolation
InterpolationConfig
constructed
Interpolation settings for missing pose data.
sampling
SamplingConfig
constructed
Frame rate and smoothing settings.
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
neck_idx
integer
3
Index of the neck keypoint in the pose array. Paired with tail_base_idx to compute heading. The direction runs from the tail-base to the neck, the direction the animal faces. It must name a selected keypoint, one listed in pose.pose_indices when that is set and one from 0 to pose.pose_n - 1 otherwise.
tail_base_idx
integer
6
Index of the tail-base keypoint in the pose array. Paired with neck_idx to compute heading. The direction runs from the tail-base to the neck, the direction the animal faces. It must name a selected keypoint, one listed in pose.pose_indices when that is set and one from 0 to pose.pose_n - 1 otherwise.
center_mode
string
"mean"
How the animal's center point is computed from its pose keypoints. Every accepted value averages all keypoints. Selecting a single keypoint by index needs an integer, which this field does not accept. Unwired: every accepted value averages all keypoints -- the per-keypoint branch needs an int this field refuses.
InterpolationConfig
Parameter
Type
Default
Constraints
Description
linear_interp_limit
integer
10
>= 1
Maximum run of consecutive missing values filled by linear interpolation. [frames]
edge_fill_limit
integer
3
>= 0
Maximum run of consecutive missing values forward-filled, then back-filled, after interpolation leaves them unresolved. [frames]
max_missing_fraction
number
0.1
>= 0.0, <= 1.0
Fraction of missing columns above which a row is dropped entirely.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
SamplingConfig
Parameter
Type
Default
Constraints
Description
fps_default
number
30.0
> 0
Frame rate used when the data's fps column is absent or does not resolve to exactly one value. [fps]
smooth_win
integer
0
>= 0
Length of the moving-average window applied to smooth position and angle values before feature computation. A value of 1 or less disables smoothing. [frames]
Version 0.2 · mosaic.behavior.feature_library.pair_facing.PairFacing
Per-frame directional facing metric for all ordered pairs of individuals.
Parameter
Type
Default
Constraints
Description
angle_thresh_deg
number
45.0
The maximum angle_diff_deg for is_facing to be true. [deg]
dist_thresh
number
3.0
The maximum distance for is_facing to be true, in the unit distance is reported in: centimeters when cm_per_pixel is set, pixels otherwise.
cm_per_pixel
number | None
null
The scale factor applied to the head-to-head distance. Unset, distance and dist_thresh are read directly from the tracks with no scaling applied. [cm/px]
pose_head_index
integer
3
The pose keypoint index for the head. The default of 3 assumes the TRex bee layout described in the class docstring.
pose_abdomen_index
integer
5
The pose keypoint index for the abdomen tip. The default of 5 assumes the TRex bee layout described in the class docstring.
body_axis_from
string
"head_to_abdomen"
The direction convention for the body axis. Known values are head_to_abdomen and abdomen_to_head. head_to_abdomen reproduces the original BeesInADish computation, under which is_facing triggers when the abdomen points toward the target. abdomen_to_head flips the sign so is_facing triggers when the head points toward the target.
Version 0.2 · mosaic.behavior.feature_library.pair_interaction_filter.PairInteractionFilter
Detect pairwise interaction segments from trajectory data.
Parameter
Type
Default
Constraints
Description
shift_dist
number
15.0
Distance each individual is shifted forward along its heading before the distance check runs. 0 uses raw positions with no shift. [px]
max_dist
number
40.0
The maximum distance between the shifted positions for a pair to count as interacting. [px]
require_facing
boolean
true
Require the pair to also face each other, using max_inv_orientation_diff_deg. False filters on distance alone.
max_inv_orientation_diff_deg
number
80.0
The maximum inverse-orientation angle for a pair to count as facing. Ignored when require_facing is false. [deg]
min_run_frames
integer
250
The minimum length a continuous run meeting the interaction criteria needs to count as a segment. [frames]
frame_padding
integer
10
Padding added before and after each detected segment. [frames]
morphological_structure_size
integer
25
Length of the structuring element used to close and open the per-frame interaction mask before segment extraction. 0 disables morphological filtering. [frames]
px_scale
number
1.0
Scale factor applied to shift_dist and max_dist, for videos recorded at a different pixel resolution.
use_pixel_coords
boolean
true
Use pixel-coordinate columns for the distance calculation. With pose_head_index set, uses that pose keypoint. Otherwise uses x_pixels/y_pixels where present, falling back to X/Y. False always uses X/Y.
pose_head_index
integer | None
null
Pose keypoint index used as the position for the distance calculation, together with use_pixel_coords. Unset, x_pixels/y_pixels or X/Y is used instead.
Version 0.3 · mosaic.behavior.feature_library.pairposedistancepca.PairPoseDistancePCA
Principal components of the pose distances between every pair, per frame.
Parameter
Type
Default
Constraints
Description
interpolation
InterpolationConfig
constructed
Interpolation settings for missing pose data.
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
include_intra_A
boolean
true
Include the pairwise distances between the focal individual's own keypoints.
include_intra_B
boolean
true
Include the pairwise distances between the other individual's own keypoints.
include_inter
boolean
true
Include the pairwise distances between the focal individual's keypoints and the other individual's keypoints.
duplicate_perspective
boolean
true
Emit each pair twice, once with each individual as id1, the focal one. False emits only the row set with the lower-id individual as id1.
n_components
integer
6
The number of principal components the PCA retains.
batch_size
integer
5000
> 0
How many pair-frames are read per batch. duplicate_perspective doubles the rows each IncrementalPCA partial_fit or transform call receives.
model
PairPoseDistancePCAModelArtifact | None
null
The model.joblib of a pair-posedistance-pca run to apply, such as a fit step scoped to the training groups. Unset, the feature fits a basis on the entries in its scope. Set, nothing is fitted, the entries in scope leave the run id, and n_components, pose, include_intra_A, include_intra_B and include_inter must equal the fit's.
InterpolationConfig
Parameter
Type
Default
Constraints
Description
linear_interp_limit
integer
10
>= 1
Maximum run of consecutive missing values filled by linear interpolation. [frames]
edge_fill_limit
integer
3
>= 0
Maximum run of consecutive missing values forward-filled, then back-filled, after interpolation leaves them unresolved. [frames]
max_missing_fraction
number
0.1
>= 0.0, <= 1.0
Fraction of missing columns above which a row is dropped entirely.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
PairPoseDistancePCAModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"pair-posedistance-pca"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
Version 0.1 · mosaic.behavior.feature_library.pair_position.PairPositionFeatures
Egocentric and kinematic features of every pair, from position and heading.
Parameter
Type
Default
Constraints
Description
interpolation
InterpolationConfig
constructed
Interpolation settings for missing position data.
sampling
SamplingConfig
constructed
Frame rate and smoothing settings.
InterpolationConfig
Parameter
Type
Default
Constraints
Description
linear_interp_limit
integer
10
>= 1
Maximum run of consecutive missing values filled by linear interpolation. [frames]
edge_fill_limit
integer
3
>= 0
Maximum run of consecutive missing values forward-filled, then back-filled, after interpolation leaves them unresolved. [frames]
max_missing_fraction
number
0.1
>= 0.0, <= 1.0
Fraction of missing columns above which a row is dropped entirely.
SamplingConfig
Parameter
Type
Default
Constraints
Description
fps_default
number
30.0
> 0
Frame rate used when the data's fps column is absent or does not resolve to exactly one value. [fps]
smooth_win
integer
0
>= 0
Length of the moving-average window applied to smooth position and angle values before feature computation. A value of 1 or less disables smoothing. [frames]
Version 0.1 · mosaic.behavior.feature_library.pair_wavelet.PairWavelet
CWT spectrograms on PairPoseDistancePCA outputs. It is the spectral stage of a behavioral map that follows the approach of Berman et al. (2014), MotionMapper.
Parameter
Type
Default
Constraints
Description
sampling
SamplingConfig
constructed
Frame rate settings. Only fps_default is read; smoothing is not applied.
f_min
number
0.2
> 0
Minimum frequency of the CWT frequency band. [Hz]
f_max
number
5.0
> 0
Maximum frequency of the CWT frequency band. [Hz]
n_freq
integer
25
> 0
Number of frequency bins, dyadically spaced between f_min and f_max.
wavelet
string
"cmor1.5-1.0"
Name of the PyWavelets continuous wavelet used for the CWT.
log_floor
number
-3.0
Minimum value the log-power spectrogram is clamped to.
pc_prefix
string
"PC"
Column-name prefix used to auto-detect principal-component input columns, for example PC0 and PC1.
cols
list of string | None
null
Explicit input column names for the CWT. Unset, columns are auto-detected via pc_prefix, falling back to every numeric feature column when none match.
SamplingConfig
Parameter
Type
Default
Constraints
Description
fps_default
number
30.0
> 0
Frame rate used when the data's fps column is absent or does not resolve to exactly one value. [fps]
smooth_win
integer
0
>= 0
Length of the moving-average window applied to smooth position and angle values before feature computation. A value of 1 or less disables smoothing. [frames]
Version 0.2 · mosaic.behavior.feature_library.scale_to_cm.ScaleToCm
Convert every length-bearing column into centimeters.
Parameter
Type
Default
Constraints
Description
cm_per_pixel
number | None
null
Override the dataset's recorded scale. A value that is not positive raises when the feature runs. Unset, the scale is read from the media index for the sequence being processed. [cm/px]
mode
"derive" | "convert"
"derive"
derive emits only the scaled columns, each under its own name plus suffix, alongside the metadata. convert returns the whole table with every length column converted in place, under its own name.
suffix
string
"_cm"
Appended to each scaled column's name in derive mode. Names nothing in convert mode, which refuses a non-default value.
columns
list of string | None
null
Replace the length-bearing classifier with exactly these columns, scaling every one whether or not it is length-bearing. A name absent from the table raises. In convert mode, naming a subset leaves the unnamed length columns in pixels inside a table whose other lengths are centimeters.
Version 0.2 · mosaic.behavior.feature_library.speed_angvel.SpeedAngvel
Per-sequence feature computing translational speed and angular velocity.
Parameter
Type
Default
Constraints
Description
step_size
integer | None
null
The additional frame step for speed_step and angvel_step, computed beside the one-frame difference. Unset, neither column is emitted. [frames]
smooth_window
integer | None
null
The window Savitzky-Golay smoothing (polyorder 1) averages speed over to produce speed_smooth. Unset, speed_smooth is not emitted. Smoothing raises when the window is 0 or 1, or when it exceeds the individual's row count. [frames]
fps
number | None
null
Frames per second. When set, dt is derived from the frame column (frame_diff / fps) instead of the time column. This is more robust for constant-fps data where the time column may contain irregular real timestamps.
Version 0.5 · mosaic.behavior.feature_library.temporal_stacking.TemporalStackingFeature
Build temporal context windows over per-sequence feature data.
Parameter
Type
Default
Constraints
Description
half
integer
60
>= 0
Half-width of the temporal window. The full window spans -half to +half around each row. [frames]
skip
integer
5
>= 1
The step between time offsets in the stacking window. [frames]
use_temporal_stack
boolean
true
Concatenate Gaussian-smoothed copies of each feature column at every offset in the window. False passes through the input feature columns unchanged.
sigma_stack
number
30.0
The Gaussian smoothing sigma applied before stacking. 0 disables smoothing. [frames]
add_pool
boolean
true
Append pooled statistics computed over a sliding Gaussian window.
pool_stats
list of string
["mean"]
Which pooled statistics to compute over the sliding Gaussian window. Known values are mean, std and variance, matched case-insensitively. Any other value is ignored.
sigma_pool
number
30.0
The Gaussian sigma for the pooling window. Non-positive computes it from win_sec and fps. [frames]
fps
number
30.0
> 0
The frame rate used to convert win_sec into a frame count. [fps]
win_sec
number
0.5
> 0
The width of the pooling window. [s]
pair_filter
NNResult | None
null
Unset, every row is read. A nearest-neighbor result narrows the input, while it loads, to rows where one individual in the pair is the other's nearest neighbor. On an input without id1/id2 columns, the filter has no effect.
columns
list of string | None
null
The payload columns to read. Unset, every numeric payload column of the input is read. A name of an identity or bookkeeping column, or of a column that the input lacks, raises. The order and repeats of the names do not change the run.
NNResult
Parameter
Type
Default
Constraints
Description
feature
string
"nearest-neighbor"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
Version 0.2 · mosaic.behavior.feature_library.track_subsample.TrackSubsample
Subsample tracks rows for downstream per-row features.
Parameter
Type
Default
Constraints
Description
method
"kmeans" | "uniform" | "clips"
"kmeans"
How rows are chosen. kmeans clusters frames in body-canonical pose space and keeps the frame closest to each cluster centroid. uniform picks frames at constant stride. clips picks seeds via kmeans and takes clip_len consecutive frames from each.
target_frames
integer
300
>= 1
The target output row count per sequence. For method=clips this is n_clips * clip_len. The number of clips is target_frames // clip_len. [frames]
clip_len
integer
8
>= 2
The clip length, for method=clips. Ignored otherwise. [frames]
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
seed
integer
42
The random seed for k-means.
drop_nan
boolean
true
Drop rows with non-finite required columns from the output. Unwired: no code path reads it; apply() emits whatever rows the chosen method selects.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
Version 0.1 · mosaic.behavior.feature_library.trajectory_smooth.TrajectorySmooth
Per-sequence feature that smooths and interpolates trajectory positions.
Parameter
Type
Default
Constraints
Description
speed_threshold
number | None
null
Unset, no frame is flagged bad. Set, a frame is flagged when its displacement from the previous frame exceeds this value, per frame or, when fps is set, per second. Uses the same unit as the X and Y position columns.
fps
number | None
null
Unset, speed_threshold is read as a per-frame value. Set, converts speed_threshold from a per-second rate to per-frame. [fps]
interpolate_centroid
boolean
true
Replace a bad frame's centroid position with linear interpolation between the nearest good frames, extending from the nearest good value at a sequence edge. Every remaining gap in each column is filled the same way.
interpolate_pose
boolean
false
Replace a bad frame's pose keypoint positions with linear interpolation between the nearest good frames, extending from the nearest good value at a sequence edge. Every remaining gap in each column is filled the same way.
expand_frames
integer
2
How far on each side of a speed-flagged row the bad flag extends. [frames]
savgol_window
integer | None
null
Unset, no Savitzky-Golay smoothing is applied. Set, smooths centroid and pose positions regardless of the interpolation settings, for an id with at least this many rows. Must exceed savgol_polyorder. An odd value keeps the filter window centered. [frames]
savgol_polyorder
integer
2
The polynomial order of the Savitzky-Golay filter.
Version 0.1 · mosaic.behavior.feature_library.collective_motion_metrics.CollectiveMotionMetrics
Per-frame group-level collective-motion metrics (Tunstrom et al. 2013).
Parameter
Type
Default
Constraints
Description
heading_source
"auto" | "orientation" | "velocity"
"auto"
Where each unit heading comes from. auto uses the ANGLE column when it has at least one finite value, falling back to the direction of travel otherwise -- what simulation output with no body orientation needs. orientation raises when no usable ANGLE column is present. velocity always uses the direction of travel.
subgroup_col
string | None
null
The column partitioning each frame into subgroups, typically event from ffgroups. Prefer it over group_membership, whose per-frame connected-component label assigns no identity across frames. Using group_membership instead differences two possibly disjoint sets of animals in centroid_speed and group_angvel. Unset, metrics are computed over the whole group.
area_method
"convex_hull" | "alpha_shape" | "none"
"convex_hull"
How area and density are computed. none skips both.
alpha
number | None
null
The circumradius cutoff for the alpha shape, in the input's position units. Required when area_method is alpha_shape.
min_individuals
integer
3
>= 1
Below this count of individuals, the relational metrics are NaN and state is Undefined. The default of 3 rather than 2 accounts for N=2, where the two radial vectors are antiparallel by construction. Rotation contains no information there, yet still feeds the classifier.
fps
number | None
null
The frame rate. With a frame order column, this derives dt as frame_diff / fps, immune to the jittery wall-clock timestamps some trackers embed. Unset, dt falls back to the time column, then to one order step. [fps]
max_frame_gap
integer | None
null
Frame steps wider than this yield NaN velocity instead of averaging across a gap the shoal reorganized over. Unset, every step is accepted. [frames]
min_group_speed
number
0.0
>= 0
centroid_heading is NaN at or below this speed. A mill's centroid is stationary by definition, and arctan2(0, 0) returns 0 (due east) with no error, which the threshold guards against.
speed_col
string | None
null
The per-individual speed column averaged into mean_speed. Worth setting under subgroup_col, where frame-aggregate cannot express a per-subgroup mean. Unset, mean_speed is not emitted.
filter_expr
string | None
null
A DataFrame.query expression applied first, before every other computation. The way to drop event == -1, the pooled non-event pseudo-group from ffgroups: that sentinel is an integer, which exclude_cols cannot express.
exclude_cols
list of string
constructed
Boolean columns whose truthy rows are dropped after filter_expr and before every other computation. One bad position moves the centroid, and with it every radial vector, the covariance and the hull. A name absent from the input is skipped rather than raising.
Version 0.2 · mosaic.behavior.feature_library.evaluate_labels.EvaluateLabels
Score a run's predictions or clusters against ground-truth labels.
Parameter
Type
Default
Constraints
Description
labels
GroundTruthLabelsSource
constructed
The ground-truth labels to score against, resolved from labels//index.csv. A {step: } reference names a convert-labels step and pins the variant it writes.
mode
"classifier" | "clusters"
"classifier"
classifier scores predicted classes. clusters scores a clustering, mapped to classes through label_map, or through the majority class of each cluster among the scored rows when label_map is unset.
prediction_column
string
"predicted_label"
The column holding each row's prediction: a class in classifier mode, a cluster in clusters mode.
probability_prefix
string
"prob_"
The prefix of the per-class probability columns that perspective 'mean_probabilities' averages, as prob_.
perspective
integer | "pooled" | "mean_probabilities" | None
null
Which rows of a pair output to score: the rows of one perspective, 'pooled' for both, or 'mean_probabilities' to average a pair's two rows. Unset scores the rows as they are, and refuses rows of more than one perspective.
missing_label
"drop" | "error"
"drop"
What an entry with no label file does: drop leaves it out of the score, error fails the run.
classes
list of integer | None
null
The classes to report, in order. Unset reports every class a row holds or is predicted as.
average_over
list of integer | None
null
The classes macro_f1 averages over, such as every behavior but other. Unset averages over every class a scored row holds.
label_map
LabelMapRef | None
null
In clusters mode, a cluster to class map fitted on other entries by fit-label-map, as {step: }.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
LabelMapRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"label_map.parquet"
Glob pattern. Auto-derived from load.kind when empty.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
Version 0.1 · mosaic.behavior.feature_library.ffgroups_metrics.FFGroupsMetrics
Per-sequence summary of focal-individual group metrics.
Parameter
Type
Default
Constraints
Description
group_col
string
"event"
Name of the column identifying group-membership events, for example the one FFGroups writes. Read only when present in the input.
speed_col
string
"speed"
The per-frame speed column.
time_chunk_sec
number | None
null
Splits each sequence into chunks of this duration, summarized separately. Unset, each sequence is summarized whole. Takes precedence over frame_chunk when both are set and the input has a time column. [s]
frame_chunk
integer | None
null
Splits each sequence into chunks of this size, summarized separately. Unset, each sequence is summarized whole. Ignored when time_chunk_sec is also set and the input has a time column. [frames]
centroid_heading_col
string
"centroid_heading"
The centroid-heading column, used to rotate positions into the group's reference frame. Read only when present in the input.
exclude_cols
list of string
constructed
Names of boolean columns whose truthy rows are dropped before computation, for example bad_frame.
The aggregation applied across ids within each frame. Known values are mean, median, min, max, std, sum and count. Without threshold, NaN values are skipped for every mode, and count counts non-null values.
output_column
string | None
null
The name of the result column. Unset, it is column followed by agg, joined with an underscore.
filter_expr
string | None
null
A pandas DataFrame.query expression applied before aggregation, for example to dedupe pair-perspective input or drop flagged frames.
threshold
number | None
null
Aggregate the boolean column > threshold instead of the raw column. Unset, the raw column is aggregated. With agg set to mean, the result is the fraction of ids in the frame exceeding the threshold.
transform
"none" | "abs"
"none"
A transform applied to the column before aggregation. Known values are none and abs. abs aggregates the column's absolute value, useful for a magnitude such as angular velocity.
Where the label sequence comes from. A GroundTruthLabelsSource reads each entry's converted label file from labels//, through its segment sidecar when the file carries one. A ResultColumn reads a categorical column of a feature's output: from that feature's run when it names a feature, from this feature's own input when its feature is left empty.
states
list of (string | integer) | None
null
The fixed vocabulary of states, which gives every entry the same columns -- a state an entry never shows reads as zero usage and zero bouts rather than as a missing column. An integer names a label id or a numeric column value, a string a label name or a text column value. Required for a ResultColumn source. Unset for ground truth, it is the label file's label_names in file order, less those in ignore, so it can differ between entries whose files were converted with different vocabularies; a file that declares no label_ids has no vocabulary to default to and is refused. A label or value in neither states nor ignore is refused.
fps
number | None
null
The frame rate that converts bout lengths to seconds and bout counts to rates. Unset, it is read from the input's time column as the median time step per frame, and an input without a usable time column is refused. [fps]
min_bout_frames
integer
1
>= 1
The shortest bout kept. A shorter run or segment is dropped before any statistic is taken: it adds to no usage and no coverage, is no bout, and takes part in no transition. A bout's length is its rows times the entry's frame step, so on a table holding every k-th frame a bout of r rows has length r*k. [frames]
ignore
list of (string | integer)
constructed
Labels or values excluded from every statistic, such as a background or an unsure class. Their frames count toward n_frames only, so they lower coverage. Matched as states are, and refused when also listed in states.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
ResultColumn
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
Version 0.1 · mosaic.behavior.feature_library.nn_delta_bins.NearestNeighborDeltaBins
Bin nearest-neighbor response fields (dangle, dspeed) over neighbor position.
Parameter
Type
Default
Constraints
Description
nbins
integer
45
> 0
The count of bin edges spanning -binmax to binmax. The two-dimensional y axis takes one more edge than this.
binmax
number
14.0
> 0
Maximum absolute neighbor offset spanned by a bin edge, in position units.
max_for_avg
number
5.0
> 0
Maximum neighbor offset, in position units, used to select rows for the turn response, keeping neighbor_x between 0 and max_for_avg. The speed response uses half this value, keeping neighbor_y within plus or minus max_for_avg / 2.
antisymm
boolean
true
Duplicate each row of the turn (dangle) response with its y offset and its dangle negated, which folds that response left-right. The speed (dspeed) response never reads this flag. Its one-dimensional bins select a symmetric band of y offsets, and its two-dimensional bins mirror-duplicate every row with the dspeed value unchanged.
focal_category_col
string
"Focal_fish"
The column recording the focal animal's category flag.
neighbor_category_col
string
"neighbor_focal"
The column recording the neighbor's category flag.
group_size_col
string
"group_size"
The column recording the group size.
exp_col
string
"Exp"
The column recording the experimental condition identifier.
trial_col
string
"Trial"
The column recording the trial identifier.
category_specs
list of any
constructed
Specs for derived category columns, each a dict with source_col, new_col, quantile (default 0.75) and op, adding a boolean column comparing source_col against its quantile. Any op value other than <= is treated as >, without raising. The specs run in list order before the category columns are read, and a spec may write focal_category_col or neighbor_category_col. A run refuses an entry whose table lacks the source_col of a spec, unless an earlier spec writes it.
exclude_cols
list of string
constructed
Column names whose truthy rows are dropped before computation. A name absent from the table is skipped rather than raising.
nonfocal_flag_col
string
"Focal_fish"
Name of the column flagging nonfocal animals, read only when focal_category_col is unset or absent from the table.
nonfocal_flag_value
boolean
false
The value in nonfocal_flag_col marking an animal as nonfocal, read under the same condition as nonfocal_flag_col.
Version 0.2 · mosaic.behavior.feature_library.social_motion_summary.SocialMotionSummary
Per-fish summary of social-interaction and locomotor-style metrics.
Parameter
Type
Default
Constraints
Description
fps
number
30.0
The frame rate, used to convert frame steps to seconds for the derivative-based metrics and for kick_rate's time base. [fps]
speed_col
string
"speed"
The per-frame speed column. speed_smooth is the better input for the derivative-based metrics, but it exists only when speed-angvel ran with smooth_window set. It is not the default for that reason.
social_min_group_size
integer
2
The minimum group_size for a frame to count as social, when group_size is available.
subgroup_col
string | None
null
The column that splits the output into one row per id per value, typically group_size from ffgroups. Unset, the output has one row per id.
compute_burst_coast
boolean
false
Also emit a burst-and-coast gait summary: kick_rate and burst_coast_ratio.
Version 0.3 · mosaic.behavior.feature_library.arhmm.ArHmmFeature
AR-HMM behavioral syllable discovery as a pipeline feature.
Parameter
Type
Default
Constraints
Description
model
ArHmmModelArtifact | None
null
A pre-fitted model artifact to load, skipping the fit. Unset, the model is fit from scratch.
pca_dim
integer | None
null
How many components PCA reduces the feature dimensionality to before fitting. Unset, no PCA is applied. Ignored when it is not smaller than the number of input feature columns.
n_states
integer
50
>= 2
The maximum number of HMM states. Pruned down after fitting when prune_threshold is greater than zero.
n_lags
integer
1
>= 1
How many preceding steps enter as autoregressive regressors. A step is one row of the sequence being fit, which is a video frame unless downsample_rate has subsampled it.
sticky_weight
number
100.0
>= 0
The extra pseudo-count added to the diagonal of the transition matrix, encouraging state persistence across frames.
n_iter
integer
200
>= 1
The ceiling on EM passes per restart. [iterations]
tol
number
0.0001
> 0
The convergence tolerance on the relative change in log-likelihood between EM iterations.
n_restarts
integer
1
>= 1
How many random restarts run, keeping the one with the best log-likelihood.
standardize
boolean
true
Z-score every feature column before fitting. When False, raw values are used directly.
downsample_rate
integer | None
null
The factor every sequence is subsampled by before fitting and inference, upsampled back afterward. Unset, no downsampling is applied.
prune_threshold
number
0.01
>= 0, <= 1
The posterior mass fraction below which a fitted state is dropped.
random_state
integer
42
The random seed for PCA and model fitting, for reproducible restarts.
columns
list of string | None
null
The payload columns to read. Unset, every numeric payload column of the input is read. A name of an identity or bookkeeping column, or of a column that the input lacks, raises. The order and repeats of the names do not change the run.
backend
"auto" | "numpy" | "numba" | "jax"
"auto"
The compute backend for EM and Viterbi decoding. auto uses the numba-compiled path when available and falls back to numpy. numba requests the compiled path but still falls back to numpy if it raises. numpy forces the plain path. jax is reserved for a future GPU path and is not implemented yet.
ArHmmModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"arhmm"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"arhmm_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
Version 0.6 · mosaic.behavior.feature_library.extract_labeled_templates.ExtractLabeledTemplates
Extract labeled, split-annotated templates from upstream features.
Parameter
Type
Default
Constraints
Description
labels
GroundTruthLabelsSource
required
Reference to the per-frame ground-truth labels used to assign a class to every template, resolved from labels//index.csv.
strategy
"random" | "farthest_first"
"random"
How templates are selected from the candidate pool. random keeps the pool as collected. farthest_first reduces an oversized pool to the target count per class by iterative farthest-point sampling.
n_per_class
integer | object | None
null
The number of templates per class. An int applies the same count to every class, and a mapping sets each class's count individually. A mapping value of null (or the string all) takes every labeled frame of that class, and a class the mapping does not list contributes none. Exactly one of n_per_class and n_total must be set.
n_total
integer | None
null
The total number of templates, split equally across the classes seen. Each class gets n_total // (number of classes), so the realized total can be lower. Exactly one of n_per_class and n_total must be set.
pool
PoolConfig
constructed
Candidate pool size and per-entry allocation strategy consulted before final template selection.
test_fraction
number
0.2
>= 0.0, <= 1.0
Fraction of sequences held out for the test split.
random_state
integer
42
The random seed for split assignment and template sampling.
columns
list of string | None
null
The payload columns to read. Unset, every numeric payload column of the input is read. A name of an identity or bookkeeping column, or of a column that the input lacks, raises. The order and repeats of the names do not change the run.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
PoolConfig
Parameter
Type
Default
Constraints
Description
size
integer | None
null
Number of candidates collected before selecting the final templates. Unset sets the pool to the target template count, so selection makes no reduction.
allocation
"reservoir" | "exact"
"reservoir"
How the per-entry quota for the pool is computed. reservoir performs weighted reservoir sampling in one pass. exact counts rows first, then samples a second pass with proportional quotas.
max_entry_fraction
number | None
null
Cap on one entry's contribution to the pool, as a fraction of the pool size. Unset applies no cap, so each entry's share is proportional to its row count. The effective cap never drops below one divided by the number of entries seen so far, so the pool can still fill completely.
Version 0.4 · mosaic.behavior.feature_library.extract_templates.ExtractTemplates
Subsample per-sequence data into a representative template matrix.
Parameter
Type
Default
Constraints
Description
strategy
"random" | "farthest_first"
"random"
The template selection method. random keeps the candidate pool as the templates. farthest_first reduces the pool to n_templates by greedy farthest-point sampling.
n_templates
integer
required
>= 1
The number of templates to select. Under farthest_first the candidate pool is reduced to this many. Under random it sets the pool size only when pool.size is unset, and pool.size then decides how many templates are written.
pool
PoolConfig
constructed
The candidate pool's size, allocation strategy, and per-entry caps.
random_state
integer
42
The random seed for the feature's random sampling steps.
pair_filter
NNResult | None
null
Unset, every row is read. A nearest-neighbor result narrows the input, while it loads, to rows where one individual in the pair is the other's nearest neighbor. On an input without id1/id2 columns, the filter has no effect.
columns
list of string | None
null
The payload columns to read. Unset, every numeric payload column of the input is read. A name of an identity or bookkeeping column, or of a column that the input lacks, raises. The order and repeats of the names do not change the run.
NNResult
Parameter
Type
Default
Constraints
Description
feature
string
"nearest-neighbor"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
PoolConfig
Parameter
Type
Default
Constraints
Description
size
integer | None
null
Number of candidates collected before selecting the final templates. Unset sets the pool to the target template count, so selection makes no reduction.
allocation
"reservoir" | "exact"
"reservoir"
How the per-entry quota for the pool is computed. reservoir performs weighted reservoir sampling in one pass. exact counts rows first, then samples a second pass with proportional quotas.
max_entry_fraction
number | None
null
Cap on one entry's contribution to the pool, as a fraction of the pool size. Unset applies no cap, so each entry's share is proportional to its row count. The effective cap never drops below one divided by the number of entries seen so far, so the pool can still fill completely.
Version 0.2 · mosaic.behavior.feature_library.feral_feature.FeralFeature
FERAL vision-transformer behavior classifier as a pipeline feature.
Parameter
Type
Default
Constraints
Description
feral_code_dir
string | None
null
A local checkout of the feral package, prepended to sys.path. Unset, the pip-installed package is used.
model_name
string
"facebook/vjepa2-vitl-fpc32-256-diving48"
The FERAL backbone: a feral.backbones.BACKBONES key or the matching HuggingFace slug. An unrecognized value raises. The choice also selects the pretrained weights FERAL downloads from the HuggingFace hub on first use, whose license differs per entry; see NOTICE. In inference mode this is a fallback: a loaded config.json or checkpoint that records its own backbone overrides it.
predict_per_item
integer
64
How many predictions the model emits per chunk. In inference mode this is a fallback: a loaded config.json or checkpoint that records its own value overrides it.
chunk_length
integer
64
The length of one video chunk. [frames]
chunk_shift
integer
32
The stride between the starts of consecutive chunks. [frames]
chunk_step
integer
1
The stride sampled within one chunk. [frames]
resize_to
integer
256
The side a frame is resized to before the model reads it. [px]
device
string
"cuda"
Which PyTorch device loads and runs the model.
infer_batch_size
integer
4
How many chunks apply() reads in one forward pass. Batching is numerically equivalent to reading them one at a time.
inference_autocast
boolean
false
Run apply()'s forward pass under bfloat16 autocast instead of full float32, roughly 1.5-2x faster at the cost of a recall shift of around 10 percentage points on threshold-sensitive classes. Read only when the model is on a CUDA device.
class_names
object | None
null
Class index to name mapping, keyed by string index. Unset, the mapping is read from the model config or checkpoint metadata.
decision_threshold
number | None
null
The minimum probability considered when choosing the predicted class. Unset, the predicted class is the argmax over every class. Set, the predicted class is default_class when every class falls below it.
default_class
integer
0
The class returned when decision_threshold is set and every class falls below it.
model_dir
string | None
null
A directory containing model_best.pt and config.json from a prior training run, for inference mode.
video_dir
string | None
null
The directory containing crop videos, for training mode.
label_json
string | None
null
A FERAL-format label JSON declaring class_names and splits, for training mode.
training
FeralTrainingConfig | None
null
Training hyperparameters. Required to train. Unset without a model to load, fit raises.
FeralTrainingConfig
Parameter
Type
Default
Constraints
Description
epochs
integer
10
>= 1
The number of training epochs.
train_bs
integer
4
>= 1
The number of samples per training batch.
val_bs
integer
8
>= 1
The number of samples per validation batch, also used when evaluating a test partition after training.
num_workers
integer
4
>= 0
The number of subprocess workers for the training and validation data loaders, also used when evaluating a test partition after training.
lr
number
4e-05
> 0
The learning rate for the AdamW optimizer.
weight_decay
number
0.1
>= 0
The weight decay for the AdamW optimizer.
label_smoothing
number
0.1
>= 0, <= 1
The label smoothing factor for the cross-entropy loss, applied only when the label JSON is not multilabel.
fc_drop_rate
number
0.5
>= 0, <= 1
The dropout rate before the classifier head.
freeze_encoder_layers
integer
14
>= 0
The number of backbone encoder layers frozen during fine-tuning.
class_weights
string
"inv_freq_sqrt"
The class-weighting scheme forwarded to FERAL's get_weights when building the loss weight tensor.
ema_decay
number | None
0.999
The decay rate for an exponential moving average of the model weights. Unset, no EMA model is tracked.
mixup_alpha
number | None
0.8
The alpha parameter for torchvision's MixUp training-batch augmentation. Unset, no MixUp is applied.
part_warmup
number
0.2
>= 0, <= 1
The fraction of total training steps spent in the cosine learning-rate schedule's warmup phase.
patience
integer | None
null
The number of epochs without validation improvement before training stops early. Unset, training never stops early.
compile
boolean
true
Compile the model with torch.compile before training.
do_aa
boolean
true
Apply data augmentation when building the training dataset.
seed
integer
0
The random seed applied to torch and numpy before training starts.
part_sample
number
1.0
> 0, <= 1
The fraction of the training partition included when building the training dataset.
multilabel_threshold
number
0.85
>= 0, <= 1
The probability threshold used when computing F1 metrics, applied only when the label JSON is multilabel.
wandb_project
string | None
null
The Weights & Biases project name recorded in the saved training config. Unwired: Stored in the saved config; nothing in this file starts a Weights & Biases run with it..
Version 0.2 · mosaic.behavior.feature_library.fit_label_map.FitLabelMap
Map each cluster to the behavior class its labeled rows favor.
Parameter
Type
Default
Constraints
Description
labels
GroundTruthLabelsSource
constructed
The ground-truth labels the map is fitted on, resolved from labels//index.csv. A {step: } reference names a convert-labels step and pins the variant it writes.
prediction_column
string
"cluster"
The column holding each row's cluster.
perspective
integer | "pooled" | None
null
Which rows of a pair output to fit on: the rows of one perspective, or 'pooled' for both. Unset fits on the rows as they are, and refuses rows of more than one perspective.
weighting
"count" | "class_balanced"
"count"
How a cluster picks its class. count takes the class with the most rows in the cluster, which follows prevalence and can leave a rare class with no cluster. class_balanced takes the class whose rows the cluster holds the largest share of.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
Version 0.1 · mosaic.behavior.feature_library.dinov2_temporal_identity_model.GlobalIdentityDinoV2Temporal
Train a DINOv2 + temporal identity model from individual sequences.
Parameter
Type
Default
Constraints
Description
model
DinoV2TemporalIdentityArtifact | None
null
A pre-fitted DinoV2TemporalIdentityArtifact to load, skipping the fit. Unset, the model fits from the crops named by identities or group_as_identity.
identities
object | None
null
An explicit mapping of identity name to the sequences containing that individual alone. Takes precedence over group_as_identity.
group_as_identity
boolean
false
Treat each sequence's group name as its identity, instead of listing sequences explicitly under identities. Ignored when identities is set.
backbone
string
"dinov2_vits14"
The DINOv2 model name loaded from the facebookresearch/dinov2 torch hub repository.
temporal_head
"gru" | "perceiver" | "pool"
"gru"
Which head aggregates a clip's per-frame embeddings into one clip embedding.
embedding_dim
integer
128
>= 16
The output dimension of the temporal head.
clip_len
integer
8
>= 2
The length of one clip. [frames]
clip_stride
integer | None
null
Unset, the step between clip starts equals clip_len and clips do not overlap. Set, it names that step directly. [frames]
image_size
tuple of (integer, integer)
[224, 224]
min items 2, max items 2
The crop resize target as (height, width). Must be a multiple of 14 for the DINOv2 patch size. [px]
channels
integer
3
How many channels the crop image is read from disk with. 1 reads grayscale and is replicated to 3 channels before the backbone reads it. Any other value reads 3-channel RGB.
epochs
integer
50
>= 1
The number of optimization passes over the training clips. [epochs]
learning_rate
number
0.001
The Adam learning rate.
batch_size
integer
32
>= 1
The training batch size.
val_split
number
0.2
>= 0.0, < 1.0
The fraction of clips set aside for validation. 0 disables the split.
max_clips_per_identity
integer
500
>= 1
The cap on training clips kept per identity.
weights_name
string
"dinov2_temporal_identity"
The filename stem for the exported checkpoint, written as .pth.
crop_root
string | None
null
Override for the directory EgocentricCrop output is read from. When it names no readable directory, the directory the loaded input came from is used instead.
DinoV2TemporalIdentityArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-identity-dinov2-temporal"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"dinov2_temporal_identity_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
A bare timm architecture tag or a Hugging Face hub id naming the frozen embedding backbone. Mosaic ships no weights, and whatever this names downloads at run time under its own license.
image_size
tuple of (integer, integer) | None
null
The crop resize target as (height, width). Unset, this follows the backbone's declared input size. [px]
channels
integer
3
How many channels the crop image is read from disk with. 1 reads grayscale and is replicated to 3 channels before the backbone reads it. Any other value reads 3-channel RGB.
batch_size
integer
32
>= 1
The embedding batch size.
max_images_per_identity
integer
2000
>= 1
The cap on training crops kept per identity.
weights_name
string
"identity_embedding"
The filename stem for the exported checkpoint, written as .pth.
crop_root
string | None
null
Override for the directory EgocentricCrop output is read from. When it names no readable directory, the directory the loaded input came from is used instead.
EmbeddingIdentityArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-identity-embedding"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"identity_embedding_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
Version 0.3 · mosaic.behavior.feature_library.identity_model.GlobalIdentityModel
Train a visual identity model from individual animal sequences.
Parameter
Type
Default
Constraints
Description
model
ClassifierIdentityArtifact | None
null
Unset, the fit runs over the input scope. Set to a pre-fitted ClassifierIdentityArtifact, the fit is skipped and the referenced training run enters this run's identity beside its scope.
identities
object | None
null
The mapping from an identity name to the group/sequence entries that contain that individual alone. Takes priority over group_as_identity when both are given.
group_as_identity
boolean
false
Derive one identity per group name from the input entries, used only when identities is unset. Fitting raises when neither is given.
A bare timm architecture tag or a Hugging Face hub id naming the backbone. Mosaic ships no weights, and whatever is named here is downloaded at run time under its own license.
image_size
tuple of (integer, integer) | None
null
Unset, the backbone's declared input size is used. Set, crops are resized to (height, width) before the backbone reads them. [px]
channels
integer
3
The number of channels read from disk: 1 for grayscale, 3 for RGB. A grayscale image is replicated to 3 channels for the backbone.
freeze_backbone
boolean
true
Train the classification head alone, leaving the pretrained backbone untouched. False fine-tunes the whole network end to end, which needs considerably more data.
epochs
integer
30
How long the classifier trains. [epochs]
learning_rate
number
0.001
The Adam learning rate.
batch_size
integer
32
How many crops the classifier reads per training batch.
val_split
number
0.2
>= 0.0, < 1.0
The fraction of collected images held out for validation. 0 disables the split, training on every image.
max_images_per_identity
integer
2000
>= 1
The most images used per identity. Balances classes by capping how many crops from any one identity enter training.
weights_name
string
"identity_classifier"
The stem of the exported .pth checkpoint filename.
crop_root
string | None
null
Unset, the EgocentricCrop output root is resolved from the input Result. Set, crops are read from this path first, falling back to that resolution when the derived directory does not exist.
ClassifierIdentityArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-identity-model"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"identity_classifier_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
Version 0.4 · mosaic.behavior.feature_library.global_kmeans.GlobalKMeansClustering
Global K-Means clustering on templates loaded via load_state. Per-sequence cluster assignment is done in apply(). It is a clustering stage of a behavioral map that follows the approach of Berman et al. (2014), MotionMapper, with k-means in place of a watershed segmentation of a density estimate.
Parameter
Type
Default
Constraints
Description
templates
TemplatesRef | None
null
The templates artifact to fit the k-means model on. Mutually exclusive with model.
model
KMeansModelArtifact | None
constructed
A pre-fitted k-means model artifact to load instead of fitting one. Mutually exclusive with templates.
k
integer
100
>= 1
The number of clusters. Fitting raises when the templates have fewer rows than k.
random_state
integer
42
The random seed for k-means initialization.
n_init
"auto" | integer
"auto"
How many k-means initializations to run, keeping the best. auto leaves the count to scikit-learn's own default policy.
max_iter
integer
300
>= 1
The maximum number of iterations for one k-means run.
device
string
"cpu"
The compute device. cuda uses cuML's GPU-accelerated k-means when available, falling back to scikit-learn otherwise.
label_artifact_points
boolean
true
Assign a cluster label to every template point used for fitting.
pair_filter
NNResult | None
null
Unset, every row is read. A nearest-neighbor result narrows the input, while it loads, to rows where one individual in the pair is the other's nearest neighbor. On an input without id1/id2 columns, the filter has no effect.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
KMeansModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-kmeans"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
NNResult
Parameter
Type
Default
Constraints
Description
feature
string
"nearest-neighbor"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
TemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.2 · mosaic.behavior.feature_library.global_scaler.GlobalScaler
Fit a StandardScaler on templates and scale per-sequence data.
Parameter
Type
Default
Constraints
Description
templates
TemplatesRef | None
null
The templates artifact to fit the scaler on. Mutually exclusive with model.
model
ScalerModelArtifact | None
constructed
A pre-fitted scaler model artifact to load, skipping the fit. Mutually exclusive with templates.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
ScalerModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-scaler"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"scaler.joblib"
Glob pattern. Auto-derived from load.kind when empty.
TemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.4 · mosaic.behavior.feature_library.global_tsne.GlobalTSNE
Fit an openTSNE embedding on templates and map per-sequence data.
Parameter
Type
Default
Constraints
Description
templates
TemplatesRef | None
null
The templates artifact to fit from. Exactly one of templates and model must be given.
model
TSNEModelArtifact | None
null
A pre-fitted t-SNE embedding artifact to load, skipping the fit. Exactly one of templates and model must be given.
random_state
integer
42
The random seed for PCA initialization and embedding optimization, and for neighbor search when knn_method is annoy.
perplexity
integer
50
>= 1
The t-SNE perplexity, controlling the effective number of nearest neighbors each point is compared against.
knn_method
string
"annoy"
The nearest-neighbor backend. Known values are annoy, faiss and faiss-gpu. Any other value falls back to annoy.
n_jobs
integer
8
>= 1
How many parallel jobs openTSNE uses for the neighbor search and optimization.
fit
TSNEFitConfig
constructed
The embedding-fit parameters: learning rate, exaggeration schedule and momentum.
mapping
TSNEMapConfig
constructed
The parameters for mapping new points into the fitted embedding: neighbor count, iterations, learning rate, exaggeration, momentum and chunk size.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
TSNEFitConfig
Parameter
Type
Default
Constraints
Description
learning_rate
number | string
"auto"
The learning rate for the t-SNE embedding, forwarded to both optimization phases. The value auto makes openTSNE compute one from the template count and the exaggeration factor.
exaggeration_iters
integer
250
>= 1
The number of iterations in the early exaggeration phase.
exaggeration
number
12
> 0
The exaggeration factor during the early exaggeration phase, increasing the attractive force between nearby points to form more compact clusters.
exaggeration_momentum
number
0.5
>= 0
The momentum during the early exaggeration phase.
iters
integer
750
>= 1
The number of iterations in the refinement phase that follows early exaggeration.
momentum
number
0.8
>= 0
The momentum during the refinement phase.
TSNEMapConfig
Parameter
Type
Default
Constraints
Description
k
integer
25
>= 1
The number of nearest neighbors used to place a new point's initial position in the fitted embedding.
iters
integer
100
>= 1
The number of optimization iterations when mapping a new point into the fitted embedding.
learning_rate
number
1.0
> 0
The learning rate used when mapping a new point into the fitted embedding.
exaggeration
number
2.0
> 0
The exaggeration factor used when mapping a new point into the fitted embedding.
momentum
number
0.0
>= 0
The momentum used when mapping a new point into the fitted embedding.
chunk_size
integer
50000
>= 1
How many rows apply() maps into the fitted embedding in one prepare_partial call.
TSNEModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-tsne"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"embedding.joblib"
Glob pattern. Auto-derived from load.kind when empty.
TemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.3 · mosaic.behavior.feature_library.global_ward.GlobalWardClustering
Ward hierarchical clustering on templates with per-sequence 1-NN assignment. It is a clustering stage of a behavioral map that follows the approach of Berman et al. (2014), MotionMapper, with Ward clustering in place of a watershed segmentation of a density estimate.
Parameter
Type
Default
Constraints
Description
templates
TemplatesRef | None
null
The templates artifact to cluster. Mutually exclusive with model.
model
WardModelArtifact | None
constructed
A pre-fitted Ward clustering model artifact to load, skipping the fit. Mutually exclusive with templates.
n_clusters
integer
20
>= 1
The largest number of clusters cut from the Ward linkage tree.
method
string
"ward"
The linkage method passed to scipy.cluster.hierarchy.linkage. ward is the only value the fit accepts, case-insensitively. Any other value raises.
pair_filter
NNResult | None
null
Unset, every row is read. A nearest-neighbor result narrows the input, while it loads, to rows where one individual in the pair is the other's nearest neighbor. On an input without id1/id2 columns, the filter has no effect.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
NNResult
Parameter
Type
Default
Constraints
Description
feature
string
"nearest-neighbor"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
TemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
WardModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"global-ward"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.1 · mosaic.behavior.feature_library.kpms.KpmsFeature
Fits keypoint-MoSeq to keypoint tracks and applies it, in a runner process.
Parameter
Type
Default
Constraints
Description
model
KpmsModelArtifact | None
null
A pre-fitted model to load. Unset, the feature fits a new model.
kpms_python
string | None
null
Path to a Python interpreter with keypoint-moseq installed. Unset, resolution reads MOSAIC_KPMS_CONDA_ENV, then MOSAIC_KPMS_PYTHON, then the in-repository environment under feature_library/external/.venv.
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
anterior_bodyparts
list of string
required
min items 1
Bodypart names marking the front of the body, used to initialize heading for egocentric alignment.
posterior_bodyparts
list of string
required
min items 1
Bodypart names marking the back of the body, used to initialize heading for egocentric alignment.
fps
integer
30
The frame rate of the input tracks. [fps]
num_iters_ar
integer
50
>= 1
How long the AR-only fitting stage runs before the full-model stage begins. [iterations]
num_iters_full
integer
500
>= 1
How long the full-model fitting stage runs after the AR-only stage. [iterations]
kappa_ar
number | None
null
The transition stickiness hyperparameter for the AR-only fitting stage. Unset, keypoint-moseq chooses its own value.
kappa_full
number | None
null
The transition stickiness hyperparameter for the full-model fitting stage. Unset, keypoint-moseq chooses its own value.
latent_dim
integer
10
>= 1
The dimensionality of the latent pose space fit by PCA. A value exceeding (number of keypoints - 1) * 2 is refused.
location_aware
boolean
false
Constrain the centroid-movement prior, setting sigmasq_loc to 0.5 rather than 1e6. The fit then treats frame-to-frame centroid movement as informative.
outlier_scale_factor
number
6.0
> 0
The multiplier setting the outlier distance threshold when removing keypoint outliers before fitting.
remove_outliers
boolean
true
Interpolate keypoints identified as outliers by their distance from the per-frame medoid, before fitting.
mixed_map_iters
integer | None
null
How many passes jax_moseq's mixed-map dispatch performs. A value of 1 or below is ignored. [iterations]
parallel_message_passing
boolean | None
null
Use keypoint-moseq's associative-scan Kalman sampling, faster to run and slower to compile. Unset, keypoint-moseq enables it when the JAX backend is not the CPU.
resume
boolean
true
Continue fitting from a previously saved checkpoint, falling back to a fresh fit when none exists.
downsample_rate
integer | None
null
Keep every Nth frame of the tracks before fitting. Unset, no downsampling is applied. Not applied when the fitted model is later used to label new tracks.
save_every_n_iters
integer
25
>= 1
How often keypoint-moseq writes a fit checkpoint. [iterations]
num_iters_apply
integer
500
>= 1
How long resampling runs when applying a fitted model to new tracks. [iterations]
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
KpmsModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"kpms"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"kpms_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
Version 0.5 · mosaic.behavior.feature_library.lightning_action_feature.LightningActionFeature
Supervised temporal action segmentation via lightning-action.
Parameter
Type
Default
Constraints
Description
templates
LabeledTemplatesRef | None
null
The extract-labeled-templates run whose train entries the model fits on. Only its template_provenance.parquet is read: the model trains on the frame runs of those entries, not on the sampled templates. An entry the run lists as test, or does not list, never trains. Mutually exclusive with model.
model
LightningActionModelArtifact | None
constructed
Pre-fitted model to load instead of training. Mutually exclusive with templates.
labels
GroundTruthLabelsSource | None
null
The per-frame ground-truth labels that the frames of the training entries are aligned to, resolved from labels//index.csv. A frame no label covers is left out of training. Required with templates.
Number of hidden units in the temporal encoder's layers.
num_layers
integer
2
>= 1
Number of layers in the temporal encoder.
num_lags
integer
4
>= 1
Number of temporal lags the encoder convolves over. Used by the temporalmlp and dtcn heads and ignored by rnn. [frames]
activation
string
"lrelu"
Activation function used in the temporal encoder. Known values are relu, lrelu, sigmoid, tanh and linear.
dropout_rate
number
0.1
>= 0, <= 1
Dropout rate applied in the temporal encoder for regularization.
sequence_length
integer
500
>= 10
Length of each training window. A run of consecutive labeled rows trains in whole windows from its first row, so a run shorter than this does not train, and neither do the rows past a run's last whole window. It must exceed twice the context that the temporal encoder reads on each side of a row, or the loss is empty. [frames]
num_epochs
integer
200
>= 1
How long the model trains. [epochs]
batch_size
integer
32
>= 1
How many training sequence chunks the model reads in one forward pass.
learning_rate
number
0.001
> 0
The optimizer's learning rate.
weight_decay
number
0.0
>= 0
The optimizer's weight decay coefficient.
optimizer
"Adam" | "AdamW"
"Adam"
The optimizer used to train the network.
weight_classes
boolean
true
Weight the training loss by inverse class frequency, to correct for class imbalance.
device
string
"gpu"
Which device trains the model. Known values are cpu and gpu. The step is offered to a GPU worker whichever it is, and gpu trains on the CPU where the machine has no CUDA device.
random_state
integer
42
The random seed for model initialization and training.
decision_threshold
number | object | None
null
Unset, the predicted class is the class with the highest probability. A float applies one probability threshold to every class, and a mapping from class to threshold applies a per-class one. Any row where no class clears its threshold is assigned default_class.
default_class
integer
required
The class assigned to a row where decision_threshold is set and no class clears its threshold. It is also assigned to a frame that belongs to no series because its identity columns are missing.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
LabeledTemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
LightningActionModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"lightning-action"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"lightning_action_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
Version 0.1 · mosaic.behavior.feature_library.probability_ensemble.ProbabilityEnsembleFeature
Average or blend the class probabilities of several classifier runs.
Parameter
Type
Default
Constraints
Description
weights
list of number | "uniform" | "fit"
"uniform"
How the members' probabilities are combined. uniform averages them. A list gives one non-negative weight per input, in the order of inputs, and the weights are scaled to sum to 1. fit chooses the weights that maximize the pooled macro F1 of the blend over the training entries of the members' cross-validation folds, which needs labels and members fitted with cv_folds.
labels
GroundTruthLabelsSource | None
null
The ground-truth labels that fit scores the blend against. Only with weights fit.
score_classes
list of integer | None
null
The classes that the macro F1 of fit averages over, such as every behavior but other. Unset, every class that a labeled frame has. Only with weights fit.
grid_step
number
0.05
> 0.0, <= 0.5
The spacing of the weights that fit tries. It must divide 1, and the search refuses a grid of more than 20,000 weight vectors, which a coarser step or fewer members avoids. Only with weights fit.
decision_threshold
number | object | None
null
Unset, the predicted class is the class with the highest probability. A float applies one probability threshold to every class, and a mapping from class to threshold applies a per-class one. Any row where no class clears its threshold is assigned default_class.
default_class
integer
required
The class assigned to a row where decision_threshold is set and no class clears its threshold.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
Version 0.1 · mosaic.behavior.feature_library.sequence_classifier.SequenceClassifier
Neural sequence classifier: a registered head on runs of consecutive frames.
Parameter
Type
Default
Constraints
Description
templates
LabeledTemplatesRef | None
null
The extract-labeled-templates run whose train entries the model fits on. Only its template_provenance.parquet is read: the model trains on the frame runs of those entries, not on the sampled templates. An entry the run lists as test, or does not list, never trains. Mutually exclusive with model.
model
SequenceClassifierModelArtifact | None
constructed
A pre-fitted SequenceClassifierModelArtifact to load instead of training. Mutually exclusive with templates.
labels
GroundTruthLabelsSource | None
null
The per-frame ground-truth labels that the frames of the training entries are aligned to, resolved from labels//index.csv. A frame no label covers is left out of training. Required with templates.
columns
list of string | None
null
The payload columns the network reads, in this order. Unset, every numeric payload column of the inputs. A narrow set usually beats a wide one: 37 columns beat 562 to 999 on CalMS21.
head
string
"gru"
The network. A built-in head is tcn (dilated convolutions), gru (a bidirectional GRU) or mlp (each frame by itself). A head registered in Python with register_head in the process that runs the step is named here too.
head_config
HeadConfig | None
null
The options of the chosen head. Unset, the head's defaults. tcn takes hidden (128), blocks (6), kernel_size (3, odd) and dropout (0.3). gru takes hidden (128), layers (2), bidirectional (true) and dropout (0.3). mlp takes hidden (128), layers (2) and dropout (0.3). Options that the chosen head does not have are refused.
head_revision
integer
0
>= 0
A number that the author of a head registered in Python raises when its code changes, so that a changed head is a new run. A built-in head ignores it, because the version of the step names its code.
input_dropout
number
0.1
>= 0.0, < 1.0
Dropout applied to the input frames while training.
clip
number
10.0
> 0.0
Standardized values are limited to this many standard deviations on both sides, so that an outlier frame does not dominate a batch.
class_weight
"sqrt" | "balanced" | "none"
"sqrt"
How the loss weighs a class. balanced weights it by the inverse of its frequency, sqrt by the square root of that, none equally. Each is scaled to a mean of 1 over the classes of the training runs.
crop_length
integer
1024
>= 2
The longest crop of a run in a training batch. A batch is cut to its shortest crop. [frames]
batch_size
integer
16
>= 1
The crops in a training batch.
steps
integer
2500
>= 1
How long each model trains, counted in batches. [steps]
learning_rate
number
0.001
> 0.0
The peak learning rate of the one-cycle schedule.
weight_decay
number
0.001
>= 0.0
The weight decay of the AdamW optimizer.
pct_start
number
0.1
> 0.0, < 1.0
The share of the steps that the learning rate climbs for.
grad_clip
number
1.0
> 0.0
The norm that gradients are clipped to.
min_run
integer
64
>= 2
The shortest run of consecutive labeled rows that trains. A shorter run is left out. [frames]
random_state
integer
42
The random seed of the batches, the initial weights and the dropout. The model fitted on every training run uses it as it is, and each fold's model a seed derived from it.
cv_folds
integer | None
null
Unset, one model is fitted and scores every entry. Set to K (at least 2), the training entries of the templates are dealt into K folds and K more models are fitted, each on the training runs outside one fold. A training entry is then scored by the model that did not see it, and every other entry by the model fitted on all training runs. The fold of each entry is in cv_folds.parquet. Needs templates and is refused beside model.
cv_seed
integer
0
The seed that deals the training sequences into folds. It changes the folds and nothing else, so runs that differ in random_state are compared on the same folds. Other than 0 only with cv_folds.
device
"cpu" | "cuda"
"cuda"
Where the network trains. cuda needs a CUDA device and fails without one. cpu is for small fits and tests. The device is part of the run id, since a GPU and a CPU need not give one model. The step is offered to a GPU worker whichever it is.
apply_device
"cpu" | "cuda"
"cpu"
Where a fitted network predicts. A model fitted on a GPU applies on the CPU. It changes the time of an apply and not the model, so it is not part of the run id. Probabilities agree across devices to about 1e-5.
n_threads
integer
1
>= 1
The CPU threads a fit or an apply uses. It changes the time, not the model beyond rounding, so it is not part of the run id. One thread makes a CPU fit repeat across machines.
decision_threshold
number | object | None
null
Unset, the predicted class is the class with the highest probability. A float applies one probability threshold to every class, and a mapping from class to threshold applies a per-class one. Any row where no class clears its threshold is assigned default_class.
default_class
integer
required
The class assigned to a row where decision_threshold is set and no class clears its threshold.
GroundTruthLabelsSource
Parameter
Type
Default
Constraints
Description
kind
string
"behavior"
min length 1
Which labels/ subdirectory the ground-truth labels resolve to. Defaults to behavior; another kind resolves the same way, through labels//index.csv.
source
"labels"
"labels"
Fixed tag naming this as a labels/ dependency. Unwired: no code path reads this field -- a GroundTruthLabelsSource is told apart from a ResultColumn by isinstance, not by this tag.
load
NpzLoadSpec | ParquetLoadSpec | JoblibLoadSpec
constructed
How to load the matched label file. Unwired: no code path reads this field -- the resolved label file is read through load_labels_for_feature_frames, not through load_from_spec, so this load specification is never consulted.
pattern
string | None
null
Glob pattern for the label file within labels//. Unwired: no code path reads this field -- the label file is the abs_path recorded in labels//index.csv, not a file matched by a pattern.
HeadConfig
No parameters.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
LabeledTemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
NpzLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"npz"
"npz"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string
required
Array key read from the .npz archive. Raises when the archive has no such key.
transpose
boolean
false
Transpose the loaded array after it is read.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
SequenceClassifierModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"sequence-classifier"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"sequence_classifier_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.1 · mosaic.behavior.feature_library.vame_feature.VameFeature
Fits VAME to keypoint tracks, and embeds each frame in its latent space.
Parameter
Type
Default
Constraints
Description
model
VameModelArtifact | None
null
A fitted VAME model to apply. Unset, the feature fits a new model. With a model set, the settings that build and preprocess the sessions must equal the model's.
vame_python
string | None
null
Path to a Python interpreter with VAME installed. Unset, resolution reads MOSAIC_VAME_CONDA_ENV, then MOSAIC_VAME_PYTHON, then the in-repository environment under feature_library/external/vame-env/.venv.
device
"auto" | "cpu" | "cuda" | "mps"
"auto"
Where VAME trains and embeds. The value auto selects CUDA, then MPS, then the CPU, as VAME does. The run identifier leaves the device out. One recipe therefore runs under one name on any machine. Training on different devices gives different models, and only training on the CPU is bit-reproducible. The model bundle and the run log record the device that ran.
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
centered_reference_keypoint
integer | string
required
The keypoint that egocentric alignment moves to the origin of each frame, as a keypoint index or a keypoint name. It must be one of the selected keypoints.
orientation_reference_keypoint
integer | string
required
The keypoint whose direction from the centered reference keypoint egocentric alignment rotates onto a fixed axis, as a keypoint index or a keypoint name. It must be one of the selected keypoints, and a keypoint other than the centered one.
run_lowconf_cleaning
boolean
true
Replace each point whose confidence is below pose_confidence by linear interpolation along its series. Off, every frame with a missing point ends its session, and max_gap_frames must be 0.
pose_confidence
number
0.99
The confidence below which low-confidence cleaning replaces a point. Confidence is read from the confidence columns when every selected keypoint has one, and is otherwise 1 for every finite point.
run_outlier_cleaning
boolean
true
Replace each point whose absolute z-score exceeds iqr_factor times the interquartile range of its series' z-scores, by linear interpolation.
iqr_factor
integer
4
The multiple of the interquartile range beyond which outlier cleaning replaces a point.
run_savgol_filtering
boolean
true
Smooth each coordinate series with a Savitzky-Golay filter.
savgol_length
integer
5
The Savitzky-Golay filter's window. It is odd, longer than savgol_order, and at most time_window. [frames]
savgol_order
integer
2
>= 0
The order of the Savitzky-Golay filter's polynomial.
run_rescaling
boolean
false
Divide each session's positions by the session's median distance between the two reference keypoints, after the other preprocessing. A session is a run of an individual's frames that no gap longer than max_gap_frames splits.
max_gap_frames
integer
30
>= 0
The longest gap inside a session that low-confidence cleaning fills by interpolation. It must be 0 without run_lowconf_cleaning. A gap is a run in which either reference keypoint lacks a finite point at or above pose_confidence. A longer gap splits the individual into separate sessions, and a gap at its start or end is trimmed. The rows of these gaps do not receive a latent, and their count is reported. The missing points of the other keypoints are interpolated however long they run. A session is excluded when one of them lacks such a point at every frame. [frames]
test_fraction
number
0.1
> 0, < 1
The fraction of the training data that VAME sets aside to compute the test loss. Training saves the model of the epoch with the lowest test reconstruction loss, among the epochs with a KL weight above 0.99.
split_mode
"mode_1" | "mode_2"
"mode_2"
How VAME chooses the test data. mode_1 takes the first test_fraction of the concatenated sessions, and mode_2 a random contiguous chunk of that fraction from each session.
random_state
integer
42
Seeds every random number generator that training draws from.
zdims
integer
30
>= 1
The dimensions of the latent space. Each dimension is one output column, vame_z0 to vame_z{zdims-1}.
time_window
integer
30
>= 1
The window that the encoder reads to produce one latent. Each latent is written on the row at the center of its window. [frames]
max_epochs
integer
500
VAME trains from epoch 1 to epoch max_epochs - 1, unless model_convergence stops it first. VAME saves a model only from the first epoch whose KL weight exceeds 0.99. A max_epochs that ends training earlier is refused. [epochs]
model_convergence
integer
50
How long VAME may train in a row without saving a model. It stops training at the next epoch without a save. Every epoch before the first one whose KL weight exceeds 0.99 is one without a save. A model_convergence that stops training earlier is refused. [epochs]
batch_size
integer
256
>= 4
The windows of one training batch. A test batch contains a quarter of them.
beta
number
1.0
The factor on the KL loss, which the annealed KL weight scales.
learning_rate
number
0.0005
The learning rate of the Adam optimizer.
prediction_decoder
boolean
true
Train a second decoder that predicts the frames after each window. The first decoder reconstructs the window.
prediction_steps
integer
15
>= 1
How far past each window the prediction decoder predicts. It is at most time_window. [frames]
noise
boolean
false
Add Gaussian noise to the training input, scaled by each input series' standard deviation in the window.
scheduler
boolean
true
Multiply the learning rate by scheduler_gamma when the loss of each epoch's last training batch has not fallen a relative 1e-3 below its lowest value for more than scheduler_step_size epochs. Off, the learning rate stays constant.
scheduler_step_size
integer
100
How long the loss of each epoch's last training batch may go without improving. The scheduler lowers the learning rate at the next epoch without an improvement. [epochs]
scheduler_gamma
number
0.2
The factor by which the scheduler multiplies the learning rate.
softplus
boolean
false
Pass the encoder's log-variance through a softplus activation.
steps_per_epoch
integer | None
null
The batches of one training epoch, at least 1. Unset, an epoch draws one window per training frame.
hidden_size_layer_1
integer
256
The hidden units of each of the encoder's two recurrent layers.
dropout_encoder
number
0.0
The dropout probability between the encoder's two recurrent layers.
hidden_size_rec
integer
256
The hidden units of the reconstruction decoder's recurrent layer.
hidden_size_pred
integer
256
The hidden units of the prediction decoder's recurrent layer.
mse_reconstruction_reduction
"sum" | "mean"
"sum"
How the reconstruction loss reduces over a batch: sum or mean.
mse_prediction_reduction
"sum" | "mean"
"sum"
How the prediction loss reduces over a batch: sum or mean.
kmeans_loss
integer
30
The cluster loss sums the square roots of this many of the largest singular values of the latent Gram matrix.
kmeans_lambda
number
0.1
The weight of the cluster loss.
anneal_function
"linear" | "sigmoid"
"linear"
The schedule on which the KL weight rises after kl_start: linear or sigmoid.
kl_start
integer
2
>= 0
The last epoch at which the KL weight is 0. [epochs]
annealtime
integer
4
>= 1
How the KL weight rises after kl_start. The linear schedule raises the weight to 1 at epoch kl_start + annealtime, and the sigmoid schedule is centered on epoch annealtime. [epochs]
embed_batch_size
integer
64
>= 1
How many windows VAME encodes at once when it applies a model. It changes latents only at float32 rounding, about 1e-6.
embed_chunk_frames
integer
50000
>= 1
The longest stretch of a session that one embedding call reads. A longer session is embedded in chunks that overlap by time_window - 1. Chunking changes latents only at float32 rounding. Lower it when embedding runs out of memory. It is at least time_window. [frames]
scratch_dir
string | None
null
The directory in which the feature creates its exchange directory for the runner's files. Unset, the system's temporary directory.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
VameModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"vame"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"vame_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.4 · mosaic.behavior.feature_library.xgboost_feature.XgboostFeature
XGBoost behavior classifier as a pipeline feature.
Parameter
Type
Default
Constraints
Description
templates
LabeledTemplatesRef | None
null
The labeled templates artifact to train on. Mutually exclusive with model.
model
XgboostModelArtifact | None
constructed
A pre-fitted XgboostModelArtifact to load, skipping training. Mutually exclusive with templates.
strategy
"multiclass" | "one_vs_rest"
"multiclass"
The classification strategy. multiclass fits one multi-class model. one_vs_rest fits one binary classifier per class.
decision_threshold
number | object | None
null
Unset, the predicted class is the class with the highest probability. A float applies one probability threshold to every class, and a mapping from class to threshold applies a per-class one. Any row where no class clears its threshold is assigned default_class.
default_class
integer
required
The class assigned to a row where decision_threshold is set and no class clears its threshold.
class_weight
"balanced" | object | None
"balanced"
Unset, every training sample is weighted equally. balanced weights each sample inversely to its class's frequency. A mapping from class to weight gives each sample its class's weight, 1 for a class it does not list, and applies no balanced weighting.
use_smote
boolean
false
Apply SMOTE oversampling to the training set, after any undersample_ratio undersampling.
undersample_ratio
number | None
null
Unset, no class is undersampled before training. Set, every class larger than the minority class is undersampled to this many times the minority class's size, never below the minority class's own size.
n_estimators
integer
100
>= 1
The number of boosting rounds the model fits.
max_depth
integer
6
>= 1
The maximum depth of each boosted tree.
learning_rate
number
0.1
> 0
The boosting learning rate (XGBoost's eta), scaling each tree's contribution to the ensemble.
subsample
number
0.8
> 0, <= 1
The fraction of training rows sampled for each boosting round.
colsample_bytree
number
0.8
> 0, <= 1
The fraction of feature columns sampled for each tree.
random_state
integer
42
The random seed for XGBoost training and, when enabled, for undersampling and SMOTE.
cv_folds
integer | None
null
Unset, one model is fitted and scores every entry. Set to K (at least 2), the training sequences of the templates are dealt into K folds and K more models are fitted, each on the training rows outside one fold. A training sequence is then scored by the model that did not see it, and every other entry by the model fitted on all training rows. The fold of each sequence is in cv_folds.parquet. Needs templates and is refused beside model.
cv_seed
integer
0
The seed that deals the training sequences into folds. It changes the folds and nothing else, so runs that differ in random_state are compared on the same folds. Other than 0 only with cv_folds.
device
"cpu" | "cuda"
"cpu"
Which device fits the model. cpu fits on the processor. cuda fits on the GPU and fails where XGBoost has no CUDA device. The step stays on a CPU worker either way, so cuda runs only where the run is placed on a GPU machine. A fitted model predicts on the CPU, so a model fitted on cuda applies anywhere.
n_jobs
integer | None
null
The number of threads each fit uses. Unset, XGBoost uses every core. It changes the time of a fit and not its model, so it is not part of the run id.
JoblibLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"joblib"
"joblib"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
key
string | None
null
Dict key extracted from the loaded object. Unset returns the object as loaded.
LabeledTemplatesRef
Parameter
Type
Default
Constraints
Description
feature
string
""
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
ParquetLoadSpec
constructed
How to load the matched files.
pattern
string
"templates.parquet"
Glob pattern. Auto-derived from load.kind when empty.
ParquetLoadSpec
Parameter
Type
Default
Constraints
Description
kind
"parquet"
"parquet"
Fixed tag identifying this load specification's format. Selects the matching spec when a LoadSpec value is parsed.
transpose
boolean
false
Transpose the loaded table after column filtering.
columns
list of string | None
null
Column names read from the file. Unset reads every column, and numeric_only then filters the result.
drop_columns
list of string | None
null
Column names dropped after loading. A name absent from the file is ignored.
numeric_only
boolean
true
Keep only numeric-dtype columns. Ignored when columns is set.
frame_column
string | None
null
Column meant to be extracted as frame indices. Unwired: no code path reads this field -- load_from_spec's ParquetLoadSpec case does not destructure it, and nothing else reads spec.frame_column.
XgboostModelArtifact
Parameter
Type
Default
Constraints
Description
feature
string
"xgboost"
Feature name whose output to consume.
run_id
string | None
null
Specific run ID, or None for latest finished run.
execution_id
string | None
null
ULID of the attempt that produced this result. It identifies the attempt, and run_id identifies the content.
cache_hit
boolean
false
Whether the producing run was fully served from cache.
failed_entries
list of string
[]
Entry keys whose apply raised while the run continued. Empty for a clean run. A non-empty list marks a partial run, and its outputs are missing these entities.
entries_written
integer
0
How many entries have a valid output row from the producing run, cache hits included. A resumed run and a fresh one therefore report the same number over the same scope.
load
JoblibLoadSpec
constructed
How to load the matched files.
pattern
string
"xgboost_model.joblib"
Glob pattern. Auto-derived from load.kind when empty.
Version 0.3 · mosaic.behavior.visualization_library.egocentric_crop.EgocentricCrop
Generate egocentric (animal-centered) video crops.
Parameter
Type
Default
Constraints
Description
target_id
integer | None
null
ID of the individual to center the crop on. Unset, the feature processes every individual found in the tracks and writes a separate output for each.
center_mode
string | integer
"default"
How to compute the crop center. Known values are default, xy and pose0. default averages the pose points present on each row, and uses the body center where the table has no pose columns. xy uses the body center alone, even where pose points exist. pose0 uses the first pose point, and an integer names a specific pose point index. Reading the body center needs pixel coordinates, and so do the pose points. A run refuses an entry recorded in centimeters (the trex_v1 and mosaic_cm_v1 schemas) under every mode. An unrecorded schema is read as trex_v1. A schema that declares no coordinate space is cropped as pixels, with a warning.
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
crop_size
tuple of (integer, integer)
[256, 256]
min items 2, max items 2
Width and height of the output crop. [px]
rotate_to_heading
boolean
true
Rotate the crop so the animal's heading aligns with the +x axis.
heading_points
tuple of (integer, integer)
[3, 6]
min items 2, max items 2
Pose point indices used for heading, as (neck index, tail index). The heading direction runs from the tail to the neck, the direction the animal faces.
margin_factor
number
1.5
Extra margin for the pre-rotation crop, as a multiple of the final crop size.
center_offset_px
number
0.0
Offset from the computed center along the heading direction, positive toward the head. Useful for centering on a specific body part instead of the detected center. [px]
body_mask
boolean
false
Apply an elliptical mask isolating the focal individual.
body_mask_length_px
integer
96
Full length of the body mask ellipse along its major axis. [px]
body_mask_width_px
integer
64
Full length of the body mask ellipse along its minor axis. [px]
use_clahe
boolean
false
Apply CLAHE (Contrast Limited Adaptive Histogram Equalization) to the crop.
clahe_clip_limit
number
2.0
Clip limit for CLAHE contrast enhancement.
clahe_tile_grid_size
integer
25
CLAHE tile grid size, applied to both width and height.
grayscale
boolean
false
Convert the crop to single-channel grayscale.
transform_keypoints
boolean
false
Transform pose keypoints into crop coordinates and include them in the output as poseX_crop and poseY_crop.
output_mode
string
"video"
Output format. Known values are video, frames and both. video writes a single video file per individual, frames writes individual frame images, and both writes both.
output_fps
number | None
null
Output video frame rate. Unset, uses the source video frame rate. [fps]
output_root
string | None
null
Directory the crops are written to, under a __ subdirectory. Unset, the crops go to the run root, or to media/egocentric_crops/ when the feature runs outside run_feature.
frame_format
string
"png"
Image file format for frame outputs. Known values are png and jpg.
interpolation
integer
1
OpenCV interpolation flag used when rotating the crop. Known values are cv2.INTER_NEAREST, cv2.INTER_LINEAR, cv2.INTER_CUBIC, cv2.INTER_AREA and cv2.INTER_LANCZOS4.
background_color
integer
0
Fill value for pixels with no source content, for example 0 for black or 255 for white. It pads where the crop window extends past the frame, fills the border rotation introduces, and fills the whole crop where the center is not finite.
angle_col
string | None
null
Name of a track column recording a pre-computed heading angle, in degrees or radians (auto-detected). Unset, heading is derived from heading_points.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
Version 0.3 · mosaic.behavior.visualization_library.interaction_crop.InteractionCropPipeline
Generate egocentric crop videos for detected interaction segments.
Parameter
Type
Default
Constraints
Description
crop_size
tuple of (integer, integer)
[192, 192]
min items 2, max items 2
Width and height of the output crop. [px]
pose
PoseConfig
constructed
Pose keypoint configuration: indices and column prefixes.
center_mode
string | integer
"default"
How to compute the crop center. Known values are default, xy and pose0. default averages the pose points present on each row, and uses the body center where the table has no pose columns. xy uses the body center alone, even where pose points exist. pose0 uses the first pose point, and an integer names a specific pose point index. Reading the body center needs pixel coordinates, and so do the pose points. A run refuses an entry recorded in centimeters (the trex_v1 and mosaic_cm_v1 schemas) under every mode. An unrecorded schema is read as trex_v1. A schema that declares no coordinate space is cropped as pixels, with a warning.
center_offset_px
number
0.0
Offset from the computed center along the heading direction, positive toward the head. Useful for centering on a specific body part instead of the detected center. [px]
rotate_to_heading
boolean
true
Rotate the crop so the animal's heading aligns with the +x axis.
heading_points
tuple of (integer, integer)
[3, 6]
min items 2, max items 2
Pose point indices used for heading, as (neck index, tail index). The heading direction runs from the tail to the neck, the direction the animal faces.
margin_factor
number
1.5
Extra margin for the pre-rotation crop, as a multiple of the final crop size.
angle_col
string | None
null
Name of a track column recording a pre-computed heading angle, in degrees or radians (auto-detected). Unset, heading is derived from heading_points.
interpolation
integer
1
OpenCV interpolation flag used when rotating the crop. Known values are cv2.INTER_NEAREST, cv2.INTER_LINEAR, cv2.INTER_CUBIC, cv2.INTER_AREA and cv2.INTER_LANCZOS4.
background_color
integer
0
Fill value for pixels with no source content, for example 0 for black or 255 for white. It pads where the crop window extends past the frame, fills the border rotation introduces, and fills the whole crop where the center is not finite.
body_mask
boolean
false
Apply an elliptical mask isolating the focal individual.
body_mask_length_px
integer
96
Full length of the body mask ellipse along its major axis. [px]
body_mask_width_px
integer
64
Full length of the body mask ellipse along its minor axis. [px]
use_clahe
boolean
false
Apply CLAHE (Contrast Limited Adaptive Histogram Equalization) to the crop.
clahe_clip_limit
number
2.0
Clip limit for CLAHE contrast enhancement.
clahe_tile_grid_size
integer
25
CLAHE tile grid size, applied to both width and height.
grayscale
boolean
false
Convert the crop to single-channel grayscale.
crop_both_individuals
boolean
true
Crop both individuals of each pair. False keeps only the first perspective, halving the output.
output_fps
number | None
null
Output video frame rate. Unset, uses the source video frame rate. [fps]
output_root
string | None
null
Directory the crops are written to. Unset, the crops go to the run root, or to media/interaction_crops/ when the feature runs outside run_feature.
PoseConfig
Parameter
Type
Default
Constraints
Description
pose_n
integer
7
Total number of pose keypoints per individual, before any subset selection by pose_indices.
pose_indices
list of integer | None
null
Zero-based keypoint indices to use, as positions into the full keypoint set. Unset uses every keypoint counted by pose_n.
x_prefix
string
"poseX"
Column name prefix for a keypoint's X coordinate, followed by its index.
y_prefix
string
"poseY"
Column name prefix for a keypoint's Y coordinate, followed by its index.
confidence_prefix
string
"poseP"
Column name prefix for a keypoint's confidence score, followed by its index.
keypoint_names
list of string | None
null
Human-readable name for each keypoint, ordered to match the keypoint columns. Its length must equal pose_n when set. Unset, a feature that needs names generates them (kp0, kp1, ...).
Version 0.1 · mosaic.behavior.visualization_library.overlay_feature.Overlay
Render one annotated video per sequence, from tracks and drawn labels.
Parameter
Type
Default
Constraints
Description
label_kind
string | None
"behavior"
Kind of converted labels to draw as ground truth. Unset, no ground truth is drawn.
color_by
string | None
null
Name of an input feature to color the drawing by, matched without regard to case. The reserved value gt colors by the ground-truth label.
label_maps
object
constructed
Names to render per feature, keyed by the label value written as text.
hide_unlabeled
boolean
false
Skip drawing individuals that lack a label after filtering and mapping.
start
integer
0
The first frame to draw.
end
integer | None
null
The last frame to draw, inclusive. Unset, drawing continues to the end of the video.
downscale
number
1.0
Downscale factor applied to the frame, where 1.0 is no scaling.
show_individual_bboxes
boolean
true
Draw a bounding box for each individual. Pose points and labels are drawn regardless.
pair_box_feature
string | None
null
Name of the pair-label feature that determines when to draw a union box around both individuals of a pair.
pair_box_behaviors
list of string
constructed
Behavior values that trigger drawing a pair-level box.
hide_individual_bboxes_for_pair
boolean
false
Skip drawing an individual's own bounding box when it is already inside a drawn pair box.
draw_options
object
constructed
Per-frame drawing options. Recognized keys are show_labels, point_radius, bbox_thickness and font_scale.
camera
string | None
null
Which camera of a synchronized recording to draw. Required when the sequence spans more than one, because two views are not one timeline. The media resolver refuses to pick between them rather than concatenating them into a fabricated timeline.