Guides

Media

Guide	For
Prepare and index your media	Indexing, transcoding, imgstore, frame extraction, calibration
Pre-process media for a tracker	Cropping, masking, trimming and adjusting video into a variant that a tracker reads
Render an annotated video	Drawing tracks, identities and behaviors back onto the recording

Declaring a media source and scanning it is dataset setup — see The mosaic dataset.

Tracking

Guide	For
Import tracks you already have	Something else did the tracking, and you have the files
Run a tracker	You have video, and one of four integrated tools can track it
Train a pose model	No off-the-shelf model works on your animal
Write a converter	Your format is not one mosaic already reads

Every tracks table is video pixels with X/Y at the body center, whichever tracker produced it. What a tracker reports covers the units, and why a tracker may not give you a speed.

Analysis

Everything computed from tracks is a feature, and there are five kinds. Per-frame features put a value on every row — kinematics, heading, neighbors, pair geometry. Global features are fitted once across many sequences and then applied — scalers, t-SNE, AR-HMM, keypoint-MoSeq, VAME, the classifiers. Summary features reduce many rows to few. Media features write an image or video that something else reads, such as an annotated overlay. Tag features add identity columns. All five run and cache the same way, which is what lets them chain.

Guide	For
Features	What the library holds and how to run one
Import behavior annotations	You have BORIS or CalMS21 annotations to bring in
Find behaviors without labels	Embedding, clustering and state models
Train a behavior classifier	You have labels and want predictions on the rest
Identify individuals by appearance	Recovering identity from how an animal looks
Write your own feature	The library does not have the quantity you need

Pipelines

Everything that mosaic does can be composed into a pipeline, which is a step-by-step processing recipe that includes the parameters used. For example, features can be chained together to define an analysis pipeline.

Guide	For
Chain steps into a recipe	Composing steps, in a notebook or as a portable file
Run work and see what ran	Executing, scoping, and reading the outcome
Keep a dataset organized	Inventory, re-indexing, and reclaiming space