Media steps¶
7 media steps are registered. A preprocess run applies
a list of them to every frame of an entry, in order, and writes the result
as a media variant. Pre-process media for a tracker
shows how to make one and track it.
Every position a step names is a pixel of the original video, and every frame
number is a frame of the original video, whatever the step's position in the
list.
Each step is written as an object whose step key is its name, for example
{"step": "crop", "x": 100, "y": 40, "width": 640, "height": 480}.
- Moves pixels: the step changes where a point of the scene appears in
the image, as
cropdoes. - Appearance: the step changes every pixel by one rule that does not depend on the pixel's position.
A new version of a step gives every variant that uses the step a new run id.
| Step | Version | Moves pixels | Appearance | Summary |
|---|---|---|---|---|
adjust |
0.1 |
no | yes | Scale, offset and gamma-correct every pixel value through one lookup table. |
clahe |
0.1 |
no | yes | Equalize contrast locally, tile by tile, with a limit on the contrast gained. |
crop |
0.1 |
yes | no | Cut the image down to a rectangle given in source coordinates. |
decimate |
0.1 |
no | no | Keep every n-th frame, where n is every, starting with the first. |
grayscale |
0.1 |
no | yes | Convert a BGR frame to gray. The frames stay gray through later steps. |
mask |
0.1 |
no | no | Fill the pixels on one side of a polygon with black. |
trim |
0.1 |
no | no | Keep the frames from start up to, not including, stop. |
In the tables below, required means the parameter has no default and must be supplied. constructed means it is optional but its default is built by a factory rather than being a literal, so there is no value to print.
adjust¶
Version 0.1 · mosaic.core.media.preprocess.steps.AdjustStep
Scale, offset and gamma-correct every pixel value through one lookup table.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"adjust" |
"adjust" |
The step's name, which selects this step in a list of steps. | |
brightness |
number |
0.0 |
>= -255.0, <= 255.0 |
The level added to every scaled pixel value, from -255 to 255. Zero leaves the values unchanged. |
contrast |
number |
1.0 |
> 0.0 |
The factor that every pixel value is scaled by before the brightness is added. One leaves the contrast unchanged. |
gamma |
number |
1.0 |
> 0.0 |
The gamma applied after brightness and contrast, as 255 * (v / 255) ** (1 / gamma). Above one brightens the mid-tones, and one leaves them unchanged. |
clahe¶
Version 0.1 · mosaic.core.media.preprocess.steps.ClaheStep
Equalize contrast locally, tile by tile, with a limit on the contrast gained.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"clahe" |
"clahe" |
The step's name, which selects this step in a list of steps. | |
clip_limit |
number |
2.0 |
> 0.0 |
The contrast limit of the equalization. Higher equalizes harder and amplifies more noise. |
tile_grid_size |
integer |
8 |
>= 1 |
The number of tiles along each side of the image. Each tile is equalized on a histogram of its pixels alone. |
crop¶
Version 0.1 · mosaic.core.media.preprocess.crop.CropStep
Cut the image down to a rectangle given in source coordinates.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"crop" |
"crop" |
The step's name, which selects this step in a list of steps. | |
x |
integer |
required | >= 0 |
The source column of the rectangle's left edge. [px] |
y |
integer |
required | >= 0 |
The source row of the rectangle's top edge. [px] |
width |
integer |
required | >= 4 |
The rectangle's width. Even, because the variant is encoded as yuv420p, and at least the AV1 encoder's minimum. [px] |
height |
integer |
required | >= 4 |
The rectangle's height. Even, because the variant is encoded as yuv420p, and at least the AV1 encoder's minimum. [px] |
decimate¶
Version 0.1 · mosaic.core.media.preprocess.frames.DecimateStep
Keep every n-th frame, where n is every, starting with the first.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"decimate" |
"decimate" |
The step's name, which selects this step in a list of steps. | |
every |
integer |
required | >= 2 |
Keep one frame in this many, starting with the first. |
grayscale¶
Version 0.1 · mosaic.core.media.preprocess.steps.GrayscaleStep
Convert a BGR frame to gray. The frames stay gray through later steps.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"grayscale" |
"grayscale" |
The step's name, which selects this step in a list of steps. |
mask¶
Version 0.1 · mosaic.core.media.preprocess.mask.MaskStep
Fill the pixels on one side of a polygon with black.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"mask" |
"mask" |
The step's name, which selects this step in a list of steps. | |
polygon |
list of tuple of (integer, integer) |
required | min items 3 |
The polygon's vertices as (x, y) source coordinates, at least three. It may extend outside the image but may not lie wholly outside it. [px] |
keep |
boolean |
true |
Keep the pixels inside the polygon and black out the rest. False blacks out the inside instead. |
trim¶
Version 0.1 · mosaic.core.media.preprocess.frames.TrimStep
Keep the frames from start up to, not including, stop.
| Parameter | Type | Default | Constraints | Description |
|---|---|---|---|---|
step |
"trim" |
"trim" |
The step's name, which selects this step in a list of steps. | |
start |
integer |
required | >= 0 |
The first source frame of the range kept. |
stop |
integer |
required | The source frame that the kept range stops before. |