Formats

AvialSync is intentionally open to lab-specific formats. It includes common video and tabular-data support, while plugins can add other recordings without changing the application itself.

Video

AvialSync decodes video itself and needs nothing installed alongside it, so what it can open is whatever its bundled FFmpeg supports — 272 video decoders, including H.264, HEVC, VP8/VP9, AV1, MPEG-4, MJPEG, ProRes, DNxHD, FFV1, and DV. This no longer varies with how a machine’s own FFmpeg happened to be compiled.

Common containers — .mp4, .m4v, .mov, .mkv, .webm, .avi, .mpg, .ts, .mts, .m2ts, .wmv, .flv, .ogv, .3gp, .dv, .mxf, .vob, .y4m — are recognised from the file name. A recording with an unfamiliar extension is still opened: AvialSync reads its header and accepts it if it genuinely holds video, so a rig that names its files something of its own works without anyone extending a list.

Still images are deliberately refused even though FFmpeg would open a PNG as a one-frame video, and tables, arrays, and audio are left to the loaders that understand them.

Raw formats such as .bin are the exception, and unavoidably so: they carry no header describing resolution, pixel format, or rate, so nothing can infer how to read them. Those belong in a plugin, which can declare the parameters and convert on import.

Timing comes from the frames, not the container

AvialSync reads every frame’s own presentation timestamp rather than trusting the rate a container declares. That matters more often than it sounds: a camera running at a varying exposure trigger routinely writes a file claiming a constant 30 fps, and its own timestamps prove otherwise.

Those timestamps decide whether a recording is treated as CFR or VFR, where a frame step lands, and which frame is named at any moment. They are cached in the per-user cache folder after the first read, so opening it again does not walk the file a second time.

AOL camera sessions and microscope trials

An AOL camera session can contain .mp4 or .avi cameras beside matching <camera>-relative times.txt files. Matching uses the exact camera stem, including underscores; when both extensions share a stem, the MP4 is selected and the AVI is reported as a duplicate. The relative-times file supplies every AOL camera frame time in milliseconds, for both MP4 and AVI. The camera timing report is read when present to compare its frame counts and rate with the timing rows and video headers. Unreadable files and disagreements are reported while the rest of the session continues loading.

The wall-clock stamps in the per-camera files have no time-zone marker. AvialSync treats the first stamp as the camera PC’s local wall-clock fields and uses the median camera start as the shared session start. This follows the rig’s external-trigger setup; the camera stop is not triggered, so the cameras may end on different frames. A spread larger than one frame period is shown as a quality warning. The timestamp is not evidence of UTC synchronization between computers.

A microscope trial folder (HH-MM-SS) or experiment folder (experiment_N) opens as one imaging source: every RibbonScan_ROI_*.mat tile of the scan in one mosaic, laid out as the reconstructed dendritic tree (branches as columns, their segments stacked, population patches beside them) and oriented as the lab’s own figures show it. Each frame carries the midpoint of its line scan; the timing note gives how far its tiles were scanned either side. An experiment folder places each trial at its own STARTTIME, so the time between trials stays on the timeline as a gap, and the trials are drawn as a Trials lane in Data Streams and as dashed boundaries at both edges of each trial on the plots. Each camera’s per-trial recordings become one video on the same timeline, each started on its trial’s trigger and trimmed to the trial. Between trials nothing is shown: the cameras show No Footage, the imaging viewer says there is no imaging data, and the plots have a gap. A trial without a STARTTIME, or one that starts inside the trial before it, is left out and named; only when no trial recorded a STARTTIME do the trials play back to back, and the warnings say so. The recordings are not changed or copied.

The camera PC names its folders by its own clock, so they never match the trial names (12-21-24 for trial 12-21-28). AvialSync pairs them by when they recorded: the cameras’ first-frame stamp against each trial’s STARTTIME, after removing the controller’s time zone (from its folder names) and the offset most recordings agree on between the two PCs’ clocks. Pairs are one to one and in time order, within Preferences → AOL → Camera-to-trial match tolerance. A lone recording left between two matched trials, beside a lone unmatched trial, is paired with it “by order only” and reported. A camera that started more than a second from its trial’s trigger is placed where its own clock says rather than on the trigger, and only its frames inside the trial are joined; one that recorded entirely outside its trial is named and left out, as is a recording no trial was running for. Every such case is listed in the status bar’s warnings. Cameras are looked for inside and beside the trial folders, under AOL camera saved-data folder, and in any folder beside the date folder that mirrors <date>/<experiment> (for example videos/2026-09-03/experiment_2 next to 2026-09-03).

What you drop decides what is joined. One trial folder opens that trial and its cameras; an experiment folder, or several trial folders selected together, open together, each trial at its STARTTIME. Trial and camera folders (or an experiment folder and its camera experiment folder) dropped together are paired with each other, wherever the camera folders are. Camera folders dropped onto trials already open join those trials where they now sit; trials dropped onto an open camera session are placed on its cameras. Several camera folders dropped on their own (or the camera PC’s experiment folder) load as their trials when AvialSync can find them, beside the camera tree or under AOL microscope saved-data folder: the imaging is offered in the import review, where you can skip it. Without trials, each camera’s recordings join into one video, each recording at its first-frame stamp on the camera PC’s clock. Either way the same cameras are one video each, not one per recording. Channels named Green and Red start in those colours, taken from the lab’s record (index 1 green, index 0 red, unless the trial’s analysis says otherwise). When a trial’s line clock and duration are missing, the controller’s Log.txt rate for that trial is used before AvialSync asks for one. A single RibbonScan_ROI_*.mat can also be opened on its own like any HDF5 or TIFF stack: its stored 16-bit pixels, both channels, at that ROI’s exact line-clock times when the trial’s params.mat is beside it. The lab’s ROI-analysis files (roi_activity/, thin_mask.mat masks) are not loaded. When the controller logged the running wheel into the trial’s params.mat (behaviour/encoder), its speed is plotted in rpm, on the trial’s clock, each trial at its start; with the stimulus TTL the trial commanded beside it. The controller saves its stimulus trigger settings with each trial – whether it fires during acquisition, the delay from the trial’s start, the pulse width, and the number and period of pulses – and AvialSync rebuilds the 0/1 trace from them, on the trial’s clock and ending with the trial, so the row has a gap between trials: on 2026-09-03 experiment_2 one pulse 4 s into every trial from 12-21-28 on, 1 s long (0.1 s in the last two), exactly as Log.txt notes. No edge is recorded, so this is the commanded schedule rather than a measured one; the source is labelled “stimulus TTL” and its pulses are found by the Stimulus Grid’s threshold scan at the default 0.5. Wheel speed and the TTL come from one trial file and form one group in the grid export, where streams of different units are stacked, each on its own scale. An AOL camera session’s encoder_log.txt is plotted instead when one is open, and its angle is loaded hidden by default so the existing wheel prop can still use it.

With an AOL camera session loaded, Align → Find microscope trial… searches the dated folder, matches the camera wall clock against the trial’s UTC STARTTIME, and presents the evidence for review. The derived time-zone offset assumes both PCs use the same local zone. Accepting places the trial on camera frame zero as one undoable change; it does not alter the already-open camera session. Clock matching was checked against a real experiment (2026-09-03, experiment_2: 18 of 19 camera recordings paired with 24 trials, the camera PC 0.71 s ahead of the controller, two recordings off their trigger). AvialSync does not apply the controller’s optional clock calibration correction.

Two-photon imaging

Open .h5, .hdf5, .tif, .tiff or .nwb through File → Open Imaging or drag the stack into the window. Imaging is a separate time-indexed source, shown below the 3D view when both are present. It follows the same master playhead as video and plots. PyAV continues to decode camera videos only.

The HDF5 reader selects a three-dimensional TYX dataset when the file has exactly one plausible stack; otherwise it lists the candidates to choose from. Four- and five-dimensional datasets need an axis order (T, C, Z, Y, X, from an axes attribute or asked at import). A Z axis with more than one plane asks which depth to show; a C axis is shown as overlaid channels. The TIFF reader supports page-per-time TYX, QYX, or IYX series, OME series with T, optional C/Z, then YX axes, and ScanImage files, whose channel- and slice-interleaved pages are separated using ScanImage’s own frame data. Other TIFF layouts need a format plugin. These choices are saved in the session file.

Timing comes from, in order: a frame-time table you supply, a per-frame timestamp table in the file (an HDF5 frame_times, timestamps or time dataset beside the images, or OME per-plane DeltaT), a frame rate you typed, and finally a rate the file states (HDF5 fps/frame_rate/dt attributes, OME TimeIncrement, ScanImage’s scan frame or volume rate). A stack with none of these asks for its frame rate rather than guessing one.

The viewer reads only the requested HDF5 hyperslab or TIFF page on a background thread and keeps recently read planes under a fixed memory budget, so large stacks do not need to fit in RAM. Brightness, contrast, channel colours and moving averages change the displayed picture only; the raw pixels are unchanged. TIFF compression that needs an unavailable codec is reported for the affected page.

Sensor and tracking data

Delimited text data can be imported through the guided importer. It lets you identify the time column, time units, units for channels, missing-value sentinels, and timestamp details. Tracking CSV files can be treated as frame-indexed when that is how the source was produced.

Pose estimates

DeepLabCut and LightningPose multi-index CSVs are recognised by their own header — a scorer row above a bodyparts row — rather than by their file name, and are read as frame-indexed. Each body part becomes a channel; any complete name_x / name_y / name_z triplet also becomes a point in the 3D pane.

Correcting a point by hand writes pose_csv_avialfix.csv (for a pose.csv) beside the original — a plain frame,bodypart,x,y table with a commented header, readable with pd.read_csv(path, comment="#"). The pose file itself is never modified, so deleting the corrections file restores exactly what the model predicted. Corrections can be exported either as a corrected copy of the pose CSV, which anything that read the original will read unchanged, or as a retraining set for DeepLabCut or single-view Lightning Pose: a new folder whose contents are copied into the project root. See Correcting a tracked point.

Vicon Nexus motion capture

Drop a Vicon session folder containing .c3d trials, matching .xcp calibration files, and the camera AVI files. AvialSync reads marker positions directly from C3D; it does not use companion CSV exports. The XCP video-camera calibration projects the 3D markers into that camera’s image, and the session scanner pairs each trial to its AVI by the calibrated camera’s device ID. Intrinsics, Vicon radial distortion, and video-resolution scaling are applied before projection. The markers are sampled onto the video frame grid and use the normal per-source tracking overlay controls. A trial without a usable calibrated video-camera entry or a uniquely matching AVI is reported and left out of the overlay rather than projected with guessed calibration. When importing a C3D separately, assign its matching AVI as the overlay target; AvialSync reads that video’s dimensions and scales the XCP projection to its pixel grid.

Each Vicon trial is placed where it was captured: its .xcp records every camera’s capture start and end to the millisecond, so the trials of a session sit one after another with the time between them kept. Each camera’s trials load as one video and its markers as one tracking source – one pane, one Data Streams lane, one object in the 3D view – with each trial still projected through its own calibration. A marker a trial did not track is missing for that trial, not interpolated. Those stamps are the capture PC’s local time without a time zone, so they order and space the trials but the clock shows time from the first trial rather than wall-clock time. A video’s file name carries the time it was exported, not captured, and is not used.

To open one trial, copy its .c3d, matching .xcp, and camera AVI into a separate folder and drop that folder onto AvialSync. The review dialog lists the paired video and tracking file, with the Vicon labels, combined 3D-pose/2D-overlay role, and XCP calibration preselected. Confirm the rows to load native XYZ markers in the 3D view and their calibrated projection on the paired video. If you drop the files individually, choose 3D Marker Tracking for the .c3d, select the XCP in the Calibration (XCP) column, then choose the combined 3D-pose and 2D-overlay role for its AVI. The .xcp is not imported as tracking data, and .x2d is not currently supported as a tracking source.

Acquisition recordings

Electrophysiology and instrument recordings are read through neo, so the formats it supports arrive in the same shape as everything else: continuous streams become plots, and TTL lines become a square wave drawn from their edges rather than a dense trace.

Open Ephys folders are recognised as whole sessions. Drop the recording folder — or the folder holding it and the cameras recorded beside it — and the streams, TTL lines, and videos are laid out together on the recording’s own clock, with wall-clock time taken from the recording’s own sync_messages.txt rather than assumed. A folder holding several recordings names each one in the Trials lane where its samples sit; the time between recordings stays a gap.

Both Open Ephys layouts are read, including the free text each one stores:

• Binary format (GUI v0.6 and later) — messages typed into the GUI’s Message Center.

• Original format (.continuous files, earlier versions) — the messages saved in messages.events.

In both cases the recording’s sync preamble is used to place the clock and is not listed as a message: it is what the software wrote about the recording, not something a person typed.

Neurodata Without Borders (NWB)

An .nwb file holds a whole session — electrophysiology, imaging, behaviour, trials — and AvialSync opens it as one. Drop the file (or choose it with File → Open) and the review lists:

• NWB Time Series: every time series in the file, one plot row per column, named by where the series sits (ophys.DfOverF.RoiResponseSeries.roi12, Position.SpatialSeries.x) so that two series with the same name stay apart, and grouped in the channel list on those dots. Columns are named after what they measure where the file says: electrodes by their id (ch17), ROIs by theirs (roi12), spatial series by axis. Values are scaled by the file’s conversion, channel_conversion and offset, and shown in its units (volts reads as V).

• NWB Imaging: each imaging series stored in the file (two-photon, one-photon, any ImageSeries) opens in the imaging viewer, named after the series. It reads one plane at a time in its stored pixel type, so channels, display levels and centred averaging work without making a copy of the full stack. The separate video-compatible proxy reader remains available to plugins; its lossless copy reports progress and completes an in-progress encoding before a cancellation takes effect.

• Videos the file points to: an ImageSeries that names an external video file is opened from beside the NWB file, placed at the time the series says its first frame was.

Everything is placed on the file’s own clock, with no alignment step: every NWB timestamp counts from the file’s timestamps_reference_time (normally its session_start_time), which becomes the session’s zero, and times read as wall clock.

Trials, epochs, and any other interval table become a row that is 1 inside an interval and 0 outside it, plus one message per row listing its columns (trials 3 · 12.3–14.1 s · condition=left). The file’s trials table is also drawn as the Trials lane in Data Streams, each trial where it ran, with dashed boundaries on the plots. IntervalSeries (reward, licking epochs) become the same kind of row. Spike-sorted units become one row per unit, high for an instant at each spike. Text annotations become messages.

What cannot be shown is listed when the file opens rather than skipped silently: series whose data is not a signal over time (spike waveforms, frequency decompositions), a video the file names but that is not beside it, and a start time recorded without a time zone (read as UTC).

Versions: NWB 1.x and 2.x HDF5 files, and NWB Zarr v2/v3 folders. Extension types are recognised from the specification each file carries when available. NWB 1.x series under acquisition/timeseries use the same plotted and imaging routes. Choose File → Open NWB… and browse to a local file, or paste a DANDI asset download URL or its public S3 content URL; the app streams the requested HDF5 ranges or Zarr chunks and keeps a small link in your application-data folder for session restore. External videos referenced by a remote file must be opened locally. Imaging creates a lossless cached copy and shows percentage progress while it is encoded; that encoding finishes once started.

Trigger and TTL files

Delimited text carrying pulses rather than data: a DAQ export with a time column and one or more logical lines, or a camera log with one timestamp per exposure. Open these through Align → Open Trigger Evidence… rather than as sensor data — they are evidence about when things happened, and nothing in them is plotted.

AvialSync reads the header and offers every column that looks like a trigger line. You then say what each one is, because the file cannot: whether a line is a camera’s exposure strobe, an external frame trigger, a shared sync pulse train, or a handful of landmarks. That declaration decides which alignment models are available, so nothing is assumed for you — every column is offered as a sync train until you say otherwise. See the alignment tutorial for what each kind licenses.

One file may carry several lines about different cameras. What you declared is saved with the session; the pulses are re-read from the file each time it loads.

Lab formats

Ask your lab for its AvialSync plugin, or see the plugin guide to write one. Plugins describe how to recognise, read, and label a format; they do not need to change the core app.