Skip to content

Behavioral recordings

Example datasets

Datasets containing behavioral data can be found in the BIDS examples repository and can be used as helpful guidance when curating new datasets.

Template:

sub-<label>/
    [ses-<label>/]
        beh/
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>]_beh.json
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>]_beh.tsv
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_audio.flac
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_audio.json
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_audio.mp3
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_audio.ogg
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_audio.wav
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_video.avi
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_video.json
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_video.mkv
            sub-<label>[_ses-<label>][_task-<label>][_acq-<label>][_run-<index>][_split-<index>][_recording-<label>]_video.mp4
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>]_events.json
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>]_events.tsv
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>][_recording-<label>]_physio.json
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>][_recording-<label>]_physio.tsv.gz
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>][_recording-<label>]_stim.json
            sub-<label>[_ses-<label>]_task-<label>[_acq-<label>][_run-<index>][_recording-<label>]_stim.tsv.gz
Legend:

  • For more information about filename elements (for example, entities, suffixes, extensions), follow the links embedded in the filename template.

  • <matches> is a placeholder to denote an arbitrary (and valid) sequence of entities and labels at the beginning of the filename (only BIDS "raw").

  • <source-entities> is a placeholder to denote an arbitrary sequence of entities and labels at the beginning of the filename matching a source file from which the file derives (only BIDS-Derivatives).

  • Filename entities or directories between square brackets (for example, [_ses-<label>]) are OPTIONAL.

  • Some entities may only allow specific values, in which case those values are listed in <>, separated by |.

  • _<suffix> means that there are several (>6) valid suffixes for this filename pattern.

  • .<extension> means that there are several (>6) valid extensions for this file type.

  • [.gz] means that both the unzipped and gzipped versions of the extension are valid.

The beh directory MAY store behavioral recordings such as audio (_audio.*) and video (_video.*) recordings, physiological (_physio.*) recordings, and other continuous recordings (_stim.tsv.gz, _stim.json). Audio and video recordings MAY be of subjects performing tasks, resting-state behavior, or recordings of stimuli being presented to the subject. Audio/video recordings MAY occur simultaneously with other recordings, such as BOLD or EEG. Relative timing between files may be determined by consulting the scans.tsv file. If no scans.tsv file is present, the alignment is undefined. The beh directory MAY also contain event timing files (_events.tsv) and their associated metadata (_events.json) for behavioral experiments that do not have corresponding neuroimaging or functional data.

Additionally, events files that do not include the mandatory onset and duration columns MAY be included, but MUST be labeled _beh.tsv rather than _events.tsv.

The following OPTIONAL columns are pre-defined for behavioral data files:

Column name Requirement Level Data type Description
trial_type OPTIONAL string Primary categorisation of each trial to identify them as instances of the experimental conditions. For example: for a response inhibition task, it could take on values go and no-go to refer to response initiation and response inhibition experimental conditions.
response_time OPTIONAL number Response time measured in seconds. A negative response time can be used to represent preemptive responses and n/a denotes a missed response.
HED OPTIONAL string Hierarchical Event Descriptor (HED) tags. See the HED Appendix for details.
stim_file OPTIONAL string Represents the location of the stimulus file (such as an image, video, or audio file) presented at the given onset time. There are no restrictions on the file formats of the stimuli files, but they should be stored in the /stimuli/ directory (under the root directory of the dataset; with OPTIONAL subdirectories). The values under the stim_file column correspond to a path relative to /stimuli. For example images/cat03.jpg will be translated to /stimuli/images/cat03.jpg.
Additional Columns OPTIONAL n/a Additional columns are allowed.

Sidecar JSON (*_beh.json)

In addition to the metadata that is either:

  • RECOMMENDED for sidecar JSON files for tabular data, or

  • REQUIRED for some data that can be found in the beh directory (for example SamplingFrequency and StartTime for *_<physio|stim>.tsv.gz files),

it is RECOMMENDED to add the following metadata to the JSON files of this directory.

Task information

Key name Requirement Level Data type Description
TaskName RECOMMENDED string Name of the task. No two tasks should have the same name. The task label included in the filename MAY be derived from this "TaskName" field by removing all non-alphanumeric or + characters (that is, all except those matching [0-9a-zA-Z+]), and potentially replacing spaces with + to ease readability. For example "TaskName" "faces n-back" or "head nodding" could correspond to task labels faces+n+back or facesnback and head+nodding or headnodding, respectively.
Instructions RECOMMENDED string Text of the instructions given to participants before the recording.
TaskDescription RECOMMENDED string Longer description of the task.
CogAtlasID RECOMMENDED string URI of the corresponding Cognitive Atlas Task term.
CogPOID RECOMMENDED string URI of the corresponding CogPO term.

Institution information

Key name Requirement Level Data type Description
InstitutionName RECOMMENDED string The name of the institution in charge of the equipment that produced the measurements.
InstitutionAddress RECOMMENDED string The address of the institution in charge of the equipment that produced the measurements.
InstitutionalDepartmentName RECOMMENDED string The department in the institution in charge of the equipment that produced the measurements.

Audio and video recordings

Audio and video recordings of behaving subjects MAY be stored in the beh directory using the _audio and _video suffixes respectively. These recordings are typically used to capture vocalizations, speech, facial expressions, body movements, or other behavioral aspects during experimental tasks or rest periods.

Privacy and personally identifiable information

Audio and video recordings of human subjects often contain personally identifiable information (PII) such as faces, voices, and other identifying features. Data curators MUST take special care to ensure compliance with applicable privacy regulations (such as HIPAA in the United States, GDPR in the European Union, or other local data protection laws) when handling these recordings.

These recordings are generally more suitable for internal use or for sharing non-human subject data, unless appropriate privacy protections are implemented.

File formats

Audio recordings MUST use one of the following extensions:

  • .flac - Free Lossless Audio Codec
  • .mp3 - MPEG Audio Layer III
  • .ogg - Ogg Vorbis
  • .wav - Waveform Audio File Format

Video recordings MUST use one of the following extensions:

  • .mp4 - MPEG-4 Part 14
  • .mkv - Matroska video container
  • .avi - Audio Video Interleave

Entities

Audio and video files MAY use the following entities:

  • task - OPTIONAL for audio and video recordings
  • acq - OPTIONAL, can distinguish different recording setups
  • run - OPTIONAL, for multiple recordings with identical parameters
  • recording - OPTIONAL, to differentiate simultaneous recordings from different angles, locations, or devices
  • split - OPTIONAL, for continuous recordings split into multiple files

Examples

└─ sub-01/
   └─ beh/
      ├─ sub-01_task-rest_video.mp4 
      ├─ sub-01_task-rest_video.json 
      ├─ sub-01_task-stroop_recording-face_video.mp4 
      ├─ sub-01_task-stroop_recording-face_video.json 
      ├─ sub-01_task-stroop_recording-room_video.mp4 
      ├─ sub-01_task-stroop_recording-room_video.json 
      ├─ sub-01_task-vocalization_audio.wav 
      └─ sub-01_task-vocalization_audio.json

For continuous recordings split into multiple files:

└─ sub-01/
   └─ ses-01/
      └─ beh/
         ├─ sub-01_ses-01_task-freeplay_run-01_split-001_video.mp4 
         ├─ sub-01_ses-01_task-freeplay_run-01_split-002_video.mp4 
         ├─ sub-01_ses-01_task-freeplay_run-01_split-003_video.mp4 
         └─ sub-01_ses-01_task-freeplay_run-01_video.json

Sidecar JSON for audio and video recordings

The following metadata fields are available for audio and video recordings:

Key name Requirement Level Data type Description
Device OPTIONAL string Free-form description of the device used to record the data (for example, "iPhone 12", "Canon EOS R5").
DeviceSerialNumber OPTIONAL string The serial number of the equipment that produced the measurements. A pseudonym can also be used to prevent the equipment from being identifiable, so long as each pseudonym is unique within the dataset.
Key name Requirement Level Data type Description
AudioChannelCount OPTIONAL integer Number of audio channels in the recording (for example, 2 for stereo).

Must be a number greater than or equal to 1.
AudioSampleRate OPTIONAL number Sample rate of the audio recording in Hertz (for example, 44100).

Must be a number greater than 0.
FrameRate OPTIONAL number Frame rate of the video recording in frames per second (for example, 30.0).

Must be a number greater than 0.
Height OPTIONAL integer Height of the video in pixels (for example, 1920).

Must be a number greater than or equal to 1.
Width OPTIONAL integer Width of the video in pixels (for example, 1080).

Must be a number greater than or equal to 1.
Duration OPTIONAL number Total duration of the audio or video recording in seconds.

Must be a number greater than 0.

Example video sidecar JSON

For a video file containing both video and audio streams:

{
  "TaskName": "RestingState",
  "Device": "Sony FDR-AX53",
  "AudioChannelCount": 2,
  "AudioSampleRate": 48000,
  "FrameRate": 30.0,
  "Height": 1920,
  "Width": 1080,
  "Duration": 600.5
}

Example audio sidecar JSON

For an audio-only recording:

{
  "TaskName": "Vocalization",
  "Device": "Zoom H6 Handy Recorder",
  "AudioChannelCount": 2,
  "AudioSampleRate": 44100,
  "Duration": 300.2
}

Annotations and events

Behavioral annotations or event markers for audio and video recordings SHOULD be stored in accompanying _events.tsv files following the standard events file format. These events files use the same filename entities as the audio/video file they describe, but with the _events suffix.

For example:

└─ sub-01/
   └─ beh/
      ├─ sub-01_task-speech_audio.wav 
      ├─ sub-01_task-speech_audio.json 
      ├─ sub-01_task-speech_events.tsv 
      └─ sub-01_task-speech_events.json

Example _beh.tsv

tsv trial response response_time stim_file congruent red 1.435 images/word-red_color-red.jpg incongruent red 1.739 images/word-red_color-blue.jpg

In the accompanying JSON sidecar, the trial column might be documented as follows:

{
   "TaskName": "Stroop",
   "trial": {
      "LongName": "Trial name",
      "Description": "Indicator of the type of trial",
      "Levels": {
         "congruent": "Word and font color match.",
         "incongruent": "Word and font color do not match."
      }
   }
}