catalystneuro / ndx-miniscope

Neurodata Without Borders extension for metadata relevant for miniscope acquisition

ndx-miniscope Extension for NWB

This is a Neurodata Extension (NDX) for Neurodata Without Borders (NWB) 2.0 for Miniscope acquisition data.

Miniscope extends the Device core NWB neurodata_type by including additional metadata for the Miniscope. Depending on the version of the acquisition software the data structure can be quite different.

Miniscope V4 format

The data recorded by the software is saved in a folder structure similar to this:

    C6-J588_Disc5/ (main folder)
    ├── 15_03_28/ (subfolder corresponding to the recording time)
    │   ├── Miniscope/ (subfolder containing the microscope video stream)
    │   │   ├── 0.avi (microscope video)
    │   │   ├── metaData.json (metadata for the microscope device)
    │   │   └── timeStamps.csv (timing of this video stream)
    │   ├── BehavCam_2/ (subfolder containing the behavioral video stream)
    │   │   ├── 0.avi (bevavioral video)
    │   │   ├── metaData.json (metadata for the behavioral camera)
    │   │   └── timeStamps.csv (timing of this video stream)
    │   └── metaData.json (metadata for the recording, such as the start time)
    ├── 15_06_28/
    │   ├── Miniscope/
    │   ├── BehavCam_2/
    │   └── metaData.json
    └── 15_12_28/

Miniscope V3 format

The Miniscope V3 acquisition software generally outputs the following files:

• msCam[##].avi
• behavCam[##].avi
• timestamp.dat
• settings_and_notes.dat

python

Convert to NWB using NeuroConv

Use the MiniscopeConverter from NeuroConv to easily convert Miniscope acquisition data to NWB.

Install NeuroConv with the additional dependencies necessary for reading Miniscope data.

pip install neuroconv[miniscope]

The MiniscopeConverter combines the imaging and behavior data streams into a single conversion.

from dateutil import tz
from neuroconv.converters import MiniscopeConverter

# The 'folder_path' is the path to the main Miniscope folder containing both the recording and behavioral data streams in separate subfolders.
folder_path = "C6-J588_Disc5/"
converter = MiniscopeConverter(folder_path=folder_path, verbose=False)

metadata = converter.get_metadata()
# For data provenance we can add the time zone information to the conversion if missing
session_start_time = metadata["NWBFile"]["session_start_time"]
tzinfo = tz.gettz("US/Pacific")
metadata["NWBFile"].update(session_start_time=session_start_time.replace(tzinfo=tzinfo))

# Choose a path for saving the nwb file and run the conversion
nwbfile_path = "miniscope.nwb"
converter.run_conversion(nwbfile_path=nwbfile_path, metadata=metadata)

Access the data from NWB

Access the Miniscope devices from the in-memory NWBFile.

from pynwb import NWBHDF5IO

nwbfile_path = "miniscope.nwb"
with NWBHDF5IO(nwbfile_path, "r") as io:
    nwbfile_in = io.read()
    # Access the device with the microscope metadata
    nwbfile_in.devices["Miniscope"]
    # Access the device that holds the metadata for the behavior camera
    nwbfile_in.devices["BehavCam2"]

Miniscope abc.Miniscope at 0x5775754960
Fields:
  compression: FFV1
  deviceType: Miniscope_V3
  frameRate: 15FPS
  framesPerFile: 1000
  gain: High
  led0: 47

BehavCam2 abc.Miniscope at 0x5775972816
Fields:
  ROI: <HDF5 dataset "ROI": shape (2,), type "<i8">
  compression: MJPG
  deviceType: WebCam-1920x1080
  framesPerFile: 1000

The imaging data was added to the NWBFile as OnePhotonSeries which can be accessed from the file as the follows:

from pynwb import NWBHDF5IO

nwbfile_path = "miniscope.nwb"
with NWBHDF5IO(nwbfile_path, "r") as io:
    nwbfile_in = io.read()
    # Access the OnePhotonSeries that holds the imaging data from the microscope.
    nwbfile.acquisition["OnePhotonSeries"]

OnePhotonSeries pynwb.ophys.OnePhotonSeries at 0x5775755728
Fields:
  comments: no comments
  conversion: 1.0
  data: <HDF5 dataset "data": shape (100, 752, 480), type "|u1">
  description: Imaging data from one-photon excitation microscopy.
  dimension: <HDF5 dataset "dimension": shape (2,), type "<i8">
  imaging_plane: ImagingPlane pynwb.ophys.ImagingPlane at 0x5775755200
Fields:
  conversion: 1.0
  description: The plane or volume being imaged by the microscope.
  device: Miniscope abc.Miniscope at 0x5775754960
Fields:
  compression: FFV1
  deviceType: Miniscope_V3
  frameRate: 15FPS
  framesPerFile: 1000
  gain: High
  led0: 47
  excitation_lambda: nan
  imaging_rate: 15.0
  indicator: unknown
  location: unknown
  optical_channel: (
    OpticalChannel <class 'pynwb.ophys.OpticalChannel'>
  )
  unit: meters
  interval: 1
  offset: 0.0
  resolution: -1.0
  timestamps: <HDF5 dataset "timestamps": shape (100,), type "<f8">
  timestamps_unit: seconds
  unit: px

The behavior camera data was added to the NWBFile as ImageSeries which can be accessed from the file as the follows:

from pynwb import NWBHDF5IO

nwbfile_path = "miniscope.nwb"
with NWBHDF5IO(nwbfile_path, "r") as io:
    nwbfile_in = io.read()
    # Access the ImageSeries that holds the behavior data.
    nwbfile.acquisition["BehavCamImageSeries"]

BehavCamImageSeries pynwb.image.ImageSeries at 0x5775971616
Fields:
  comments: no comments
  conversion: 1.0
  data: <HDF5 dataset "data": shape (0, 0, 0), type "|u1">
  description: no description
  device: BehavCam2 abc.Miniscope at 0x5775972816
Fields:
  ROI: <HDF5 dataset "ROI": shape (2,), type "<i8">
  compression: MJPG
  deviceType: WebCam-1920x1080
  framesPerFile: 1000
  dimension: <HDF5 dataset "dimension": shape (2,), type "<i8">
  external_file: <StrDataset for HDF5 dataset "external_file": shape (5,), type "|O">
  format: external
  interval: 1
  offset: 0.0
  resolution: -1.0
  starting_frame: [   0  690 1383 2073 2763]
  timestamps: <HDF5 dataset "timestamps": shape (3453,), type "<f8">
  timestamps_unit: seconds
  unit: px

For more information about accessing data in NWB, visit the File Basics tutorial. To learn more about NeuroConv, visit this documentation page.

Installing ndx-miniscope

Get most recent release:

pip install ndx-miniscope

Install latest:

git clone https://github.com/catalystneuro/ndx-miniscope.git
cd ndx-miniscope
pip install -e .

The following code demonstrates the usage of this extension to convert Miniscope acquisition data into NWB.

Usage

from datetime import datetime
from dateutil.tz import tzlocal
import glob
import os
from pynwb import NWBFile, NWBHDF5IO
from pynwb.image import ImageSeries
from natsort import natsorted

from ndx_miniscope.utils import (
    add_miniscope_device,
    get_starting_frames,
    get_timestamps,
    read_miniscope_config,
    read_notes,
)

# The main folder that contains subfolders with the Miniscope data
folder_path = "C6-J588_Disc5/"

# Create the NWBFile
session_start_time = datetime(2017, 4, 15, 12, tzinfo=tzlocal())
nwbfile = NWBFile(
    session_description="session_description",
    identifier="identifier",
    session_start_time=session_start_time,
)

# Load the miscroscope settings
miniscope_folder_path = "C6-J588_Disc5/15_03_28/Miniscope/"
miniscope_metadata = read_miniscope_config(folder_path=miniscope_folder_path)
# Create the Miniscope device with the microscope metadata and add it to NWB
add_miniscope_device(nwbfile=nwbfile, device_metadata=miniscope_metadata)

# Load the behavioral camera settings
behavcam_folder_path = "C6-J588_Disc5/15_03_28/BehavCam_2/"
behavcam_metadata = read_miniscope_config(folder_path=behavcam_folder_path)
# Create the Miniscope device with the behavioral camera metadata and add it to NWB
add_miniscope_device(nwbfile=nwbfile, device_metadata=behavcam_metadata)

# Loading the timestamps
behavcam_timestamps = get_timestamps(folder_path=folder_path, file_pattern="BehavCam*/timeStamps.csv")
# Load the starting frames of the video files
# Note this function requires to have `cv2` installed
starting_frames = get_starting_frames(folder_path=folder_path, video_file_pattern="*/BehavCam*/*.avi")


# Legacy usage for Miniscope V3

ms_files = natsorted(glob(os.path.join(folder_path, 'msCam*.avi')))
nwbfile.add_acquisition(
    ImageSeries(
        name='OnePhotonSeries',  # this is not recommended since pynwb has native OnePhotonSeries
        format='external',
        external_file=[os.path.split(x)[1] for x in ms_files],
        timestamps=get_timestamps(folder_path=folder_path, cam_num=1),
        starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern="msCam*.avi"),
    )
)

behav_files = natsorted(glob(os.path.join(folder_path, 'behavCam*.avi')))
nwbfile.add_acquisition(
    ImageSeries(
        name='behaviorCam',
        format='external',
        external_file=[os.path.split(x)[1] for x in behav_files],
        timestamps=get_timestamps(folder_path=folder_path, cam_num=2),
        starting_frame=get_starting_frames(folder_path=folder_path, video_file_pattern="behavCam*.avi"),
    )
)

annotations = read_notes(folder_path=folder_path)
if annotations is not None:
    nwbfile.add_acquisition(annotations)


save_path = os.path.join(folder_path, "test_out.nwb")
with NWBHDF5IO(save_path, "w") as io:
    io.write(nwbfile)

# test read
with NWBHDF5IO(save_path, "r") as io:
    nwbfile_in = io.read()

MATLAB:

Installation

git clone https://github.com/bendichter/ndx-miniscope.git

generateExtension('path/to/ndx-miniscope/spec');

Usage

under construction...

ndx-miniscope's Issues

rmv H5DataIO

ndx-miniscope/src/pynwb/ndx_miniscope/utils/nwb.py

Line 111 in e775f2d

timestamps=H5DataIO(timestamps, compression=True),

We are in the process of refactoring NeuroConv to be able to assemble NWB files in a way that is backend-agnostic. We should refactor the utilities of neurodata extension packages in the same way. Once this is incorporated into a DataInterface, the application of dataset configurations should be taken care of automatically. See catalystneuro/neuroconv#475

Structure of paths

Currently using the miniscope v4.4 with the latest software from Aharoni. Unfortunately, the saved structure mentioned in the ndx-miniscope intro does not match the structure of the saved data (attached image). The error (assert miniscope_folder_paths error below line 26) mentioned was solved by adding the code below to the init (see second image). But then I discovered this issue was present in other files and so wanted to create an issue.

dir_contents = sorted(os.listdir(folder_path)); miniscope_name = [i for i in dir_contents if 'miniscope' in i.lower()][0]; miniscope_folder_paths = list(Path(folder_path).rglob(miniscope_name))

Of course I could always change the structure of my paths, but wanted to mention this!

Thanks as always,
-John

Updated PyPI release?

The current README instructions suggest calling functions that aren't available in the latest version available via pip install (0.2.2).

from ndx-miniscope.miniscope import get_starting_frames

Are there plans to update the release? If not, I'll plan to add the repo to the requirements on my end (i.e., git+https:// ...)