Miniscope V4 data structure documentation? #60
Labels
documentation
Improvements or additions to documentation
Comments
sneakers-the-rat
added
the
documentation
|
ping on this. We could help documenting this but we would need some guidance. Maybe some pointer to where in the code this is executed and confirmation once we write some sort documentation. |
|
we're working on a new i/o framework that will output NWB directly, but yes some docs for the data structures would be nice. @fnsangiul (or others in this thread) would it be possible to zip a small data directory and post it here and i can make some schema definitions for it? (can't run the DAQ on my mac, am at home atm) I think the main thing that's undocumented there is the and written here: Miniscope-DAQ-QT-Software/source/datasaver.cpp Lines 277 to 351 in 0fecb97 Here's what I have last time i looked at this for a set of models and schema. I think it differs slightly from the literal configs but i tried to be as faithful as i could expand/collapse pydantic modelsfrom typing import List, Optional, ClassVar, Tuple
from enum import Enum
from pydantic import BaseModel, Field
from pydantic import DirectoryPath, FilePath, confloat, conint
# --------------------------------------------------
# Generic types
# --------------------------------------------------
class Executable(BaseModel):
"""
An executable to be run during some phase of recording!
"""
enabled: bool = Field(
description="This enables or disabled the .exe file from being called."
)
filePath: FilePath = Field(
description="This is an absolute path to a .exe file you want to have executed at the start of recording. This is usually used to syncronize some third party hardware."
)
arguments: Optional[List[str]] = Field(
default=None,
description="This is an array of strings that will be sent as arguments to the .exe listed in 'filePath'."
)
class Window(BaseModel):
"""
Parameterizes a plotting window within the GUI
"""
windowX: int = Field(
default=0,
description="These 'window' keys: value pairs affect just the display of the behavior tracker. Set these to fit well within the rest of your GUI. This is the top left location of the GUI window."
)
windowY: int = Field(
default=0,
description="These 'window' keys: value pairs affect just the display of the behavior tracker. Set these to fit well within the rest of your GUI. This is the top left location of the GUI window."
)
windowWidth: int = 256 # just putting in random value for now as a demo
windowHeight: int = 256 # just putting in random value for now as a demo
windowScale: float = Field(
default=1,
description="These 'window' keys: value pairs affect just the display of the behavior tracker. Set these to fit well within the rest of your GUI. This sets the scale of the window relative to the size of the actual video data it is processing. A value of 1 is a good place to start."
)
class ROI(BaseModel):
left: int
top: int
width: int
height: int
# --------------------------------------------------
# plot types
# --------------------------------------------------
class Plot(BaseModel):
"""
Metaclass for different kinds of plots used in the tracker. Just used to group like types
"""
enabled: bool = Field(
default=True,
description="Enable or disable this plot!"
)
window: Window = Field(
default_factory=Window,
description="Parameterize the display in the miniscope behavior tracker"
)
class OccupancyPlot(Plot):
"""
The behavior tracker is capable of generating a 2D occupancy plot based on DLC pose data and overlaying it within the behavior tracker GUI."
"""
poseIdxToUse: List[int] = Field(
description="The 2D occupancy plotter will use the mean of the DLC pose/joint indices listed in this array to fill the 2D occupancy bins. You need to list at least 1 indice for the plotter to work."
)
numBinsX: int = Field(
default=20,
description="This is the number of horizontal spatial bins used in the 2D occupancy calculation. Usually somewhere between 20 and 100 works well."
)
numBinsY: int = Field(
default=20,
description="This is the number of vertical spatial bins used in the 2D occupancy calculation. Usually somewhere between 20 and 100 works well."
)
class POSE_OVERLAY_TYPE(str, Enum):
point = 'point'
line = 'line'
ribbon = 'ribbon'
class PoseOverlay(Plot):
type: POSE_OVERLAY_TYPE = 'point'
numOfPastPoses: int = Field(
description="The overlay can display recent past positions of joints and fade them out as time goes one. This value sets how many past poses to display in the overlay. Setting this to 0 will only show the most recent DLC pose data. A valuse of 0 to 20 is resonable here."
)
markerSize: int = Field(
description="The markerSize sets how large the points, lines, or ribbons in the overlay will be. Reasonable values are between 3 and 20."
)
skeleton: Optional[List[List[int]]] = Field(
default=None,
description = "This is an array of arrays. Each internal array defines the set of connected pose/joint indices to be connected and displayed."
)
class TRACE_DISPLAY_TYPE(str, Enum):
scrolling = 'scrolling'
"""
Some description of what this trace display type does
"""
class TRACE_DISPLAY_FEATURE(str, Enum):
w = 'w'
"""
width, or x position
"""
h = 'h'
"""
height, or y position
"""
wh = 'wh'
"""
Both I think?
"""
class TraceDisplay(Plot):
poseIdx: Optional[List[Tuple[int,TRACE_DISPLAY_FEATURE]]] = Field(
default = None,
description="The behavior tracker can also plot the positional time trace of any of the DLC pose/joints within the trace display. This array will setup time trace plotting for specific pose indices. You must give it a the index number along with 'w', 'h', or 'wh'. 'w' means it will plot the width or x position. 'h' means it will plot the height or y position."
)
type: TRACE_DISPLAY_TYPE = 'scrolling'
class ORIENTATION(str, Enum):
roll = 'roll'
pitch = 'pitch'
yaw = 'yaw'
class HeadOrientation(Plot):
filterBadData:bool = Field(
default=True,
description="Set this to true if you have a V4 or newer Miniscope. It will acquire and display head-orinetation data from the on-board BNO sensor."
)
plotTrace: List[ORIENTATION] = Field(
default_factory=list,
description = "Add or remove 'roll' 'pitch' and/or 'yaw' to define which dimensions get plotted in the time trace display."
)
# --------------------------------------------------
# Behavior Trackers
# --------------------------------------------------
class BehaviorTracker(BaseModel):
"""
Metaclass for different types of behavior trackers.
Empty for now since I am not sure what the desired structure of this class tree is, but
present so that we can specify in the main config that we want a subclass of this class.
"""
enabled: bool = Field(
default=True,
description="This enables or disabled the entire behavior tracker module within the software. Enable this for the behavior tracker to run and generate a GUI."
)
type: ClassVar[str] = Field(
description="Right now the behavior tracker only supports \"DeepLabCut-Live\" as type. For info on setting up DLC-Live go here, https://github.com/DeepLabCut/DeepLabCut-live"
)
class DLCLive(BehaviorTracker):
type: ClassVar[str] = "DeepLabCut-Live"
pyEnvPath: DirectoryPath = Field(
description="The behavior tracker needs the absolute path to a functional Python virtual environment which can successfully run DeepLabCut-Live. The Miniscope Software initially checks this folder to make sure it contains this exact subfolder, '/Lib/site-packages/dlclive'"
)
modelPath: DirectoryPath = Field(
description="This is the absolute path to the DLC model you want to use."
)
resize: confloat(ge=0, le=1) = Field(
description="This number will resize the behavior camera image to lower resolution if your CPU/GPU is having a hard time keeping up with the video stream. This number shouldn't ever be set above 1."
)
pCutoffDisplay: confloat(ge=0, le=1) = Field(
description="All DLC pose data will be recording but this pCufOff will set the minimum p value for data to be plotted within the GUI."
)
window: Window = Field(
default_factory=Window,
description="Parameterize the display in the miniscope behavior tracker"
)
occupancyPlot: Optional[OccupancyPlot] = None
poseOverlay: Optional[PoseOverlay] = None
# --------------------------------------------------
# Devices
# --------------------------------------------------
class Device(BaseModel):
"""Same thing here, keeping device types in a tree"""
class MINISCOPE_TYPE(str, Enum):
Miniscope_V4_BNO = "Miniscope_V4_BNO"
Miniscope_V3 = "Miniscope_V3"
class COMPRESSION(str, Enum):
GREY = "GREY"
"""
No compression
"""
FFV1 = "FFV1"
"""
Lossless compression
"""
class GAIN(str, Enum):
Low = "Low"
"""
1x
"""
Medium = "Medium"
"""
2x
"""
High = "High"
"""
3x
"""
class Camera(Device):
deviceType: str = Field(
description="This should be a supported deviceType listed in the deviceconfig/videoDevice.json file. Supported types include \"Miniscope_V4_BNO\" and \"Miniscope_V3\"."
)
deviceID: int = Field(
description="All UVC camera devices on your computer have a unique deviceID starting with '0'. Try numbers until you find the one that connects to the correct Miniscope. This number will never be more than one minus the number of cameras devices (including Miniscopes) connected to your computer."
)
showSaturation: bool = Field(
description="If set to true, the video stream window will show red pixels for any fully saturated pixels. This can also be turned on and off within the video display window."
)
compression: COMPRESSION = Field(
default='GREY',
description="Imaging data is save in .avi file format. You can choose what type of video compression to apply when saving data. We suggest using a lossless compression CODEC for Miniscope data. This would be either GREY or FFV1. GREY does no compression. FFV1 losslessly compresses the data but can be CPU intensive. We generally use FFV1 is our computer can keep up with it. If you notice the frame buffer filling up completely while recording you should switch to GREY."
)
framesPerFile: int = Field(
description="We generally set this to 1000. It defines how large each .avi file will be in your recording."
)
ROI: Optional[ROI] = None
window: Window = Field(
default_factory=Window
)
class Miniscope(Camera):
deviceType: MINISCOPE_TYPE = Field(
description="This should be a supported deviceType listed in the deviceconfig/videoDevice.json file. Supported types include \"Miniscope_V4_BNO\" and \"Miniscope_V3\"."
)
gain: GAIN = Field(
default="Medium",
description="Acceptable values are \"Low\", \"Medium\", \"High\". For the V4 Miniscope these correspond to 1x, 2x, and 3.5x respectively."
)
ewl: int = Field(
description="This sets the initial position of the electro-tunable lens focal plane."
)
led0: conint(ge=0, le=100) = Field(
description="This sets the initial excitation LED intensity. Values can be from 0 to 100 but many Miniscope configurations have trouble with values over 70 or 80 due to current limits through certain coax cable lengths."
)
frameRate: conint(ge=5, le=30, multiple_of=5) = Field(
description="This sets the initial FPS of the video device. Acceptable values usually go in increments of 5 between 10 and 30 and have the form of: \"20FPS\" for example."
)
headOrientation: Optional[HeadOrientation] = None
class MiniscopeConfig(BaseModel):
dataDirectory: DirectoryPath = Field(
description="This is an absolute path to the base directory you want to save all data to. It should use '/' and not '\\'."
)
directoryStructure: List[str] = Field(
description="This is an array of strings which defines the structure of the save directory. 'Time' and 'Date' will generate the exact time and data when recording starts. All other entries can be whatever you want but they also need to have their own \"key\": \"value\" pair defined within this file."
)
researcherName: str = Field(
description="This will be saved in the recording's meta data file as well as can be used as an entry into the 'directoryStructure' array."
)
experimentName: str = Field(
description="This will be saved in the recording's meta data file as well as can be used as an entry into the 'directoryStructure' array."
)
animalName: str = Field(
description="This will be saved in the recording's meta data file as well as can be used as an entry into the 'directoryStructure' array."
)
recordLengthinSeconds: int = Field(
default=0,
description="This is an integer value. A value of '0' does not limit recording length. An integer value above 0 will stop the recording at that amount of seconds."
)
executableOnStartRecording: Optional[Executable] = Field(
default=None,
description="An executable to be run when the recording starts"
)
executableOnStopRecording: Optional[Executable] = Field(
default=None,
description="An executable to be run when the recording stops!"
)
behaviorTracker: Optional[BehaviorTracker] = None
traceDisplay: Optional[TraceDisplay]
devices: List[Miniscope]
cameras: Optional[List[Camera]] = Nonejson schema: expand/collapse json schema{
"title": "MiniscopeConfig",
"type": "object",
"properties": {
"dataDirectory": {
"title": "Datadirectory",
"description": "This is an absolute path to the base directory you want to save all data to. It should use '/' and not '\\'.",
"format": "directory-path",
"type": "string"
},
"directoryStructure": {
"title": "Directorystructure",
"description": "This is an array of strings which defines the structure of the save directory. 'Time' and 'Date' will generate the exact time and data when recording starts. All other entries can be whatever you want but they also need to have their own \"key\": \"value\" pair defined within this file.",
"type": "array",
"items": {
"type": "string"
}
},
"researcherName": {
"title": "Researchername",
"description": "This will be saved in the recording's meta data file as well as can be used as an entry into the 'directoryStructure' array.",
"type": "string"
},
"experimentName": {
"title": "Experimentname",
"description": "This will be saved in the recording's meta data file as well as can be used as an entry into the 'directoryStructure' array.",
"type": "string"
},
"animalName": {
"title": "Animalname",
"description": "This will be saved in the recording's meta data file as well as can be used as an entry into the 'directoryStructure' array.",
"type": "string"
},
"recordLengthinSeconds": {
"title": "Recordlengthinseconds",
"description": "This is an integer value. A value of '0' does not limit recording length. An integer value above 0 will stop the recording at that amount of seconds.",
"default": 0,
"type": "integer"
},
"executableOnStartRecording": {
"title": "Executableonstartrecording",
"description": "An executable to be run when the recording starts",
"allOf": [
{
"$ref": "#/definitions/Executable"
}
]
},
"executableOnStopRecording": {
"title": "Executableonstoprecording",
"description": "An executable to be run when the recording stops!",
"allOf": [
{
"$ref": "#/definitions/Executable"
}
]
},
"behaviorTracker": {
"$ref": "#/definitions/BehaviorTracker"
},
"traceDisplay": {
"$ref": "#/definitions/TraceDisplay"
},
"devices": {
"title": "Devices",
"type": "array",
"items": {
"$ref": "#/definitions/Miniscope"
}
},
"cameras": {
"title": "Cameras",
"type": "array",
"items": {
"$ref": "#/definitions/Camera"
}
}
},
"required": [
"dataDirectory",
"directoryStructure",
"researcherName",
"experimentName",
"animalName",
"devices"
],
"definitions": {
"Executable": {
"title": "Executable",
"description": "An executable to be run during some phase of recording!",
"type": "object",
"properties": {
"enabled": {
"title": "Enabled",
"description": "This enables or disabled the .exe file from being called.",
"type": "boolean"
},
"filePath": {
"title": "Filepath",
"description": "This is an absolute path to a .exe file you want to have executed at the start of recording. This is usually used to syncronize some third party hardware.",
"format": "file-path",
"type": "string"
},
"arguments": {
"title": "Arguments",
"description": "This is an array of strings that will be sent as arguments to the .exe listed in 'filePath'.",
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"enabled",
"filePath"
]
},
"BehaviorTracker": {
"title": "BehaviorTracker",
"description": "Metaclass for different types of behavior trackers.\n\nEmpty for now since I am not sure what the desired structure of this class tree is, but\npresent so that we can specify in the main config that we want a subclass of this class.",
"type": "object",
"properties": {
"enabled": {
"title": "Enabled",
"description": "This enables or disabled the entire behavior tracker module within the software. Enable this for the behavior tracker to run and generate a GUI.",
"default": true,
"type": "boolean"
}
}
},
"Window": {
"title": "Window",
"description": "Parameterizes a plotting window within the GUI",
"type": "object",
"properties": {
"windowX": {
"title": "Windowx",
"description": "These 'window' keys: value pairs affect just the display of the behavior tracker. Set these to fit well within the rest of your GUI. This is the top left location of the GUI window.",
"default": 0,
"type": "integer"
},
"windowY": {
"title": "Windowy",
"description": "These 'window' keys: value pairs affect just the display of the behavior tracker. Set these to fit well within the rest of your GUI. This is the top left location of the GUI window.",
"default": 0,
"type": "integer"
},
"windowWidth": {
"title": "Windowwidth",
"default": 256,
"type": "integer"
},
"windowHeight": {
"title": "Windowheight",
"default": 256,
"type": "integer"
},
"windowScale": {
"title": "Windowscale",
"description": "These 'window' keys: value pairs affect just the display of the behavior tracker. Set these to fit well within the rest of your GUI. This sets the scale of the window relative to the size of the actual video data it is processing. A value of 1 is a good place to start.",
"default": 1,
"type": "number"
}
}
},
"TRACE_DISPLAY_FEATURE": {
"title": "TRACE_DISPLAY_FEATURE",
"description": "An enumeration.",
"enum": [
"w",
"h",
"wh"
],
"type": "string"
},
"TRACE_DISPLAY_TYPE": {
"title": "TRACE_DISPLAY_TYPE",
"description": "An enumeration.",
"enum": [
"scrolling"
],
"type": "string"
},
"TraceDisplay": {
"title": "TraceDisplay",
"description": "Metaclass for different kinds of plots used in the tracker. Just used to group like types",
"type": "object",
"properties": {
"enabled": {
"title": "Enabled",
"description": "Enable or disable this plot!",
"default": true,
"type": "boolean"
},
"window": {
"title": "Window",
"description": "Parameterize the display in the miniscope behavior tracker",
"allOf": [
{
"$ref": "#/definitions/Window"
}
]
},
"poseIdx": {
"title": "Poseidx",
"description": "The behavior tracker can also plot the positional time trace of any of the DLC pose/joints within the trace display. This array will setup time trace plotting for specific pose indices. You must give it a the index number along with 'w', 'h', or 'wh'. 'w' means it will plot the width or x position. 'h' means it will plot the height or y position.",
"type": "array",
"items": {
"type": "array",
"minItems": 2,
"maxItems": 2,
"items": [
{
"type": "integer"
},
{
"$ref": "#/definitions/TRACE_DISPLAY_FEATURE"
}
]
}
},
"type": {
"default": "scrolling",
"allOf": [
{
"$ref": "#/definitions/TRACE_DISPLAY_TYPE"
}
]
}
}
},
"MINISCOPE_TYPE": {
"title": "MINISCOPE_TYPE",
"description": "An enumeration.",
"enum": [
"Miniscope_V4_BNO",
"Miniscope_V3"
],
"type": "string"
},
"COMPRESSION": {
"title": "COMPRESSION",
"description": "An enumeration.",
"enum": [
"GREY",
"FFV1"
],
"type": "string"
},
"GAIN": {
"title": "GAIN",
"description": "An enumeration.",
"enum": [
"Low",
"Medium",
"High"
],
"type": "string"
},
"ORIENTATION": {
"title": "ORIENTATION",
"description": "An enumeration.",
"enum": [
"roll",
"pitch",
"yaw"
],
"type": "string"
},
"HeadOrientation": {
"title": "HeadOrientation",
"description": "Metaclass for different kinds of plots used in the tracker. Just used to group like types",
"type": "object",
"properties": {
"enabled": {
"title": "Enabled",
"description": "Enable or disable this plot!",
"default": true,
"type": "boolean"
},
"window": {
"title": "Window",
"description": "Parameterize the display in the miniscope behavior tracker",
"allOf": [
{
"$ref": "#/definitions/Window"
}
]
},
"filterBadData": {
"title": "Filterbaddata",
"description": "Set this to true if you have a V4 or newer Miniscope. It will acquire and display head-orinetation data from the on-board BNO sensor.",
"default": true,
"type": "boolean"
},
"plotTrace": {
"description": "Add or remove 'roll' 'pitch' and/or 'yaw' to define which dimensions get plotted in the time trace display.",
"type": "array",
"items": {
"$ref": "#/definitions/ORIENTATION"
}
}
}
},
"Miniscope": {
"title": "Miniscope",
"description": "Same thing here, keeping device types in a tree",
"type": "object",
"properties": {
"deviceType": {
"description": "This should be a supported deviceType listed in the deviceconfig/videoDevice.json file. Supported types include \"Miniscope_V4_BNO\" and \"Miniscope_V3\".",
"allOf": [
{
"$ref": "#/definitions/MINISCOPE_TYPE"
}
]
},
"deviceID": {
"title": "Deviceid",
"description": "All UVC camera devices on your computer have a unique deviceID starting with '0'. Try numbers until you find the one that connects to the correct Miniscope. This number will never be more than one minus the number of cameras devices (including Miniscopes) connected to your computer.",
"type": "integer"
},
"showSaturation": {
"title": "Showsaturation",
"description": "If set to true, the video stream window will show red pixels for any fully saturated pixels. This can also be turned on and off within the video display window.",
"type": "boolean"
},
"compression": {
"description": "Imaging data is save in .avi file format. You can choose what type of video compression to apply when saving data. We suggest using a lossless compression CODEC for Miniscope data. This would be either GREY or FFV1. GREY does no compression. FFV1 losslessly compresses the data but can be CPU intensive. We generally use FFV1 is our computer can keep up with it. If you notice the frame buffer filling up completely while recording you should switch to GREY.",
"default": "GREY",
"allOf": [
{
"$ref": "#/definitions/COMPRESSION"
}
]
},
"framesPerFile": {
"title": "Framesperfile",
"description": "We generally set this to 1000. It defines how large each .avi file will be in your recording.",
"type": "integer"
},
"ROI": {
"title": "Roi",
"type": "null"
},
"window": {
"$ref": "#/definitions/Window"
},
"gain": {
"description": "Acceptable values are \"Low\", \"Medium\", \"High\". For the V4 Miniscope these correspond to 1x, 2x, and 3.5x respectively.",
"default": "Medium",
"allOf": [
{
"$ref": "#/definitions/GAIN"
}
]
},
"ewl": {
"title": "Ewl",
"description": "This sets the initial position of the electro-tunable lens focal plane.",
"type": "integer"
},
"led0": {
"title": "Led0",
"description": "This sets the initial excitation LED intensity. Values can be from 0 to 100 but many Miniscope configurations have trouble with values over 70 or 80 due to current limits through certain coax cable lengths.",
"minimum": 0,
"maximum": 100,
"type": "integer"
},
"frameRate": {
"title": "Framerate",
"description": "This sets the initial FPS of the video device. Acceptable values usually go in increments of 5 between 10 and 30 and have the form of: \"20FPS\" for example.",
"minimum": 5,
"maximum": 30,
"multipleOf": 5,
"type": "integer"
},
"headOrientation": {
"$ref": "#/definitions/HeadOrientation"
}
},
"required": [
"deviceType",
"deviceID",
"showSaturation",
"framesPerFile",
"ewl",
"led0",
"frameRate"
]
},
"Camera": {
"title": "Camera",
"description": "Same thing here, keeping device types in a tree",
"type": "object",
"properties": {
"deviceType": {
"title": "Devicetype",
"description": "This should be a supported deviceType listed in the deviceconfig/videoDevice.json file. Supported types include \"Miniscope_V4_BNO\" and \"Miniscope_V3\".",
"type": "string"
},
"deviceID": {
"title": "Deviceid",
"description": "All UVC camera devices on your computer have a unique deviceID starting with '0'. Try numbers until you find the one that connects to the correct Miniscope. This number will never be more than one minus the number of cameras devices (including Miniscopes) connected to your computer.",
"type": "integer"
},
"showSaturation": {
"title": "Showsaturation",
"description": "If set to true, the video stream window will show red pixels for any fully saturated pixels. This can also be turned on and off within the video display window.",
"type": "boolean"
},
"compression": {
"description": "Imaging data is save in .avi file format. You can choose what type of video compression to apply when saving data. We suggest using a lossless compression CODEC for Miniscope data. This would be either GREY or FFV1. GREY does no compression. FFV1 losslessly compresses the data but can be CPU intensive. We generally use FFV1 is our computer can keep up with it. If you notice the frame buffer filling up completely while recording you should switch to GREY.",
"default": "GREY",
"allOf": [
{
"$ref": "#/definitions/COMPRESSION"
}
]
},
"framesPerFile": {
"title": "Framesperfile",
"description": "We generally set this to 1000. It defines how large each .avi file will be in your recording.",
"type": "integer"
},
"ROI": {
"title": "Roi",
"type": "null"
},
"window": {
"$ref": "#/definitions/Window"
}
},
"required": [
"deviceType",
"deviceID",
"showSaturation",
"framesPerFile"
]
}
}
} |
|
Sure @sneakers-the-rat, I'm happy to help. Just to be sure, what you need is the folder structure that gets generated each time a recording is done? |
|
Thanks for the detailed response. The pointer to the code is very useful and I will read both the code and the schema you shared in detail. To add context, for us (@weiglszonja @alessandratrapani) there are two things that are not clear:
Currently, in the wiki there is the following documentation: http://miniscope.org/index.php/Data_Acquisition_Software This has a section "Data Structure" but I believe this is outdated and probably only applies to version 3? Something like that for the latest version would be very useful.
I have one file where the top level json has the session start time like this: {
"animalName": "REDACTED",
"baseDirectory": "REDACTED",
"cameras": [
],
"day": 15,
"experimentName": "Ca_EEG2",
"hour": 10,
"miniscopes": [
"Miniscope"
],
"minute": 33,
"month": 10,
"msec": 10,
"msecSinceEpoch": 1634308400010,
"researcherName": "REDACTED",
"second": 20,
"year": 2021
}
and another one where there is a key to it: "animalName": "",
"baseDirectory": "REDACTED",
"cameras": [
"BehavCam 2"
],
"experimentName": "",
"miniscopes": [
"Miniscope"
],
"nameExpMouse": "REDACTED",
"recordingStartTime": {
"day": 7,
"hour": 15,
"minute": 3,
"month": 10,
"msec": 635,
"msecSinceEpoch": 1633644208635,
"second": 28,
"year": 2021
},
"researcherName": ""
}So, in the second instance the session start time is nested but not in the first. Is this a natural variation that we can expect? Is this for different versions? |
|
Fascinating. It absolutely should be nested since having all those loose keys is uh not usable. There has only ever been one version of this software as far as I know, so it must be behaving differently for different configs, which is... bad. This software (linked to from the docs) is very much abandoned and should not be used. https://github.com/daharoni/Miniscope_DAQ_Software Sorry the docs are wrong, we're in the process of rearchitecting all these systems to prevent exactly this problem of multiple versions of abandoned software and out of date docs, but there's a lot to do. This software (in this repo) is also marked as deprecated because we are moving to a system where the I/O is decoupled from the GUI and so we can make it work consistently across all miniscopes. I know that doesnt help much for now but soon we'll be able to address all these problems. For these samples: were they both collected with this software, or were some using the previous softw as re? Were each of those collected with a different camera config? Or what was different between those sessions? |
|
OK, thanks a lot. Also, thanks a lot for your work on making this happen, decoupling the GUI from I/O seems like the way to go. Looking forward to see the final product. Concerning the samples. They are from different labs so presumably different cam and software configurations. It is not clear to me which version of the software was used to collect them, is there a way for me to figure out from the json?. For the non-nested version, I will inquire with the lab and come back to you. The other json is is older and I am not sure we have that provenance but both of them have the file structure that @weiglszonja described above if that is useful somehow. |
|
@sneakers-the-rat the version of the software that you (and @h-mayorquin) linked is outdated and not used anymore. The only current version is this one. I'm finishing something right now, but as soon as I finish I'll upload a sample folder of the current Miniscope Software. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hey,
I am working on a project to convert Miniscope data to NWB format.
The configuration file that I received ("metaData.json") contains a "deviceType" parameter which is "Miniscope_V3".
After looking at the data structure that is described for V3 here is different from what I have:
C6-J588/(main folder)15_03_28/(subfolder corresponding to the recording time, there are multiple subfolders like this)Miniscope/(subfolder containing the microscope video stream)0.avi(file containing the microscope video)metaData.json(file containing the metadata for the microscope device)timeStamps.csv(file containing the timing of this video stream)BehavCam_2/(subfolder containing the behavioral video stream)0.avi(file containing the bevavioral video)metaData.json(file containing the metadata for the behavioral camera)timeStamps.csv(file containing the timing of this video stream)metaData.json(file containing metadata for the recording, such as the start time)I'm confused now which version of Miniscope it is.
Is the Miniscope V4 data structure is documented somewhere that I missed?
The text was updated successfully, but these errors were encountered: