Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update methods for writing n-dimensional data #33

Merged
merged 32 commits into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
b7439ed
detect dataset or group type before opening
stephprince Jul 17, 2024
fcf0c60
extend writeDataBlock for arbitrary n dimensions - wip
stephprince Jul 17, 2024
ce27f3c
remove inaccurate buffer naming
stephprince Jul 18, 2024
7a530ac
extend writeDataBlock for arbitrary n dimensions
stephprince Jul 18, 2024
19e501d
update create dataset for arbitrary n dimensions
stephprince Jul 18, 2024
c68705f
add tests for multiple dimensions of datablock writing
stephprince Jul 18, 2024
ade6460
fix formatting and spelling
stephprince Jul 18, 2024
521914a
move and consolidate data and timestamps writing methods to TimeSeries
stephprince Jul 18, 2024
0731871
update TimeSeries to create data and timestamps datasets on initializ…
stephprince Jul 18, 2024
41f89df
add 1D mock data test util function
stephprince Jul 18, 2024
bd911bf
add tests for TimeSeries write data method
stephprince Jul 18, 2024
85b191f
get data and timestamp datatypes from timeseries object
stephprince Jul 18, 2024
e28569a
add writeChannel method for electrical series
stephprince Jul 18, 2024
3e93e0e
add ElectricalSeries tests
stephprince Jul 22, 2024
4f0224a
fix formatting
stephprince Jul 22, 2024
8507d8b
move data transform to utils
stephprince Jul 22, 2024
db9dfe2
set timestamps type to float64
stephprince Jul 22, 2024
7f84c9e
add RecordingContainers to manage getting and storing TimeSeries
stephprince Jul 23, 2024
cc6f854
update cmakelists for new boost policy
stephprince Jul 23, 2024
611472f
Remove extra comment from BaseIO.hpp
stephprince Jul 25, 2024
c0c1261
fix comment in BaseIO about position variable
stephprince Jul 25, 2024
daa015a
update docstring for HDF5RecordingData
stephprince Jul 25, 2024
d9f01e8
remove containerSize as member variable
stephprince Jul 25, 2024
9204038
add comments attribute to channel
stephprince Jul 25, 2024
b3c1110
move readDataBlock to testUtils
stephprince Jul 25, 2024
5fddba2
add conditional compiling for H5 object type
stephprince Jul 25, 2024
e174f26
fix formatting
stephprince Jul 25, 2024
1fb512a
Merge branch 'main' into refactor-data-write
stephprince Jul 25, 2024
b971e63
rename functions and methods for clarity
stephprince Jul 26, 2024
6273c6c
update NWBFile.recordingContainers data type
stephprince Jul 26, 2024
4f928aa
fix formatting
stephprince Jul 26, 2024
c71ec22
rename startRecording to createElectricalSeries
stephprince Jul 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
cmake_minimum_required(VERSION 3.14)
cmake_minimum_required(VERSION 3.15)
if(POLICY CMP0167)
cmake_policy(SET CMP0167 NEW)
endif()

include(cmake/prelude.cmake)

Expand Down
6 changes: 4 additions & 2 deletions src/BaseIO.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,11 @@ BaseRecordingData::BaseRecordingData() {}

BaseRecordingData::~BaseRecordingData() {}

Status BaseRecordingData::writeDataBlock(const SizeType& xDataSize,
// Overload that uses the member variable position (works for simple data
// extension)
Status BaseRecordingData::writeDataBlock(const std::vector<SizeType>& dataShape,
const BaseDataType& type,
const void* data)
{
return writeDataBlock(xDataSize, size[1], type, data);
return writeDataBlock(dataShape, position, type, data);
}
43 changes: 14 additions & 29 deletions src/BaseIO.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -357,64 +357,49 @@ class BaseRecordingData
virtual ~BaseRecordingData();

/**
* @brief Writes a 1D block of data (samples).
* @param xDataSize The size of the data block in the x dimension (samples).
* @brief Writes a block of data using the stored position information.
* @param dataShape The size of the data block.
* @param type The data type of the elements in the data block.
* @param data A pointer to the data block.
* @return The status of the write operation.
*/
Status writeDataBlock(const SizeType& xDataSize,
Status writeDataBlock(const std::vector<SizeType>& dataShape,
stephprince marked this conversation as resolved.
Show resolved Hide resolved
const BaseDataType& type,
const void* data);

/**
* @brief Writes a 2D block of data (samples x channels).
* @param xDataSize The size of the data block in the x dimension (samples).
* @param yDataSize The size of the data block in the y dimension (channels).
* @brief Writes a block of data (any number of dimensions).
* @param dataShape The size of the data block.
* @param positionOffset The position of the data block to write to.
* @param type The data type of the elements in the data block.
* @param data A pointer to the data block.
* @return The status of the write operation.
*/
virtual Status writeDataBlock(const SizeType& xDataSize,
const SizeType& yDataSize,
virtual Status writeDataBlock(const std::vector<SizeType>& dataShape,
const std::vector<SizeType>& positionOffset,
const BaseDataType& type,
const void* data) = 0;

/**
* @brief Writes a row of data in a 2D block.
* @param xDataSize The size of the data row in the x dimension (samples).
* @param yPosition The position of the data row in the y dimension
* (channels).
* @param type The data type of the elements in the data block.
* @param data A pointer to the data block.
* @return The status of the write operation.
*/
virtual Status writeDataRow(const SizeType& xDataSize,
const SizeType& yPos,
const BaseDataType& type,
const void* data) = 0;

protected:
/**
* @brief The current position in the x dimension.
*/
SizeType xPos;

/**
* @brief The size of the data block in each dimension.
* @brief The size of the dataset in each dimension.
*/
SizeType size[3];
std::vector<SizeType> size;

/**
* @brief The number of dimensions in the data block.
* @brief The current position in the dataset.
*/
SizeType dimension; /**< The number of dimensions in the data block. */
std::vector<SizeType> position;

/**
* @brief The position in the x dimension of samples written for each row
* (channel).
* @brief The number of dimensions in the data block.
*/
std::vector<uint32_t> rowXPos;
SizeType nDimensions;
};

} // namespace AQNWB
4 changes: 3 additions & 1 deletion src/Channel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ Channel::Channel(const std::string name,
const float conversion,
const float samplingRate,
const float bitVolts,
const std::array<float, 3> position)
const std::array<float, 3> position,
const std::string comments)
: name(name)
, groupName(groupName)
, localIndex(localIndex)
Expand All @@ -20,6 +21,7 @@ Channel::Channel(const std::string name,
, conversion(conversion)
, samplingRate(samplingRate)
, bitVolts(bitVolts)
, comments(comments)
stephprince marked this conversation as resolved.
Show resolved Hide resolved
{
}

Expand Down
9 changes: 8 additions & 1 deletion src/Channel.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ class Channel
const float bitVolts = 0.000002f, // least significant bit needed to
// convert 16-bit int to volts
// currently a placeholder
const std::array<float, 3> position = {0.f, 0.f, 0.f});
const std::array<float, 3> position = {0.f, 0.f, 0.f},
const std::string comments = "no comments");

/**
* @brief Destructor
Expand All @@ -45,6 +46,7 @@ class Channel
* @return The samplingRate value.
*/
float getSamplingRate() const;

/**
* @brief Getter for bitVolts
* @return The bitVolts value.
Expand Down Expand Up @@ -76,6 +78,11 @@ class Channel
*/
std::array<float, 3> position;

/**
* @brief Comments about the channel.
*/
std::string comments;

private:
/**
* @brief Conversion factor.
Expand Down
2 changes: 1 addition & 1 deletion src/Types.hpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#pragma once

#include <vector>
#include <cstddef>
#include <vector>

namespace AQNWB
{
Expand Down
25 changes: 25 additions & 0 deletions src/Utils.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,29 @@ inline std::unique_ptr<BaseIO> createIO(const std::string& type,
throw std::invalid_argument("Invalid IO type");
}
}

inline std::unique_ptr<int16_t[]> transformToInt16(SizeType numSamples,
float conversion_factor,
const float* data)
{
std::unique_ptr<float[]> scaledData = std::make_unique<float[]>(numSamples);
std::unique_ptr<int16_t[]> intData = std::make_unique<int16_t[]>(numSamples);

// copy data and multiply by scaling factor
double multFactor = 1 / (32767.0f * conversion_factor);
std::transform(data,
data + numSamples,
scaledData.get(),
[multFactor](float value) { return value * multFactor; });

// convert float to int16
std::transform(
scaledData.get(),
scaledData.get() + numSamples,
intData.get(),
[](float value)
{ return static_cast<int16_t>(std::clamp(value, -32768.0f, 32767.0f)); });

return intData;
}
} // namespace AQNWB
Loading
Loading