Skip to content

Commit

Permalink
Add feature to choose voxel pooling mode when creating VoxelGrid from…
Browse files Browse the repository at this point in the history 
… PointCloud (isl-org#6937)

* add feature to choose voxel color mode in VoxelGrid

* update CHANGELOG.md

* update tutorial for VoxelGrid color_mode change

* update CHANGELOG.md

* rename VoxelColorMode to VoxelPoolingMode

* update CHANGELOG

* apply style

---------

Co-authored-by: Benjamin Ummenhofer <[email protected]>
  • Loading branch information
@rxba @benjaminum
rxba and benjaminum authored Dec 19, 2024
1 parent 0ae7b1a commit ba2a6b1
Show file tree
Hide file tree
Showing 5 changed files with 118 additions and 18 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@
- Split pybind declarations/definitions to avoid C++ types in Python docs (PR #6869)
- Fix minimal oriented bounding box of MeshBase derived classes and add new unit tests (PR #6898)
- Fix projection of point cloud to Depth/RGBD image if no position attribute is provided (PR #6880)
- Add choice of voxel pooling mode when creating VoxelGrid from PointCloud (PR #6937)
- Support lowercase types when reading PCD files (PR #6930)
- Fix visualization/draw ICP example and add warnings (PR #6933)
- Unified cloud initializer pipeline for ICP (fixes segfault colored ICP) (PR #6942)
Expand All @@ -55,6 +56,7 @@
- Fix infinite loop in segment_plane if num_points < ransac_n (PR #7032)
- Add select_by_index method to Feature class (PR #7039)


## 0.13

- CUDA support 10.1 -> 11.0. Tensorflow 2.3.1 -> 2.4.1. PyTorch 1.6.0 -> 1.7.1 (PR #3049). This requires a custom PyTorch wheel from <https://github.com/isl-org/open3d_downloads/releases/tag/torch1.7.1> due to PyTorch issue #52663
Expand Down
78 changes: 74 additions & 4 deletions cpp/open3d/geometry/VoxelGrid.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -189,30 +189,42 @@ class VoxelGrid : public Geometry3D {
double height,
double depth);

/// \enum VoxelPoolingMode
///
/// \brief Possible ways of determining voxel color from PointCloud.
enum class VoxelPoolingMode { AVG, MIN, MAX, SUM };

/// Creates a VoxelGrid from a given PointCloud. The color value of a given
/// voxel is the average color value of the points that fall into it (if the
/// voxel is determined by the VoxelPoolingMode, e.g. by default the average
/// color value of the points that fall into it (if the
/// PointCloud has colors).
/// The bounds of the created VoxelGrid are computed from the PointCloud.
///
/// \param input The input PointCloud.
/// \param voxel_size Voxel size of of the VoxelGrid construction.
/// \param color_mode Mode of determining color for each voxel.
static std::shared_ptr<VoxelGrid> CreateFromPointCloud(
const PointCloud &input, double voxel_size);
const PointCloud &input,
double voxel_size,
VoxelPoolingMode color_mode = VoxelPoolingMode::AVG);

/// Creates a VoxelGrid from a given PointCloud. The color value of a given
/// voxel is the average color value of the points that fall into it (if the
/// voxel is determined by the VoxelPoolingMode, e.g. by default the average
/// color value of the points that fall into it (if the
/// PointCloud has colors).
/// The bounds of the created VoxelGrid are defined by the given parameters.
///
/// \param input The input PointCloud.
/// \param voxel_size Voxel size of of the VoxelGrid construction.
/// \param min_bound Minimum boundary point for the VoxelGrid to create.
/// \param max_bound Maximum boundary point for the VoxelGrid to create.
/// \param color_mode Mode of determining color for each voxel.
static std::shared_ptr<VoxelGrid> CreateFromPointCloudWithinBounds(
const PointCloud &input,
double voxel_size,
const Eigen::Vector3d &min_bound,
const Eigen::Vector3d &max_bound);
const Eigen::Vector3d &max_bound,
VoxelPoolingMode color_mode = VoxelPoolingMode::AVG);

/// Creates a VoxelGrid from a given TriangleMesh. No color information is
/// converted. The bounds of the created VoxelGrid are computed from the
Expand Down Expand Up @@ -294,5 +306,63 @@ class AvgColorVoxel {
Eigen::Vector3d color_;
};

/// \class AggColorVoxel
///
/// \brief Class to aggregate color values from different votes in one voxel
/// Can be used to compute min, max, average and sum voxel color.
class AggColorVoxel {
public:
AggColorVoxel()
: num_of_points_(0),
color_(0.0, 0.0, 0.0),
min_color_(Eigen::Vector3d::Constant(
std::numeric_limits<double>::max())),
max_color_(Eigen::Vector3d::Constant(
std::numeric_limits<double>::lowest())) {}

public:
void Add(const Eigen::Vector3i &voxel_index) {
if (num_of_points_ > 0 && voxel_index != voxel_index_) {
utility::LogWarning(
"Tried to aggregate ColorVoxel with different "
"voxel_index");
}
voxel_index_ = voxel_index;
}

void Add(const Eigen::Vector3i &voxel_index, const Eigen::Vector3d &color) {
Add(voxel_index);
color_ += color;
num_of_points_++;
min_color_ = min_color_.cwiseMin(color);
max_color_ = max_color_.cwiseMax(color);
}

Eigen::Vector3i GetVoxelIndex() const { return voxel_index_; }

Eigen::Vector3d GetAverageColor() const {
if (num_of_points_ > 0) {
return color_ / double(num_of_points_);
} else {
return color_;
}
}

Eigen::Vector3d GetMinColor() const { return min_color_; }

Eigen::Vector3d GetMaxColor() const { return max_color_; }

Eigen::Vector3d GetSumColor() const { return color_; }

public:
int num_of_points_;
Eigen::Vector3i voxel_index_;
Eigen::Vector3d color_;

private:
Eigen::Vector3d min_color_;
Eigen::Vector3d max_color_;
};

} // namespace geometry
} // namespace open3d
23 changes: 16 additions & 7 deletions cpp/open3d/geometry/VoxelGridFactory.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
const PointCloud &input,
double voxel_size,
const Eigen::Vector3d &min_bound,
const Eigen::Vector3d &max_bound) {
const Eigen::Vector3d &max_bound,
VoxelGrid::VoxelPoolingMode pooling_mode) {
auto output = std::make_shared<VoxelGrid>();
if (voxel_size <= 0.0) {
utility::LogError("voxel_size <= 0.");
Expand All @@ -57,7 +58,7 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
}
output->voxel_size_ = voxel_size;
output->origin_ = min_bound;
std::unordered_map<Eigen::Vector3i, AvgColorVoxel,
std::unordered_map<Eigen::Vector3i, AggColorVoxel,
utility::hash_eigen<Eigen::Vector3i>>
voxelindex_to_accpoint;
Eigen::Vector3d ref_coord;
Expand All @@ -76,9 +77,15 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
}
for (auto accpoint : voxelindex_to_accpoint) {
const Eigen::Vector3i &grid_index = accpoint.second.GetVoxelIndex();
const Eigen::Vector3d &color =
has_colors ? accpoint.second.GetAverageColor()
: Eigen::Vector3d(0, 0, 0);
// clang-format off
const Eigen::Vector3d &color = has_colors ?
(pooling_mode == VoxelPoolingMode::AVG ? accpoint.second.GetAverageColor()
: pooling_mode == VoxelPoolingMode::MIN ? accpoint.second.GetMinColor()
: pooling_mode == VoxelPoolingMode::MAX ? accpoint.second.GetMaxColor()
: pooling_mode == VoxelPoolingMode::SUM ? accpoint.second.GetSumColor()
: Eigen::Vector3d::Zero())
: Eigen::Vector3d::Zero();
// clang-format on
output->AddVoxel(geometry::Voxel(grid_index, color));
}
utility::LogDebug(
Expand All @@ -88,12 +95,14 @@ std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloudWithinBounds(
}

std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromPointCloud(
const PointCloud &input, double voxel_size) {
const PointCloud &input,
double voxel_size,
VoxelGrid::VoxelPoolingMode pooling_mode) {
Eigen::Vector3d voxel_size3(voxel_size, voxel_size, voxel_size);
Eigen::Vector3d min_bound = input.GetMinBound() - voxel_size3 * 0.5;
Eigen::Vector3d max_bound = input.GetMaxBound() + voxel_size3 * 0.5;
return CreateFromPointCloudWithinBounds(input, voxel_size, min_bound,
max_bound);
max_bound, pooling_mode);
}

std::shared_ptr<VoxelGrid> VoxelGrid::CreateFromTriangleMeshWithinBounds(
Expand Down
31 changes: 25 additions & 6 deletions cpp/pybind/geometry/voxelgrid.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,15 @@ void pybind_voxelgrid_definitions(py::module &m) {
static_cast<py::class_<VoxelGrid, PyGeometry3D<VoxelGrid>,
std::shared_ptr<VoxelGrid>, Geometry3D>>(
m.attr("VoxelGrid"));

py::enum_<VoxelGrid::VoxelPoolingMode> pooling_mode(
voxelgrid, "VoxelPoolingMode",
"Mode of determining color for each voxel.");
pooling_mode.value("AVG", VoxelGrid::VoxelPoolingMode::AVG)
.value("MIN", VoxelGrid::VoxelPoolingMode::MIN)
.value("MAX", VoxelGrid::VoxelPoolingMode::MAX)
.value("SUM", VoxelGrid::VoxelPoolingMode::SUM);

py::detail::bind_default_constructor<VoxelGrid>(voxelgrid);
py::detail::bind_copy_functions<VoxelGrid>(voxelgrid);
voxelgrid
Expand Down Expand Up @@ -129,19 +138,23 @@ void pybind_voxelgrid_definitions(py::module &m) {
.def_static("create_from_point_cloud",
&VoxelGrid::CreateFromPointCloud,
"Creates a VoxelGrid from a given PointCloud. The "
"color value of a given voxel is the average color "
"color value of a given voxel is determined by the "
"VoxelPoolingMode, e.g. by default the average color "
"value of the points that fall into it (if the "
"PointCloud has colors). The bounds of the created "
"VoxelGrid are computed from the PointCloud.",
"input"_a, "voxel_size"_a)
"input"_a, "voxel_size"_a,
"pooling_mode"_a = VoxelGrid::VoxelPoolingMode::AVG)
.def_static("create_from_point_cloud_within_bounds",
&VoxelGrid::CreateFromPointCloudWithinBounds,
"Creates a VoxelGrid from a given PointCloud. The "
"color value of a given voxel is the average color "
"color value of a given voxel is determined by the "
"VoxelPoolingMode, e.g. by default the average color "
"value of the points that fall into it (if the "
"PointCloud has colors). The bounds of the created "
"VoxelGrid are defined by the given parameters.",
"input"_a, "voxel_size"_a, "min_bound"_a, "max_bound"_a)
"input"_a, "voxel_size"_a, "min_bound"_a, "max_bound"_a,
"pooling_mode"_a = VoxelGrid::VoxelPoolingMode::AVG)
.def_static("create_from_triangle_mesh",
&VoxelGrid::CreateFromTriangleMesh,
"Creates a VoxelGrid from a given TriangleMesh. No "
Expand All @@ -162,6 +175,7 @@ void pybind_voxelgrid_definitions(py::module &m) {
"origin point.")
.def_readwrite("voxel_size", &VoxelGrid::voxel_size_,
"``float64`` Size of the voxel.");

docstring::ClassMethodDocInject(m, "VoxelGrid", "has_colors");
docstring::ClassMethodDocInject(m, "VoxelGrid", "has_voxels");
docstring::ClassMethodDocInject(m, "VoxelGrid", "get_voxel",
Expand Down Expand Up @@ -214,15 +228,20 @@ void pybind_voxelgrid_definitions(py::module &m) {
docstring::ClassMethodDocInject(
m, "VoxelGrid", "create_from_point_cloud",
{{"input", "The input PointCloud"},
{"voxel_size", "Voxel size of of the VoxelGrid construction."}});
{"voxel_size", "Voxel size of of the VoxelGrid construction."},
{"pooling_mode",
"VoxelPoolingMode for determining voxel color."}});
docstring::ClassMethodDocInject(
m, "VoxelGrid", "create_from_point_cloud_within_bounds",
{{"input", "The input PointCloud"},
{"voxel_size", "Voxel size of of the VoxelGrid construction."},
{"min_bound",
"Minimum boundary point for the VoxelGrid to create."},
{"max_bound",
"Maximum boundary point for the VoxelGrid to create."}});
"Maximum boundary point for the VoxelGrid to create."},
{"pooling_mode",
"VoxelPoolingMode that determines how to compute the voxel "
"color."}});
docstring::ClassMethodDocInject(
m, "VoxelGrid", "create_from_triangle_mesh",
{{"input", "The input TriangleMesh"},
Expand Down
2 changes: 1 addition & 1 deletion docs/jupyter/geometry/voxelization.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"metadata": {},
"source": [
"## From point cloud\n",
"The voxel grid can also be created from a point cloud using the method `create_from_point_cloud`. A voxel is occupied if at least one point of the point cloud is within the voxel. The color of the voxel is the average of all the points within the voxel. The argument `voxel_size` defines the resolution of the voxel grid."
"The voxel grid can also be created from a point cloud using the method `create_from_point_cloud`. A voxel is occupied if at least one point of the point cloud is within the voxel. The argument `voxel_size` defines the resolution of the voxel grid. By default, the color of the voxel is the average of all the points within the voxel. The argument `pooling_mode` can be changed to determine the color by average, min, max or sum value of the points, e.g. with `o3d.geometry.VoxelGrid.VoxelPoolingMode.MIN`."
]
},
{
Expand Down

0 comments on commit ba2a6b1

Please sign in to comment.