Skip to content

Commit

Permalink
fixing memory for normalizer
Browse files Browse the repository at this point in the history
  • Loading branch information
dkazanc committed Jul 28, 2023
1 parent 555e934 commit d057e7f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion httomolibgpu/prep/normalize.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def _normalize_max_slices(

# it also needs space for data input and output (we don't care about slice_dim)
# data: [x, 10, 20], dtype => other_dims = [10, 20]
in_slice_memory = np.prod(non_slice_dims_shape) * uint16().nbytes
in_slice_memory = np.prod(non_slice_dims_shape) * dtype.itemsize
out_slice_memory = np.prod(non_slice_dims_shape) * float32().nbytes
slice_memory = in_slice_memory + out_slice_memory
max_slices = available_memory // slice_memory # rounds down
Expand Down

0 comments on commit d057e7f

Please sign in to comment.