[BUG] TypeError when fitting a model to MaNGA Cubes

[james-trayford]

Jdaviz component

Cubeviz

Description

Perhaps a lingering issue from #3307: An issue when trying to model fit a MaNGA cube that throws a TypeError in the console log. It seems model fitting still runs (produces WARNING: Model is linear in parameters; consider using linear fitting methods. [astropy.modeling.fitting] warnings) but gives bad results.

How to Reproduce

1. Import a MaNGA Cube (e.g. cubeviz.load_data(f"https://data.sdss.org/sas/dr17/manga/spectro/redux/v3_1_1/7443/stack/manga-7443-12703-LOGCUBE.fits.gz"))
2. Open Model Fitting plug-in
3. Toggle on Cube Fit
4. Error Triggered

Full trace:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:773](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=772), in Widget._handle_msg(self, msg)
    771         if 'buffer_paths' in data:
    772             _put_buffers(state, data['buffer_paths'], msg['buffers'])
--> 773         self.set_state(state)
    775 # Handle a state request.
    776 elif method == 'request_state':

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:650](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=649), in Widget.set_state(self, sync_data)
    645         self._send(msg, buffers=echo_buffers)
    647 # The order of these context managers is important. Properties must
    648 # be locked when the hold_trait_notification context manager is
    649 # released and notifications are fired.
--> 650 with self._lock_property(**sync_data), self.hold_trait_notifications():
    651     for name in sync_data:
    652         if name in self.keys:

File [/opt/homebrew/Cellar/python](http://localhost:8891/opt/homebrew/Cellar/python)@3.11[/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py:144](http://localhost:8891/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/contextlib.py#line=143), in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    142 if typ is None:
    143     try:
--> 144         next(self.gen)
    145     except StopIteration:
    146         return False

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1510](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1509), in HasTraits.hold_trait_notifications(self)
   1508 for changes in cache.values():
   1509     for change in changes:
-> 1510         self.notify_change(change)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:701](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=700), in Widget.notify_change(self, change)
    698     if name in self.keys and self._should_send_property(name, getattr(self, name)):
    699         # Send new state to front-end
    700         self.send_state(key=name)
--> 701 super().notify_change(change)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1525](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1524), in HasTraits.notify_change(self, change)
   1523 def notify_change(self, change: Bunch) -> None:
   1524     """Notify observers of a change event"""
-> 1525     return self._notify_observers(change)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1568](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1567), in HasTraits._notify_observers(self, event)
   1565 elif isinstance(c, EventHandler) and c.name is not None:
   1566     c = getattr(self, c.name)
-> 1568 c(event)

File [~/Documents/Code/jdaviz/jdaviz/configs/default/plugins/model_fitting/model_fitting.py:324](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/configs/default/plugins/model_fitting/model_fitting.py#line=323), in ModelFitting._cube_fit_changed(self, event)
    322     self._units['y'] = sb_unit
    323     self.dataset.add_filter('is_flux_cube')
--> 324     self.dataset.remove_filter('layer_in_spectrum_viewer')
    325 else:
    326     self._units['y'] = spectral_y_unit

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:906](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=905), in SelectPluginComponent.remove_filter(self, *filters)
    905 def remove_filter(self, *filters):
--> 906     self.filters = [f for f in self.filters
    907                     if (f not in filters and getattr(f, '__name__', '') not in filters)]

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:685](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=684), in BasePluginComponent.__setattr__(self, attr, value, force_super)
    683 def __setattr__(self, attr, value, force_super=False):
    684     if attr[0] == '_' or force_super or attr not in self._plugin_traitlets.keys():
--> 685         return super().__setattr__(attr, value)
    687     return setattr(self._plugin, self._plugin_traitlets.get(attr), value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:716](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=715), in TraitType.__set__(self, obj, value)
    714 if self.read_only:
    715     raise TraitError('The "%s" trait is read-only.' % self.name)
--> 716 self.set(obj, value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:3635](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=3634), in List.set(self, obj, value)
   3633     return super().set(obj, [value])  # type:ignore[list-item]
   3634 else:
-> 3635     return super().set(obj, value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:706](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=705), in TraitType.set(self, obj, value)
    702     silent = False
    703 if silent is not True:
    704     # we explicitly compare silent to True just in case the equality
    705     # comparison above returns something other than True[/False](http://localhost:8891/False)
--> 706     obj._notify_trait(self.name, old_value, new_value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1513](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1512), in HasTraits._notify_trait(self, name, old_value, new_value)
   1512 def _notify_trait(self, name: str, old_value: t.Any, new_value: t.Any) -> None:
-> 1513     self.notify_change(
   1514         Bunch(
   1515             name=name,
   1516             old=old_value,
   1517             new=new_value,
   1518             owner=self,
   1519             type="change",
   1520         )
   1521     )

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1525](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1524), in HasTraits.notify_change(self, change)
   1523 def notify_change(self, change: Bunch) -> None:
   1524     """Notify observers of a change event"""
-> 1525     return self._notify_observers(change)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1568](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1567), in HasTraits._notify_observers(self, event)
   1565 elif isinstance(c, EventHandler) and c.name is not None:
   1566     c = getattr(self, c.name)
-> 1568 c(event)

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:3671](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=3670), in DatasetSelect._on_data_changed(self, msg)
   3668 manual_items = [{'label': label} for label in self.manual_options]
   3669 self.items = manual_items + [_dc_to_dict(data) for data in self.app.data_collection
   3670                              if self._is_valid_item(data)]
-> 3671 self._apply_default_selection()
   3672 # future improvement: only clear cache if the selected data entry was changed?
   3673 self._clear_cache(*self._cached_properties)

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:984](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=983), in SelectPluginComponent._apply_default_selection(self, skip_if_current_valid)
    982 default_empty = [] if self.is_multiselect else ''
    983 if self.default_mode == 'first':
--> 984     self.selected = self.labels[0] if len(self.labels) else default_empty
    985 elif self.default_mode == 'default_text':
    986     self.selected = self._default_text if self._default_text else default_empty

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:687](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=686), in BasePluginComponent.__setattr__(self, attr, value, force_super)
    684 if attr[0] == '_' or force_super or attr not in self._plugin_traitlets.keys():
    685     return super().__setattr__(attr, value)
--> 687 return setattr(self._plugin, self._plugin_traitlets.get(attr), value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:716](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=715), in TraitType.__set__(self, obj, value)
    714 if self.read_only:
    715     raise TraitError('The "%s" trait is read-only.' % self.name)
--> 716 self.set(obj, value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:706](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=705), in TraitType.set(self, obj, value)
    702     silent = False
    703 if silent is not True:
    704     # we explicitly compare silent to True just in case the equality
    705     # comparison above returns something other than True[/False](http://localhost:8891/False)
--> 706     obj._notify_trait(self.name, old_value, new_value)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1513](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1512), in HasTraits._notify_trait(self, name, old_value, new_value)
   1512 def _notify_trait(self, name: str, old_value: t.Any, new_value: t.Any) -> None:
-> 1513     self.notify_change(
   1514         Bunch(
   1515             name=name,
   1516             old=old_value,
   1517             new=new_value,
   1518             owner=self,
   1519             type="change",
   1520         )
   1521     )

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py:701](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/ipywidgets/widgets/widget.py#line=700), in Widget.notify_change(self, change)
    698     if name in self.keys and self._should_send_property(name, getattr(self, name)):
    699         # Send new state to front-end
    700         self.send_state(key=name)
--> 701 super().notify_change(change)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1525](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1524), in HasTraits.notify_change(self, change)
   1523 def notify_change(self, change: Bunch) -> None:
   1524     """Notify observers of a change event"""
-> 1525     return self._notify_observers(change)

File [~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py:1568](http://localhost:8891/lab/tree/notebook/~/Documents/Code/strauss_jdat/vslim/lib/python3.11/site-packages/traitlets/traitlets.py#line=1567), in HasTraits._notify_observers(self, event)
   1565 elif isinstance(c, EventHandler) and c.name is not None:
   1566     c = getattr(self, c.name)
-> 1568 c(event)

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:2940](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=2939), in NonFiniteUncertaintyMismatchMixin._check_non_finite_uncertainty_mismatch(self, event)
   2936 if not hasattr(self, 'dataset') or self.dataset_selected == '':
   2937     # during initial init, this can trigger before the component is initialized
   2938     return
-> 2940 spec = self.dataset.selected_spectrum
   2942 if spec.uncertainty is None:
   2943     self.non_finite_uncertainty_mismatch = False

File [/opt/homebrew/Cellar/python](http://localhost:8891/opt/homebrew/Cellar/python)@3.11[/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py:1001](http://localhost:8891/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/functools.py#line=1000), in cached_property.__get__(self, instance, owner)
    999 val = cache.get(self.attrname, _NOT_FOUND)
   1000 if val is _NOT_FOUND:
-> 1001     val = self.func(instance)
   1002     try:
   1003         cache[self.attrname] = val

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:3551](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=3550), in DatasetSelect.selected_spectrum(self)
   3549 @cached_property
   3550 def selected_spectrum(self):
-> 3551     return self.get_selected_spectrum(use_display_units=True)

File [~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py:3545](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/template_mixin.py#line=3544), in DatasetSelect.get_selected_spectrum(self, use_display_units)
   3541     spec_extract = self.app._jdaviz_helper.plugins['Spectral Extraction']._obj
   3542     sp = spec_extract._extract_in_new_instance(self.selected,
   3543                                                function=self._spectral_extraction_function,
   3544                                                add_data=False)
-> 3545     return self.plugin._specviz_helper._handle_display_units(sp, use_display_units)
   3546 return self.plugin._specviz_helper.get_data(data_label=self.selected,
   3547                                             use_display_units=use_display_units)

File [~/Documents/Code/jdaviz/jdaviz/core/helpers.py:479](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/core/helpers.py#line=478), in ConfigHelper._handle_display_units(self, data, use_display_units)
    477     new_uncert = uncertainty
    478 if ('_pixel_scale_factor' in data.meta):
--> 479     new_uncert_converted = flux_conversion(new_uncert.quantity.value,
    480                                            new_uncert.unit, y_unit, spec=data)
    481     new_uncert = StdDevUncertainty(new_uncert_converted, unit=y_unit)
    482 else:

File [~/Documents/Code/jdaviz/jdaviz/utils.py:442](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/utils.py#line=441), in flux_conversion(values, original_units, target_units, spec, eqv, slice)
    434 eqv += _eqv_flux_to_sb_pixel()
    436 # when angle<>pixel translations are enabled
    437 # eqv += _eqv_sb_per_pixel_to_per_angle(u.Jy)
    438 
    439 # indirect units cannot be directly converted, and require
    440 # additional conversions to reach the desired end unit.
    441 # if spec_unit in [original_units, target_units]:
--> 442 result = _indirect_conversion(
    443             values=values, orig_units=orig_units, targ_units=targ_units,
    444             eqv=eqv, spec_unit=spec_unit
    445         )
    447 if result and len(result) == 2:
    448     values, updated_units = result

File [~/Documents/Code/jdaviz/jdaviz/utils.py:512](http://localhost:8891/lab/tree/notebook/~/Documents/Code/jdaviz/jdaviz/utils.py#line=511), in _indirect_conversion(values, orig_units, targ_units, eqv, spec_unit, image_data)
    508 if ((u.Unit(targ_units) in indirect_units()) or
    509    (u.Unit(orig_units) in indirect_units())):
    510     # SB -> Flux -> Flux -> SB
    511     temp_orig = orig_units * solid_angle_in_orig
--> 512     temp_targ = targ_units * solid_angle_in_targ
    514     # Convert Surface Brightness to Flux, then Flux to Flux
    515     values = (values * orig_units).to_value(temp_orig, equivalencies=eqv)

TypeError: unsupported operand type(s) for *: 'CompositeUnit' and 'NoneType'

Expected behavior

Should run error free and produce appropriate fit

Browser

Chrome

Jupyter

Selected Jupyter core packages...
IPython          : 8.26.0
ipykernel        : 6.29.5
ipywidgets       : 8.1.3
jupyter_client   : 8.6.2
jupyter_core     : 5.7.2
jupyter_server   : 2.14.1
jupyterlab       : 4.2.3
nbclient         : 0.7.4
nbconvert        : 7.16.4
nbformat         : 5.10.4
notebook         : 7.2.1
qtconsole        : not installed
traitlets        : 5.14.3

Software versions

Versions:

macOS-14.2-arm64-arm-64bit
Python 3.11.9 (main, Apr  2 2024, 08:25:04) [Clang 15.0.0 (clang-1500.3.9.4)]
Numpy 2.0.0
astropy 6.1.1
matplotlib 3.8.4
scipy 1.14.0
scikit-image 0.24.0
asdf 3.2.0
stdatamodels 2.0.0
gwcs 0.21.0
regions 0.9
specutils 1.19.0
specreduce 1.4.1
photutils 1.13.0
astroquery 0.4.7
pyyaml 6.0.1
asteval 1.0.0
idna 3.7
traitlets 5.14.3
bqplot 0.12.43
bqplot-image-gl 1.5.0
glue-core 1.21.0
glue-jupyter 0.23.1
glue-astronomy 0.10.0
echo 0.8.0
ipyvue 1.11.1
ipyvuetify 1.9.4
ipysplitpanes 0.2.0
ipygoldenlayout 0.4.0
ipypopout 2.0.0
Jinja2 3.1.4
solara 1.41.0
vispy 0.14.3
sidecar 0.7.0
Jdaviz 0.1.dev5434+g1408806

[james-trayford]

RE optical IFU surveys, I've alsohad automatic spectral extraction errors for SAMI (e.g. file) and CALIFA (e.g. file) cubes. Was there any plan to support these?

[kecnry]

are you on jdaviz main or the released version? #3307 should fix this on main, but from the description there it wasn't expected to be a problem in 4.0 so we didn't issue a bugfix. cc @rosteen

[james-trayford]

@kecnry this is on current main - note I can now open MaNGA cubes with automatic spectral extraction, the errror above is when you try and run the model fitting. It's SAMI and CALIFA that have the same problem as the original MaNGA issue in #3307

[pllim]

What is the cube data unit?

It is triggered here but somehow solid_angle_in_targ is None.

jdaviz/jdaviz/utils.py

Line 512 in 527d1f5

temp_targ = targ_units * solid_angle_in_targ

[james-trayford]

@pllim cube BUNIT is '1E-17 erg/s/cm^2/Angstrom/spaxel' (see #3307 (comment))

RE solid_angle_in_targ - noticed this - I also noticed this conditional above:

jdaviz/jdaviz/utils.py

Lines 506 to 507 in 527d1f5

	if not solid_angle_in_targ:
	targ_units /= solid_angle_in_spec

which should catch this case but couldn't work out why - perhaps a clue?

[pllim]

which should catch this case

L507 acknowledges that solid_angle_in_targ could be None but still tries to stuff it into a unit on L512, which does not make sense to me. Perhaps a bug of not handling this edge case properly?

[gibsongreen]

I coincidentally noticed this bug sometime last week and currently have a PR that is currently in review for the exact lines mentioned in utils.py. I tested it on a few manga cubes including the one provided here and this PR should do the trick.