Background Fitting Failure after cluster_analysis of hyperspy

[liuchzzyy]

Hi, all
I am just starting to EELS curve fitting with the data obtained after cluster_analysis of hyperspy, but resulting in a bad Background fitting. 

and also if considering m.print_current_values(), I could have that information:

EELSModel: EELS HL SI
CurrentComponentValues: PowerLaw
Active: True
Parameter Name | Free | Value | Std | Min | Max | Linear
============== | ======= | ========== | ========== | ========== | ========== | ======
A | True | 3.03929749 | 8.80313056 | 0.0 | None | True
left_cutoff | False | 0.0 | None | None | None | False
origin | False | 0.0 | None | None | None | False
r | True | 14.2606613 | 0.47342670 | 1.0 | 5.0 | False

CurrentComponentValues: O_K
Active: True
Parameter Name | Free | Value | Std | Min | Max | Linear
============== | ======= | ========== | ========== | ========== | ========== | ======
intensity | True | -558.15145 | 24.3702120 | 0.0 | None | True
fine_structure | True | (-2.193871 | (19.421992 | None | None | False
effective_angl | False | 53.7034119 | None | None | None | False
onset_energy | False | 532.0 | None | None | None | False

CurrentComponentValues: Mn_L3
Active: True
Parameter Name | Free | Value | Std | Min | Max | Linear
============== | ======= | ========== | ========== | ========== | ========== | ======
intensity | True | -343.88464 | 14.0597759 | 0.0 | None | True
fine_structure | True | (-82.25430 | (37.348486 | None | None | False
effective_angl | False | 53.7047657 | None | None | None | False
onset_energy | False | 640.0 | None | None | None | False

CurrentComponentValues: Mn_L2
Active: True
Parameter Name | Free | Value | Std | Min | Max | Linear
============== | ======= | ========== | ========== | ========== | ========== | ======
intensity | Twinned | -343.88464 | None | 0.0 | None | True
fine_structure | True | (-68.27276 | (44.552646 | None | None | False
effective_angl | False | 53.7049174 | None | None | None | False
onset_energy | Twinned | 651.0 | None | None | None | False

Then I tried the remove_background of my data obtained from cluster_analysis, but it does not plot properly, which should show background and the data after background removal, but not, as below:

also with or without the parameters signal_range and fast = False, they don't work,,, :( (which mentioned in Hyperspy issues)

To think out where the problem is from, first I tried the sample and code from exspy demo, and it works well, which means the issue does not stem from the package, I guess. (sample and example code I used)

I have no idea what else I could attempt further, hence want to ask a question here,
any idea would be appreciative.
And also, I attached one high loss spectrum and related low loss from cluster_analysis of hyperspy in case you could have a trrrry.
Clusters.zip

Here is my code for EELS curve fitting

## data import
data = hs.load(path.joinpath(file_path, r'STEM SI.dm4'))
data[-2].align_zero_loss_peak(subpixel=True, also_align=[data[-1]])   # data[-2] is low loss, data[-1] is high loss

## cluster_analysis
s = data[-1].deepcopy()
s.decomposition(algorithm="SVD", centre="signal")  
num_clusters = s.estimate_number_of_clusters(cluster_source="decomposition", preprocessing="norm", algorithm="kmeans")
s.cluster_analysis(cluster_source="decomposition", n_clusters=num_clusters, preprocessing="norm", algorithm="kmeans", n_init=8)
cluster_signals = s.get_cluster_signals(signal='mean')
cluster_labels = s.get_cluster_labels()

metadata = data[-1].metadata.as_dictionary()
cluster_signals.metadata.add_dictionary(metadata)
cluster_signals.add_elements(('Mn', 'O'))

cluster_signals.inav[1].save(path.joinpath(file_path, r'Clusters_High_Loss.hspy'))
data[-2].inav[27,27].save(path.joinpath(file_path, r'Clusters_Low_Loss.hspy'))

## EELS curve fitting
EELS = cluster_signals.inav[1].deepcopy()  # using one spectrum to do EELS fitting
m = EELS.create_model(low_loss=data[-2].inav[27, 27])
m.enable_fine_structure()
m.smart_fit()
m.plot(plot_components=False)
m.print_current_values()

It is also showing some outputs like:

WARNING | Hyperspy | Covariance of the parameters could not be estimated. Estimated parameter standard deviations will be np.nan. (hyperspy.model:1409)
WARNING | Hyperspy | `m.fit()` did not exit successfully. Reason: Number of calls to function has reached maxfev = 600. (hyperspy.model:1875)

[CSSFrancis]

I'm going to leave the discussion to people who are a bit more knowledgeable in doing eels. @francisco-dlp and @ericpre might have good suggestions. The background for your data doens't look like power law function. It could be because you are in the more linear reagion that fitting the background is struggling.

There are lots of different options for background subtraction you could try:

https://hyperspy.org/hyperspy-doc/current/reference/api.signals/Signal1D.html#hyperspy.api.signals.Signal1D.remove_background

Maybe start with a polynomial of degree 2-3 and see if that works. A better discussion of the different types of backgrounds common to electron microscopy might be a useful addition to the documentation.

This PR might also be relevent https://github.com/hyperspy/hyperspy/pull/3381/files/2ddac67596e725517a6d2d937e0f4c279dcf331c..4175422e6f52e85ec3fb09b3bb8a086b670e556f

[ericpre]

Similarly as @CSSFrancis already said, it seems that the spectrum you get from clustering is not suitable for what you are trying to do:

• there are negative values

• it doesn't follow a power law: after adding an offset, this is better but still the power law doesn't describe well the background:

[liuchzzyy]

Thanks @ericpre and @CSSFrancis! and it work although not perfecto anyway.