Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop Process - same functionality as USIDataset #12

Open
ssomnath opened this issue Jun 12, 2020 · 3 comments
Open

Develop Process - same functionality as USIDataset #12

ssomnath opened this issue Jun 12, 2020 · 3 comments
Assignees
Labels
core core classes or infrastructure enhancement New feature or request

Comments

@ssomnath
Copy link
Member

ssomnath commented Jun 12, 2020

  1. What changes would you like to make to this class if you could - this is your chance.
  2. __init__ can't be taken more or less as is -
    • would need some way to figure out how to flatten the data into two axes - examples vs features (over which data would be processed)
  3. Take these functions as is:
    • assign job indices
    • estimate compute time per pixel
    • get pixels in current batch
    • test
    • use_partial_Computation
    • set_cores
    • set_memory_and_cores
    • map_function
    • write_results_chunk
    • create_results_dataset
    • write_source_dset_provenance
    • get_existing_datasets
    • _unit_computation
  4. Rewrite the following:
    • check for duplicates
    • set_memory
    • read_data_chunk
    • create_compute_status_dataset
  5. Carefully rewrite compute()
@ssomnath
Copy link
Member Author

Process class will expect NSIDataset to arise from HDF5 (data should exist safely on disk)

@ssomnath ssomnath added this to the Bells and Whistles milestone Jul 24, 2020
@ssomnath
Copy link
Member Author

We will use @gduscher 's problem - Gaussian mixtures model applied to denoising EELS spectra as the basis for developing the new Process class. I am still not entirely sure where this class will live. There may be a possibility to generalize the Process class by finding commonalities with pyUSID.Process

@ssomnath
Copy link
Member Author

ssomnath commented Sep 4, 2020

This issue may need to be moved to sidpy

@ssomnath ssomnath added core core classes or infrastructure enhancement New feature or request labels Sep 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core core classes or infrastructure enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants