Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Models methods & properties (WIP) #93

Merged
merged 28 commits into from
Jun 22, 2024

Conversation

alanvww
Copy link
Member

@alanvww alanvww commented Jun 17, 2024

This PR is for updating methods and properties for each models. @ziyuan-linn Peter, thanks for the amazing works of the library and commenting everything! This update is based on the those content and also the latest examples in the collection. Feel free to review and leave any comments!
@QuinnHe @myrahsa

Progress

  • BodyPose
  • HandPose
  • FaceMesh
  • Body Segmentation
  • Image Classifier
  • Sound Classifier 🚧
  • Sentiment
  • NN

Copy link

netlify bot commented Jun 17, 2024

Deploy Preview for ml5-next-gen-docs ready!

Name Link
🔨 Latest commit b833365
🔍 Latest deploy log https://app.netlify.com/sites/ml5-next-gen-docs/deploys/6676ebecac0a4c0008b5af32
😎 Deploy Preview https://deploy-preview-93--ml5-next-gen-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 18, 2024

Hi @alanvww , thanks for the amazing updates! One thing I noticed, can we include options that are shared by MoveNet and Blaz
Screenshot 2024-06-17 at 8 44 38 PM
ePose. See the screenshot for reference.

@alanvww
Copy link
Member Author

alanvww commented Jun 18, 2024

@QuinnHe got it! I will work on it now :)

@alanvww
Copy link
Member Author

alanvww commented Jun 20, 2024

Quick update for the docs! Just confirmed with Peter, all score labels will be replaced with confidence in 1.0. Check PR here @QuinnHe @myrahsa @MOQN

@alanvww alanvww marked this pull request as ready for review June 20, 2024 18:37
@alanvww
Copy link
Member Author

alanvww commented Jun 20, 2024

Resolved conflicts for updated to main and also included #112

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

@alanvww some bodyseg really new PRs and Issues for your reference, the second one seems more urgent to be included:
switch default runtime to tfjs: ml5js/ml5-next-gen#140
BodySegmentation: Add raw values of the detected body parts to result: ml5js/ml5-next-gen#139

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

@alanvww some bodyseg really new PRs and Issues for your reference, the second one seems more urgent to be included: switch default runtime to tfjs: ml5js/ml5-next-gen#140 BodySegmentation: Add raw values of the detected body parts to result: ml5js/ml5-next-gen#139

I can fix it quickly now. But I didn't realize NN's workload was so huge. I better skip Sound Classifier and focus on this.

In the meantime, would you please also check the preview deployment of this PR to see the new content I added? I tried my best to match the source code, but it might still be missing something.

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

@alanvww some bodyseg really new PRs and Issues for your reference, the second one seems more urgent to be included: switch default runtime to tfjs: ml5js/ml5-next-gen#140 BodySegmentation: Add raw values of the detected body parts to result: ml5js/ml5-next-gen#139

I can fix it quickly now. But I didn't realize NN's workload was so huge. I better skip Sound Classifier and focus on this.

In the meantime, would you please also check the preview deployment of this PR to see the new content I added? I tried my best to match the source code, but it might still be missing something.

Yes, I will go through them after I complete BodySeg other sections.

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

@QuinnHe Got it! For the runtime change, I added the Runtime section in the glossary update. #84 Would you please review and merge this as well? I corrected the content and runtime is the only new section there. I don't think I'll add anything else before tomorrow's launch.

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

@QuinnHe Got it! For the runtime change, I added the Runtime section in the glossary update. #84 Would you please review and merge this as well? I corrected the content and runtime is the only new section there. I don't think I'll add anything else before tomorrow's launch.

Thanks Alan, will review soon!

docs/reference/bodypose.md Outdated Show resolved Hide resolved

[More info on options for BodyPix model.](https://github.com/tensorflow/tfjs-models/blob/master/body-segmentation/src/body_pix/README.md#create-a-detector)

- **callback(bodySegmentation, error)**: OPTIONAL. A function to run once the model has been loaded. Alternatively, call `ml5.bodySegmentation()` within the p5 `preload` function.
- **callback(bodySegmentation)**: OPTIONAL. A function to run once the model has been loaded. Alternatively, call `ml5.bodySegmentation()` within the p5 `preload` function.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remembered we still have error, it's just we don't use it in example code. We could double check this after soft launch, feel free to create a Issue for this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I will consider putting it back with the ? indicating that it is optional. It is actually quite important for debugging.


---

#### .predict()
### sentiment.predict()

> Given a number, will make magicSparkles
Copy link
Collaborator

@QuinnHe QuinnHe Jun 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

> Given a number, will make magicSparkles This does not make a lot sense to me? Maybe we can remove this? Or Alan you have any specific reasons for keeping it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this to be consistent with the methods in the other models page. But I also realize that NN has a completely different structure compared to other models. Should we use the .functionName() style for all the pages?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did this to be consistent with the methods in the other models page. But I also realize that NN has a completely different structure compared to other models. Should we use the .functionName() style for all the pages?

Oh no I refer to > Given a number, will make magicSparkles this line, maybe we should remove this line?

Copy link
Member Author

@alanvww alanvww Jun 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh got it. At least for this one, we should remove it. But the same question still, in NN a lot of > are working as method description. Should I reformat that too? I might need to work on it tomorrow morning if that is ok :/

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh got it. At least for this one, we should remove it. But the same question still, in NN a lot of > are working as method description. Should I reformat that too? I might need to work on it tomorrow morning if that is ok :/

Yes, if time is permitted, let's remove > for NN as well. We can do a merge tomorrow morning around 10-11am ish before the presentation. If can't make it, it's fine, we can keep > for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👌, that is feasible timing.

Copy link
Collaborator

@QuinnHe QuinnHe Jun 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Sentiment, let's update "Properties" style to align with "Methods". We can remove --- divider, and remove >.

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

@alanvww , I've done reviewing the models you have right now! I have to say, this is AWESOME! Thank you so much for your work Alan!!

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

@alanvww , I've done reviewing the models you have right now! I have to say, this is AWESOME! Thank you so much for your work Alan!!

Thanks Quinn! Thanks for the reviews. My schedule is too fragmented these days and sometimes I can't finish reading the codes right away, so I have to go over things again and again. I wish I didn't mess things up too much lol.

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

Hey @alanvww , just let you know that I will start to work on SoundClassifier now, let's avoid duplicated work :)

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

@alanvww , I've done reviewing the models you have right now! I have to say, this is AWESOME! Thank you so much for your work Alan!!

Thanks Quinn! Thanks for the reviews. My schedule is too fragmented these days and sometimes I can't finish reading the codes right away, so I have to go over things again and again. I wish I didn't mess things up too much lol.

Haha as always you did awesome, no worries at all!

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

Hey @alanvww , just let you know that I will start to work on SoundClassifier now, let's avoid duplicated work :)

All good!

- String: The type of model to use. Default: "MULTIPOSE_LIGHTNING".
- _enableSmoothing_ - Optional
- Boolean: Whether to smooth the pose landmarks across different input images to reduce jitter. Default: true.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may need to add Options for MoveNet model only as well. See https://github.com/ml5js/ml5-next-gen/blob/main/src/BodyPose/index.js#L27-L32 for reference

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for point that out! It was in the previous commit but I accidentally removed it when solving conflict when updating. Will fix it now.

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

@QuinnHe @MOQN Last comment before bed. I just added Properties to the model pages.

Question about NN: The origin methods section of it is more like an instruction. The way I modify it right now is to make it match with other page's style more. But I think those contents are still very helpful for beginners.

Maybe we should separate it from the methods and use it in the guide?

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

This PR is for updating methods and properties for each models. @ziyuan-linn Peter, thanks for the amazing works of the library and commenting everything! This update is based on the those content and also the latest examples in the collection. Feel free to review and leave any comments! @QuinnHe @myrahsa

Progress

  • BodyPose
  • HandPose
  • FaceMesh
  • Body Segmentation
  • Image Classifier
  • Sound Classifier 🚧
  • Sentiment
  • NN

Sound Classifier is done. #0649346

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

@QuinnHe @MOQN Last comment before bed. I just added Properties to the model pages.

Question about NN: The origin methods section of it is more like an instruction. The way I modify it right now is to make it match with other page's style more. But I think those contents are still very helpful for beginners.

Maybe we should separate it from the methods and use it in the guide?

Thanks @alanvww for the work on Properties. After soft launch, we can consider put How to initialize NN uder ml5.neuralNetwork().

One small thing - can we make sure the Properties goes before Methods?

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

@QuinnHe @MOQN Last comment before bed. I just added Properties to the model pages.

Question about NN: The origin methods section of it is more like an instruction. The way I modify it right now is to make it match with other page's style more. But I think those contents are still very helpful for beginners.

Maybe we should separate it from the methods and use it in the guide?

Thanks @alanvww for the work on Properties. After soft launch, we can consider put How to initialize NN uder ml5.neuralNetwork().

One small thing - can we make sure the Properties goes before Methods?

Got it. I am on my way to school now. Will fix that soon.

@alanvww
Copy link
Member Author

alanvww commented Jun 22, 2024

@QuinnHe @MOQN this PR is ready for merge now!

@QuinnHe
Copy link
Collaborator

QuinnHe commented Jun 22, 2024

Thanks @alanvww ! This looks great! Merged

@QuinnHe QuinnHe merged commit 1d852ab into ml5js:main Jun 22, 2024
4 checks passed
@alanvww alanvww deleted the alan-methods&properties branch August 6, 2024 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants