Update Models methods & properties (WIP)

docs/reference/body-segmentation.md

@@ -148,7 +148,7 @@ You have successfully built the BodySegmentation Mask Body Part example! Press t
 
 ## Methods
 
-#### ml5.bodySegmentation()
+### ml5.bodySegmentation()
 
 This method is used to initialize the bodySegmentation object.
 
@@ -158,9 +158,13 @@ const bodySegmentation = ml5.bodySegmentation(?modelName, ?options, ?callback);
 
 **Parameters:**
 
-- **modelName**: OPTIONAL: A string specifying which model to use, "SelfieSegmentation" or "BodyPix".
+- **modelName**: Optional. A string specifying which model to use. Types of model:
+  - _SelfieSegmentation_(default): A model that can be used to segment people from the background.
+  - _BodyPix_: A model that can be used to segment people and body parts.
 
-- **options**: OPTIONAL. An object to change the default configuration of the model. See the example options object:
+
+
+- **options**: Optional. An object to change the default configuration of the model. See the example options object:
 
   ```javascript
   {
@@ -170,18 +174,25 @@ const bodySegmentation = ml5.bodySegmentation(?modelName, ?options, ?callback);
   }
   ```
 
-  [More info on options for SelfieSegmentation model with mediaPipe runtime](https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation/src/selfie_segmentation_mediapipe#create-a-detector).
+  Important Option:
+  - **maskType**: The type of mask to output. The options are:
+    - _background_: A mask of the background. The result is an image with transparent pixels on the background and black pixels on the person.
+    - _body_: A mask of the person. The result is an image with black pixels on the background and transparent pixels on the person.
+    - _parts_: **BodyPix** only. A mask of the body parts. The result is an image with white pixels on the background and various color pixels for each body part.
+
+    [More info on options for SelfieSegmentation model with mediaPipe runtime](https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation/src/selfie_segmentation_mediapipe#create-a-detector).
 
-  [More info on options for SelfieSegmentation model with tfjs runtime](https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation/src/selfie_segmentation_tfjs#create-a-detector).
+    [More info on options for SelfieSegmentation model with tfjs runtime](https://github.com/tensorflow/tfjs-models/tree/master/body-segmentation/src/selfie_segmentation_tfjs#create-a-detector).
 
   [More info on options for BodyPix model.](https://github.com/tensorflow/tfjs-models/blob/master/body-segmentation/src/body_pix/README.md#create-a-detector)
 
-- **callback(bodySegmentation, error)**: OPTIONAL. A function to run once the model has been loaded. Alternatively, call `ml5.bodySegmentation()` within the p5 `preload` function.
+- **callback(bodySegmentation)**: OPTIONAL. A function to run once the model has been loaded. Alternatively, call `ml5.bodySegmentation()` within the p5 `preload` function.
 
 **Returns:**  
-The bodySegmentation object.
 
-#### bodySegmentation.detectStart()
+- **Object**: The bodySegmentation object. This object contains the methods to start and stop the body segment detection process.
+
+### bodySegmentation.detectStart()
 
 This method repeatedly outputs segmentation masks on an image media through a callback function.
 
@@ -193,7 +204,7 @@ bodySegmentation.detectStart(media, callback);
 
 - **media**: An HTML or p5.js image, video, or canvas element to run the segmentation on.
 
-- **callback(output, error)**: A function to handle the output of `bodySegmentation.detectStart()`. Likely a function to do something with the segmented image. Based on your options, the `output` can be a mask of the background, a mask of the person, or a mask of the body parts. Regardless of the scenario, the segmentation information deemed valuable is consistently stored within the `mask` property of the resulting object. See below for the output passed into the callback function:
+- **callback(results)**: A function to handle the output of `bodySegmentation.detectStart()`. Likely a function to do something with the segmented image. Based on your options, the `results` can be a mask of the background, a mask of the person, or a mask of the body parts. Regardless of the scenario, the segmentation information deemed valuable is consistently stored within the `mask` property of the resulting object. See below for the output passed into the callback function:
 
   ```javascript
   {
@@ -202,26 +213,15 @@ bodySegmentation.detectStart(media, callback);
   }
   ```
 
-#### bodySegmentation.detectStop()
+  _results.mask_ under different _maskType_ options:
+  - _background_: A mask of the background. _results.mask_ is an image with transparent pixels on the background and black pixels on the person.
+  - _body_: A mask of the person. _results.mask_ is an image with black pixels on the background and transparent pixels on the person.
+  - _parts_: **BodyPix** only. _results.mask_ is an image with white pixels on the background and various color pixels for each body part.
+
+### bodySegmentation.detectStop()
 
 This method can be called after a call to `bodySegmentation.detectStart` to stop the repeating pose estimation.
 
 ```javascript
 bodySegmentation.detectStop();
-```
-
-#### bodySegmentation.detect()
-
-This method asynchronously outputs a single segmentation mask on an image media when called.
-
-```javascript
-bodySegmentation.detect(media, ?callback);
-```
-
-**Parameters:**
-
-- **media**: An HTML or p5.js image, video, or canvas element to run the segmentation on.
-- **callback(output, error)**: OPTIONAL. A callback function to handle the output of the estimation, see output example above.
-
-**Returns:**  
-A promise that resolves to the segmentation output.
+```

docs/reference/bodypose.md

@@ -159,7 +159,7 @@ Within each pose, we only want to draw the skeleton connections that the model h
 
 We iterate through the connections array, with each item being a link of `pointA` and `pointB`. For instance, `connections[1]` is `[0, 2]`, where 0 is the index of `pointA` and 2 is the index of `pointB`. Thus, `let pointAIndex = connections[j][0];` means we get the starting point (pointA) of the link `j`, and `let pointBIndex = connections[j][1];` means we get the ending point (pointB) of the link `j`.
 
-Use the indices to retrieve the `pointA` and `pointB` objects from the `pose.keypoints`. As with all keypoints, `pointA` is an object with properties `x`, `y`, and `score`.
+Use the indices to retrieve the `pointA` and `pointB` objects from the `pose.keypoints`. As with all keypoints, `pointA` is an object with properties `x`, `y`, and `confidence`.
 
 ```javascript
     for (let j = 0; j < connections.length; j++) {
@@ -240,12 +240,52 @@ let bodypose = ml5.bodypose(?model, ?options, ?callback);
 
 - **options**: Optional. An object to change the default configuration of the model. The available options differ depending on which of the two underlying models are used.
 
-See See the [MoveNet documentation](https://github.com/tensorflow/tfjs-models/tree/master/pose-detection/src/movenet#create-a-detector) and the [BlazePose documentation](https://github.com/tensorflow/tfjs-models/tree/master/pose-detection/src/blazepose_tfjs#create-a-detector) for more information on available options.
+  The default and available options are:
 
-- **callback(bodypose, error)**: Optional. A "callback" function that runs when the model has been successfully loaded. Most ml5.js example call `ml5.bodyPose()` in the p5.js `preload()` function and no callback is needed.
+  ```javascript
+  {
+    modelType: "MULTIPOSE_LIGHTNING" // "MULTIPOSE_LIGHTNING", "SINGLEPOSE_LIGHTNING", or "SINGLEPOSE_THUNDE"
+    enableSmoothing: true,
+    minPoseScore: 0.25,
+    multiPoseMaxDimension: 256,
+    enableTracking: true,
+    trackerType: "boundingBox", // "keypoint" or "boundingBox"
+    trackerConfig: {},
+    modelUrl: undefined,
+  }
+  ```
+
+  Options for both models:
+  - _modelType_ - Optional
+    - String: The type of model to use. Default: "MULTIPOSE_LIGHTNING".
+  - _enableSmoothing_ - Optional
+    - Boolean: Whether to smooth the pose landmarks across different input images to reduce jitter. Default: true.
+
+  Options for the BlazePose model only:
+  - _runtime_ - Optional
+    - String: Either "tfjs" or "mediapipe". Default: "tfjs"
+  - _enableSegmentation_ - Optional
+    - Boolean: A boolean indicating whether to generate the segmentation mask.
+  - _smoothSegmentation_ - Optional
+    - Boolean: whether to filters segmentation masks across different input images to reduce jitter.
+
+  For using custom or offline models
+  - _modelUrl_ - Optional
+    - String: The file path or URL to the MoveNet model.
+  - _solutionPath_ - Optional
+    - String: The file path or URL to the mediaPipe BlazePose model.
+  - _detectorModelUrl_ - Optional
+    - String: The file path or URL to the tfjs BlazePose detector model.
+  - _landmarkModelUrl_ - Optional
+    - String: The file path or URL to the tfjs BlazePose landmark model.
+
+  See See the [MoveNet documentation](https://github.com/tensorflow/tfjs-models/tree/master/pose-detection/src/movenet#create-a-detector) and the [BlazePose documentation](https://github.com/tensorflow/tfjs-models/tree/master/pose-detection/src/blazepose_tfjs#create-a-detector) for more information on available options.
+
+- **callback(bodypose)**: Optional. A "callback" function that runs when the model has been successfully loaded. Most ml5.js example call `ml5.bodyPose()` in the p5.js `preload()` function and no callback is needed.
 
 **Returns:**  
-The bodyPose object.
+
+- **Object**: The bodyPose object. This object contains the methods to start and stop the pose detection process.
 
 ### bodypose.detectStart()
 
@@ -257,16 +297,15 @@ bodypose.detectStart(media, callback);
 
 **Parameters:**
 
-- **media**: An HMTL or p5.js image, video, or canvas element to run the estimation on.
-
-- **callback(results, error)**: A callback function to handle the results of the pose estimation. See below for an example of the model's results:
+- **media**: An HTML or p5.js image, video, or canvas element to run the estimation on.
+- **callback(results)**: A callback function to handle the results of the pose estimation. See below for an example of the model's results:
 
   ```javascript
   [
     {
       box: { width, height, xMax, xMin, yMax, yMin },
       id: 1,
-      keypoints: [{ x, y, score, name }, ...],
+      keypoints: [{ x, y, confidence, name }, ...],
       left_ankle: { x, y, confidence },
       left_ear: { x, y, confidence },
       left_elbow: { x, y, confidence },
@@ -302,8 +341,8 @@ bodypose.detectStart(media, callback);
     {
       box: { width, height, xMax, xMin, yMax, yMin },
       id: 1,
-      keypoints: [{ x, y, z, score, name }, ...],
-      keypoints3D: [{ x, y, z, score, name }, ...],
+      keypoints: [{ x, y, z, confidence, name }, ...],
+      keypoints3D: [{ x, y, z, confidence, name }, ...],
       left_ankle: { x, y, z, confidence },
       left_ear: { x, y, z, confidence },
       left_elbow: { x, y, z, confidence },
@@ -324,6 +363,26 @@ This method can be called to stop the continuous pose estimation process.
 bodypose.detectStop();
 ```
 
+For example, you can toggle the pose estimation with click event in p5.js by using this function as follows:
+
+```javascript
+// Toggle detection when mouse is pressed
+function mousePressed() {
+  toggleDetection();
+}
+
+// Call this function to start and stop detection
+function toggleDetection() {
+  if (isDetecting) {
+    bodypose.detectStop();
+    isDetecting = false;
+  } else {
+    bodyPose.detectStart(video, gotPoses);
+    isDetecting = true;
+  }
+}
+```
+
 ### bodypose.detect()
 
 This method runs the pose estimation on an image once, not continuously!
@@ -336,4 +395,39 @@ bodypose.detect(media, ?callback);
 
 - **media**: An HTML or p5.js image, video, or canvas element to run the estimation on.
 
-- **callback(output, error)**: Optional. A callback function to handle the results of the pose estimation. See the results above for an example of the model's output.
+- **callback(results)**: Optional. A callback function to handle the results of the pose estimation. See the results above for an example of the model's output.
+
+### bodypose.getSkeleton()
+
+This method returns an array of arrays, where each sub-array contains the indices of the connected keypoints.
+
+```javascript
+const connections = bodypose.getSkeleton();
+```
+
+**Returns:**  
+
+- **Array**: An array of arrays representing the connections between keypoints. For example, using BlazePose model will returns:
+
+```js
+[
+    [0,1],
+    [0,4],
+    [1,2],
+   ...
+    [28,32],
+    [29,31],
+    [30,32]
+]
+```
+
+This array represents the connections between keypoints, please refer to these images to understand the connections:
+<center>
+      <h3>MoveNet</h3>
+      <img style="display:block; max-width:30%" alt="MoveNet keypoint diagram" src="https://camo.githubusercontent.com/c3641b718d7e613b2ce111a6a4575e88ca35a60cb325efdd9113c453b2a09301/68747470733a2f2f73746f726167652e676f6f676c65617069732e636f6d2f6d6f76656e65742f636f636f2d6b6579706f696e74732d3530302e706e67">
+  </center> <br/>
+   <center>
+      <h3>BlazePose</h3>
+      <img style="display:block; max-width:30%" alt="BlazePose keypoint diagram" src="https://camo.githubusercontent.com/17082997c33fc6d2544c4aea33d9898860cf902ed5a0b865527d1dd91bbc7efa/68747470733a2f2f73746f726167652e676f6f676c65617069732e636f6d2f6d65646961706970652f626c617a65706f73652d6b6579706f696e74732d757064617465642e706e67">
+  </center>
+

docs/reference/facemesh.md

@@ -177,10 +177,10 @@ And, that's it! You have successfully built the FaceMesh Keypoints example from
 
 #### ml5.faceMesh()
 
-This method is used to initialize the facemesh object.
+This method is used to initialize the faceMesh object.
 
 ```javascript
-const facemesh = ml5.faceMesh(?options, ?callback);
+const faceMesh = ml5.faceMesh(?options, ?callback);
 ```
 
 **Parameters:**
@@ -195,26 +195,42 @@ const facemesh = ml5.faceMesh(?options, ?callback);
   }
   ```
 
+  Options for face detection:
+
+  - _maxFacess_
+    - Number: The maximum number of faces to detect. Defaults to 2.
+  - _refineLandmarks_ 
+    - Boolean: Refine the landmarks. Defaults to false.
+  - _flipHorizontal_ 
+    - Boolean: Flip the result horizontally. Defaults to false.
+  - _runtime_
+    - String: The runtime to use. "tfjs" (default) or "mediapipe".
+
+  For using custom or offline models:
+
+  - _solutionPath_
+    - String: The file path or URL to the model.
+
   More info on options [here](https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection/src/mediapipe#create-a-detector).
 
-- **callback(facemesh, error)**: Optional. A function to run once the model has been loaded. Alternatively, call `ml5.faceMesh()` within the p5 `preload` function.
+- **callback(faceMesh)**: Optional. A function to run once the model has been loaded. Alternatively, call `ml5.faceMesh()` within the p5 `preload` function.
 
 **Returns:**  
-The facemesh object.
 
-#### facemesh.detectStart()
+- **Object**: The faceMesh object. This object contains the methods to start and stop the detection process.
+
+### faceMesh.detectStart()
 
 This method repeatedly outputs face estimations on an image media through a callback function.
 
 ```javascript
-facemesh.detectStart(media, callback);
+faceMesh.detectStart(media, callback);
 ```
 
 **Parameters:**
 
-- **media**: An HMTL or p5.js image, video, or canvas element to run the estimation on.
-
-- **callback(output, error)**: A callback function to handle the output of the estimation. See below for an example output passed into the callback function:
+- **media**: An HTML or p5.js image, video, or canvas element to run the estimation on.
+- **callback(results)**: A callback function to handle the output of the estimation. See below for an example output passed into the callback function:
 
   ```javascript
   [
@@ -231,27 +247,24 @@ facemesh.detectStart(media, callback);
 
   [Here](https://github.com/tensorflow/tfjs-models/blob/master/face-landmarks-detection/mesh_map.jpg) is a diagram for the position of each keypoint (download and zoom in to see the index).
 
-#### facemesh.detectStop()
+### faceMesh.detectStop()
 
-This method can be called after a call to `facemesh.detectStart` to stop the repeating face estimation.
+This method can be called after a call to `faceMesh.detectStart` to stop the repeating face estimation.
 
 ```javascript
-facemesh.detectStop();
+faceMesh.detectStop();
 ```
 
-#### facemesh.detect()
+### faceMesh.detect()
 
 This method asynchronously outputs a single face estimation on an image media when called.
 
 ```javascript
-facemesh.detect(media, ?callback);
+faceMesh.detect(media, ?callback);
 ```
 
 **Parameters:**
 
-- **media**: An HMTL or p5.js image, video, or canvas element to run the estimation on.
+- **media**: An HTML or p5.js image, video, or canvas element to run the estimation on.
+- **callback(results)**: Optional. A callback function to handle the output of the estimation, see output example above.
 
-- **callback(output, error)**: Optional. A callback function to handle the output of the estimation, see output example above.
-
-**Returns:**  
-A promise that resolves to the estimation output.