Food on Fork Integration with Web App

[atharva-kashyap]

Describe this pull request. Link to relevant GitHub issues, if any.

[TODO]

Explain how this pull request was tested, including but not limited to the below checkmarks.

[TODO]

---

Before creating a pull request

• Format React code with npm run format
• [N/A] Format Python code by running python3 -m black . in the top-level of this repository
• Thoroughly test your code's functionality, including unintended uses.
• Fully test the responsiveness of the feature as documented in the Responsiveness Testing Guidelines. If you deviate from those guidelines, document above why you deviated and what you did instead.
• Consider the user flow between states that this feature introduces, consider different situations that might occur for the user, and ensure that there is no way for the user to get stuck in a loop.

Before merging a pull request

• Squash all your commits into one (or Squash and Merge)

[atharva-kashyap]

Note: This is not fully complete. I am hoping to get some feedback towards my approach through this draft. I understand that there aren't testing methods outlined yet. I will be doing that in a later draft. Additionally, I will be adding description in a later draft as well.

Here is my current implementation:

• When the web app reaches the bite done screen, we see two buttons (bite done and re-acquire bite). At this point, the current method is to listen to the /food_on_fork node at a rate of 1 message per second (interval = 1000ms) for approximately 10 messages (~10 seconds). Currently, I am simply checking if the value at the 9th index is less than the probability threshold, then I go ahead and change the state. Note that 10 and 9th index are simply hardcoded to verify the reliability of this approach and will be changed soon.
• My plan is to wait for approximately 1 minute (60 messages) and take the average of the last 10 messages and if the average is below the threshold, then I will call the move over plate method.
• Additionally, currently, we are only able to call one function to move back above the plate. The yellow button indicating reacquire bite is not really something we can use with this approach. Additionally, during the 60 second wait, the user is more than welcome to click one of the buttons to override listening to the FoF node.

Again, I am mainly looking to get thoughts on what you think of an approach like this. In addition to this, I am thinking the trigger for being able to subscribe is currently useEffect() of rendering the bite done page. But, I feel rather than that, the trigger should be the end of previous action of Move to Mouth instead. Feel free to express your thoughts here.

Thanks!

[amalnanavati]

See in-line comments. In addition, below are comments on your overall algorithm.

1. Waiting 1 minute to detect whether food is on the fork is WAY too long, so is 10 seconds. If the automated food-on-fork detection is not faster than it would take users to click the button themselves, they will just do that. Your node runs at 30 Hz; why do we need to wait that long to decide whether there is food on the fork or not?
2. Why did you add a throttle rate to the subscription to the ROS topic? Every time we get a new reading from the food on fork node, we should record that. By taking one message every second , you are dropping 29/30 of all the messages...
3. I assume that the reason you want to wait some time period and then take the average is to avoid noise. But remember what I said about starting with simple solutions first and then only moving to more complex solutions if they are motivated?
   1. You should start with a system that gets every message published by the node (i.e., no throttle), and if that message is lower than the threshold, immediately move above plate.
   2. And then test that out to understand its strengths and weaknesses.
   3. Only if noise is truly an issue should we move to an approach that averages readings over a window.
   4. Further, even as we move to move complex solutions, we have to be very data-driven — why do you need to wait 60 seconds to average the readings? Why not wait 0.3 secs? Every design decision you make when designing an algorithm must be justified by the data/domain. (e.g., measure the nosie you get, see how frequently you see that noise, and use that interval to determine an amount of time to average over)
4. What do you mean “we are only able to call one function to move back above the plate?” Why can’t re-acquire bite and your subscriber callback both call the same function? That way, if the user decides to move the robot above the plate before your automated system does (or your automated system is not in the threshold where it is making a decision), the user can click the button.

[amalnanavati]

Describe how your node interprets the probabilities.

[amalnanavati]

The return function gets called before useEffect gets called again. Why should we wait until them before checking whether the probability is in the range? This check should be in the subscriber's callback function.

[amalnanavati]

Please make this a separate function (use the useCallback hook), because, as I suggested below, you will be adding more lines to this function. Imo, multi-line functions are more readable when defined as separate functions, not in-line.

[amalnanavati]

Good generalization of this component!

[amalnanavati]

Please read the documentation for useEffect, specifically around the dependencies list. As I understand it, without dependencies, useEffect will be called every time a re-render happens. So every time a re-render happens you are unsubscribing and re-subscribing from the topic, which is extremely wasteful.

You should instead only subscribe to the topic once when this component starts. Because you pass a callback function to the subscriber, that callback function will be called every time a new message is received---you don't need to re-enter to useEffect call for this. And then, the return function from useEffect should unsubscribe from the topic, but that will only be called once when the component unmounts.

To achieve the above functionality of useEffect being called only once, you need to either pass in an empty list as dependencies, or all dependencies must also never change during the lifetime of this component. Reading into the documentation of each hook will help you understand when each one gets called, which will help you understand which variables are and aren't changing when.

[atharva-kashyap]

@amalnanavati A followup to your bullet point 4: What I mean by only function is the following:
Suppose there is message that the probability is 0.2. Then in that case, our prediction is that there is no FOF. So, it should automatically move above the plate (which would be a call to moveAbovePlate() function). But, as far as I understand (do correct me if I am wrong), the Take Another Bite is pretty much going to move the arm to the resting position and the fork will still have food on it. It is just so that the bite can be taken in multiple stages while the user is perhaps having a conversation. So, this functionality would not happen automatically. That would be something that the user needs to decide. In any case, FOF detector should predict FOF if there is another piece on the fork as well.
So my thought with that point was that the "take another bite" functionality will be handled by the user themselves and will not be an automatic call, if that makes sense.

[amalnanavati]

You are correct that "move to resting position" can only be initiated by the user, whereas "move above plate" can be initiated by the user or by your algorithm.

[amalnanavati]

BTW I'm confused by which page you are focusing on. In your comment, you wrote "When the web app reaches the bite done screen, we see two buttons (bite done and re-acquire bite)." On the "Bite Done" page, the buttons are "Bite Done" and "Take Another Bite." Did you mean "take another bite" instead of "re-acquire bite"? ("re-acquire bite" appears on the BiteAcquisitionCheck page)

[atharva-kashyap]

@amalnanavati yikes, yes! I meant take another bite and not reacquire bite.

[amalnanavati]

Closed by #127