How to ensure the robot's horizon is in a valid range?

[Roadsong]

Hi guys, I was playing robothor ObjNav tasks recently and it was really interesting! Thanks for your great work!

I have a few questions about the actions of the robot:

1. I checked out the example_submission file and found that the "success" of an "Stop" action can be either be true of false. Why the simple "Stop" action can produce both true or false outcomes? Does the success of the "Stop" action indicate the final success of the entire trajectory? I don't really think so because I also noticed that when "Stop"'s "success" is false, the final result is guaranteed to be false. However, when "Stop"'s "success" is true, still the final results can be false.

2. How to guarantee the horizons of the robot are in the valid range? I am aware that the expected horizon for locobot is {-30, 0, +30}, as stated in the documentation. But, how to ensure when the horizon is 30, another "LookDown" will not performed? Or those actions are deemed invalid and will not be executed? I mean, I realized that the horizon of 60, or -60 are possible, I just want to know how to ensure they are not exist in the submissions, thanks in advance!

[Lucaweihs]

Hi @Roadsong,

1. Good catch! When the agent takes a stop action it is always considered the end of the action regardless of the success/failure associated with the action. In training, we set the stop action to be "successful" when the agent successfully finds the object but this is just a convention and the leaderboard evaluator ignores this value.
2. The easiest way to check this would be run things locally:

import ai2thor.controller import Controller

c = Controller(
    commit_id="bad5bc2b250615cb766ffb45d455c211329af17e", 
    rotateStepDegrees=30,
    visibilityDistance=1.0,
    gridSize=0.25,
    agentType="stochastic",
    continuousMode=True,
    snapToGrid=False,
    agentMode="locobot",
    fieldOfView=63.453048374758716,
    width=640,
    height=480
)
c.reset("FloorPlan_Train1_1")

print(c.last_event.metadata["agent"]["cameraHorizon"], c.last_event.metadata["lastActionSuccess"])
# 0.0 True

c.step("LookDown")

print(c.last_event.metadata["agent"]["cameraHorizon"], c.last_event.metadata["lastActionSuccess"])
# 30 True

c.step("LookDown")

print(c.last_event.metadata["agent"]["cameraHorizon"], c.last_event.metadata["lastActionSuccess"])
# 30 False

for _ in range(3):
  c.step("LookUp")
  print(c.last_event.metadata["agent"]["cameraHorizon"], c.last_event.metadata["lastActionSuccess"])

# 0 True
# -30 True
# -30 False

So looking up and down will not allow the agent to look beyond the -30 to 30 range. Note that this is only true for this particular version of the AI2-THOR build (build bad5bc2b250615cb766ffb45d455c211329af17e) for newer versions of AI2-THOR, I believe the agent has more freedom of view angles so be careful if you decide to train with new versions of AI2-THOR (you might need to ensure the agent doesn't look too far up or down with some code of your own).

[Roadsong]

Thanks for your quick and detailed response, really appreciate it!

[Roadsong]

Hi @Lucaweihs I have another quick question: I noticed that in your initialization of locobot robot, the filed of view is set to 63.453048374758716 rather than 79 degrees.

According to the paper: https://arxiv.org/pdf/2006.13171.pdf section "sensor specification", the filed of view should be set to 79 and the camera's resolution (also the rgb + depth image's resolution used for training) is 480 x 640. Just trying to confirm that for the robothor ObjNav challenge, we should stick with these settings, is that correct?

[Lucaweihs]

Hi @Roadsong,

63.453048374758716 is the vertical FOV, as we're using 4:3 aspect ratio this translates to a horizontal FOV of 79. Here's some code that does this conversion in case you're interested.

Regarding resolution: yes that's right, feel free to use a smaller resolution if you'd prefer though (so long as you keep the same aspect ratio!) as this can be faster to train and use less memory (I believe we render at 300x400 and then resize to 224x224 before passing images to a pretrained cnn).