Multiplayer over the network #32

AlanAnxinChen · 2024-05-14T04:27:46Z

Overview

Implemented client and server functionality to enable cooperative multiplayer. Players are automatically matchmaked into games when enough player connect. Upon the successful conclusion of a game, a replay of the game is saved on the server. Command line arguments configure game and connection settings.

Changes Made

Implemented client and server roles: clients connect to servers
Robotouille game can be played over the internet
Recordings of successful games are saved by the server
Recordings can be played back by the application or rendered into a video
Multiagent environments are supported via multiplayer
Basic matchmaking system assigns multiple players to a game

Test Coverage

Tested manually. Use command line arguments to set up the server and connect as a client.

Next Steps

Making the netcode more resilient to network and computer failures.

Related PRs or Issues

Multiagent

Screenshots

Screenshot of different instances in the same game

…ons to env files; empty fields in problem files not not required

…or filling water

chalo2000

For context, when we tested this code there was a state bug across sessions where previously done actions were unable to be completed. I poked around with the networking and looked through the code and do not think this is a networking issue - rather an issue with the environment itself. We can ignore this for now - I will have @lsuyean look into a fix for this. In the meantime, the code needs to be reorganized. Can we have the server and client functionality split into separate files and have some documentation for the networking options, preferably in a networking README.md. Rather than intertangling networking with Robotouille, it would be good to have an option in main.py to run the default simulator which a researcher trying to run an RL agent would prefer while having an option for the networking server-client for those wanting to run user studies.

chalo2000 · 2024-06-11T20:38:52Z

robotouille/robotouille_simulator.py

+            keydown_events = list(filter(lambda e: e.type == pygame.KEYDOWN, pygame_events))
+            action, args = create_action_from_control(env, shared_state["obs"], player, mousedown_events + keydown_events, renderer)
+
+            online = not (pygame.key.get_mods() & pygame.KMOD_CAPS)


what is this for?

chalo2000 · 2024-06-11T21:17:59Z

robotouille/robotouille_simulator.py

+
+            while not done:
+                # Wait for messages from any client
+                # lol this is causing a memory leak


make this is a TODO to fix 😃 format should be # TODO(username): ...

chalo2000 · 2024-06-11T21:18:10Z

robotouille/robotouille_simulator.py

+        async for message in websocket:
+            await q.put(message)
+
+    #start_server = websockets.serve(handle_connection, "0.0.0.0", 8765)


clear comments

chalo2000 · 2024-06-11T21:18:14Z

robotouille/robotouille_simulator.py

-            continue
-        obs, reward, done, info = env.step(actions, interactive=interactive)
+    async def handle_connection(websocket):
+        # simple lobby code; will break if anyone disconnects, probably has race conditions lol, etc.


TODO to fix

chalo2000

We need one more iteration on this PR before we can merge in; mainly we need to ensure anything networking related is in the networking directory (see comments, especially on the 'biggest feedback').

Other than that, left some comments for code clarity and improvements to make the README.md better for others to quickly play with networking.

Finally, please add docstrings to all modified functions explaining the purpose of the function and a description of all parameters. Be consistent with the formatting of other functions in the repository.

chalo2000 · 2024-10-08T03:48:48Z

networking/README.md

+3. `single`
+4. `replay`
+5. `render`
+6. `simulator`


nit: call simulator local and move it to 1 so explanations are in order and the first explanation is how to run Robotouille without networking

chalo2000 · 2024-10-08T03:50:32Z

networking/server.py

+
+            while not done:
+                # Wait for messages from any client
+                # TODO(aac77): make more robust


make this TODO more specific - what does 'more robust' mean? create an actionable github issue and refer to it in this PR so we know to address this in the future.

chalo2000 · 2024-10-08T03:50:41Z

networking/server.py

+                pickle.dump(recording, f)
+
+    async def handle_connection(websocket):
+        # TODO(aac77): make more robust


chalo2000 · 2024-10-08T03:52:12Z

robotouille/robotouille_simulator.py


-def simulator(environment_name: str, seed: int=42, noisy_randomization: bool=False):
+async def single_player(environment_name: str, seed: int=42, noisy_randomization: bool=False):


can we move the networking related code to a utils directory under the networking directory (functions single_player, replay, render)

chalo2000 · 2024-10-08T03:54:49Z

robotouille/robotouille_simulator.py


+def simulator(environment_name: str, seed: int=42, role: str="simulator", display_server: bool=False, host: str="ws://localhost:8765", recording: str="", noisy_randomization: bool=False):
+    if recording != "" and role != "replay" and role != "render":


add a comment to this if statement since it takes too long to reason about why we're defaulting to replay if a recording is specified for a gameplaying role

chalo2000 · 2024-10-08T04:01:13Z

networking/README.md

@@ -0,0 +1,40 @@
+# Robotouille Networking README


I'd add some common usecase commands so others can copy-paste to get a desired result like

server command with option to display screen

client command with a specific host to connect to

how to replay game data onto the simulator

how to render game data into a video

chalo2000 · 2024-10-08T04:04:44Z

main.py

@@ -4,7 +4,11 @@
 parser = argparse.ArgumentParser()
 parser.add_argument("--environment_name", help="The name of the environment to create.", default="original")
 parser.add_argument("--seed", help="The seed to use for the environment.", default=None)
+parser.add_argument("--role", help="\"simulator\" for vanilla simulator, \"client\" if client, \"server\" if server, \"single\" if single-player, \"replay\" if replaying, \"render\" if rendering video", default="simulator")


separate the Robotouille arguments and the networking arguments with a newline and have a comment above the networking parameters explaining where to find the networking README.md

chalo2000 · 2024-10-08T04:11:32Z

robotouille/robotouille_simulator.py


+def simulator(environment_name: str, seed: int=42, role: str="simulator", display_server: bool=False, host: str="ws://localhost:8765", recording: str="", noisy_randomization: bool=False):


biggest feedback is we want to move as much of the networking related code to the networking directory. I'd like to keep this file as clean as possible to the original so someone that looks at this function can easily reason how Robotouille works without needing to have a coupled understanding on the networking. I suggest moving these if statements to a function call within the networking directory and calling simulate immediately if the role is appropriate. Here is some psuedocode for what I have in mind to

def simulator(...): """...docstring...""" if role != "local": # Run networked Robotouille networked_simulate(...) simulate(...)

roles replay and render also make sense under networking since they're utilities for replaying or rendering data generated by the networking class.

chalo2000

LGTM - left minor comments to resolve before squashing and merging.

chalo2000 · 2024-10-25T19:47:38Z

main.py

@@ -2,9 +2,17 @@
 import argparse

 parser = argparse.ArgumentParser()
+
+# Robotuille game parameters


Robotouille

chalo2000 · 2024-10-25T19:50:18Z

networking/README.md

+This mode runs Robotouille without any networking overhead.
+
+E.g.
+```python main.py --environment_name original```


try adding sh so it looks like a shell command like below (with grave accents ` replacing apostrophe ')
'''sh
python main.py --environment_name original
'''

chalo2000 · 2024-10-25T19:52:26Z

networking/server.py

+
+            while not done:
+                # Wait for messages from any client
+                # TODO(aac77): 


make a Git issue and refer to it in the PR as well in the code by the number, e.g. #32

AlanAnxinChen and others added 30 commits March 9, 2024 16:11

Lockstep synchronization server-client prototype

5202bba

rough draft of creation effect

300b934

implemented pickup and place container

c6e6227

implemented fill water

c25ce36

Made boil water: bug when creating new objects, no id assigned

7201b5d

boil soup bug fixed

4fa2078

finish implementing all actions

34058a0

implemented cook soup test

c79971f

added conditional delayed effect for cook, boil, and fry; added versi…

82a9e36

…ons to env files; empty fields in problem files not not required

Made server also render game (helps debug)

5ff34f3

Client runs responsive local environment with rollback support

34c3714

updated based on comments

0eba451

Server now records gameplay

5d3cea2

Recordings can now be replayed

704df1f

Added recording video rendering

bc539fe

edited goal such that it can include objects not in initial state

3045608

Added docker support

9c5710a

merge main

9f1c202

pddl functions uncommented

29863d2

fixed bug for station location predicates and no conditional effect f…

d7f92e3

…or filling water

Implemented multi-agent

d81deea

addressed comments, fixed bug in creation effect

2611715

fixed tests

9f3eb7e

merge goal

4c4634f

made changes based on comments

e3882b6

merge

e9e8bbd

Removed action args

05c6ea1

server and client loop support multiagent

0b3eb1c

replay and render support multiagent

18d988a

Added single player mode (only works for 1 player environments)

9bd7bfe

AlanAnxinChen added 2 commits May 7, 2024 15:39

docker build

b95ede9

merge

ec879a3

AlanAnxinChen requested a review from chalo2000 May 14, 2024 04:27

chalo2000 mentioned this pull request Jun 11, 2024

Implemented custom tileset parsing and rendering #23

Merged

chalo2000 requested changes Jun 11, 2024

View reviewed changes

lsuyean and others added 3 commits June 25, 2024 15:59

fixed state bug

da460e3

Adjust comments

58b9189

Set default behavior to run local loop

1110e7b

chalo2000 force-pushed the main branch from 0f9a050 to 8fcc19b Compare September 20, 2024 21:24

AlanAnxinChen added 2 commits September 28, 2024 13:05

Factored out server and client code

d717eb8

Add README

34faf0c

chalo2000 requested changes Oct 8, 2024

View reviewed changes

AlanAnxinChen added 4 commits October 18, 2024 16:48

Add example cmds to README

e66a556

Reorganize main

1fef98e

Refactor all other modes to networking folder

22c6ccf

Elaborated TODOs

aa26202

chalo2000 approved these changes Oct 25, 2024

View reviewed changes

AlanAnxinChen added 5 commits October 26, 2024 12:17

Documentation fixes

0c00de9

Add issues to TODO

e707039

merge

13e7f4c

Server self-updates

d78c14f

Players exchanged between server and client

dc1c7d9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Multiplayer over the network #32

Multiplayer over the network #32

AlanAnxinChen commented May 14, 2024

chalo2000 left a comment

chalo2000 Jun 11, 2024

chalo2000 Jun 11, 2024

chalo2000 Jun 11, 2024

chalo2000 Jun 11, 2024

chalo2000 left a comment •

edited

Loading

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 Oct 8, 2024

chalo2000 left a comment

chalo2000 Oct 25, 2024

chalo2000 Oct 25, 2024

chalo2000 Oct 25, 2024


		def simulator(environment_name: str, seed: int=42, noisy_randomization: bool=False):
		async def single_player(environment_name: str, seed: int=42, noisy_randomization: bool=False):


		def simulator(environment_name: str, seed: int=42, role: str="simulator", display_server: bool=False, host: str="ws://localhost:8765", recording: str="", noisy_randomization: bool=False):
		if recording != "" and role != "replay" and role != "render":

Multiplayer over the network #32

Are you sure you want to change the base?

Multiplayer over the network #32

Conversation

AlanAnxinChen commented May 14, 2024

Overview

Changes Made

Test Coverage

Next Steps

Related PRs or Issues

Screenshots

chalo2000 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chalo2000 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chalo2000 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chalo2000 left a comment •

edited

Loading