Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VR Side: Unity/Server Connections #142

Open
wants to merge 109 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
109 commits
Select commit Hold shift + click to select a range
1fa4c8e
Initial commit, set up ServerConnections.cs
jasmine-bae Sep 24, 2020
8f782f9
setup drone_v2 skeleton
Huw-man Sep 30, 2020
2ca2450
basic structure setup
PeruDayani Oct 3, 2020
3da75b4
Merge branch 'master' of https://github.com/immersive-command-system/…
jasmine-bae Oct 3, 2020
3a7afb0
Upgraded worldproperties and dronebuttons to drone_v2
jasmine-bae Oct 3, 2020
582a5d2
server connection work
PeruDayani Oct 3, 2020
c093feb
fixed merge conflicts
PeruDayani Oct 3, 2020
f23fdca
basic server connections
Huw-man Oct 3, 2020
86f56b1
Initial commit, set up ServerConnections.cs
jasmine-bae Sep 24, 2020
629731d
setup drone_v2 skeleton
Huw-man Sep 30, 2020
4969c01
basic structure setup
PeruDayani Oct 3, 2020
abab529
Upgraded worldproperties and dronebuttons to drone_v2
jasmine-bae Oct 3, 2020
548df92
broke unity but added dictionary
jasmine-bae Oct 8, 2020
3196fe9
changed to ordered dict
jasmine-bae Oct 8, 2020
a1e64e7
Merge branch 'vrSide' into fixingit
PeruDayani Oct 10, 2020
bdb802f
Merge pull request #153 from immersive-command-system/fixingit
PeruDayani Oct 10, 2020
205ea20
fixed remaining errors
PeruDayani Oct 10, 2020
3653b09
Added new dictionary in addition to queue, compatability with Drone_v2
jasmine-bae Oct 10, 2020
f6f0fb9
server add drone method
PeruDayani Oct 10, 2020
105319d
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
PeruDayani Oct 10, 2020
ac82526
Merged with Jasmine's work
PeruDayani Oct 10, 2020
9e56b8d
new tasks decided, worksession successfull
PeruDayani Oct 10, 2020
ccb3647
updated some more upgrades
jasmine-bae Oct 10, 2020
1851dd1
starting worksession oct 14
jasmine-bae Oct 15, 2020
e76b7bf
Fixed Droneproperties.cs, debugged Waypoint.cs & WaypointProperties.c…
akhilvemuri Oct 15, 2020
33effcf
Casting Drone to DroneV2 using as operator
akhilvemuri Oct 17, 2020
4f24a95
need to fixup serverconnections
jasmine-bae Oct 17, 2020
ccd3023
fixed minor errors
PeruDayani Oct 24, 2020
16f6e7b
broken testing features
PeruDayani Oct 24, 2020
e7fa009
Added some custom messages for allDronesAvailable
jasmine-bae Oct 24, 2020
dbc397f
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
jasmine-bae Oct 24, 2020
9aa3993
Custom messages added
akhilvemuri Oct 24, 2020
0f0afa9
fixing errors
PeruDayani Oct 24, 2020
8e78b11
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
PeruDayani Oct 24, 2020
5b65658
fixed errors and completed testing
PeruDayani Oct 24, 2020
edd210d
Returns -1 for _id in QueryTopicsMsg.cs
akhilvemuri Oct 24, 2020
7ceb741
set up skeleton for sensors in serverconnection.cs
jasmine-bae Nov 19, 2020
f656c06
doing lamp sensor
jasmine-bae Nov 19, 2020
3c2bec9
fixed some changes to serverconnections.cs, still doesnt display dron…
jasmine-bae Nov 21, 2020
49fafa1
Added template for SensorV2 scripts
akhilvemuri Dec 3, 2020
36d3db4
Added template for ServerV2 scripts
akhilvemuri Dec 3, 2020
b9ddbb5
Added template for SensorV2
akhilvemuri Dec 3, 2020
4331e62
changed ip address to my local
Dec 3, 2020
e56717e
Fixed all custom mesages, adding functionality to drone-server topics
Jan 16, 2021
3650021
still working on ServerConnections/drone services
Jan 17, 2021
95ad6d4
finished drone operations for serverconnections.cs, test with server …
jasmine-bae Jan 23, 2021
46d1bf7
fixed up serverconnections.cs, fixed bug with rosBridgeLib Render(), …
jasmine-bae Feb 5, 2021
a6165a4
debugged submodule
jasmine-bae Feb 6, 2021
dad0473
pushed changes to try and fix UpdateWaypoints and ControlDrone
jasmine-bae Feb 6, 2021
a04e64a
added first 3d recon mesh in Assets/Models
orrblue Feb 10, 2021
c438565
Fixed up DroneMenu (ready for testing)
jasmine-bae Feb 13, 2021
0be1dfe
starting switching to ROS #
jasmine-bae Feb 14, 2021
bb886ad
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
jasmine-bae Feb 14, 2021
eed0f24
switched to 2019 and integrated ROS Sharp
akhilvemuri Feb 21, 2021
a567185
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
jasmine-bae Feb 21, 2021
76e367a
changed to 2019 v2
akhilvemuri Feb 21, 2021
2515753
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
jasmine-bae Feb 21, 2021
e04f9bc
removed unused Assets
akhilvemuri Feb 21, 2021
1eb5f6c
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
jasmine-bae Feb 21, 2021
6fba9c7
saving mesh and main scene settings
orrblue Feb 27, 2021
9f8faa2
some changes to ros service calls, will test next week
jasmine-bae Feb 27, 2021
f860504
added UI functionality for setting drone speed
jasmine-bae Feb 28, 2021
e4d828c
shader for zed mesh
WeiyanZhu Mar 1, 2021
6436d70
add zed mesh sensor type
WeiyanZhu Mar 1, 2021
92b8c29
change sensor type name
WeiyanZhu Mar 1, 2021
4b7d5af
make shader in mesh visualizer changeable
WeiyanZhu Mar 1, 2021
fb34cd4
readded and changed .dll files
akhilvemuri Mar 2, 2021
8ed504d
Merge pull request #168 from WeiyanZhu/meshChangeMaterial
orrblue Mar 3, 2021
d58d55b
Merge branch '3d-recon' of https://github.com/immersive-command-syste…
orrblue Mar 3, 2021
0e76182
removed rossharp temporarily
akhilvemuri Mar 6, 2021
8b964e4
fixed .gitmodules ?
akhilvemuri Mar 6, 2021
9b310fc
fixed issues with submodules
jasmine-bae Mar 7, 2021
1272669
switching to ROS#, visual studio solution not working on my computer
jasmine-bae Mar 7, 2021
964c5e2
merge conflict resolved
orrblue Mar 8, 2021
bd7dc09
visualize 2 meshes
orrblue Mar 11, 2021
767efe0
added custom messages rossharp
akhilvemuri Mar 20, 2021
2b303d6
working on transferring to ros#
jasmine-bae Mar 21, 2021
4de5c0f
alldronesavailable ros# subscriber & publisher
akhilvemuri Mar 22, 2021
3d63632
alldronesavailable ros#
akhilvemuri Mar 22, 2021
b118f40
made all drones available, drone gameobject not made in unity
jasmine-bae Apr 3, 2021
34208dc
fixed infinite recursion, maybe
jasmine-bae Apr 10, 2021
b277b4e
adding uploadmission actions, fixed serverconnection2 bug
jasmine-bae Apr 10, 2021
460bf38
controlDrone action (editor version) complete
akhilvemuri Apr 11, 2021
bc301e7
sorted merge conflict
akhilvemuri Apr 11, 2021
edc9bdf
everything fixed maybe
jasmine-bae Apr 17, 2021
d198b0f
set speed action client added
akhilvemuri Apr 17, 2021
d613159
controldrone works, upload mission results aren't recieved
jasmine-bae Apr 17, 2021
2a58461
fixed set speed kinda:
akhilvemuri Apr 17, 2021
460c8a2
Merge branch 'vrSide' of https://github.com/immersive-command-system/…
akhilvemuri Apr 17, 2021
534f22d
combined set speed stuff
akhilvemuri Apr 17, 2021
ed41506
restored accidentally deleted folder
akhilvemuri Apr 17, 2021
3b2b7ab
quick name changes
akhilvemuri May 1, 2021
753f90d
added comments to controlDrone, setSpeed, uploadMission action client…
akhilvemuri May 1, 2021
2ab81d1
added more comments + Drone_v2 & ServerConnections2
akhilvemuri May 1, 2021
8c843bf
merged from master
akhilvemuri May 1, 2021
8aa9dbf
zed2marker transform matrix sets mesh location, still incorrectly
orrblue May 6, 2021
1365bed
updated rosbridge lib
orrblue May 6, 2021
f5356b6
switching to 3d recon rosbridgelib
orrblue May 7, 2021
3043592
End of day May 6th mapping flight
orrblue May 7, 2021
453e741
add RFS B100 hardcoded WPs
orrblue Jul 19, 2021
eafe209
add safe hardcoded WPs for mapping B100 at RFS
orrblue Aug 4, 2021
0d66479
transform mesh with matrix (still using hardcoded matrix)
WeiyanZhu Sep 22, 2021
d01d7c5
receive Matrix4x4 from drone
WeiyanZhu Sep 25, 2021
c5dbcb8
Merge branch '3d-recon-wayne' into vrSide
WeiyanZhu Sep 25, 2021
b603668
Merge pull request #1 from immersive-command-system/vrSide
WeiyanZhu Sep 25, 2021
cf6cb40
Merge pull request #171 from WeiyanZhu/3d-recon-wayne
WeiyanZhu Sep 26, 2021
ff6b9e6
Merge pull request #172 from immersive-command-system/wayne
WeiyanZhu Sep 26, 2021
7779691
use Main.unity that was in VRSide
WeiyanZhu Sep 26, 2021
e729c0d
Merge pull request #173 from immersive-command-system/wayne
WeiyanZhu Sep 26, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fixed up serverconnections.cs, fixed bug with rosBridgeLib Render(), …
…getting some responses from server
jasmine-bae committed Feb 5, 2021
commit 46d1bf72dcabc0e69a89d5a30ebf86a541cecfb0
111 changes: 107 additions & 4 deletions Assets/Scripts/Drone Scripts/Drone_v2.cs
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@ public class Drone_v2 : Drone
public Drone_v2(Vector3 position, int uniqueID) : base(position, uniqueID) {
Debug.Log("Drone V2 Constructor: " + uniqueID.ToString());
_id = uniqueID.ToString();
WorldProperties.AddDrone(this);
//WorldProperties.AddDrone(this);
}

public void uploadMission()
@@ -67,17 +67,17 @@ public void stopMission()

public void onStartMission()
{

//change waypoint colors?
}

public void onPauseMission()
{

//change btw pause and resume buttons?
}

public void onResumeMission()
{

//change btw pause and resume buttons?
}

public void onLandDrone()
@@ -89,5 +89,108 @@ public void onFlyHome()
{

}

//TODO: this is where unity is able to listen to changes in drone postion/etc, need to adjust it for the server itself

/*public ROSBridgeMsg OnReceiveMessage(string topic, JSONNode raw_msg, ROSBridgeMsg parsed = null)
{
ROSBridgeMsg result = null;
// Writing all code in here for now. May need to move out to separate handler functions when it gets too unwieldy.
switch (topic)
{
case "/dji_sdk/attitude":
QuaternionMsg attitudeMsg = (parsed == null) ? new QuaternionMsg(raw_msg["quaternion"]) : (QuaternionMsg)parsed;
attitude = offset * (new Quaternion(attitudeMsg.GetX(), attitudeMsg.GetY(), attitudeMsg.GetZ(), attitudeMsg.GetW()));
// Update drone transform to new quaternion
this.transform.rotation = attitude;
// this.transform.localRotation = attitude;
if (home_attitude_set == false)
{
home_attitude = attitude;
home_attitude_set = true;
// Localize sensors when both orientation and gps position is set
if (home_position_set)
{
LocalizeSensors();
}
}
result = attitudeMsg;
break;
case "/dji_sdk/battery_state":
battery_state = (parsed == null) ? new BatteryStateMsg(raw_msg) : (BatteryStateMsg)parsed;
result = battery_state;
break;
case "/dji_sdk/flight_status":
//flight_status = (FlightStatus)(new UInt8Msg(raw_msg)).GetData();
break;
case "/dji_sdk/gimbal_angle":
Vector3Msg gimbalAngleMsg = (parsed == null) ? new Vector3Msg(raw_msg["vector"]) : (Vector3Msg)parsed;
gimbal_joint_angles = new Vector3((float)gimbalAngleMsg.GetX(), (float)gimbalAngleMsg.GetY(), (float)gimbalAngleMsg.GetZ());
result = gimbalAngleMsg;
break;
case "/dji_sdk/gps_health":
gps_health = (parsed == null) ? (new UInt8Msg(raw_msg)).GetData() : ((UInt8Msg)parsed).GetData();
break;
case "/dji_sdk/imu":
imu = (parsed == null) ? new IMUMsg(raw_msg) : (IMUMsg)parsed;
result = imu;
break;
case "/dji_sdk/rc":
remote_controller_msg = (parsed == null) ? new JoyMsg(raw_msg) : (JoyMsg)parsed;
result = remote_controller_msg;
break;
case "/dji_sdk/velocity":
Vector3Msg velocityMsg = (parsed == null) ? new Vector3Msg(raw_msg["vector"]) : (Vector3Msg)parsed;
velocity = new Vector3((float)velocityMsg.GetX(), (float)velocityMsg.GetY(), (float)velocityMsg.GetZ());
result = velocityMsg;
break;
case "/dji_sdk/height_above_takeoff":
relative_altitude = (parsed == null) ? (new Float32Msg(raw_msg)).GetData() : ((Float32Msg)parsed).GetData();
break;
case "/dji_sdk/local_position":
PointMsg pointMsg = (parsed == null) ? new PointMsg(raw_msg["point"]) : (PointMsg)parsed;
local_position = new Vector3(pointMsg.GetX(), pointMsg.GetY(), pointMsg.GetZ());
result = pointMsg;
Debug.Log(result);
break;
case "/dji_sdk/gps_position":
case "/dji_sdk/rtk_position":
gps_position = (parsed == null) ? new NavSatFixMsg(raw_msg) : (NavSatFixMsg)parsed;
result = gps_position;
if (gps_position.GetLatitude() == 0.0f && gps_position.GetLongitude() == 0.0f)
{
break;
}
// TODO: Test that setting drone home latitude and longitutde as first message from drone gps position works.
if (home_position_set == false)
{
home_position = gps_position;
home_position_set = true;
}
// TODO: Complete function in World properties.
if (home_position_set)
{
this.transform.localPosition = WorldProperties.ROSCoordToUnityCoord(gps_position);
if (WorldProperties.DJI_SIM)
{
this.transform.localPosition += new Vector3(0, -100.0f, 0);
}
}
break;
default:
Debug.LogError("Topic not implemented: " + topic);
break;
}
return result;
}
*/
}
}
Original file line number Diff line number Diff line change
@@ -4,17 +4,22 @@
using System.Collections;
using SimpleJSON;
using UnityEngine;
using System;

namespace ROSBridgeLib
{

public class AllDronesAvailableMsg : ROSBridgeMsg
{
private DroneMsg[] _drones_available;
private bool _success;

public AllDronesAvailableMsg(JSONNode msg)
{
msg = msg["values"];
JSONArray temp = msg["drones_available"].AsArray;
_success = msg["success"].AsBool;
Debug.Log(_success);
_drones_available = new DroneMsg[temp.Count];

for (int i = 0; i < _drones_available.Length; i++)
@@ -25,10 +30,15 @@ public AllDronesAvailableMsg(JSONNode msg)

public static string getMessageType()
{
return "/server/all_drones_available";
return "/isaacs_server/all_drones_available";

}

public bool getSuccess()
{
return _success;
}

public DroneMsg[] getDronesAvailable()
{
return _drones_available;
15 changes: 15 additions & 0 deletions Assets/Scripts/Server Scripts/CustomMsgs/DroneCommandMsg.cs
Original file line number Diff line number Diff line change
@@ -13,13 +13,22 @@ public class DroneCommandMsg : ROSBridgeMsg
{
int _id;
string _command;
bool _success;

public DroneCommandMsg(JSONNode msg)
{
msg = msg["values"];
_id = int.Parse(msg["id"]);
_command = msg["control_task"].ToString();
_success = msg["success"].AsBool;
}

public DroneCommandMsg(String thing)
{
_id = 0;
_command = "sample";
}

public static string getMessageType()
{
return "/server/drone_command";
@@ -35,6 +44,12 @@ public int getID()
return _id;
}

public bool getSuccess()
{
return _success;
}


public void setCommand(string command)
{
_command = command;
5 changes: 3 additions & 2 deletions Assets/Scripts/Server Scripts/CustomMsgs/UploadMissionMsg.cs
Original file line number Diff line number Diff line change
@@ -17,9 +17,10 @@ public class UploadMissionMsg : ROSBridgeMsg

public UploadMissionMsg(JSONNode msg)
{
_drone_id = int.Parse(msg["id"]);
msg = msg["values"];
_drone_id = int.Parse(msg["id"]);
_success = Convert.ToBoolean(msg["success"]);
_meta_data = msg["meta_data"].ToString();
_meta_data = msg["message"].ToString();
}

public static string getMessageType()
468 changes: 234 additions & 234 deletions Assets/Scripts/Server Scripts/Sensor_v2 Scripts/LampSensor_v2.cs

Large diffs are not rendered by default.

67 changes: 54 additions & 13 deletions Assets/Scripts/Server Scripts/ServerConnections.cs
Original file line number Diff line number Diff line change
@@ -93,13 +93,25 @@ public SensorInformation(JSONNode msg)
void Start()
{
rosServerConnection = new ROSBridgeWebSocketConnection("ws://" + serverIP, serverPort);
Debug.Log("created connection with: " + serverIP + " " + serverPort);
rosServerConnection.Connect();
System.Threading.Thread.Sleep(5000);
Debug.Log("created connection with: " + serverIP + " " + serverPort);
//rosServerConnection.AddSubscriber("/isaacs_server/client_count", this);
GetAllDrones();
GetAllSensors();
}

// Update is called once per frame in Unity
void Update()
{
if (rosServerConnection != null)
{
//Debug.Log("rendered!");
rosServerConnection.Render();
}

}

//Calls the server and returns every drone in the current state
void GetAllDrones()
{
@@ -111,16 +123,23 @@ void GetAllDrones()

public static void GetAllDronesCallback(JSONNode response)
{
Debug.Log(response);
Debug.Log("response gotten");
//Debug.Log(response);

AllDronesAvailableMsg result = new AllDronesAvailableMsg(response);
DroneMsg[] lst = result.getDronesAvailable();
foreach (DroneMsg x in lst)
{
ClientMsg drone = x.getDrone();
DroneInformation droneInfo = new DroneInformation(drone.getName(), drone.getId());
InstantiateDrone(droneInfo, drone);
}
//Debug.Log(result.getSuccess());
//if (result.getSuccess())
//{
DroneMsg[] lst = result.getDronesAvailable();
Debug.Log("DroneList" + lst);
foreach (DroneMsg x in lst)
{
ClientMsg drone = x.getDrone();
//Debug.Log("hits here");
DroneInformation droneInfo = new DroneInformation(drone.getName(), drone.getId());
InstantiateDrone(droneInfo, drone);
}
//}
}

//TODO: void GetAllSensors() : will set service name similarly to GetAllDrones
@@ -162,7 +181,7 @@ public static void InstantiateDrone(DroneInformation droneInformation, ClientMsg

droneGameObject.name = drone_name;
WorldProperties.AddDrone(droneInstance);

Debug.Log("hit here");

// Get DroneMenu and instansiate. (OPTIONAL)
TopicTypesMsg[] lst = msg.getTopics();
@@ -203,12 +222,29 @@ public static void InstantiateDrone(DroneInformation droneInformation, ClientMsg
*/
}

public static void InstantiateDrone(DroneInformation droneInformation)
{
int drone_id = droneInformation.id;
string drone_name = droneInformation.drone_name;
Debug.Log("made drone: " + drone_name);

// Create a new drone
Drone_v2 droneInstance = new Drone_v2(WorldProperties.worldObject.transform.position, drone_id);
Debug.Log("Drone Created: " + droneInstance.gameObjectPointer.name);

DroneProperties droneProperties = droneInstance.droneProperties;
GameObject droneGameObject = droneInstance.gameObjectPointer;

droneGameObject.name = drone_name;
WorldProperties.AddDrone(droneInstance);
}


//calls server to upload a new or changed mission
public static void uploadMission(Drone_v2 drone, string ID, List<Waypoint> waypoints)
{
string service_name = "/isaacs_server/upload_mission ";

string service_name = "/isaacs_server/upload_mission";
Debug.Log("called service");
NavSatFixMsg[] waypointMsgs = new NavSatFixMsg[waypoints.Count];
//change each waypoint to navsatros messages
int count = 0;
@@ -227,6 +263,7 @@ public static void uploadMission(Drone_v2 drone, string ID, List<Waypoint> waypo

public static void uploadMissionCallback(JSONNode response)
{
Debug.Log(response);
UploadMissionMsg result = new UploadMissionMsg(response);
int drone_id = result.getDroneId();
bool success = result.getSuccess();
@@ -255,7 +292,7 @@ public static void controlDrone(Drone_v2 drone, string ID, string command)
{
string service_name = "/isaacs_server/control_drone";
//Debug.LogFormat();
DroneCommandMsg msg = new DroneCommandMsg("[]");
DroneCommandMsg msg = new DroneCommandMsg("");
msg.setCommand(command);
msg.setID(int.Parse(ID));
//hopefully this works!
@@ -271,6 +308,10 @@ public static void controlDroneCallback(JSONNode response)
string command = result.getCommand();
Drone_v2 drone = WorldProperties.GetDroneDict()[drone_id];
//do the command stuff
if (!result.getSuccess())
{
Debug.Log("Wrong drone callback/Failure to complete!");
}
switch (command)
{
case "start_mission":
6 changes: 3 additions & 3 deletions Assets/Scripts/Utility Scripts/Fall2020_Testing.cs
Original file line number Diff line number Diff line change
@@ -40,7 +40,7 @@ public class Fall2020_Testing : MonoBehaviour {


[Header("Drone Variable")]
public Drone_v2 drone;
public Drone_v2 drone = WorldProperties.GetSelectedDrone();

// Update is called once per frame
void Update()
@@ -49,8 +49,8 @@ void Update()
if (Input.GetKeyDown(initTestDrones))
{
// Call server connection create drone function.
ServerConnections.DroneInformation test_drone_1 = new ServerConnections.DroneInformation("jazzy", 0);
ServerConnections.DroneInformation test_drone_2 = new ServerConnections.DroneInformation("banana", 1);
//ServerConnections.DroneInformation test_drone_1 = new ServerConnections.DroneInformation("jazzy", 1);
////ServerConnections.DroneInformation test_drone_2 = new ServerConnections.DroneInformation("banana", 2);

//ServerConnections.InstantiateDrone(test_drone_1);
//ServerConnections.InstantiateDrone(test_drone_2);