[BROKEN]Add support for Apple Silicon (M1) build

Docs/build_mac.md

@@ -0,0 +1,114 @@
+# Mac build
+
+This guide details how to build CARLA from source on MacOS Monterey (Apple Silicon). This work is mostly experimental and serves primarily as a proof-of-concept for those interested in building CARLA 0.9.13 on the Apple Silicon devices. 
+
+---
+## Prerequisites
+
+### System requirements
+
+* __MacOS 12__ This is for building on MacOS Monterey (12). The latest version this was tested on is 12.1 on my M1 Max.
+* __130 GB disk space.__ Carla will take around 31 GB and Unreal Engine will take around 91 GB so have about 130 GB free to account for both of these plus additional minor software installations. 
+* __An adequate GPU.__ The M1 GPU's are very capable, though it is still recommended to get the Pro/Max's since I've found the performance to scale nearly linearly with number of GPU cores. 
+* __Two TCP ports and good internet connection.__ 2000 and 2001 by default. Make sure that these ports are not blocked by firewalls or any other applications. 
+
+..warning::
+    __If you are upgrading from CARLA 0.9.12 to 0.9.13__: you must first upgrade the CARLA fork of the UE4 engine to the latest version. See the [__Unreal Engine__](#unreal-engine) section for details on upgrading UE4
+
+**WARNING**: UE4-carla does not build on MacOS 12 by default! See [this discussion](https://github.com/carla-simulator/carla/discussions/4848) for more details.
+
+
+### Software requirements
+
+CARLA requires many different kinds of software to run. Some are built during the CARLA build process itself, such as *Boost.Python*. Others are binaries that should be installed before starting the build (*cmake*, *clang*, different versions of *Python*, etc.). To install these requirements, run the following commands:
+
+```bash
+brew update && brew upgrade
+
+brew install --build-from-source mono
+
+brew install coreutils ninja wget autoconf automake curl libtool libpng aria2 libiconv
+```
+
+I recommend installing python through conda on mac:
+```bash
+conda create --name carla python=3.8
+conda activate carla
+conda install numpy distro wheel
+pip install setuptools==47.3.1 nose2
+```
+
+(Install xcode through the app store, I used the latest (at time of writing) 12.2.1)
+And then open up xcode and install the `xcode command line developer tools`
+```bash
+xcode-select --install
+sudo xcode-select -s /Applications/Xcode.app
+```
+
+Finally, you'll also need [Rosetta 2](https://support.apple.com/en-us/HT211861) for the translation between x86 (UE4 prerequisite) and arm64 (native Apple Silicon)
+
+---
+
+## Unreal Engine
+
+For now, refer to the Linux guide for how to build the engine, but note the [strange problem](https://github.com/carla-simulator/carla/discussions/4848) with building the Carla fork (not Vanilla 4.26.2) on Apple silicon.
+
+## Build CARLA 
+
+Note: This time, the build procedure is somewhat hacky and fragmanted by nature of using different architectures, but it still works reasonably well. The procedure is as follows:
+1. Build LibCarla in x86
+2. Build the UE4 carla game (also x86)
+3. Build LibCarla (AGAIN) in arm64
+4. Build the PythonAPI in arm64
+5. Add the arm64 Python arm .egg file to your PYTHONPATH
+6. Build the UE4 package in x86 (back to x86)
+
+
+This is because the python scripts work much better in `arm64` but the game itself (which depends on UE4) must be built for x86 and run on rosetta2.
+
+The way I recommend you go about this is as follows:
+```bash
+# git clone and enter mac branch
+./Update.sh # wait a while for the asset fetching to complete
+
+# go to Util/Buildtools/Environment.sh
+# make sure the ARCH (for IS_MAC) is set to "x86_64"
+
+make LibCarla # in x86
+...
+
+make launch # in x86 (uses UE4 build and xcode cmd line)
+...
+
+# go to Util/Buildtools/Environment.sh
+# make sure the ARCH (for IS_MAC) is set to "arm64"
+mv Build Build.x86 # denoting the current (x86) build as such
+
+make LibCarla # now in arm64!
+...
+
+make PythonAPI # now in arm64
+...
+
+make check # unit tests to verify everything works!
+...
+
+mv Build Build.arm64
+
+# now you can switch between (re)building arm64 and x86 by renaming
+# the Build.XYZ to just Build
+# example:
+
+mv Build.x86 Build
+
+make package
+...
+
+```
+
+Also, as an aside, since (almost) all the PythonAPI scripts look for a `'linux-x86_64'` `.egg` file, you'll not be able to find it unless you modify EVERY single file, since this is too much work, you can simply add this to your PYTHONPATH via:
+```bash
+export PYTHONPATH="${PYTHONPATH}:/PATH/TO/CARLA/PythonAPI/carla/dist/carla-0.9.13-py3.8-macosx-11.0-arm64.egg"
+# I simply put this in my .zshrc file
+```
+Then you should be able to quickly `import carla` without hassle!

LibCarla/source/carla/Buffer.h

@@ -71,7 +71,7 @@ namespace carla {
         _data(std::make_unique<value_type[]>(size)) {}
 
     /// @copydoc Buffer(size_type)
-    explicit Buffer(uint64_t size)
+    explicit Buffer(size_t size)
       : Buffer([size]() {
           if (size > max_size()) {
             throw_exception(std::invalid_argument("message size too big"));
@@ -319,7 +319,7 @@ namespace carla {
     template <typename T>
     typename std::enable_if<boost::asio::is_const_buffer_sequence<T>::value>::type
     copy_from(size_type offset, const T &source) {
-      reset(boost::asio::buffer_size(source) + offset);
+      reset(static_cast<size_type>(boost::asio::buffer_size(source)) + offset);
       DEBUG_ASSERT(boost::asio::buffer_size(source) == size() - offset);
       DEBUG_ONLY(auto bytes_copied = )
       boost::asio::buffer_copy(buffer() + offset, source);

LibCarla/source/carla/MsgPack.h

@@ -20,7 +20,8 @@ namespace carla {
       namespace mp = ::clmdep_msgpack;
       mp::sbuffer sbuf;
       mp::pack(sbuf, obj);
-      return Buffer(reinterpret_cast<const unsigned char *>(sbuf.data()), sbuf.size());
+      return Buffer(reinterpret_cast<const unsigned char *>(sbuf.data()), 
+                    static_cast<Buffer::size_type>(sbuf.size()));
     }
 
     template <typename T>

LibCarla/source/carla/MsgPackAdaptors.h

@@ -115,7 +115,7 @@ namespace adaptor {
       v = o.via.array.ptr[1].as<T>();
     }
 
-    template <uint64_t... Is>
+    template <size_t... Is>
     static void copy_to_variant(
         const uint64_t index,
         const clmdep_msgpack::object &o,

LibCarla/source/carla/road/element/Waypoint.cpp

@@ -13,12 +13,12 @@ namespace std {
   using WaypointHash = hash<carla::road::element::Waypoint>;
 
   WaypointHash::result_type WaypointHash::operator()(const argument_type &waypoint) const {
-    WaypointHash::result_type seed = 0u;
+    std::size_t seed = 0u;
     boost::hash_combine(seed, waypoint.road_id);
     boost::hash_combine(seed, waypoint.section_id);
     boost::hash_combine(seed, waypoint.lane_id);
     boost::hash_combine(seed, static_cast<float>(std::floor(waypoint.s * 200.0)));
-    return seed;
+    return static_cast<WaypointHash::result_type>(seed);
   }
 
 } // namespace std

LibCarla/source/carla/sensor/s11n/DVSEventArraySerializer.h

@@ -55,7 +55,7 @@ namespace s11n {
     };
 
     /// Reset the output buffer
-    output.reset(sizeof(DVSHeader) + (events.size() * sizeof(data::DVSEvent)));
+    output.reset(static_cast<Buffer::size_type>(sizeof(DVSHeader) + (events.size() * sizeof(data::DVSEvent))));
 
     /// Pointer to data in buffer
     unsigned char *it = output.data();

LibCarla/source/test/common/test_buffer.cpp

@@ -114,9 +114,9 @@ TEST(buffer, memcpy) {
 
 #ifndef LIBCARLA_NO_EXCEPTIONS
 TEST(buffer, message_too_big) {
-  ASSERT_THROW(Buffer(4294967296ul), std::invalid_argument);
+  ASSERT_THROW(Buffer(static_cast<size_t>(4294967296ul)), std::invalid_argument);
   Buffer buf;
-  ASSERT_THROW(buf.reset(4294967296ul), std::invalid_argument);
+  ASSERT_THROW(buf.reset(static_cast<uint64_t>(4294967296ul)), std::invalid_argument);
 }
 #endif // LIBCARLA_NO_EXCEPTIONS
 

LibCarla/source/test/common/test_streaming.cpp

@@ -55,12 +55,12 @@ TEST(streaming, low_level_sending_strings) {
 
   io_context_running io;
 
-  Server<tcp::Server> srv(io.service, TESTING_PORT);
+  carla::streaming::low_level::Server<tcp::Server> srv(io.service, TESTING_PORT);
   srv.SetTimeout(1s);
 
   auto stream = srv.MakeStream();
 
-  Client<tcp::Client> c;
+  carla::streaming::low_level::Client<tcp::Client> c;
   c.Subscribe(io.service, stream.token(), [&](auto message) {
     ++message_count;
     ASSERT_EQ(message.size(), message_text.size());
@@ -90,10 +90,10 @@ TEST(streaming, low_level_unsubscribing) {
 
   io_context_running io;
 
-  Server<tcp::Server> srv(io.service, TESTING_PORT);
+  carla::streaming::low_level::Server<tcp::Server> srv(io.service, TESTING_PORT);
   srv.SetTimeout(1s);
 
-  Client<tcp::Client> c;
+  carla::streaming::low_level::Client<tcp::Client> c;
   for (auto n = 0u; n < 10u; ++n) {
     auto stream = srv.MakeStream();
     std::atomic_size_t message_count{0u};

Makefile

@@ -2,5 +2,5 @@ include Util/BuildTools/Vars.mk
 ifeq ($(OS),Windows_NT)
 include Util/BuildTools/Windows.mk
 else
-include Util/BuildTools/Linux.mk
+include Util/BuildTools/Unix.mk
 endif

PythonAPI/carla/setup.py

@@ -34,8 +34,8 @@ def walk(folder, file_filter='*'):
     if os.name == "posix":
         import distro
 
-        linux_distro = distro.linux_distribution()[0]
-        if linux_distro.lower() in ["ubuntu", "debian", "deepin"]:
+        if distro.id().lower() in ["ubuntu", "debian", "deepin", "darwin"]:
+            is_mac = bool(distro.id().lower() == "darwin")
             pwd = os.path.dirname(os.path.realpath(__file__))
             pylib = "libboost_python%d%d.a" % (sys.version_info.major,
                                                sys.version_info.minor)
@@ -63,6 +63,9 @@ def walk(folder, file_filter='*'):
                 '-Wconversion', '-Wfloat-overflow-conversion',
                 '-DBOOST_ERROR_CODE_HEADER_ONLY', '-DLIBCARLA_WITH_PYTHON_SUPPORT'
             ]
+            if is_mac:
+                extra_compile_args += ['-stdlib=libc++'] # compile same as boost
+                # see: https://stackoverflow.com/questions/35006614/what-does-symbol-not-found-expected-in-flat-namespace-actually-mean
             if is_rss_variant_enabled():
                 extra_compile_args += ['-DLIBCARLA_RSS_ENABLED']
                 extra_compile_args += ['-DLIBCARLA_PYTHON_MAJOR_' +  str(sys.version_info.major)]
@@ -91,7 +94,10 @@ def walk(folder, file_filter='*'):
                 extra_link_args += ['-ljpeg', '-ltiff']
                 extra_compile_args += ['-DLIBCARLA_IMAGE_WITH_PNG_SUPPORT=false']
             else:
-                extra_link_args += ['-lpng', '-ljpeg', '-ltiff']
+                if is_mac:
+                    extra_link_args += ['-lpng'] # TODO: find a suitable MacOS replacement for libjpeg and libtiff
+                else:
+                    extra_link_args += ['-lpng', '-ljpeg', '-ltiff']
                 extra_compile_args += ['-DLIBCARLA_IMAGE_WITH_PNG_SUPPORT=true']
             # @todo Why would we need this?
             # include_dirs += ['/usr/lib/gcc/x86_64-linux-gnu/7/include']

Unreal/CarlaUE4/Config/DefaultEngine.ini

@@ -57,6 +57,27 @@ OcclusionPlugin=
 +TargetedRHIs=SF_VULKAN_SM5
 +TargetedRHIs=GLSL_430
 
+[/Script/IOSRuntimeSettings.IOSRuntimeSettings]
+bSupportsPortraitOrientation=False
+bSupportsUpsideDownOrientation=False
+bSupportsLandscapeLeftOrientation=True
+PreferredLandscapeOrientation=LandscapeLeft
+
+[/Script/MacTargetPlatform.MacTargetSettings]
+-TargetedRHIs=SF_METAL_SM5
++TargetedRHIs=SF_METAL_SM5
+MaxShaderLanguageVersion=3
+UseFastIntrinsics=False
+EnableMathOptimisations=True
+AudioSampleRate=0
+AudioCallbackBufferFrameSize=0
+AudioNumBuffersToEnqueue=0
+AudioMaxChannels=0
+AudioNumSourceWorkers=0
+SpatializationPlugin=
+ReverbPlugin=
+OcclusionPlugin=
+
 [/Script/Engine.PhysicsSettings]
 DefaultGravityZ=-980.000000
 DefaultTerminalVelocity=4000.000000
@@ -110,3 +131,4 @@ DefaultGraphicsRHI=DefaultGraphicsRHI_DX12
 +DefaultChannelResponses=(Channel=ECC_GameTraceChannel3,DefaultResponse=ECR_Overlap,bTraceType=True,bStaticObject=False,Name="OverlapChannel")
 +EditProfiles=(Name="BlockAll",CustomResponses=((Channel="SensorObject"),(Channel="SensorTrace")))
 +EditProfiles=(Name="OverlapAll",CustomResponses=((Channel="SensorObject",Response=ECR_Overlap),(Channel="SensorTrace",Response=ECR_Overlap)))
+

Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/CarlaActor.h

@@ -166,13 +166,13 @@ class FCarlaActor
   template<typename T>
   T* GetActorData()
   {
-    return dynamic_cast<T*>(ActorData.Get());
+    return static_cast<T*>(ActorData.Get());
   }
 
   template<typename T>
   const T* GetActorData() const
   {
-    return dynamic_cast<T*>(ActorData.Get());
+    return static_cast<T*>(ActorData.Get());
   }
 
   // Actor function interface ----------------------

Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Carla.Build.cs

@@ -114,6 +114,12 @@ public Carla(ReadOnlyTargetRules Target) : base(Target)
     AddCarlaServerDependency(Target);
   }
 
+  private bool IsMac(ReadOnlyTargetRules Target)
+  {
+    return (Target.Platform == UnrealTargetPlatform.Mac);
+  }
+
+
   private bool UseDebugLibs(ReadOnlyTargetRules Target)
   {
     if (IsWindows(Target))
@@ -231,6 +237,7 @@ private void AddCarlaServerDependency(ReadOnlyTargetRules Target)
     PublicDefinitions.Add("LIBCARLA_NO_EXCEPTIONS");
     PublicDefinitions.Add("PUGIXML_NO_EXCEPTIONS");
     PublicDefinitions.Add("BOOST_DISABLE_ABI_HEADERS");
+    PublicDefinitions.Add("MSGPACK_DISABLE_LEGACY_NIL");
     PublicDefinitions.Add("BOOST_TYPE_INDEX_FORCE_NO_RTTI_COMPATIBILITY");
   }
 }

Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Sensor/WorldObserver.cpp

@@ -277,7 +277,7 @@ static carla::Buffer FWorldObserver_Serialize(
   auto total_size = sizeof(Serializer::Header) + sizeof(ActorDynamicState) * Registry.Num();
   auto current_size = 0;
   // Set up buffer for writing.
-  buffer.reset(total_size);
+  buffer.reset(static_cast<carla::Buffer::size_type>(total_size));
   auto write_data = [&current_size, &buffer](const auto &data)
   {
     auto begin = buffer.begin() + current_size;

Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Vehicle/CarlaWheeledVehicle.cpp

@@ -593,7 +593,7 @@ float ACarlaWheeledVehicle::GetWheelSteerAngle(EVehicleWheelLocation WheelLocati
   }
   else
   {
-    return VehicleAnim->GetWheelRotAngle((uint8)WheelLocation);
+      return VehicleAnim->GetWheelRotAngle((uint8)WheelLocation);
   }
 }