Storage rewrite - Phase 1 (#4065)

* Phase 1 - wip

* Add some tests

* wip

* Save waypoints

* Implement backpacks

* Add tests for waypoints

* Changes to ADR

* Small changes

* Fix englandish and some small changes to UUID in PlayerProfile

* Fix saving of player data in a few cases

* Documentation around deprecated

* Add some more tests

* Some small doc updates

* Make old Waypoint constructor deprecated and fix javadocs

.adr-dir

@@ -0,0 +1 @@
+docs/adr

.gitignore

@@ -5,6 +5,7 @@
 /.settings/
 /.idea/
 /.vscode/
+/data-store/
 
 dependency-reduced-pom.xml
 

docs/adr/0001-storage-layer.md

@@ -0,0 +1,129 @@
+# 1. Storage layer
+
+Date: 2023-11-15
+Last update: 2023-12-27
+
+**DO NOT rely on any APIs introduced until we finish the work completely!**
+
+## Status
+
+Work in progress
+
+## Context
+
+Slimefun has been around for a very long time and due to that, the way we
+wrote persistence of data has also been around for a very long time.
+While Slimefun has grown, the storage layer has never been adapted.
+This means that even all these years later, it's using the same old saving/loading.
+This isn't necessarily always bad, however, as Slimefun has grown both in terms of content
+and the servers using it - we've seen some issues.
+
+Today, files are saved as YAML files (sometimes with just a JSON object per line),
+which is good for a config format but not good for a data store. It can create very large files
+that can get corrupted, the way we've been saving data often means loading it all at once as well
+rather than lazy-loading and generally isn't very performant.
+
+For a long time we've been talking about rewriting our data storage in multiple forms
+(you may have seen this referenced for "BlockStorage rewrite" or "SQL for PlayerProfiles", etc.).
+Now is the time we start to do this, this will be a very large change and will not be done quickly or rushed.
+
+This ADR talks about the future of our data persistence. 
+
+## Decision
+
+We want to create a new storage layer abstraction and implementations
+which will be backwards-compatible but open up new ways of storing data
+within Slimefun. The end end goal is we can quickly and easily support
+new storage backends (such as binary storage, SQL, etc.) for things like
+[PlayerProfile](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java), [BlockStorage](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java), etc.
+
+We also want to be generally more efficient in the way we save and load data.
+Today, we load way more than is required.
+We can improve memory usage by only loading what we need, when we need it.
+
+We will do this incrementally and at first, in an experimental context.
+In that regard, we should aim to minimise the blast radius and lift as much
+as possible.
+
+### Quick changes overview
+
+* New abstraction over storage to easily support multiple backends.
+* Work towards moving away from the legacy YAML based storage.
+* Lazy load and save data to more efficiently handle the data life cycle.
+
+### Implementation details
+
+There is a new interface called [`Storage`](TBD) which is what all storage
+backends will implement.
+This will have methods for loading and saving things like
+[`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java) and [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java).
+
+Then, backends will implement these
+(e.g. [`LegacyStorageBackend`](TBD) (today's YAML situation))
+in order to support these functions.
+Not all storage backends are required support each data type.
+e.g. SQL may not support [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java).
+
+
+## Addons
+
+The goal is that Addons will be able to use and implement new storage backends
+if they wish and also be extended so they can load/save things as they wish.
+
+The first few iterations will not focus on Addon support. We want to ensure
+this new storage layer will work and supports what we need it to today.
+
+This ADR will be updated when we get to supporting Addons properly.
+
+## Considerations
+
+This will be a big change therefore we will be doing it as incrementally as
+possible.
+Changes will be tested while in the PR stage and merged into the Dev releases when possible.
+We may do an experimental release if required.
+
+Phases do not (and very likely will not) be done within a single PR. They will also not have any timeframe attached to them.
+
+The current plan looks like this:
+
+* Phase 1 - Implement legacy data backend for [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java).
+  * We want to load player data using the new storage layer with the current
+    data system.
+  * We'll want to monitor for any possible issues and generally refine 
+    how this system should look
+* Phase 2 - Implement new experimental binary backend for [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java).
+  * Create a new backend for binary storage
+  * Implement in an experimental capacity and allow users to opt-in
+    * Provide a warning that this is **experimental** and there will be bugs.
+  * Implement new metric for storage backend being used
+* Phase 3 - Mark the new backend as stable for [`PlayerProfile`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/io/github/thebusybiscuit/slimefun4/api/player/PlayerProfile.java).
+  * Mark it as stable and remove the warnings once we're sure things are
+    working correctly
+  * Create a migration path for users currently using "legacy".
+  * Enable by default for new servers
+* Phase 4 - Move [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java) to new storage layer.
+  * The big one! We're gonna tackle adding this to BlockStorage.
+    This will probably be a large change and we'll want to be as 
+    careful as possible here.
+  * Implement `legacy` and `binary` as experimental storage backends
+    for BlockStorage and allow users to opt-in
+    * Provide a warning that this is **experimental** and there will be bugs.
+* Phase 5 - Mark the new storage layer as stable for [`BlockStorage`](https://github.com/Slimefun/Slimefun4/blob/bbfb9734b9f549d7e82291eff041f9b666a61b63/src/main/java/me/mrCookieSlime/Slimefun/api/BlockStorage.java).
+  * Mark it as stable and remove the warnings once we're sure things are
+    working correctly
+  * Ensure migration path works here too.
+  * Enable by default for new servers
+* Phase 6 - Finish up and move anything else we want over
+  * Move over any other data stores we have to the new layer
+  * We should probably still do experimental -> stable but it should have
+    less of a lead time.
+
+## State of work
+
+* Phase 1: In progress
+  * https://github.com/Slimefun/Slimefun4/pull/4065
+* Phase 2: Not started
+* Phase 3: Not started
+* Phase 4: Not started
+* Phase 5: Not started
+* Phase 6: Not started

docs/adr/README.md

@@ -0,0 +1,11 @@
+# ADR
+
+An ADR (Architecture Decision Record) is a document describing large changes, why we made them, etc.
+
+## Making a new ADR
+
+If you're making a large change to Slimefun, we recommend creating an ADR
+in order to document why this is being made and how it works for future contributors.
+
+Please follow the general format of the former ADRs or use a tool 
+such as [`adr-tools`](https://github.com/npryce/adr-tools) to generate a new document.

src/main/java/io/github/thebusybiscuit/slimefun4/api/gps/GPSNetwork.java

@@ -331,7 +331,7 @@ public void addWaypoint(@Nonnull Player p, @Nonnull String name, @Nonnull Locati
                         }
                     }
 
-                    profile.addWaypoint(new Waypoint(profile, id, event.getLocation(), event.getName()));
+                    profile.addWaypoint(new Waypoint(p.getUniqueId(), id, event.getLocation(), event.getName()));
 
                     SoundEffect.GPS_NETWORK_ADD_WAYPOINT.playFor(p);
                     Slimefun.getLocalization().sendMessage(p, "gps.waypoint.added", true);

src/main/java/io/github/thebusybiscuit/slimefun4/api/gps/Waypoint.java

@@ -1,11 +1,13 @@
 package io.github.thebusybiscuit.slimefun4.api.gps;
 
 import java.util.Objects;
+import java.util.UUID;
 
 import javax.annotation.Nonnull;
 import javax.annotation.ParametersAreNonnullByDefault;
 
 import org.apache.commons.lang.Validate;
+import org.bukkit.Bukkit;
 import org.bukkit.Location;
 import org.bukkit.World.Environment;
 import org.bukkit.entity.Player;
@@ -30,14 +32,14 @@
  */
 public class Waypoint {
 
-    private final PlayerProfile profile;
+    private final UUID ownerId;
     private final String id;
     private final String name;
     private final Location location;
 
     /**
      * This constructs a new {@link Waypoint} object.
-     * 
+     *
      * @param profile
      *            The owning {@link PlayerProfile}
      * @param id
@@ -46,28 +48,62 @@ public class Waypoint {
      *            The {@link Location} of the {@link Waypoint}
      * @param name
      *            The name of this {@link Waypoint}
+     *
+     * @deprecated Use {@link #Waypoint(UUID, String, Location, String)} instead
      */
+    @Deprecated
     @ParametersAreNonnullByDefault
     public Waypoint(PlayerProfile profile, String id, Location loc, String name) {
-        Validate.notNull(profile, "Profile must never be null!");
+        this(profile.getUUID(), id, loc, name);
+    }
+
+    /**
+     * This constructs a new {@link Waypoint} object.
+     * 
+     * @param ownerId
+     *            The owning {@link Player}'s {@link UUID}
+     * @param id
+     *            The unique id for this {@link Waypoint}
+     * @param loc
+     *            The {@link Location} of the {@link Waypoint}
+     * @param name
+     *            The name of this {@link Waypoint}
+     */
+    @ParametersAreNonnullByDefault
+    public Waypoint(UUID ownerId, String id, Location loc, String name) {
+        Validate.notNull(ownerId, "owner ID must never be null!");
         Validate.notNull(id, "id must never be null!");
         Validate.notNull(loc, "Location must never be null!");
         Validate.notNull(name, "Name must never be null!");
 
-        this.profile = profile;
+        this.ownerId = ownerId;
         this.id = id;
         this.location = loc;
         this.name = name;
     }
 
     /**
-     * This returns the owner of the {@link Waypoint}.
+     * This returns the owner's {@link UUID} of the {@link Waypoint}.
      * 
+     * @return The corresponding owner's {@link UUID}
+     */
+    @Nonnull
+    public UUID getOwnerId() {
+        return this.ownerId;
+    }
+
+    /**
+     * This returns the owner of the {@link Waypoint}.
+     *
      * @return The corresponding {@link PlayerProfile}
+     *
+     * @deprecated Use {@link #getOwnerId()} instead
      */
     @Nonnull
+    @Deprecated
     public PlayerProfile getOwner() {
-        return profile;
+        // This is jank and should never actually return null
+        return PlayerProfile.find(Bukkit.getOfflinePlayer(ownerId)).orElse(null);
     }
 
     /**
@@ -126,7 +162,7 @@ public ItemStack getIcon() {
      */
     @Override
     public int hashCode() {
-        return Objects.hash(profile.getUUID(), id, name, location);
+        return Objects.hash(this.ownerId, this.id, this.name, this.location);
     }
 
     /**
@@ -139,7 +175,9 @@ public boolean equals(Object obj) {
         }
 
         Waypoint waypoint = (Waypoint) obj;
-        return profile.getUUID().equals(waypoint.getOwner().getUUID()) && id.equals(waypoint.getId()) && location.equals(waypoint.getLocation()) && name.equals(waypoint.getName());
+        return this.ownerId.equals(waypoint.getOwnerId())
+            && id.equals(waypoint.getId())
+            && location.equals(waypoint.getLocation())
+            && name.equals(waypoint.getName());
     }
-
 }

src/main/java/io/github/thebusybiscuit/slimefun4/api/items/SlimefunItem.java

@@ -1160,11 +1160,11 @@ public final int hashCode() {
     }
 
     /**
-     * Retrieve a {@link Optional}<{@link SlimefunItem}> by its id.
+     * Retrieve a {@link Optional} {@link SlimefunItem} by its id.
      *
      * @param id
      *            The id of the {@link SlimefunItem}
-     * @return The {@link Optional}<{@link SlimefunItem}> associated with that id. Empty if non-existent
+     * @return The {@link Optional} {@link SlimefunItem} associated with that id. Empty if non-existent
      */
     public static @Nonnull Optional<SlimefunItem> getOptionalById(@Nonnull String id) {
         return Optional.ofNullable(getById(id));
@@ -1193,11 +1193,11 @@ public final int hashCode() {
     }
 
     /**
-     * Retrieve a {@link Optional}<{@link SlimefunItem}> from an {@link ItemStack}.
+     * Retrieve a {@link Optional} {@link SlimefunItem} from an {@link ItemStack}.
      *
      * @param item
      *            The {@link ItemStack} to check
-     * @return The {@link Optional}<{@link SlimefunItem}> associated with this {@link ItemStack} if present, otherwise empty
+     * @return The {@link Optional} {@link SlimefunItem} associated with this {@link ItemStack} if present, otherwise empty
      */
     public @Nonnull Optional<SlimefunItem> getOptionalByItem(@Nullable ItemStack item) {
         return Optional.ofNullable(getByItem(item));