L2JS Binary Schema

L2JS Binary Schema

Lineage 2 JS Binary Schema is a JavaScript library implemented in TypeScript for fast two-way serialization of structured data for use in communications protocols, data storage, and more. It creates a binary payload much smaller than XML or JSON by using a shared schema.

Although this library can be extended and used for general purposes, it has been built specifically for serializing Lineage 2 server-to-client and client-to-server communication protocols. Hence, the library natively implements the following data types:

• byte - 8 bit integer - 1 byte
• word - 16-bits integer - 2 bytes
• dword - 32-bits integer - 4 bytes
• double - 64-bits integer (long) - 8 bytes
• float - signed 64-bit float (LE) - 8 bytes
• bytes - an array of bytes. Uses Uint8Array as a container
• ntstring - null-terminated UTF-16 (LE) string
• array - list of structured elements
• branch - conditional structure

Example Usage :

import { Schema, Stream } from "l2js-client/mmocore/schema/Schema";

const shm = Schema({
  b1: {
    $type: "byte",
  },
  d1: {
    $type: "dword",
  },
  s1: {
    $type: "ntstring",
  },
});

const buffer = [];
const writer = (b) => buffer.push(b);

// Serialize
shm.write({ b1: 16, d1: 40, s1: "admin" })(writer);

// buffer should have value of:
// [16, 40, 0, 0, 0, 97, 0, 100, 0, 109, 0, 105, 0, 110, 0, 0, 0]

// De-serialize
const stream = Stream(Uint8Array.from([16, 40, 0, 0, 0, 97, 0, 100, 0, 109, 0, 105, 0, 110, 0, 0, 0]));
const parsed = shm.read(stream).unwrap();

// parsed should have value of:
// { b1: 16, d1: 40, s1: "admin" }

Schema

A schema in this library is a vocabulary that allows you to annotate, and later serialize and de-serialize structured data. The schema should be defined in a JSON object.

The schema should contain zero or more JSON objects with user-defined keys, which keys later can be used for accessing the data value. Each object describes a single element of the structured data, and has a mandatory descriptor called $type, defining the type of the data located at this position of the structure. Examples is:

{
    my_var: {
        "$type": "word"
    }
}

The order of the nested objects is important as well, since this will the be order of the serialized binary representation. Example:

{
    a1: {
        "$type": "byte"
    },
    a2: {
        "$type": "word"
    }
}

with data:

{ a1: 5, a2: 9 }

serializes in:

0x05 0x09 0x00

3 bytes in this order, because a1 has type of byte (a single byte) - 0x05; and a2 has type word (16-bits, or 2 bytes) - 0x09 0x00

Descriptors

---

$type - defines the types of the element. Supported values are : "byte", "word", "dword", "float", "double", "ntstring", "bytes", "array" or "branch".

---

$default - not mandatory; if defined, the value is used whenever the input data is missing.

example:

{
    my_var: {
        "$type": "byte",
        "$default": 5
    },
    my_blob: {
        "$type": "bytes",
        "$length": 5,
        "$default": [ 7, 19, 33, 2, 8]
    }
}

---

$length - defines the length of elements of type array or bytes. Can be defined with either a specific integer value, or by a reference to another data element by using an $id descriptor.

examples:

{
    my_blob: {
        "$type": "bytes",
        "$length": 15
    }
}

{
    blob_size: {
        "$type": "byte"
    },
    my_blob: {
        "$type": "bytes",
        "$length": { "$id": "blob_size" }
    }
}

---

$id - defines a reference to another data element. Can be used either with $length descriptor, or with elements with type branch and $condition descriptor.

example:

{
    info: {
        "$type": "byte"
    },
    my_branch: {
        "$type": "branch",
        "$id": "info",
        "$condition": 5,
        "$schema": {
            my_var: {
                "$type": "byte",
                "$default": 5
            }
        }
    }
}

---

$condition - can be used in elements with type of "branch", and specify the exact condition the branch structure will be taken into account. Can be defined with an integer or a string value, or with in a combination of the following supported operators:

• $eq - equals (=)
• $neq - not-equals (!=)
• $gt- greater-than (>)
• $gte - greater-than or equal (>=)
• $lt - less-than (<)
• $lte - less-than or equal (<=)
• $and - logical and. Expects and array.
• $or - logical or. Expects an array.

example:

{
    info: {
        "$type": "byte"
    },
    my_branch: {
        "$type": "branch",
        "$id": "info",
        "$condition": { "$or": [ { "$eq": 5 }, { "$eq": 8 } ] },
        "$schema": {
            my_var: {
                "$type": "byte",
                "$default": 5
            }
        }
    }
}

---

$wrapper - a boolean (true or false) descriptor defines whether the current element is a wrapper, and it's schema should not be nested. This descriptor has a default value of true for elements of type branch.

---

$schema - defines a nested schema. Used in elements of type array and branch. The structure within $schema follows the same rules as per the main schema. Multiple schema levels can be nested.