[Draft] Package file format specification

Package file format specification

Revision:

• 0.1 - TDEngine2 0.5.x [2022-01-25]

Main structure

First of all all numeric values are stored in little endian bytes order as it's done for all other in-engine storage formats. The basic structure of the package file is same as Quake's family of PAK formats with some project changes.

The packages support both compressed and uncompressed files no matter of their content. Inside an application they could be mount to virtual paths and all the data could be accessed in simple manner as they lie in physical file system.

The basic structure of the package file is presented below

HEADER - contains meta/helper information about stored data
FILES_DATA - contiguous block of memory that contains files data
FILES_ENTRIES - a block that consists of small blocks that describes packed file's properties

Section review

Header

TAG                  (CHAR[4] bytes)   (contains "PAK" string null terminated string
VERSION              (U16 (2/4) bytes) (equals to 0x100 under 0.5 - 0.6 versions)
UNUSED_PADDING       (U16 (2/4) bytes) (used just for padding)
ENTITIES_COUNT       (U32 (4/8) bytes) (an amount of files that's packed into the archive)
FILES_ENTRIES_OFFSET (U64 (8/16) bytes) (an offset from file's beginning in bytes to start of files table description)
FILES_ENTRIES_SIZE   (U64 (8/16) bytes) (size of the files table)

Files entries

FILE_ENTRY_0
...
FILE_ENTRY_N

File entry

FILENAME_LENGTH       (U64 (8/16) bytes) (length of a CHARs array that represents a filename)
FILENAME_DATA         (CHAR[FILENAME_LENGTH] bytes) (filename string)
DATA_BLOCK_OFFSET     (U64 (8/16) bytes) (offset from file's beginning to the data of the file)
DATA_BLOCK_SIZE       (U64 (8/16) bytes) (size of the uncompressed file's data / if the data is compressed the field contains original size)
COMPRESSED_BLOCK_SIZE (U64 (8/16) bytes) (size of the compressed file's data / equals to 0 if the file is not compressed)
IS_COMPRESSED_FLAG    (U64 (8/16) bytes) (equals to 0x1 if the file's data is compressed)

File data

The data at the segment could be placed in different ways based on data's format, its nature, etc. It also could be stored in compressed way.