UPBGE: seamless world system

[vlad0337187]

There are some thoughts about seamless world system, which fits well to current UPBGE's logic system.
It can also be used to get better detalization instead\with current LOD system.

I suggest all you to discuss it, make corrections and, if it'll be opportune, than start discussing how and when to implement it.

It can be used to create seamless territories of any configuration: as open, so closed.

Main idea of the system: #1 (comment)
More widely about system: #1 (comment)
Details about UI for it: #1 (comment)
Other technical details: #1 (comment)
Advantages and disadvantages: #1 (comment)
Using examples: #1 (comment)

[vlad0337187]

return back: #1 (comment)

Main idea:

• We add special kind of empty object - "area empty". They'll represent certain territories.
• In 3d View and while runtime when you come to certain distance, specified for each "area empty", it'll asynchronously load objects from specified scene, and place to current scene so, that center of specified scene will match position of "area empty".
• When you leave certain distance, all data from loaded scene will be removed.

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

More widely about system:

• We add special kind of empty object - "area mark". They'll represent certain territories.
  • There will be 3 display modes for "area marks": 1. Like general empty; 2. Like wireframe box; 3. Like wireframe sphere.
    (1 will be needed to do not disturb us with drawing area's borders. 2 will be needed to represent area's borders in box mode (when you're inside the box - scene is loaded). 3 will be needed to represent area's borders in sphere mode (when we come closer, that specified radius - scene is loaded))
  • You can also specify "open and close distances", scene, to which this "area mark" will be referencing (where it'll save and from where will read data).
  • All that will be available from Properties window from Data tab.
• In 3d View or while runtime when you come to certain distance, specified for each "area mark", area will be opened, it'll asynchronously load objects from specified scene, and place to current scene so, that center of specified scene will match position of "area mark".
  • (there is no need to store to objects inverse transformation matrix because: > "area marks" cannot be moved while runtime; > not to get errors while parenting; > make loaded objects be the same, like general objects from the main scene.)
  • If it's in the 3d View, objects will be added to outliner and will be displayed like children of "area mark" (to be almost seamless with them), but they'll be marked with special marker (or color) for user to know, that they are "area objects".
  • You could also select in Outliner's display mode "Areas" and there will be displayed "area marks" with all their objects by hierarchy (when you open "area mark", it's objects will be appeared just like when you open "scene" in "All scenes" display mode).
  • While runtime from Python API, or in 3d Viewer from Properties window, Data section you could turn off automatic loading\unloading of areas. Also you could load\unload areas from there manually.
• Loaded objects in runtime and 3d VIew will be the same, as general top-level objects from main scene. They'll have coordinates in main scene's world coordinates. When you move "area mark" in 3D View, there will be launched function, that moves all it's objects to the same distance.
  • Between loaded objects could be other "area marks", that are referring to other scenes, which could be as in the same file, from where "area mark" was loaded, so in other .blend file (but not in one of .blend files, from where it's parent areas are (for not to make circle references and have bugs while saving).
    So this is semi-recursive system (in one area can be several other areas) (not recursive to not have many bugs, not non-recursive to have enough flexibility and ability to divide game into seamless parts).
• When you add new objects in 3d View from the top-level .blend file, top-level scene, they are added to top-level scene. But in property window in Object tab you could specify: to which "area empty" it belongs. After you specified that, it'll belong to it (and will not be saved to top-level scene). After you press "save", all modified scenes will be written to their files.
• When you leave certain distance (specified for each empty, different from loading distance), "area mark" is closed and all data from loaded scene will be removed.
  • If it's while runtime, loaded scenes will not change, there will be nothing to modify.
  • If it's in 3d View and scene was placed in different .blend file, from each object's coordinates are subtracted it's "area empty" coordinates (to move objects to the center of it's scene) and all data is saved to .blend1 file. This file is also added to the "changed" list, when you press "save" button in the main .blend file, all changed .blend files will be rewritten from corresponding .blend1 files.
  • If it's in 3d View and scene was placed in the same .blend file, it just will be changed and will save after pressing "save" button, as usually.
• If you'll return back to closed but not saved "area empty", it'll load asynchronously data from .blend1 file, because it has a mark: "modified".

Summary:

• All current objects (including objects from different areas) physically are present in RAM (objects from different areas physically belong to current scene, but marked, as objects from different areas) (or some kind of swap, if Blender provides this).
• Some of them will be written to .blend1 file of that scene, for which they belong to, if their "area mark" was modified and closed.
• Only when you press "save" button, all data from current scene:
  • which belongs to current scene will be written there
  • which belongs to other scene (including area scenes) in current .blend file will be written there
  • which belongs to other scene (including area scenes) in other .blend file will be written there
  • if some area scenes were modified and closed - they will be written to their .blend file from .blend1 file, if they were modified but not closed - from current scene.

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

df

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

Details about UI for it:

• In Properties window in Data tab (When "Area mark" is selected):
  • you can switch between 3 display modes for "area marks": 1. Like general empty; 2. Like wireframe box; 3. Like wireframe sphere. Each mode will be for different loading "area scenes" modes.
  • You can also specify "open and close distances" for "area marks" (open - objects are loaded, closed - removed).
  • "File", from where "area mark" will load scene.
  • "Scene", to which this "area mark" will be referencing (where it'll save and from where will read data).
  • you can turn on and off automatic opening and closing of this area.
  • you can manually load or unload objects from this mark (automatic mode will be switched off).
  • you can look to "area mark"'s state: > objects are loaded, area is open; > objects are unloaded, area is closed; > objects are loading or unloading;
  • you can to errors of this area, if they're present.
• In "Outliner":
  • In "All scenes", "Scene", other displayed modes:
    • Objects (groups, animations, etc), marked as belonging to some areas, will be displayed just like other objects, but marked with special marker or color.
    • Also general objects (with meshes, lamps, cameras) will be displayed, like children of "area mark" (for convenience and clearness) (but they're not actually children for not storing there inverse transformation matrices > because when they'll be written to their scenes\files, such information will be not needed > also not to have many bugs with it > also to add ability to make them parents\children of other objects while runtime (in 3d View you cannot do so - after closing and re-opening areas, cross-area relationships will be removed (because scenes cannot save such information))
    • Near each "area mark" (at right, near eye, cursor, photo buttons) will be color mark: red; yellow; green; grey. Red means that there is an error loading scene (.blend file or scene in it are absent). Yellow means that loading objects is proceeding. Green means that objects were successfully loaded and "area mark" is open. Grey means that "area mark" is closed and objects are unloaded.
    • Also each "area mark" will be hightlighted with star (*) after it's name if it was modified.
  • In "Areas" display mode:
    • will be displayed all "area marks" with all their objects by hierarchy. At first - categories: groups, animations, objects, etc, in each category will be appropriate objects (when you open "area mark", it's objects will be appeared just like when you open "scene" in "All scenes" display mode.
  • In "Current Area" display mode:
    • You can select between one of current (opened) areas from list, which is similar to "display mode" list, than will be displayed all categories (animations, objects, groups, etc) and their objects, which belong to selected area.

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

Other technical details:

• "Area marks" will be active only in Game Engine mode, in other modes they will act like general empties. This is for compatibility with other Blender's modes.
• Of course, such system could be implemented only in Python by analyzing current position and using libLoad() and libFree(), but it would be very inconvenient. Game developer cannot create good game, when he does not see it. He cannot create different areas separately to when he'll combine them, they looked well. That's why it's important to make loading objects system same as in 3d View while game creation process, so while game running process.
• "Area marks" would have in-game KX_AreaMark Python object. It'll have such properties (except general properties like worldPosition, worldOrientation):
  • state (bool). Possible values: True or False. True - current area is open (active), False - current area is closed (inactive).
  • errors (bool). Possible values: True or False. True - there were errors while objects loading (or specified .blend file, or scene is missing); False - errors are absent;
  • loaded (bool). Possible values: True or False. True - all objects from specified scene were loaded; False - objects are not loaded yet or already unloaded.
  • automatic (bool). Possible values: True or False. True - area is opened and closed automatically, depending changing active camera's position. False - area is opened and closed manually from Python API by .open() and .close() methods.
  • box_mode (bool) (read only). Possible values: True or False. True - box mode on, areas will be opened when active camera will be inside of activating box (deactivating box can differ). False - box mode off (simple sphere mode), areas will be opened when active camera will be nearer than specified opening radius (closing radius can differ)
  • scene (string) (read only). Contains name of the bounded scene, which contains objects of this area.
  • file (string) (read only). Name of file, in which is bounded scene. If bounded scene is in current file, string is empty.
  • objects (list of strings) (read only). List of objects, that were loaded from bounded scene. If Area is closed - list is empty.
  • open() (method). Opens manually this area, it's objects will start loading. Also automatic property will be set as False.
  • close() (method). Closes manually this area, it's objects will start unloading. Also automatic property will be set as False.
• All "area mark" objects could be reached KX_Scene.areas, all active "area mark" objects could be reached from KX_Scene.active_areas

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

Advantages and disadvantages:

Advantages:

• There are no loading screens, there is no need to wait time while it'll load. Loading will take time when player is playing game;
• Lesser RAM consumption: there is no need to load all objects from large location to RAM, at one moment of time there are objects only of needed part of location;
• Lesser size of .blend files. (lesser size of main .blend file leads for lesser loading time, this relates to all .blend files. Loading time will be spent while player is playing);
• It does not enforce or impose user to use game engine with this system. There is no need to rebuild current levels\games to fit this system. User can simply make all like he was doing before, just if he'll not create "area marks";
• For game engine developers it does not touch many parts of code: almost all it's parts will work, as they did this before, no need to rewrite\adapt them.
• Lesser loading time of Blender itself and game, because at one time it proceeds only limited amount of objects, independently of location size;
• Clear and intuitive system for game developer, it simplifies game creation process. Every part of game world can be edited from one place (main scene).
• Result world can have any shape in 3D space.

Disadvantages:

• Extra work of game engine developers.
• Can be received additional amount of bugs.
• Need to move to certain area to see it's objects in outliner and edit them (can be avoided if to edit area scenes separately).

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

Using examples:

• Possible variants of placing areas in scene:
  • Classic square nested scheme:
    
    • Here main scene was subdivided on 2 main areas, which can have it's own main objects like ground. They are placed in overlap for there will not be break: when player came to territory, but it has not loaded yet. Nested areas can have detailed objects (for example, rooms interior). Also they could be placed overlap (I did not do that on sketch to avoid confusing)
  • Before overlapping region in area 1 could be also placed area, that places LOD objects to coordinates of area 2 (red) and vice versa (light blue) (nobody said, that objects must not leave it's area's coordinates):
    
    When player come from area 1 to area 2, he'll go to 1-2_lod area, which will load LOD objects of territory 2 (if they're too large), after coming from that area to area 2, when it'll be opened, lod objects could be removed.

return back: #1 (comment)

[vlad0337187]

return back: #1 (comment)

Notes on using this system:

• Bigger area nesting levels can be used for bigger territories, lesser - for more detailed.
• "Area marks" could be placed in arbitrary places of scene, with arbitrary nesting levels.
• "Area marks" could be reached from Python API while runtime. Also it would be convenient to reach them from Debug mode.

return back: #1 (comment)

[vlad0337187]

What to add:
ability manually choose object, which coordinates will open and close areas.