No memory management is slowing the engine and causing memory fragmentation

[twaritwaikar]

Details of why this is happening


We need a memory management system as explained in the image source article

[twaritwaikar]

One of the methods I had found that dealt with this problem is basically allocating a HUGE chunk of heap memory and distributing it as per the engine's convenience amongst its subsystems.

But there is a downside in this method that there might not always be that much memory to spare for the computer. Also, unless the entire engine's core systems are complete, we don't even know how much memory we are going to need.

And sometimes this memory requirement will be different on different platforms.

[asutoshpalai]

How did you detect this? The memory manager of OS is supposed to prevent this. I suspect that it is happening as you have shown in the diagram because generally (at least Linux) memory allocators divide the memory into different pools for different chunk sizes. Thus when you do malloc(8), it is allocated from a different pool than if you do malloc(256). So, it does not fragment as you have shown in the illustration. There can be many other reasons for slowing down your engine, use perf tools to find out where the time is actually spent. If it is spent on memory allocations (calls to memory allocator/deallocator functions), then you should proceed in this direction.

Also, you can plug in third-party memory allocators without writing your own. This is a very common practice in game development but do deliberate on if you actually need that. Although a very old paper, you might want to take a look at https://people.cs.umass.edu/~emery/pubs/berger-oopsla2002.pdf

[twaritwaikar]

Thanks for the information!

Although I had found an article in Jason Gregory's Game Engine Architecture that claimed that poor class design in C++ can affect CPU performance. It basically said that sometimes even rearranging the order in which data members are listed in a class can help memory jumps because of the kind of arithmetic operations required while accessing them.

This sort of a attention was never given while writing classes for Rubeus, which raised suspicion for performance left to be etched out of the system.

[twaritwaikar]

Another thing that article mentioned was that data members of a class can't be allocated in a way such that the OS can help in reducing fragmentation. Space for individual variables can be allocated while preventing this but a classes data members need to be allocated in a continuous array, which is also distributed in multiple 8-byte blocks.

[DhavalKapil]

No memory management should not slow the engine, it should just waste more space. Are you sure this is the bottleneck? Also, what are you using currently for memory management?

[twaritwaikar]

I think I can elaborate on what I mean by memory management. I am referring to the absence of a central pool of memory and a distributor of this memory for the engine sub-systems.

Calling the OS for allocating space for heavyweight classes during the game loop is something I am considering for something that could reduce performance. If we allocate a continuous block a memory big enough to fit the entire engine could potentially solve this problem.

However, I have not been seeing any clues of slowing down due to this reason. This is something I speculated a while back pre v1.0

[twaritwaikar]

I have been using Visual Studio's Code profilers as of now

[DhavalKapil]

The standard allocator 'malloc' is a user level allocator. It gets a huge chunk from the OS one time and keeps on returning memory from within it. Not sure what happens in Windows.

Malloc seems to be more optimized towards using less space than speed. If you're open to waste space to get benefits in speed, see jemalloc.

If we allocate a continuous block a memory big enough to fit the entire engine could potentially solve this problem.

Do you have an approximation how much memory would that be?

[twaritwaikar]

I will have to check the exact amount but I have seen some engines perform memory management for themselves (rather than calling the OS every time a new allocation needs to be made). Rubeus currently requires above 100 MB while executing but this can easily change if the game resources are different.

[DhavalKapil]

Calling malloc is not calling the OS for every allocation. Performing memory management for themselves makes sense if the current need is not fulfilled by other available allocators. A good way is to log all malloc/free calls with sizes and then see which allocator fits (if any) according to the pattern.

[twaritwaikar]

Ohkay this looks like we need to verify whether we need an allocator in the first place. I will add this to the list of things to do in the sprint next week.