Observing memory leak with ruleset creation

[GAdoring]

Creating and removing a ruleset is causing a memory leak in version 3.3

I am observing a memory leak with the operator objects not being freed, once the ruleset is cleaned up.
With a test program that initializes modsecurity, reads a ruleset and cleans up the ruleset

Code fragment:
global_modsec = msc_init();
global_rules = msc_create_rules_set();
const char* error = NULL;
msc_rules_add_file(global_rules, "rules.conf", &error);
msc_rules_cleanup(global_rules);
msc_cleanup(global_modsec);

I see memory leaks:

==1836==
==1836== 142,421,048 bytes in 30,350 blocks are possibly lost in loss record 410 of 412
==1836== at 0x4C2C089: calloc (vg_replace_malloc.c:762)
==1836== by 0x4FBAFDD: acmp_add_pattern (acmp.cc:517)
==1836== by 0x4FA4CBB: modsecurity::operators::Pm::init(std::string const&, std::string*) (pm.cc:136)
==1836== by 0x4EFF823: yy::seclang_parser::parse() (seclang-parser.yy:874)
==1836== by 0x4F39373: modsecurity::Parser::Driver::parse(std::string const&, std::string const&) (driver.cc:145)
==1836== by 0x4F39697: modsecurity::Parser::Driver::parseFile(std::string const&) (driver.cc:189)
==1836== by 0x4F50CC6: modsecurity::RulesSet::loadFromUri(char const*) (rules_set.cc:53)
==1836== by 0x4F52542: msc_rules_add_file (rules_set.cc:296)
==1836== by 0x4011D7: process_rules (modsec_memory.c:15)
==1836== by 0x40126C: main (modsec_memory.c:33)
==1836==
==1836== 569,323,113 (1,344 direct, 569,321,769 indirect) bytes in 21 blocks are definitely lost in loss record 412 of 412
==1836== at 0x4C2A593: operator new(unsigned long) (vg_replace_malloc.c:344)
==1836== by 0x4EF56B1: yy::seclang_parser::parse() (seclang-parser.yy:1032)
==1836== by 0x4F39373: modsecurity::Parser::Driver::parse(std::string const&, std::string const&) (driver.cc:145)
==1836== by 0x4F39697: modsecurity::Parser::Driver::parseFile(std::string const&) (driver.cc:189)
==1836== by 0x4F50CC6: modsecurity::RulesSet::loadFromUri(char const*) (rules_set.cc:53)
==1836== by 0x4F52542: msc_rules_add_file (rules_set.cc:296)
==1836== by 0x4011D7: process_rules (modsec_memory.c:15)
==1836== by 0x40126C: main (modsec_memory.c:33)

How do we deallocate the memory used by the parser?

the 'pm' operator, if used with a large set of phrases can cause a huge leak.

[zimmerle]

Hi @GAdoring,

The rules creation is associated with the loading of ModSecurity itself; it is held in Memory till ModSecurity closes. This piece of Memory is used as-is during the entire execution. Therefore, it is not something that impacts the production environment.

[GAdoring]

It is not released when modsecurity closes.

global_modsec = msc_init();
:
:
msc_cleanup(global_modsec);

After this call to cleanup modsecurity, the memory is still left behind.
Is this expected?
What if the production system wants to dynamically load and unload a ruleset? Then this will be leaked each time..

[zimmerle]

It is not released when modsecurity closes.

global_modsec = msc_init();
:
:
msc_cleanup(global_modsec);

After this call to cleanup modsecurity, the memory is still left behind.
Is this expected?
What if the production system wants to dynamically load and unload a ruleset? Then this will be leaked each time..

That is indeed true, this code was migrated from ModSecurity version 2 where memory cleanups were held by deleting a memory pool. We expect to have this code replaced with pull request #2551.