[Bug] r.out.gdal: export of COG and GPKG fails with segfault

[ninsbl]

Describe the bug

r.out.gdal runs into a segmentation fault, when exporting raster data to COG or GPKG. The problem is likely that these formats are not written directly but go through the memory driver.
The issue is seemingly not triggered if nodata and data type checks are skipped.

After some debuging it looks to me that the issue is related to the new PARSON library (see valgrind output below). However, parson is present also in the 8.4 release branch and there COG export works...

To reproduce

valgrind --leak-check=full --track-origins=yes -s r.out.gdal input=aspect output=/tmp/test_raster.tif format=COG createopt="COMPRESS=LZW" overviews=2 --o --q

Expected behavior

COG and GPKG are quite central formats and should work without issues.

System description

• Operating System: Linux [Ubuntu 22.04, Fedora 42]
• GRASS version: 8.5

Additional context

Here is a piece of valgrind output from debuging the issue:

==282957== 1 errors in context 1 of 67:
==282957== Invalid read of size 8
==282957==    at 0x6BAAD49: json_object_get_cell_ix (parson.c:632)
==282957==    by 0x6BAB04A: json_object_getn_value (parson.c:698)
==282957==    by 0x6BAE2FB: json_object_get_value (parson.c:1629)
==282957==    by 0x6BAE320: json_object_get_string (parson.c:1634)
==282957==    by 0x4BC36A0: CPLJSONObject::ToString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const (cpl_json.cpp:1075)
==282957==    by 0x4BD39FB: CPLJSONObject::GetString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const (cpl_json.cpp:1059)
==282957==    by 0x5C253B2: gdal::TileMatrixSet::parse(char const*) (tilematrixset.cpp:192)
==282957==    by 0x4E49980: GDALCOGDriver::InitializeCreationOptionList() (cogdriver.cpp:1556)
==282957==    by 0x4E289AF: GDALCOGDriver::GetMetadataItem(char const*, char const*) (cogdriver.cpp:1404)
==282957==    by 0x5AF219D: GDALValidateCreationOptions (gdaldriver.cpp:2119)
==282957==    by 0x5B0EFDA: GDALDriver::CreateCopy(char const*, GDALDataset*, int, char const* const*, int (*)(double, char const*, void*), void*) (gdaldriver.cpp:1334)
==282957==    by 0x405130: main (main.c:782)
==282957==  Address 0xdd353a9072c1a08 is not stack'd, malloc'd or (recently) free'd

[ninsbl]

@cwhite911 could you have a look into this to see whether it is indeed PARSON related?

[ninsbl]

See also: #6170

[nilason]

This seems to be related to #4697.

[tmszi]

I confirm this bug.

Debugged with GNU Debugger:

GRASS nc_basic_spm_grass7/PERMANENT:~ > gdb `which r.out.gdal`
GNU gdb (Gentoo 16.3 vanilla) 16.3
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-pc-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://bugs.gentoo.org/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/lib64/grass85/bin/r.out.gdal...
Reading symbols from /usr/lib/debug/usr/lib64/grass85/bin/r.out.gdal.debug...
(gdb) run input=elevation output=/tmp/elevation.gpkg format=GPKG
Starting program: /usr/lib64/grass85/bin/r.out.gdal input=elevation output=/tmp/elevation.gpkg format=GPKG
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Checking GDAL data type and nodata value...
 100%
Using GDAL data type <Float32>

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5eb44f9 in json_object_get_cell_ix (object=object@entry=0x5555559292e0, key=0x7fffffffc9b0 "\340\222\222UUU", key_len=6, hash=6958333955944, 
    out_found=out_found@entry=0x7fffffffc944) at parson.c:628
628	        cell = object->cells[ix];
(gdb) bt full
#0  0x00007ffff5eb44f9 in json_object_get_cell_ix (object=object@entry=0x5555559292e0, key=0x7fffffffc9b0 "\340\222\222UUU", key_len=6, hash=6958333955944, 
    out_found=out_found@entry=0x7fffffffc944) at parson.c:628
        cell_ix = 2474102620256
        cell = <optimized out>
        ix = <optimized out>
        i = 0
        hash_to_check = <optimized out>
        key_to_check = <optimized out>
        key_to_check_len = <optimized out>
#1  0x00007ffff5eb45e9 in json_object_getn_value (object=0x5555559292e0, name=<optimized out>, name_len=<optimized out>) at parson.c:698
        hash = <optimized out>
        found = 0
        cell_ix = 0
        item_ix = 0
#2  0x00007ffff5eb4d86 in json_object_get_value (object=<optimized out>, name=<optimized out>) at parson.c:1629
No locals.
#3  0x00007ffff5eb51ad in json_object_get_string (object=<optimized out>, name=<optimized out>) at parson.c:1634
No locals.
#4  0x00007ffff6afae1f in CPLJSONObject::ToString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const ()
   from /usr/lib64/libgdal.so.35
No symbol table info available.
#5  0x00007ffff6afc6f9 in CPLJSONObject::GetString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const () from /usr/lib64/libgdal.so.35
No symbol table info available.
#6  0x00007ffff78d149f in gdal::TileMatrixSet::parse(char const*) () from /usr/lib64/libgdal.so.35
No symbol table info available.
#7  0x00007ffff7663b7e in ?? () from /usr/lib64/libgdal.so.35
No symbol table info available.
#8  0x00007ffff76649f8 in ?? () from /usr/lib64/libgdal.so.35
No symbol table info available.
#9  0x00007ffff77bf578 in GDALValidateCreationOptions () from /usr/lib64/libgdal.so.35
No symbol table info available.
#10 0x00007ffff77c041d in GDALDriver::Create(char const*, int, int, int, GDALDataType, char const* const*) () from /usr/lib64/libgdal.so.35
No symbol table info available.
#11 0x00005555555593bc in main (argc=<optimized out>, argv=<optimized out>) at main.c:675
        module = <optimized out>
        flag_l = <optimized out>
        flag_c = 0x55555558bc60
        flag_m = 0x55555558bdf0
        flag_f = <optimized out>
        flag_t = 0x55555558be40
        input = <optimized out>
        format = 0x55555558c0d0
        type = <optimized out>
        output = 0x55555558c010
        createopt = 0x555555613b20
        metaopt = 0x555555613bc0
        nodataopt = 0x555555613c60
        overviewopt = 0x555555613d00
        offsetopt = <optimized out>
        scaleopt = 0x555555613e60
        cellhead = {format = 0, compressed = -1, rows = 1350, rows3 = 1350, cols = 1500, cols3 = 1500, depths = 1, proj = 99, zone = 0, ew_res = 10, ew_res3 = 10, 
          ns_res = 10, ns_res3 = 10, tb_res = 1, north = 228500, south = 215000, east = 645000, west = 630000, top = 1, bottom = 0}
        ref = {nfiles = 1, file = 0x555555615200, red = {table = 0x0, index = 0x555555589450 "", buf = 0x0, fd = 774492672, min = 1830485141, max = 1431870176, n = -1}, 
          grn = {table = 0x0, index = 0x2c <error: Cannot access memory at address 0x2c>, buf = 0x2b <error: Cannot access memory at address 0x2b>, fd = -160938944, 
            min = 32767, max = -201353684, n = -1}, blu = {table = 0x0, index = 0x420 <error: Cannot access memory at address 0x420>, buf = 0x7ffff633cac0 "", 
            fd = -165699930, min = 32767, max = 1040, n = -1}}
        mapset = <optimized out>
        gdal_formats = 0x5555556136d0 "VRT,GTiff,COG,NITF,HFA,ELAS,AAIGrid,DTED,PNG,JPEG,MEM,GIF,XPM,BMP,PCIDSK,PCRaster,ILWIS,SGI,SRTMHGT,Leveller,Terragen,ISIS3,ISIS2,PDS4,VICAR,ERS,FIT,GRIB,RMF,WMS,RST,GSAG,GSBG,GS7BG,R,KMLSUPEROVERLAY,"...
        maptype = <optimized out>
        testmaptype = <optimized out>
        bHaveMinMax = <optimized out>
        dfCellMin = 55.578792572021484
        export_min = <optimized out>
        dfCellMax = 156.32986450195312
        export_max = <optimized out>
        sRange = {min = 55.578792572021484, max = 156.32986450195312, first_time = 0, rstats = {sum = -nan(0xfffffffffffff), sumsq = -nan(0xfffffffffffff), count = 0}}
        retval = <optimized out>
        n_overviews = 0
        projinfo = <optimized out>
        projunits = <optimized out>
        projepsg = <optimized out>
        srswkt = 0x555555928550 "PROJCRS[\"NAD83(HARN) / North Carolina\",\n    BASEGEOGCRS[\"NAD83(HARN)\",\n        DATUM[\"NAD83 (High Accuracy Reference Network)\",\n", ' ' <repeats 12 times>, "ELLIPSOID[\"GRS 1980\",6378137,298.257222101,\n", ' ' <repeats 16 times>...
        indef = <optimized out>
        hDriver = 0x5555555f4790
        hMEMDriver = 0x0
        datatype = <optimized out>
        band = <optimized out>
        nodataval = nan(0x8000000000000)
        default_nodataval = <optimized out>
        offsetval = 0
        scaleval = 1
        papszOptions = 0x0
        hCurrDS = 0x0
        hMEMDS = 0x0
        hDstDS = 0x0
        adfGeoTransform = {0, 6.9533476211707112e-310, 6.9533558072666312e-310, 5.1382827167489641e-321, 6.9533476864866836e-310, 5.0442126177807907e-319}
        ret = <optimized out>
(gdb) frame 2
#2  0x00007ffff5eb4d86 in json_object_get_value (object=<optimized out>, name=<optimized out>) at parson.c:1629
1629	    return json_object_getn_value(object, name, strlen(name));
(gdb) l
1624	JSON_Value *json_object_get_value(const JSON_Object *object, const char *name)
1625	{
1626	    if (object == NULL || name == NULL) {
1627	        return NULL;
1628	    }
1629	    return json_object_getn_value(object, name, strlen(name));
1630	}
1631	
1632	const char *json_object_get_string(const JSON_Object *object, const char *name)
1633	{
(gdb) 

[petrasovaa]

Unless somebody uses some C magic to fix the gjson wrapper, I would suggest to rename everything in the parson library (similarly to the wrapper, in an automated way), drop the gjson wrapper and fix all the parson library usage in the tools.
Any other options, @nilason , @NishantBansal2003 , @wenzeslaus ?

[nilason]

Unless somebody uses some C magic to fix the gjson wrapper, I would suggest to rename everything in the parson library (similarly to the wrapper, in an automated way), drop the gjson wrapper and fix all the parson library usage in the tools. Any other options, @nilason , @NishantBansal2003 , @wenzeslaus ?

I will need to investigate this a bit first. But instinctively, I'd say the other way around, now the parson library is exposed via:

grass/lib/external/parson/gjson.h

Line 4 in 23b5a22

#include "parson.h"

and we still make direct use of parsons functions, objects. I think we should strive to wrap it completely.

[wenzeslaus]

While the current wrapping is certainly limited since it is using the object names, including the .h and thus also putting it to the includes, I don't know if just cleaning up includes will be sufficient. Isn't the issue with linking, rather than header files?

As I mentioned in #4697, my understanding is that the functions are in the library and are exposed whether or not you have a function declaration at hand. If you provide a declaration, by an include statement or by writing the declaration by hand, you can use the function. Is our library case different than the following example?

// function_definition.c
#include <stdio.h>

void greet(const char* name) {
    printf("Hello, %s!\n", name);
}

// main_program.c
void greet(const char* name);

int main() {
    greet("World");
    return 0;
}

gcc function_definition.c main_program.c -o greet_program

I don't see that would be currently use __attribute__((visibility("..."))), __declspec, or some other "export" control feature, but that's also where my understanding ends.