how to make Ghidra use a function's complete/original stackframe for decompiled code

[wothke]

I have a case where some function allocates/uses a 404 bytes temporary structure on the stack for its internal calculations (the function is self-contained and shuffles data around within that data structure). Conceptually the respective structure seems to consist of some 32-bit counters followed by an int[15] and a byte[80] array, and then an area that might or might not actually be used. Some of the generated data in the tables seems to represent offsets that are again used by the function to navigate within the temporary structure.

Unfortunately Ghidra's decompiler makes a total mess while trying to make sense of the function: In particular it creates separate "local_.." int-vars (and then uses a pointer to that var) for what should correctly be a pointer into the function's original data-structure (e.g. pointing into one of the arrays), e.g.:

    undefined4 local_17f;
...
    dest= &local_17f;
    for (i = 0xf; i != 0; i = i + -1) {
      *dest = 0;
      dest = dest + 1;
    }

Ghidra does not seem to understand that an array based data access is actually being used at that point. Ghirda's decompiler then also generates a local auStack316[316] variable which unfortunately seems to cover only a part of the respective local data structure used by the original ASM code (at least Ghidra actually did notice that a temporary memory buffer is used). As a result the decompiled code basically uses two overlapping (and broken) shadow data structures that should correctly just be the same block of memory.

Is there some way to make Ghidra's decompiler use the complete 404 bytes block allocated by the function as an auStack404 thus bypassing Ghidra's flawed interpretation logic and actually preserve the original functionality of the ASM code?

[wothke]

I think I found something.. In the "Listing" view the used local-variable layout is shown as a comment under the function's header. It seems that by right clicking on a respective local-var line in that comment, "set data type" can be applied to a respective local variable. Ah, and then there is what I've been looking for under "Function/"Edit stack frame" :-)

[wothke]

PS; to bad it seems that Ghidra is actually unable to properly use a respective local stack space even when it has been declared; it messes up respective address calculations, introduces shadow vars that overlap with the original "local" vars, etc... this just sucks and the decompiled code is often completely broken
!

[gtackett]

Have you tried tinkering with calling conventions in the processor’s language files? I’m trying this out, but don’t have a good understanding of how to do it. I see function calls, typically to RTL routines, that may mix passing a few arguments on the stack, with one or two passed in registers. Sometimes a value is passed in, or returned via a fixed memory address. Still unsure how to set up appropriate calling conventions for the processors I’ve been using (currently 8051 based architectures but several others in the past.)

Some functions (like integer division) may return more than one value, e.g., quotient in one register [or subregister], remainder in another, or a structure that’s spread across adjacent registers. I’d like to know how to handle that as well.

Have you dealt successfully with such call or return conventions?

[wothke]

I am a rookie Ghidra user and I've only used features that where accessible via some 'point & click' GUI.. it worked as far as params via FPU and CPU registers were involved. Most of the problems I encountered had to do with Ghidra not handling the stack vars correctly, e.g. making a total mess when alloc_probe is involved, creating additional shadow-vars for specifically defined local-vars, not actually using the declared calling convention for function-table based calls at all - consequently not calculating the offsets to respective parts of the local-var correctly (other problems; incorrectly mixing up the sequence of CPU and FPU driven operations, completely ignoring certain CPU operations thus generating absolute decompile garbage, etc). It seems to me that there are quite a few severe bugs that should rather be fixed.

But yet I managed to get a compileable / running C code export from my toy project - which is more than what I had dared to expect. Still I am glad that it's done, and I don't have any additional reverse engineering projects on my todo list in the near future.. :-)