Prevent return of dangling Vector/QAngle to VScript #321
Conversation
When a Vector or QAngle rvalue reference is returned from a VScript function, the constructed ScriptVariant_t must not store the pointer to it since it is, per convention, a temporary reference. Only do that for lvalue-references, but do a copy when constructing from or assigning a rvalue reference.
|
Also, specifically addressing #104 (comment)
I didn't see any crashes calling |
|
That comment was poorly worded. It was more about being explicit that the only effect of
I'm curious what were the problematic functions that required this. |
I don't know about 'HSCRIPT
I didn't even check that.
Specifically I noticed it on
That still uses the |
z33ky
force-pushed
the
dangling-vscript-vector
branch
2 times, most recently
from
f9d4f33
to
0551292
Compare
|
I think it makes more sense to change
Stack allocating a |
I could
Oh sorry, it it
I don't know what branch you've linked in #321 (comment), but there is different code in both master and develop, which does return a |
|
I'm using # 260, but it's my mistake. I incorrectly put |
|
I think I have been chasing some ghost bugs due to corrupted object files or something. |
Instead of temporary allocating dynamic memory, which subsequently is copied into another dynamically allocated piece of memory, provide a temporary buffer on the stack. The script value inserted into the VM still needs to allocate dynamically though... A few sites are adapted to not create a ScriptVariant_t from a temporary Vector to avoid dynamic allocation there too. cf. mapbase-source#321
z33ky
force-pushed
the
dangling-vscript-vector
branch
from
0551292
to
a88fc1e
Compare
|
I've had a family emergency and some stuff got left by the wayside. |
z33ky
force-pushed
the
dangling-vscript-vector
branch
from
a88fc1e
to
11167f6
Compare
Instead of temporary allocating dynamic memory, which subsequently is copied into another dynamically allocated piece of memory, provide a temporary buffer on the stack. The script value inserted into the VM still needs to allocate dynamically though... A few sites are adapted to not create a ScriptVariant_t from a temporary Vector to avoid dynamic allocation there too. cf. mapbase-source#321
z33ky
force-pushed
the
dangling-vscript-vector
branch
from
11167f6
to
11232d9
Compare
|
Hm, I attempted to fix some sort of syntax error MSVC has with the placement new, but it still complains. gcc accepts both versions. I don't know what its problem is, nor how else I could attempt to fix it :/ |
Instead of temporary allocating dynamic memory, which subsequently is copied into another dynamically allocated piece of memory, provide a temporary buffer on the stack. The script value inserted into the VM still needs to allocate dynamically though... A few sites are adapted to not create a ScriptVariant_t from a temporary Vector to avoid dynamic allocation there too. cf. mapbase-source#321
z33ky
force-pushed
the
dangling-vscript-vector
branch
from
11232d9
to
7ef11da
Compare
Instead of temporary allocating dynamic memory, which subsequently is copied into another dynamically allocated piece of memory, provide a temporary buffer on the stack. The script value inserted into the VM still needs to allocate dynamically though... A few sites are adapted to not create a ScriptVariant_t from a temporary Vector to avoid dynamic allocation there too. cf. mapbase-source#321
z33ky
force-pushed
the
dangling-vscript-vector
branch
from
7ef11da
to
45c0490
Compare
|
Well, why would I expect |
|
I built |
Conceptually yes (unless
Perhaps another change in the commit is responsible? I don't have time to do more testing today; Instead of reverting the whole commit I manually inserted the intermediary diff --git a/sp/src/public/vscript/vscript_templates.h b/sp/src/public/vscript/vscript_templates.h
index 9f2054cc3..4edf38002 100644
--- a/sp/src/public/vscript/vscript_templates.h
+++ b/sp/src/public/vscript/vscript_templates.h
@@ -374,8 +374,10 @@ inline FUNCPTR_TYPE ScriptConvertFuncPtrFromVoid( void *p )
{ \
return false; \
} \
- new ( &temporaryReturnStorage.m_vec ) Vector( ((FUNC_TYPE)pFunction)( SCRIPT_BINDING_ARGS_##N ) ); \
- *pReturn = temporaryReturnStorage.m_vec; \
+ Vector *vec = (Vector*)malloc( sizeof( Vector ) ); \
+ new ( vec ) Vector( ((FUNC_TYPE)pFunction)( SCRIPT_BINDING_ARGS_##N ) ); \
+ *pReturn = vec; \
+ pReturn->m_flags |= SV_FREE; \
return true; \
} \
}; \
@@ -394,8 +396,10 @@ inline FUNCPTR_TYPE ScriptConvertFuncPtrFromVoid( void *p )
{ \
return false; \
} \
- new ( &temporaryReturnStorage.m_ang ) QAngle( ((FUNC_TYPE)pFunction)( SCRIPT_BINDING_ARGS_##N ) ); \
- *pReturn = temporaryReturnStorage.m_ang; \
+ QAngle *ang = (QAngle*)malloc( sizeof( QAngle ) ); \
+ new ( ang ) QAngle( ((FUNC_TYPE)pFunction)( SCRIPT_BINDING_ARGS_##N ) ); \
+ *pReturn = ang; \
+ pReturn->m_flags |= SV_FREE; \
return true; \
} \
}; \
@@ -452,8 +456,10 @@ inline FUNCPTR_TYPE ScriptConvertFuncPtrFromVoid( void *p )
{ \
return false; \
} \
- new ( &temporaryReturnStorage.m_vec ) Vector( (((OBJECT_TYPE_PTR)(pContext))->*ScriptConvertFuncPtrFromVoid<FUNC_TYPE>(pFunction))( SCRIPT_BINDING_ARGS_##N ) ); \
- *pReturn = temporaryReturnStorage.m_vec; \
+ Vector *vec = (Vector*)malloc( sizeof( Vector ) ); \
+ new ( vec ) Vector( (((OBJECT_TYPE_PTR)(pContext))->*ScriptConvertFuncPtrFromVoid<FUNC_TYPE>(pFunction))( SCRIPT_BINDING_ARGS_##N ) ); \
+ *pReturn = vec; \
+ pReturn->m_flags |= SV_FREE; \
return true; \
} \
}; \
@@ -472,8 +478,10 @@ inline FUNCPTR_TYPE ScriptConvertFuncPtrFromVoid( void *p )
{ \
return false; \
} \
- new ( &temporaryReturnStorage.m_ang ) QAngle( (((OBJECT_TYPE_PTR)(pContext))->*ScriptConvertFuncPtrFromVoid<FUNC_TYPE>(pFunction))( SCRIPT_BINDING_ARGS_##N ) ); \
- *pReturn = temporaryReturnStorage.m_ang; \
+ QAngle *ang = (QAngle*)malloc( sizeof( QAngle ) ); \
+ new ( ang ) QAngle( (((OBJECT_TYPE_PTR)(pContext))->*ScriptConvertFuncPtrFromVoid<FUNC_TYPE>(pFunction))( SCRIPT_BINDING_ARGS_##N ) ); \
+ *pReturn = ang; \
+ pReturn->m_flags |= SV_FREE; \
return true; \
} \
}; \
diff --git a/sp/src/vscript/vscript_squirrel.cpp b/sp/src/vscript/vscript_squirrel.cpp
index 47364e007..153dbe2a4 100644
--- a/sp/src/vscript/vscript_squirrel.cpp
+++ b/sp/src/vscript/vscript_squirrel.cpp
@@ -1428,7 +1428,6 @@ SQInteger function_stub(HSQUIRRELVM vm)
PushVariant(vm, retval);
- Assert(!(retval.m_flags & SV_FREE));
retval.Free();
return pFunc->m_desc.m_ReturnType != FIELD_VOID;Could you test this on your system? Also, just making sure, you are profiling a release-mode build? My results look like (I assume t2 and t3 are the same loop, just more iterations.) I'll test with fully reverting 7efec77 tomorrow. The only sensible explanation I can come up with would be that passing the |
|
That malloc patch is slower in the same way as yours. Even removing the unused I am of course testing on release with no (native) debugger attached, and t2, t3 are more iterations of the same loop. |
Type FIELD_HSCRIPT is removed, it never sets SV_FREE. All dynamic memory is now free`d via free(). For strdup(), this would be the conformant way in standard C++. It matters not in Source, since both use the same global allocator of the engine, but it is tidier. Vector and QAngle are now allocated via malloc() instead of new to provide symmetry.
When assigning a null value in getVariant(), make sure a previous SV_FREE is not carried over. In SqurrelVM::ReleaseValue(), make sure SV_FREE is not kept after Free()ing a value.
Instead of temporary allocating dynamic memory, which subsequently is copied into another dynamically allocated piece of memory, provide a temporary buffer on the stack. The script value inserted into the VM still needs to allocate dynamically though... A few sites are adapted to not create a ScriptVariant_t from a temporary Vector to avoid dynamic allocation there too. cf. mapbase-source#321
They wouldn't even work.
Free values before assignment.
z33ky
force-pushed
the
dangling-vscript-vector
branch
from
45c0490
to
cdb8de8
Compare
|
I got it, it wins by cheating. If you do So it's not slower, but the speedup looks to be within statistical noise... |
|
Its impact was more noticable in my test For reference, testing const ref return (EyePosition) Related, return value should only be pushed to script if there is a return value. Otherwise it's unnecessarily pushing null to stack all the time. diff --git a/sp/src/vscript/vscript_squirrel.cpp b/sp/src/vscript/vscript_squirrel.cpp
index 47364e00..b2a361b1 100644
--- a/sp/src/vscript/vscript_squirrel.cpp
+++ b/sp/src/vscript/vscript_squirrel.cpp
@@ -1426,10 +1426,13 @@ SQInteger function_stub(HSQUIRRELVM vm)
return sq_throwobject(vm);
}
- PushVariant(vm, retval);
+ if ( pFunc->m_desc.m_ReturnType != FIELD_VOID )
+ {
+ PushVariant(vm, retval);
- Assert(!(retval.m_flags & SV_FREE));
- retval.Free();
+ Assert(!(retval.m_flags & SV_FREE));
+ retval.Free();
+ }
return pFunc->m_desc.m_ReturnType != FIELD_VOID;
}And profile of |
Cool :)
That's neat. I'll commit this too then. One more question, I'm seeing the |
|
Yes, retval should be freed on error as well since the error state doesn't change the return type and allocation. No place where RaiseException is used returns any freeable value though.
With the stack allocation of Vector now, when is retval ever going to need freeing, what strings will be allocated? Strings returned from bound functions are owned by the callees, neither |
Sounds fair.
Oh you're right, nothing does. I incorrectly thought the |
|
Those metamethods indeed leak. They are also completely broken, they get the instance itself again instead of the second parameter. Apparently no one tested or used them :P
diff --git a/sp/src/vscript/vscript_squirrel.cpp b/sp/src/vscript/vscript_squirrel.cpp
index 47364e00..fec6a6d1 100644
--- a/sp/src/vscript/vscript_squirrel.cpp
+++ b/sp/src/vscript/vscript_squirrel.cpp
@@ -1586,7 +1586,7 @@ SQInteger add_stub(HSQUIRRELVM vm)
sq_getinstanceup(vm, 1, (SQUserPointer*)&classInstanceData, 0);
ScriptVariant_t var;
- getVariant( vm, 1, var );
+ getVariant( vm, -1, var );
if (classInstanceData &&
classInstanceData->instance &&
@@ -1611,7 +1611,7 @@ SQInteger sub_stub(HSQUIRRELVM vm)
sq_getinstanceup(vm, 1, (SQUserPointer*)&classInstanceData, 0);
ScriptVariant_t var;
- getVariant( vm, 1, var );
+ getVariant( vm, -1, var );
if (classInstanceData &&
classInstanceData->instance &&
@@ -1636,7 +1636,7 @@ SQInteger mul_stub(HSQUIRRELVM vm)
sq_getinstanceup(vm, 1, (SQUserPointer*)&classInstanceData, 0);
ScriptVariant_t var;
- getVariant( vm, 1, var );
+ getVariant( vm, -1, var );
if (classInstanceData &&
classInstanceData->instance &&
@@ -1661,7 +1661,7 @@ SQInteger div_stub(HSQUIRRELVM vm)
sq_getinstanceup(vm, 1, (SQUserPointer*)&classInstanceData, 0);
ScriptVariant_t var;
- getVariant( vm, 1, var );
+ getVariant( vm, -1, var );
if (classInstanceData &&
classInstanceData->instance && |
|
|
When a Vector or QAngle rvalue reference is returned from a VScript function, the constructed ScriptVariant_t must not store the pointer to it since it is, per convention, a temporary reference. Only do that for lvalue-references, but do a copy when constructing from or assigning a rvalue reference.
I'm not sure
ScriptVairant_treally needs to try and prevent copies ofVectorandQAngle. They're pretty small and trivial classes, so are cheap to copy. When a copy must be made it surely beats a dynamic allocation & deallocation.Free()refers toFIELD_HSCRIPT, although it will never setSV_FREE. Perhaps this is just outdated?Strings will need to be copied (
strdup()ed) due to their dynamic length, but it could try to do a small-string optimization.The types of
newanddeleteare misaligned, sinceFree()only ever doesdelete m_pszString. SinceHSCRIPT,VectorandQAngleare all trivial to destruct this shouldn't have any ill effects.At most I'm worried about
strdup()being paired withdeleteinstead offree().Anyway, I wanted to hear your thoughts on this before I change more stuff. This change alone sufficiently fixes issues I've observed in return-values playing a Mapbase-based mod that uses VScripts (https://www.moddb.com/mods/sourceworld).
Does this PR close any issues?
PR Checklist
developbranch OR targets another branch with a specific goal in mind