Stale files in structures/ and modules/dfhack.lua

[ChrisJohnsen]

Thank you for putting this type definition library together. The autocomplete functionality it enables is quite handy!

---

The Great Reorganization restructured much of DFHack's DF type library. Many types are now defined in new XML files and many XML files were renamed. The update-definitions workflow doesn't start from a blank slate, so definitions that were extracted from files that no longer exist are still included in the published dist/library.

I cleared out the dist/library subdirectories (rm -r dist/library/*/*) and regenerated. That cleaned up the obsolete dist/library/structures files, but it also uncovered a few other "stale" items:

• dist/library/lua/plugins/rename.lua
  • plugins/rename.cpp was recently deleted
  • This one isn't super important since no types were actually generated from rename.cpp, but it is probably worth clearing out.
• dist/library/modules/console.lua
  • I went back to the file's origin commit (8261c29), but failed to reproduce its generation.
  • It seems like it has always been explicitly ignored (ignored_modules)?
• dist/library/modules/dfhack.lua
  • This one was generable at 8261c29, but not on current main. A manual bisect points to c5de45d as the commit that stopped generating this file.

Possible fix for dfhack.lua (re-enable capture of dfhack_module and dfhack_funcs; diff against 2be8ed0):

diff --git a/lib/cpp.rb b/lib/cpp.rb
index 12811b8..618ea64 100644
--- a/lib/cpp.rb
+++ b/lib/cpp.rb
@@ -29,7 +29,9 @@ module DFHackLuaDefinitions
         file = File.read(entry_point)
         directory = File.dirname(entry_point)
 
-        modules = file.scan(/^static.*dfhack_(.*)(?:_funcs|_module)\[\][\s\S]+?};/).flatten.uniq.reject do |name|
+        modules = file.scan(/^static.*(dfhack(?:_.+)?)(?:_funcs|_module)\[\][\s\S]+?};/).flatten.uniq.map do |name|
+          name.sub /^dfhack_/, ''
+        end.reject do |name|
           ignored_modules.include? name
         end
 
@@ -48,7 +50,7 @@ module DFHackLuaDefinitions
             output << "---@meta\n\n"
 
             prefix = module_name == 'dfhack' ? '' : 'dfhack.'
-            namespace = is_module ? "#{module_name.capitalize}::" : ''
+            namespace = is_module && module_name != 'dfhack' ? "#{module_name.capitalize}::" : ''
 
             output << "---@class #{prefix}#{module_name}\n"
 

With dfhack_modules and dfhack_funcs back in action, everything in subdirs except console.lua seems to generate correctly. Would it make sense to move console.lua to be next to the "manual" files (i.e., directly in dist/library, not in a subdir), so that everything in subdirs could be regularly deleted in update-definitions (or rake build)?

[vallode]

Thank you for the great write up! I've been out of the loop for a few months so I'll have to catch up but it looks like you did most of the work for me here 😄 I appreciate that!

I'll make it a point to action everything here in the next few days.

• update-definitions should completely re-generate the definitions library if changes are detected, to avoid stale files as mentioned above
• Investigate console.lua, I can't remember any details around why it was ignored. Regardless it should be noted as manually written and moved away from the automatically generated files
• Commit the fix for dfhack.lua module

Again, I really appreciate the in-depth issue!

[ChrisJohnsen]

👍 No rush from me! Just some things to consider the next time you have a chance to work on this stuff.