Preparation of version 1.0 release

README.md

@@ -1,13 +1,181 @@
-Qt plugins for DFHack
-=====================
+DwarfExplorer
+=============
+A DFHack plugin for browsing Dwarf Fortress internal structures (Windows/Linux).
 
-DFHack plugins for using Qt widgets.
+![](images/dwarf_explorer.png)
 
-How to build
-------------
+*NOTE:* Linux version is done but it will be out in a few weeks.
 
-Copy/link/clone this directory as a subdirectory in dfhack's `plugins`
-directory.  Add the subdirectory (e.g. a `add_subdirectory(qt)` line) in
-`plugins/CMakeLists.custom.txt`. Then follow DFHack instructions for building
-plugins.
+Features
+--------
 
+* Graphical viewer for all Dwarf Fortress structures known from DFHack.
+* Open any individual structure in a new window for easy understanding.
+* All structure types supported (vectors, arrays, pointers, enums, unions, bitfields, etc).
+* Hexadecimal memory viewer for Dwarf Fortress process memory.
+* Show comments and 'refers-to' (if defined) attributes for fields in structures.
+* Show the dfhack xml file where each structure is defined.
+* Stores opened df structures so you can track evolution of the structures in time (rather primitive right now).
+* Automatic decoding of enums, bitfields, coordinates and language names.
+* Decoding of derived classes showing its hyerarachy.
+
+
+
+
+Motivation
+----------
+We all know gm-editor, the wonderful in game browser invaluable for researching Dwarf Fortress internal structures. Dwarf Explorer does something similar with the following advantages:
+
+* A treeview allows a better understanding of a structure. You can expand/compact fields as you wish.<br>
+gm-editor changes to a new screen each time you move down in the structure.
+* DFHack has a hex memory viewer, but is a command line one. Dwarf Explorer allows you to open a hex memory viewer for each field of a df structure and even follow pointers.
+
+Dwarf Explorer has also some disavantadges:
+* Right now you can't change values in Dwarf Fortress memory.
+  This is something that will change in future versions of the plugin.<br>
+  Use gm-editor for that task.
+
+
+
+Qt DFHack plugins, what sorcery is this?
+----------------------------------------
+You need to thanks Clement. Yes that Clement, the one that maintains Dwarf Therapist.
+
+He found a method for loading Qt inside DFHack, so you can create multiplatform graphical applications running in the same process as Dwarf Fortress.
+
+Think about that. You get access to all the abilities that DFHack provides for plugins, and you can show all that information graphically!
+
+
+
+How do I install the plugin?
+----------------------
+
+First, download the plugin from github. You need to download the plugin for the dfhack version that you have installed in your computer.
+
+You will get four files:
+* `dwarfexplorer.plug.dll` (.so in Linux).
+* `Qt5Core.dll`
+* `Qt5Gui.dll`
+* `Qt5Widgets.dll`
+
+The first one is the dfhack plugin and you need to put in the folder `/hack/plugins` of your Dwarf Fortress directory.
+
+The other ones are Qt dynamic libraries that need to be installed in your Dwarf Fortress directory.
+
+How do I run the plugin?
+----------------------
+
+<b>Dwarf Explorer only works in PAUSED games</b>.
+
+There's no way that the plugin can track all the data that changes in Dwarf Fortress in real time, so the plugin automatically pauses the game, gets the data from DF and shows it.
+
+Launch Dwarf Fortress (and DFHack of course) and load your world.
+In the dfhack console type:
+
+`enable qapplication`
+
+This loads the Qt dynamic libraries into Dwarf Fortress process.
+
+After that, launch the graphical viewer typing
+
+`dwarfexplorer`
+
+The plugin window will be displayed without any data, just telling you that Dwarf Fortress is running.
+
+![](images/1.gif)
+
+
+You need to pause Dwarf Fortress using the Suspend button in the toolbar. When you do this, Dwarf Fortress will pause and the plugin window will display all the known global variables (aka df.global).
+
+![](images/2.gif)
+
+You can now browse all the data, view memory, open new windows, etc. When you are done, press the resume button in the toolbar and continue playing.
+
+![](images/6.gif)
+
+Where's the data?
+----------------------
+The most important global variable is world (df.global.world). This structure has all the data that your fortress uses. So, navigate to the bottom of the windows and locate world.
+
+Then expand the tree and marvel about all the glorious Dwarf Fortress data.
+
+If your tree gets convoluted with too much data, you can also open any structure in a new window by selecting it and using the menu `Window->Open in new window`.
+
+![](images/3.gif)
+
+Some DF data examples
+----------------------
+
+* Where are my dwarves?<br>
+  They are in the vector df.global.world.units.active
+
+* How much vermin exists in my embark?<br>
+  Check vector df.global.world.vermin.all.
+
+* What about the fortress buildings?<br>
+  vector df.global.world.buildings.all
+
+* Where is the rest of the world data?<br>
+  It's in the pointer df.global.world.world_data
+
+* What's the tallest mountain in the world?<br>
+  Look for the field height in df.global.world.world_data.mountain_peaks vector
+
+The memory viewer
+----------------------  
+As you browse Dwarf Fortress data, you will encounter a lot of unknow structures (data where we know the C++ type of the field but we don't know its meaning related to the game).
+
+For every field or df structure, you can always see the Dwarf Fortress process memory where this data resides. To do that, select the field and click in the menu `Window->Open address in hex viewer`.
+
+A memory viewer will open at the address where the field is stored.
+
+![](images/4.gif)
+
+For some C++ structures like vectors or pointers, the data has a another level of indirection. 
+
+If you open a memory viewer for a vector you will not get the real data. This is because the internal structure of a vector consists of three pointers, being the first pointer the one that points to the data beginning.
+
+To do this operation easier, just select `Window->Open destination address in hex viewer` and you will get the hex memory viewer pointing to the real data.
+
+![](images/5.gif)
+
+Does it run with the LNP (Lazy Newcomers Pack)?
+----------------------------------------------
+Yes as long as your LNP version matches the plugin version.
+
+If the plugin is compiled for DFHack 0.44.12, your LNP must be 0.44.12 also.
+
+No MacOS?
+----------------------------------------------
+I've zero knowledge of Macs. In theory the code should be portable across operating systems (Qt strenght). But it seems that there are problems in that platform.
+
+What happens when Dwarf Fortress / DFHack changes?
+----------------------
+The plugin code is generated automatically from DFHack `codegen.out.xml` file using a tool.<br>
+Each time that a new DFHack version will be released, I'll update the plugin accordingly.
+
+Bugs, suggestion, etc
+----------------------
+You can use github issues or contact me by irc in `#dfhack` in freenode.
+
+Closing thoughts
+----------------------
+
+*The real *FUN* is not in your fortress but in the code!!!!*
+
+
+I hope this tool will encourage more people to do researching in Dwarf Fortress. It'a fascinating puzzle ready to be assembled.
+You can try to find relationship between known and unknown data, discover new structures or fields, etc. Then, open a pull request in https://github.com/DFHack/df-structures
+for providing that valuable info to the community.
+
+When you starts digging into Dwarf Fortress internals, you wil be hooked. You can even try to advance to the next level and do some reverse engineering of Dwarf Fortress code using IDA or Ghidra.
+
+
+
+
+
+Thanks
+----------------------
+Clement (https://github.com/cvuchener) for providing the wonderful framework in which this plugin runs.
+
+lethosor (https://github.com/lethosor) and all the wonderful people of `#dfhack` channel in freenode.

dwarfexplorer/CMakeLists.txt

@@ -21,6 +21,7 @@ set(DWARFEXPLORER_SOURCES
     EventProxy.cpp
     MainWindow.cpp
     dfstructure_window.cpp
+    hexviewer_window.cpp
     QtModel/df_model.cpp
     QtModel/df_model_data_from_structure.cpp
     QtModel/df_model_data_from_type.cpp
@@ -30,6 +31,7 @@ set(DWARFEXPLORER_SOURCES
     QtModel/df_model_pointer.cpp
     QtModel/df_model_util.cpp
     QtModel/df_model_df_array.cpp
+    QtModel/df_proxy_model.cpp
     generated/offsets_cache.cpp
     generated/fill_nodes.cpp
     generated/fill_offsets.cpp
@@ -59,7 +61,9 @@ qt5_wrap_cpp(DWARFEXPLORER_MOC_SOURCES
     EventProxy.h
     MainWindow.h
     df_model.h
+    QtModel/df_proxy_model.h
     dfstructure_window.h
+    hexviewer_window.h
     QHexView/document/commands/hexcommand.h
     QHexView/document/commands/insertcommand.h
     QHexView/document/commands/removecommand.h
@@ -82,6 +86,6 @@ qt5_wrap_ui(DWARFEXPLORER_UI_SOURCES
 
 
 DFHACK_PLUGIN(dwarfexplorer
-    ${DWARFEXPLORER_SOURCES} ${DWARFEXPLORER_MOC_SOURCES} ${DWARFEXPLORER_UI_SOURCES}
+    ${DWARFEXPLORER_SOURCES} ${DWARFEXPLORER_MOC_SOURCES} ${DWARFEXPLORER_UI_SOURCES} ${RSCS}
     LINK_LIBRARIES Qt5::Core Qt5::Widgets)
 target_include_directories(dwarfexplorer PRIVATE ${CMAKE_CURRENT_BINARY_DIR})

dwarfexplorer/MainWindow.cpp

@@ -149,9 +149,10 @@ void MainWindow::on_suspend_action_triggered()
 //
 void MainWindow::on_resume_action_triggered()
 {
-    for (DFStructure_Window* child_window: m_child_window_list)
-        if (child_window != nullptr)
-            child_window->set_outdated();
+//    for (DFStructure_Window* child_window: m_child_window_list)
+//        if (child_window != nullptr)
+//            child_window->set_outdated();
+    emit resumed_signal();
 
     ui->stackedWidget->setCurrentIndex(0);
     ui->resume_action->setEnabled(false);
@@ -165,24 +166,6 @@ void MainWindow::on_resume_action_triggered()
     m_suspended = false;
 }
 
-//
-//---------------------------------------------------------------------------------------
-//
-void MainWindow::remove_child_window(DFStructure_Window *p_child_window)
-{
-    for (auto i = 0; i < m_child_window_list.size(); i++)
-        if (m_child_window_list[i] == p_child_window)
-            m_child_window_list[i] = nullptr;
-}
-
-//
-//---------------------------------------------------------------------------------------
-//
-void MainWindow::add_child_window(DFStructure_Window *p_child_window)
-{
-    m_child_window_list.push_back(p_child_window);
-}
-
 //
 //---------------------------------------------------------------------------------------
 //
@@ -264,7 +247,7 @@ void MainWindow::on_actionOpen_in_new_window_triggered()
     new_window_name.append(n_root->m_path);
 
     // Add the window to the list of child windows
-    m_child_window_list.push_back(new_window);
+    //m_child_window_list.push_back(new_window);
 
     // Show the window
     new_window->setWindowTitle(QString::fromStdString(new_window_name));
@@ -292,24 +275,13 @@ void MainWindow::on_actionOpen_in_hex_viewer_triggered()
     // Get the selected node
     rdf::NodeBase* node = dynamic_cast<rdf::NodeBase*>(model->nodeFromIndex(selected_node));
 
-    auto the_data = reinterpret_cast<char*>(node->m_address);
-    auto the_size = std::min(std::size_t(4096), size_of_DF_Type(node->m_df_type));
-    the_size = std::max(std::size_t(4096), the_size);
-
-    QHexDocument* document = QHexDocument::fromMemory<QMemoryBuffer>(the_data, the_size);
-    QHexView* hexview = new QHexView();
-    hexview->setDocument(document);
-    hexview->setReadOnly(true);
-    document->setBaseAddress(node->m_address);
-    //QHexViewer_Window* hex_window = new QHexViewer_Window(this);
-
-    //QHexView* hex_view = hex_window->get_hexview();
-    //hex_view->set_base_address(node->m_address);
-    //hex_view->setData(new QHexView::DataStorageArray(data));
+    uint64_t the_address = node->m_address;
+    auto     the_data    = reinterpret_cast<char*>(node->m_address);
+    auto     the_size    = std::min(std::size_t(4096), size_of_DF_Type(node->m_df_type));
+    the_size             = std::max(std::size_t(4096), the_size);
 
-    // window title
-    auto the_number = QString::fromStdString(to_hex(node->m_address));
-    hexview->setWindowTitle("Memory viewer - " + the_number);
+    // Create the window and show it
+    auto hexview = new QHexViewer_Window(this, the_address, the_data, the_size);
     hexview->show();
 }
 
@@ -335,21 +307,14 @@ void MainWindow::on_actionOpenPointer_in_hex_viewer_triggered()
     rdf::NodeBase* node_base = model->nodeFromIndex(selected_node);
 
     uint64_t* pointer_address = reinterpret_cast<uint64_t*>(node_base->m_address);
-    uint64_t  item_address    = *pointer_address;
+    uint64_t  the_address     = *pointer_address;
 
-    auto the_data = reinterpret_cast<char*>(item_address);
+    auto the_data = reinterpret_cast<char*>(the_address);
     auto the_size = std::min(std::size_t(4096), size_of_DF_Type(node_base->m_df_type));
     the_size = std::max(std::size_t(4096), the_size);
-    QHexDocument* document = QHexDocument::fromMemory<QMemoryBuffer>(the_data, the_size);
-    document->setBaseAddress(item_address);
 
-    QHexView* hexview = new QHexView();
-    hexview->setDocument(document);
-    hexview->setReadOnly(true);
-
-    // Window tittle
-    auto the_number = QString::fromStdString(to_hex(item_address));
-    hexview->setWindowTitle("Memory viewer - " + the_number);
+    // Create the window and show it
+    auto hexview = new QHexViewer_Window(this, the_address, the_data, the_size);
     hexview->show();
 }
 
@@ -364,3 +329,7 @@ void MainWindow::closeEvent(QCloseEvent* p_event)
     // Do the thing
     p_event->accept();
 }
+
+//void MainWindow::resumed_signal()
+//{
+//}

dwarfexplorer/MainWindow.h

@@ -24,16 +24,23 @@
 #include <QMainWindow>
 
 #include <Core.h>
-
+#include <QCloseEvent>
 #include <QSortFilterProxyModel>
 
 #include <memory>
 #include <utility>
+#include <vector>
 #include "df_model.h"
+#include "QtModel/df_proxy_model.h"
 
     class EventProxy;
 
-    namespace Ui { class MainWindow; }
+    namespace Ui
+    {
+        class MainWindow;
+    }
+
+    class DFStructure_Window;
 
     class MainWindow: public QMainWindow
     {
@@ -42,20 +49,26 @@
         explicit MainWindow(std::shared_ptr<EventProxy> &&proxy, QWidget *parent = nullptr);
         ~MainWindow() override;
 
+    signals:
+        void resumed_signal();
+
     private slots:
         void on_suspend_action_triggered();
         void on_resume_action_triggered();
-
-        void updateUnitModel();
-        void clearUnitModel();
         void on_treeView_expanded(const QModelIndex& p_index);
         void on_actionOpen_in_new_window_triggered();
         void on_actionOpen_in_hex_viewer_triggered();
         void on_actionOpenPointer_in_hex_viewer_triggered();
+        void on_filter_textChanged(const QString &arg1);
+    protected:
+        void closeEvent(QCloseEvent* p_close_event);
     private:
-        std::unique_ptr<Ui::MainWindow> ui;
-        std::shared_ptr<EventProxy> event_proxy;
-        DF_Model* model;
+        std::unique_ptr<Ui::MainWindow>        ui;
+        std::shared_ptr<EventProxy>            event_proxy;
+        std::unique_ptr<DF_Model>              m_model;
+        std::unique_ptr<QSortFilterProxyModel> m_proxy_model;
+        DFHack::CoreSuspender*                 m_core_suspender;
+        bool                                   m_suspended;
     };
 
 #endif