Add tileable RR Graph #3134

amin1377 · 2025-06-11T18:14:24Z

Merging OpenFPGA branch into master branch. PR #2135 explains features of OpenFPGA.

…le info

…rilog-to-routing into add_tileable_rr_graph

amin1377 · 2025-06-19T22:37:00Z

This PR is ready for your review. Since you're familiar with the RR Graph code, it would be great if you could take a look at the tileable RR Graph implementation.

…rilog-to-routing into add_tileable_rr_graph

AlexandreSinger

Hi @amin1377 thanks for bringing this in! I recognize not all of this is your code. Overall the code is well structured however it needs some code style and data structure cleanup so it can fit in better with the rest of the VTR flow.

Some. of the data structure changes can be made into issues; however, the coding style things should probably be fixed now.

doc/src/arch/reference.rst

doc/src/vpr/VIB.rst

vpr/src/route/rr_graph_generation/tileable_rr_graph/tileable_rr_graph_gsb.cpp

vpr/src/route/rr_graph_generation/tileable_rr_graph/tileable_rr_graph_node_builder.h

vpr/src/route/rr_graph_generation/tileable_rr_graph/tileable_rr_graph_node_builder.cpp

amin1377 · 2025-06-23T15:06:17Z

QoR comparison:
Titan: Link
Large VTR: Link

…rilog-to-routing into add_tileable_rr_graph

… code

amin1377 · 2025-06-24T13:29:02Z

Thanks @AlexandreSinger; reviewing this amount of code is no small task, and I really appreciate your time. I've addressed your comments. Regarding your suggestions about replacing the vector of vectors with VTR data structures: while I agree with them in principle, I didn’t apply all of them since this part of the code is not performance-critical.

I think the code is now ready for the next round of reviews. I’ve also added @AmirhosseinPoolad to help with it.

soheilshahrouz

Thanks for this PR.
Since it's a large one, I’ve reviewed some files for now and will go through the rest soon.

libs/libarchfpga/src/physical_types.h

libs/libarchfpga/src/arch_util.cpp

libs/librrgraph/src/base/get_parallel_segs.h

libs/librrgraph/src/base/get_parallel_segs.cpp

libs/librrgraph/src/base/rr_graph_storage.h

libs/librrgraph/src/base/rr_graph_storage.cpp

libs/libarchfpga/src/read_xml_arch_file.cpp

…rilog-to-routing into add_tileable_rr_graph

amin1377 · 2025-06-25T21:31:09Z

@soheilshahrouz: Thank you for taking the time to review this PR. I’ve addressed all your comments, and I think this PR is now ready for the next round of review.

AlexandreSinger

Hi Amin, thanks for fixing my prior comments. The reward for good work is more work my friend. I am just kidding; I gave this another pass focusing more on the documentation since I only glanced over it in my last review.

AlexandreSinger · 2025-06-27T01:54:37Z

doc/src/VIB.rst

+
+VIB Architecture
+============
+The VIB architecture adds modeling support for double-level MUX topology and bent wires. In past, switch blocks have only one level of routing MUXes, whose inputs are driven by outputs of programmable blocks and routing tracks. Now outputs of programmable blocks can shape the first level of routing MUXes, while the inputs of second level involves the outputs of first level and other routing tracks. This can reduce the number and input sizes of routing MUXes.


"In past" -> "In the past"
"switch blocks have" -> "switch blocks had"
"whose inputs are driven" -> "whose inputs were driven"

AlexandreSinger · 2025-06-27T01:55:55Z

doc/src/VIB.rst

+============
+The VIB architecture adds modeling support for double-level MUX topology and bent wires. In past, switch blocks have only one level of routing MUXes, whose inputs are driven by outputs of programmable blocks and routing tracks. Now outputs of programmable blocks can shape the first level of routing MUXes, while the inputs of second level involves the outputs of first level and other routing tracks. This can reduce the number and input sizes of routing MUXes.
+
+Figure 1 shows the proposed VIB architecture which is tile-based. Each tile is composed of a CLB and a VIB. Each CLB can interact with the corresponding VIB which contains all the routing programmable switches in one tile. Figure 2 shows an example of the detailed interconnect architecture in VIB. The CLB input muxes and the driving muxes of wire segments can share the same fanins. A routing path of a net with two sinks is presented red in the Figure.


I think it would be a good idea to spell out what the acronym "VIB" means in this context. "Versatile Interconnect Block (VIB)" or something just so its written down somewhere on this page.

AlexandreSinger · 2025-06-27T01:57:57Z

doc/src/VIB.rst

+
+FPGA Architecture File Modification (.xml)
+--------------------------
+For original tags of FPGA architecture file see :ref:`fpga_architecture_description`.


I suggest: "For the original tags available for the standard FPGA architecture file see"

I feel like it can get confusing with all the architecture file words being thrown around.

AlexandreSinger · 2025-06-27T01:59:01Z

doc/src/VIB.rst

+    For example, a length 4 wire has a bent pattern of ``- - U``.
+    A ``-`` indicates no bent at this position and a ``U`` indicates a conterclockwise bent at the position. (``D`` indicates a clockwise bent.)
+
+    .. note:: A bent wire should remain consistent in both the x and y axes.


What does consistent mean in this context? If you know it may be a good idea to expand on this if this is important.

AlexandreSinger · 2025-06-27T02:00:03Z

doc/src/VIB.rst

+.. arch:tag:: <multistage_muxs>content</multistage_muxs>
+
+    :req_param content:
+    	The detaild information for first and second MUXes.


"detaild" -> "detailed"

AlexandreSinger · 2025-06-27T02:26:59Z

doc/src/arch/reference.rst

+    <direct name="scff_chain" from_pin="clb.sc_out" to_pin="clb.sc_in" x_offset="0" y_offset="-1" z_offset="0"/>
+  </directlist>
+
+In OpenFPGA architecture:


"In a tileable architecture:"

AlexandreSinger · 2025-06-27T02:28:36Z

doc/src/arch/reference.rst

+
+In this figure, the red arrows represent the initial direct connection. The green arrows represent the point to point connection to connect all the columns of CLB.
+
+A point to point connection can be applied in different ways than showed in the example section. To help the designer implement his point to point connection, a truth table with our new parameters id provided below.


"implement his point to point connection" -> "implement their point to point connection"

Anyone can be designers 😄

"a truth table with our new parameters id provided below". Something is off about this sentence.

AlexandreSinger · 2025-06-27T02:34:09Z

libs/libarchfpga/src/read_xml_arch_file_vib.h

@@ -0,0 +1,11 @@
+#include <vector>


This file needs a pragma once.

Explain what this file is for (VIB-specific architecture file reading code. And a VIB is ....)

AlexandreSinger · 2025-06-27T02:41:13Z

libs/librrgraph/src/base/rr_graph_view.h

+        return node_storage_.edge_sink_node(edge);
+    }
+
+    /** @brief Get the source node for the iedge'th edge from specified RRNodeId.


Missing a return-carriage just before the "@brief"

AlexandreSinger · 2025-06-27T02:54:55Z

vpr/src/base/setup_vib_grid.cpp

+
+    if (priority < max_priority_type_loc.priority) {
+        //Lower priority, do not override
+#ifdef VERBOSE


I am wondering if using VTR_LOG_DEBUG would be more appropriate and easier to work with.

soheilshahrouz · 2025-06-27T20:20:32Z

libs/libarchfpga/src/arch_util.cpp

            InstPort in_pins(annot_in_str);
-            for (const auto& annot_out_str : vtr::split(annotation.output_pins)) {
+            for (const auto& annot_out_str : vtr::StringToken(annotation.output_pins).split(" \t\n")) {


soheilshahrouz · 2025-06-27T20:21:13Z

libs/libarchfpga/src/parse_switchblocks.cpp

@@ -201,7 +201,7 @@ t_wire_switchpoints parse_wireconn_from_to_node(pugi::xml_node node, const pugiu
    wire_switchpoints.segment_name = get_attribute(node, "type", loc_data).value();

    auto points_str = get_attribute(node, "switchpoint", loc_data).value();
-    for (const auto& point_str : vtr::split(points_str, ",")) {
+    for (const auto& point_str : vtr::StringToken(points_str).split(",")) {


auto in lines 203 and 204

soheilshahrouz · 2025-06-27T20:21:21Z

libs/libarchfpga/src/parse_switchblocks.cpp

@@ -229,7 +229,7 @@ static void parse_switchpoint_order(const char* order, SwitchPointOrder& switchp
 /* parses the wire types specified in the comma-separated 'ch' char array into the vector wire_points_vec.
 * Spaces are trimmed off */
 static void parse_comma_separated_wire_types(const char* ch, std::vector<t_wire_switchpoints>& wire_switchpoints) {
-    auto types = vtr::split(ch, ",");
+    auto types = vtr::StringToken(ch).split(",");


soheilshahrouz · 2025-06-27T20:21:30Z

libs/libarchfpga/src/parse_switchblocks.cpp

@@ -245,7 +245,7 @@ static void parse_comma_separated_wire_types(const char* ch, std::vector<t_wire_

 /* parses the wirepoints specified in the comma-separated 'ch' char array into the vector wire_points_vec */
 static void parse_comma_separated_wire_points(const char* ch, std::vector<t_wire_switchpoints>& wire_switchpoints) {
-    auto points = vtr::split(ch, ",");
+    auto points = vtr::StringToken(ch).split(",");


soheilshahrouz · 2025-06-27T20:22:05Z

libs/libarchfpga/src/physical_types.h

@@ -2262,7 +2282,7 @@ struct t_noc_inf {
    std::string noc_router_tile_name;
 };

-/*   Detailed routing architecture */
+/* Detailed routing architecture */


replace with // or write doxygen comment

soheilshahrouz · 2025-06-27T21:17:55Z

libs/libvtrutil/src/vtr_util.cpp

+
+/* Split the string using a given delim */
+std::vector<std::string> StringToken::split(const std::string& delims) const {
+    /* Return vector */


block comments
several instacnes in this file

soheilshahrouz · 2025-06-27T21:18:43Z

libs/libvtrutil/src/vtr_util.cpp

+    return split(delims);
+}
+
+/* Split the string using a given delim */


can be moved to the header file as doxygen comment for the method
several instances

soheilshahrouz · 2025-06-27T21:20:13Z

utils/fasm/src/fasm.cpp

@@ -782,7 +782,7 @@ void FasmWriterVisitor::output_fasm_mux(std::string_view fasm_mux_str,
      bool root_level_connection = interconnect->parent_mode->parent_pb_type ==
          mux_input_pin->parent_node->pb_type;

-      auto fasm_features_str = vtr::join(vtr::split(mux_parts[1], ","), "\n");
+      auto fasm_features_str = vtr::join(vtr::StringToken(mux_parts[1]).split(","), "\n");


auto
a few instances in this file

soheilshahrouz · 2025-06-27T21:20:47Z

vpr/src/base/place_and_route.cpp

@@ -84,6 +84,10 @@ int binary_search_place_and_route(const Netlist<>& placement_net_list,
        graph_directionality = e_graph_type::BIDIR;
    } else {
        graph_type = (det_routing_arch.directionality == BI_DIRECTIONAL ? e_graph_type::BIDIR : e_graph_type::UNIDIR);
+        /* Branch on tileable routing */


soheilshahrouz · 2025-06-27T21:22:32Z

vpr/src/base/setup_clocks.cpp

@@ -136,7 +136,7 @@ void setup_clock_connections(const t_arch& Arch, FormulaParser& p) {
            if (RoutingToClockConnection* routing_to_clock = dynamic_cast<RoutingToClockConnection*>(clock_connections_device.back().get())) {
                //TODO: Add error check to check that clock name and tap name exist and that only
                //      two names are returned by the below function
-                auto names = vtr::split(clock_connection_arch.to, ".");
+                auto names = vtr::StringToken(clock_connection_arch.to).split(".");


auto
a few cases

vaughnbetz

Done reviewing up to the start of rr_spatial_lookup (more to do).
Lots of work here, thanks Amin!

Big picture comments:
Memory: don't allocate things we don't need to (bends, ptc_num array). Instead, allocate only if tileable is on, and add checking to the get/set functions for these.
Clarity: ptc_nums is confusing. Need to give it a different name to avoid confusion (can't have two ptc_nums). I suggest tileable_track_num or something like that.
bend_start / bend_end are mysterious. Need to completely explain what they are returning.

Various other comments in the text.
QoR: looks mostly good, but Titan routing slowed down 3%. May be due to the memory footprint --> making the allocation changes above will hopefully fix.
Longer term: doesn't have to be in this PR, but right afterwards we should get rid of the tiling data we added earlier to the RRgraph in an attempt to unify a bit of OpenFPGA and VTR code. Clearly we won't use that, so storing something of O(rr_nodes) is not a good idea when no one uses it.

vaughnbetz · 2025-06-27T23:38:44Z

libs/libarchfpga/src/physical_types.h

@@ -2272,12 +2292,43 @@ struct t_arch {
    /// Secure hash digest of the architecture file to uniquely identify this architecture
    char* architecture_id;

+    // Options for tileable routing architectures


You need to explain what tileable routing architectures are / what this means to the code. I.e., these are used for an alternative, tilable, rr-graph generator that can produce OpenFPGA-compatible FPGAs that can be implemented to silicon via the OpenFPGA flow. This seems like it might be a logical place for such a high-level comment and you have to have one like that somewhere.

vaughnbetz · 2025-06-27T23:39:23Z

libs/libarchfpga/src/physical_types.h

    std::vector<t_switchblock_inf> switchblocks;
    float R_minW_nmos;
    float R_minW_pmos;
    int Fs;
+    int sub_fs;


Comment what this is. Comment that it is only valid with (VIB? tileable?)

vaughnbetz · 2025-06-27T23:40:35Z

libs/libarchfpga/src/physical_types.h

    std::string device_layout;

+    /// VIB grid layouts


Needs a more extensive comment, or a pointer to one. What is a VIB? Why does it have a different layout?
(The current comment pretty much removes _ from the variable name, which is not very helpful).

vaughnbetz · 2025-06-27T23:42:01Z

libs/libarchfpga/src/physical_types.h

    t_clock_arch_spec clock_arch; // Clock related data types

    /// Stores NoC-related architectural information when there is an embedded NoC
    t_noc_inf* noc = nullptr;
+
+    // added for vib


Need a high-level comment on VIB. What are vib_infs? Also, if you're going to add various VIB data I suggest putting it all together (group the variables together) and put a high-level comment on what VIB is above it.

Explain the individual variables as much as you can, but at the least group them and document as a group as I listed above.

vaughnbetz · 2025-06-27T23:48:30Z

libs/libarchfpga/src/read_xml_arch_file.cpp

+     * Note that sub_type does not support custom switch block pattern!!!
+     * If 'sub_type' is specified, the custom switch block for 'type' is not allowed! 
+     */
+    std::string sub_type_str = get_attribute(Cur, "sub_type", loc_data, BoolToReqOpt(false)).as_string("");


I think it would be good to move the sub_type and sub_fs processing into a subroutine, to keep this routine from getting to long and to help localize the extra tilable rr-graph code.

If that routine could go in the vib-specific reader file that would be even better (to localize the VIB code).

vaughnbetz · 2025-06-28T00:59:24Z

libs/librrgraph/src/base/rr_graph_view.h

@@ -222,6 +229,24 @@ class RRGraphView {
    inline short node_layer(RRNodeId node) const {
        return node_storage_.node_layer(node);
    }
+
+    /** 
+     * @brief Return the bend start of a specified node.


Again, need to really define the use and the return value meaning.

vaughnbetz · 2025-06-28T01:01:33Z

libs/librrgraph/src/base/rr_graph_view.h

+
+    /** 
+     * @brief Return incoming edges for a given routing resource node 
+     * Require build_in_edges() to be called first


Require -> Requires.
Also all 3 functions below require build_in_edges() to be called first, so that should be commented on each.

vaughnbetz · 2025-06-28T01:02:11Z

libs/librrgraph/src/base/rr_graph_view.h

+    /** @brief Validate if all the fan-in edge lists are valid */
+    bool validate_in_edges() const;
+    /** @brief Count the number of incoming edges for all the nodes */
+    size_t in_edges_count() const;


I need to call the build_in_edges function first for these?

vaughnbetz · 2025-06-28T01:02:36Z

libs/librrgraph/src/base/rr_graph_view.h

+
+    /** A list of incoming edges for each routing resource node. This can be built optionally, as required by applications.
+     *  By default, it is empty! Call build_in_edges() to construct it!!! */
+    const vtr::vector<RRNodeId, std::vector<RREdgeId>>& node_in_edges_;


Remove ! and !!

vaughnbetz · 2025-06-28T01:04:00Z

libs/librrgraph/src/base/rr_graph_view.h

+     *  By default, it is empty! Call build_in_edges() to construct it!!! */
+    const vtr::vector<RRNodeId, std::vector<RREdgeId>>& node_in_edges_;
+
+    /** A list of extra ptc numbers for each routing resource node. See details in RRGraphBuilder class */


Rename to node_tileable_track_nums.
Say this is only loaded for tilable rr-graphs.
Change implementation to check if it is loaded before set/get operations.

Longer term: maybe we can store this more efficiently (it is a modolo computation, so I don't think we need to store it on every location along a wire).

amin1377 added 19 commits June 11, 2025 09:55

[vpr][rr_graph] add tileable rr graph dir

b6371d2

[vpr][rr_graph] move tileable rr graph dir uner rr_graph_generation

77781b4

[vpr][route] update rr_graph generation with tileable rr graph

cf95e2c

[vpr][util] add is_inter_cluster_node for vib arch

431923c

[vpr][route] update router lookahead with tileable rr graph

d62fb79

[vpr][blif] use regex to find param val

02a56e9

[vpr][base] update with tileable rr graph

204a794

[vpr][route] remove unused param

7edf56d

[lib][arch] add vib processing

f5e8061

[lib][arch] add vib_inf

14064c7

[libs][rr_graph] update lib rr graph with tileable

49b04b0

[libs][rr graph][io] update read/write rr graph functions with tileab…

c6df91a

…le info

[libs][rr graph][io] update rr_grpah utils with tileable info

bf19150

[lib][rr_graph] add vtr tokenizer

3970db7

[lib][util] update capnp and util with open fpga

797b488

add openfpga doc

cf2dbc8

[vpr][base] add vib grid

0e03dc7

add VIB doc

f8fae78

[vpr][route] fix alloc_and_load_rr_switch_inf definition

5462ccd

github-actions bot added VPR VPR FPGA Placement & Routing Tool libarchfpga Library for handling FPGA Architecture descriptions docs Documentation lang-cpp C/C++ code libvtrutil labels Jun 11, 2025

amin1377 added 6 commits June 11, 2025 14:15

make format

7cdf5bf

[vtr_flow][test] add openfpga arch

5ad8303

[CI] add openfpga tests

c79ab8f

fix formatting

8eccbcc

Merge branch 'master' of https://github.com/verilog-to-routing/vtr-ve…

0cc6f10

…rilog-to-routing into add_tileable_rr_graph

[doc][arch] add tileable doc

5a492c8

amin1377 requested a review from AlexandreSinger June 19, 2025 22:36

amin1377 requested a review from soheilshahrouz June 19, 2025 22:37

Merge branch 'master' of https://github.com/verilog-to-routing/vtr-ve…

7c6a979

…rilog-to-routing into add_tileable_rr_graph

AlexandreSinger requested changes Jun 20, 2025

View reviewed changes

amin1377 added 10 commits June 23, 2025 12:43

[libs][arch] fix c-style code in read_xml_arch_file

ca95008

[lib][arch] fix process_vib_block_type_locs style

bb9a338

[lib][arch] pass set funcitons params by reference

af27706

Apply code review comment

4194e81

make format

561a0d5

Merge branch 'master' of https://github.com/verilog-to-routing/vtr-ve…

07d4bac

…rilog-to-routing into add_tileable_rr_graph

fix a typo

4e38274

[libs][arch] fix clang-17 warning

f324567

[vpr][route][tileable] replace VTR_LOG to VTR_LOG_DEBUG & fix c-style…

1f929ea

… code

[libs][vtrutil] fix string_view error

bc7ffcd

amin1377 requested a review from AmirhosseinPoolad June 24, 2025 13:29

soheilshahrouz requested changes Jun 24, 2025

View reviewed changes

amin1377 added 5 commits June 25, 2025 12:23

Merge branch 'master' of https://github.com/verilog-to-routing/vtr-ve…

9cf3775

…rilog-to-routing into add_tileable_rr_graph

[vpr][utils] move StringToken under vtr util

5f1939c

[vpr][base] remove unnecessary auto

bd62d9f

[vpr] apply code format rules to tileable RR Graph files

1d1318d

[libs][arch] move vib arch functions to a separate file

4c60c9d

make format

70836c5

amin1377 requested a review from vaughnbetz June 26, 2025 17:38

AlexandreSinger reviewed Jun 27, 2025

View reviewed changes

soheilshahrouz reviewed Jun 27, 2025

View reviewed changes

vaughnbetz requested changes Jun 28, 2025

View reviewed changes


		In this figure, the red arrows represent the initial direct connection. The green arrows represent the point to point connection to connect all the columns of CLB.

		A point to point connection can be applied in different ways than showed in the example section. To help the designer implement his point to point connection, a truth table with our new parameters id provided below.

Add tileable RR Graph #3134

Are you sure you want to change the base?

Add tileable RR Graph #3134

Uh oh!

Conversation

amin1377 commented Jun 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

amin1377 commented Jun 19, 2025

Uh oh!

AlexandreSinger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amin1377 commented Jun 23, 2025

Uh oh!

amin1377 commented Jun 24, 2025

Uh oh!

soheilshahrouz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amin1377 commented Jun 25, 2025

Uh oh!

AlexandreSinger left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

amin1377 commented Jun 11, 2025 •

edited

Loading