Is there a way to have arrows to and from Clusters #17

alexandrnikitin · 2020-02-13T20:29:52Z

I want to have arrows from and to Clusters, e.g.

    ELB("lb") >> Cluster("cluster1")
    // OR
    Cluster("cluster1") >> Cluster("cluster2")
    // OR
    with Cluster("cluster1"):
        cluster1 = [EC2("node1"),
                    EC2("node2"),
                    EC2("node3")]

    with Cluster("cluster2"):
        cluster2 = [EC2("node1"),
                    EC2("node2"),
                    EC2("node3")]
    cluster1 >> cluster2

All of them throw an exception right now. Is this use case supported or planned? Is there a workaround?

mingrammer · 2020-02-14T01:24:28Z

Inter-cluster edge is not supported now. Cluster class is for just grouping the nodes now. Let me research if it is possible (I think it could be achieved lhead/ltail options of Graphviz)

And 3rd way is not possible because Python does not support shift operation between lists. To support this method, we should create a grouping purpose class like “NodeList” so that let someone connects the nodes between clusters.

For now, you can connect the nodes between clusters in this way: https://diagrams.mingrammer.com/docs/examples#clustered-web-services

jbpratt · 2020-02-15T18:21:29Z

I've played around this morning with diagrams and have ran into a few issues with arrow direction. @mingrammer please let me know if I am doing something wrong.

From the step function, it should go into the first X. From the very last X lambda, it should go to both the MoveToOrg lambda as well as the S3 bucket -> SNS.

I also haven't found an appealing way to represent StepFunctions map state feature / concurrency within it

    with Diagram("AcctCreation", show=False):
        begin = (
            SQS("KickoffQueue")
            >> Lambda("AcctCreate")
            >> StepFunctions("StepFunctions")
        )

        with Cluster("StateMchne"):
            with Cluster("Cncrrntx3.1"):
                lambdas = Lambda("X") >> Lambda("X")

            x = lambdas >> Lambda("X") >> Lambda("X")

            with Cluster("Cncrrntx3"):
                z = x >> Lambda("X") >> Lambda("X")

            z >> [S3("X") >> SNS("Notif"), Lambda("MoveToOrg")]

        begin >> lambdas

Really enjoying this package, great job! Excited to share this with the team.

mingrammer · 2020-02-16T06:18:27Z

@jbpratt78 >> operation returns last node. So the return value of Lambda("X") >> Lambda("X2") is Lambda("X2"), therefore begin >> lambdas will connect the begin to Lambda("X2").

In this case, nodes that directly connect to the preceding node should be declared separately. You can write like this:

with Diagram("AcctCreation", show=False):
    begin = (
        SQS("KickoffQueue")
        >> Lambda("AcctCreate")
        >> StepFunctions("StepFunctions")
    )

    with Cluster("StateMchne"):
        with Cluster("Cncrrntx3.1"):
            entry_lambda = Lambda("X")
            out_lambda = Lambda("X")
            entry_lambda >> out_lambda

        inter_lambdas = out_lambda >> Lambda("X") >> Lambda("X")

        with Cluster("Cncrrntx3"):
            post_lambdas = inter_lambdas >> Lambda("X") >> Lambda("X")

        store = S3("X")
        store >> SNS("Notif")
        post_lambdas >> [store, Lambda("MoveToOrg")]

    begin >> entry_lambda

kontrafiktion · 2020-02-16T20:30:32Z

This issue now seems to contain two different issues.
I am getting back to the issue of arrows to "clusters".

Yes, graphviz can have arrows between groups/clusters, but it is kind of a hack: You declare an edge between two nodes and add an annotation that you want to start/end at the cluster (https://stackoverflow.com/questions/2012036/graphviz-how-to-connect-subgraphs)

If you would want to implement that in mingrammer, e.g.:

    lb = ELB("lb")
    services = Cluster("Services")
    with services:
        svc_group = [ECS("web1"),
                     ECS("web2"),
                     ECS("web3")]

    lb >> services

you need to get one node from the services cluster, so that the correct code could be generated

To get the node, you either have to implicitly retrieve any one node from the cluster – so the cluster must keep all node objects (currently the node is immediately destructured when adding it to the cluster, AFAICS).

Or the node must be a assigned to a variable (e.g. svc_group_node1 and then something like:

    lb >> svc_group_node1.cluster()

jbpratt · 2020-02-16T22:42:30Z

@kontrafiktion my apologies, this was due to my lack of understanding as I thought my cluster usage was related. Thank you for the help @mingrammer

ringods · 2020-02-22T15:07:56Z

I'm with @kontrafiktion on this one. I would also like to see arrows between clusters. But allow also for the more Pythonic way of writing this:

lb = ELB("lb")
with Cluster("Services") as services:
    ECS("web1")
    ECS("web2")
    ECS("web3")

    lb >> services

BTW, have a look at #34.

IaroslavR · 2020-02-26T00:31:36Z

@kontrafiktion @ringods
Can't reproduce this trick. diagrams 0.6.3. Full code of script:

from diagrams import Cluster, Diagram
from diagrams.aws.compute import ECS
from diagrams.aws.network import ELB

with Diagram("Cluster link", show=False):
    lb = ELB("lb")
    with Cluster("Services") as services:
        ECS("web1")
        ECS("web2")
        ECS("web3")
        lb >> services

failed with

Traceback (most recent call last):
  File "/home/iaro/PycharmProjects/diagrams_test/tmp.py", line 12, in <module>
    lb >> services
  File "/home/iaro/miniconda3/envs/diagrams/lib/python3.7/site-packages/diagrams/__init__.py", line 326, in __rshift__
    return self.connect(other)
  File "/home/iaro/miniconda3/envs/diagrams/lib/python3.7/site-packages/diagrams/__init__.py", line 372, in connect
    self._diagram.connect(self, node, directed)
  File "/home/iaro/miniconda3/envs/diagrams/lib/python3.7/site-packages/diagrams/__init__.py", line 168, in connect
    self.dot.edge(node.hashid, node2.hashid, **attrs)
AttributeError: 'Cluster' object has no attribute 'hashid'

ringods · 2020-02-26T07:03:48Z

@IaroslavR my code snippet was a proposal of how I would like to use it. It doesn't work like that (yet).

IaroslavR · 2020-02-27T01:27:40Z

@ringods Got it

Karreg · 2020-04-21T08:49:51Z

I +1 this! When you have 2 clusters, each one accessing 2 other clusters with many components in it, it makes the diagram less readable.

Also, it prevents from linking clusters directly.

juv · 2020-05-01T17:21:10Z

Another use case: I wanted to have a Kafka cluster and a Zookeeper cluster with 5 nodes each. All of the Kafka nodes are connected with - and the Zookeeper nodes as well. Now, when trying to visualize that there is a connection between the Kafka and Zookeeper cluster, one can either connect the two Clusters together (preferred) or visualize the connection from each Kafka Node to each Zookeeper node, as all Kafka brokers could could theoretically connect to each Zookeeper node. The latter generates a mess on my end so the proposed solution in this issue would be a great feature.

mingrammer · 2020-05-01T18:30:29Z

I'll review this proposal in a week.

meshuga · 2020-06-05T18:49:22Z

👍 for the feature.

I work on a tool to automate AWS infrastructure detection and draw diagram of the detected infrastructure. Right now, there's a high number of duplicates caused by redundant connections to multiple subnets from EC2 instances, Lambda functions etc.

Having this feature in place, the diagrams would look much cleaner and easier to understand.

jaywarfield · 2020-10-01T14:28:36Z

I use a corresponding feature on PlantUML and would add my request here for this feature too. Great tool - thanks!

kjh235 · 2020-10-08T17:33:13Z

My current work around :
Make the cluster a list, then point the middle elements together.
It is a little more defining if you want multiple elements to have arrows, or just the cluster as a single arrow.

with Cluster("cluster1"):
        cluster1 = [EC2("node1"),
                    EC2("node2"),
                    EC2("node3")]

    with Cluster("cluster2"):
        cluster2 = [EC2("node1"),
                    EC2("node2"),
                    EC2("node3")]
    cluster1[1] >> cluster2[1]

clayms · 2020-11-05T00:20:10Z

@alexandrnikitin

I want to have arrows from and to Clusters, e.g.

Below is a work-around:

from diagrams import Diagram, Cluster, Edge, Node

graph_attr = {
    "layout":"neato",
    }

scaling_clus_attr = {
    "bgcolor":"transparent",
    "pencolor":"blue",
    "penwidth":"4.0"
    }

with Diagram("\n\nNodes on Cluster boundary\nwith connecting arrow", show=False, graph_attr=graph_attr) as diag:
    with Cluster("cluster 1"):
        A_UpLf = Node("", shape="plaintext", pin="true", pos="0,4")
        A_LwRt = Node("", shape="plaintext", pin="true", pos="4,0")

    with Cluster("cluster 2", graph_attr=scaling_clus_attr):
        B_UpLf = Node("", shape="plaintext", pin="true", pos="8,3")
        B_LwRt = Node("", shape="plaintext", pin="true", pos="10,1")

    A_tail = Node("", shape="plaintext", pin="true", pos="4,2")
    B_head = Node("", shape="plaintext", pin="true", pos="8,2")

    A_tail >> B_head

diag

Karreg · 2020-11-05T10:07:39Z

Is it working with some components inside the cluster? I tried your code but only ended with an arrow, no square...

clayms · 2020-11-05T13:42:37Z

@Karreg did you change the layout engine to "neato" ?

graph_attr = {"layout":"neato", }

clayms · 2020-11-07T00:43:20Z

Here is another way. It only works with the 'dot' layout engine, which is the default for this library, but I explicitly specified it for clarity.

The main thing to notice is that this library automatically sets the name attribute of the Cluster to be "cluster_" + self.label .
See

diagrams/diagrams/__init__.py

Line 223 in dfd8e0a

self.name = "cluster_" + self.label

You must keep this in mind when referring to the Cluster name, in the ltail and lhead attributes of Edge.

Below is a replication of Figure 22 from https://www.graphviz.org/pdf/dotguide.pdf using this library. This is a good example of how the Edge's head and tail are truncated at the Cluster boundary.

from diagrams import Diagram, Cluster, Edge, Node

graph_attr = {
    "layout":"dot",
    "compound":"true",
    "splines":"spline",
    }

node_attr = {
    "shape":"ellipse", 
    "height":"0.8",
    "labelloc":"c"
}

with Diagram("\nCluster to Cluster Edges", show=False, direction="TB", graph_attr=graph_attr) as diag:
    with Cluster("0"):
        a = Node("a", **node_attr)
        b = Node("b", **node_attr)
        c = Node("c", **node_attr)
        d = Node("d", **node_attr)
    
    with Cluster("1"):
        e = Node("e", **node_attr)
        f = Node("f", **node_attr)
        g = Node("g", **node_attr)

    h = Node("h", **node_attr)

    a >> [b, c]
    [b, c] >> d
    e >> [f, g]
    
    b >> Edge(lhead="cluster_1") >> f
    c >> Edge(ltail="cluster_0", lhead="cluster_1") >> g
    c >> Edge(ltail="cluster_0") >> e
    d >> [e, h]

diag

clayms · 2020-11-08T11:33:31Z

To have a more meaningful Cluster label, but still keep the Cluster name simple, just add the Cluster keyword argument graph_attr dictionary, and then set the "label" key to the desired text value.

with Diagram("\nCluster to Cluster Edges", show=False, direction="TB", graph_attr=graph_attr) as diag:
    with Cluster("0", graph_attr={"label":"More meaningful label"}):
        a = Node("a", **node_attr)
        b = Node("b", **node_attr)
        c = Node("c", **node_attr)
        d = Node("d", **node_attr)
    
    with Cluster("1", graph_attr={"label":"This is a longer label"}):
        e = Node("e", **node_attr)
        f = Node("f", **node_attr)
        g = Node("g", **node_attr)

    h = Node("h", **node_attr)

    a >> [b, c]
    [b, c] >> d
    e >> [f, g]
    
    b >> Edge(lhead="cluster_1") >> f
    c >> Edge(ltail="cluster_0", lhead="cluster_1") >> g
    c >> Edge(ltail="cluster_0") >> e
    d >> [e, h]

diag

geoffreywiseman · 2020-12-30T22:12:34Z

Sorry, does workaround still work? I've tried this as an experiment and I'm not seeing the arrows stop at the cluster edges:

with Diagram("Cluster Arrows", show=False):
    with Cluster("1a"):
        pub1a = PublicSubnet("1b")
    with Cluster("1b"):
        pub1b = PublicSubnet("1a")

    tgw = TransitGateway("tgw")
    tgw >> Edge(lhead='cluster_1a') >> pub1a
    tgw >> Edge(lhead='cluster_1b') >> pub1b

clayms · 2020-12-31T12:31:22Z

@geoffreywiseman
You need to include the graph_attr dictionary, and set the "compound" key to "true".

See https://graphviz.gitlab.io/doc/info/attrs.html#d:compound

compound : bool, default: false
If true, allow edges between clusters.
See lhead and ltail below.
Valid for: Graphs. Note: dot only

Example:

from diagrams import Diagram, Cluster, Edge, Node
from diagrams.aws.network import PublicSubnet, TransitGateway

graph_attr = {
    "layout":"dot",
    "compound":"true",
    }

with Diagram("Cluster Arrows", show=False, graph_attr=graph_attr) as diag:
    with Cluster("1a"):
        pub1a = PublicSubnet("1b")
    with Cluster("1b"):
        pub1b = PublicSubnet("1a")
    
    tgw = TransitGateway("tgw")
    tgw >> Edge(lhead='cluster_1a') >> pub1a
    tgw >> Edge(lhead='cluster_1b') >> pub1b


diag

geoffreywiseman · 2020-12-31T17:36:01Z

Ah, yes, I tried reproducing most of what you had, but the compound part I guess I didn't try (or failed to see that it worked).
So that helps but not if I want two-way arrows:

graph_attr = {
    "layout":"dot",
    "compound":"true",
    }
    
with Diagram("Cluster Arrows", show=False, direction='TB', graph_attr=graph_attr):
    with Cluster("vpc"):
        with Cluster("1a"):
            pub1a = PublicSubnet("pub")
            priv1a = PrivateSubnet("priv")
        with Cluster("1b"):
            pub1b = PublicSubnet("pub")
            priv1b = PrivateSubnet("priv")
    
    tgw = TransitGateway("tgw")
    tgw << Edge(lhead='cluster_1a') >> priv1a
    tgw >> Edge(lhead='cluster_1b') << pub1b

clayms · 2021-01-01T12:37:23Z

@geoffreywiseman
You'll need a Cluster for the ltails as well. Set the Clusters graph_attr as below if you don't want to see it.

from diagrams import Diagram, Cluster, Edge, Node
from diagrams.aws.network import PublicSubnet, TransitGateway

graph_attr = {
    "layout":"dot",
    "compound":"true",
    }

tgw_clus_attr = {
    "label":"",
    "bgcolor":"transparent",
    "penwidth":"0.0"    
}

with Diagram("Cluster Arrows", show=False, direction="TB", graph_attr=graph_attr) as diag:
    with Cluster("vpc"):
        with Cluster("1a"):
            pub1a = PublicSubnet("1b")
        with Cluster("1b"):
            pub1b = PublicSubnet("1a")
    
    with Cluster("tgw", graph_attr=tgw_clus_attr):
        tgw = TransitGateway("tgw")
    
    tgw << Edge(ltail="cluster_tgw", lhead='cluster_1a', minlen="2") >> pub1a
    tgw << Edge(ltail="cluster_tgw", lhead='cluster_1b', minlen="2") >> pub1b

diag

bkmeneguello · 2021-01-06T14:49:32Z

#407 does that nativelly, please check. Nodes can behave as Clusters.

geoffreywiseman · 2021-01-06T15:26:25Z

#407 does that nativelly, please check. Nodes can behave as Clusters.

Yes, #407 definitely looks like it'll do what I want in what seems like a simpler way; looking forward to that making it into a release. ;)

chadfurman · 2021-06-10T14:49:46Z

This is technically supported in Graphviz, looks like there might already be a PR for this #439 which replaces #407

deeplook · 2023-10-24T10:49:11Z

@mingrammer What is the current state of this ticket?

mingrammer added kind/feature New feature or request question Further information is requested labels Feb 14, 2020

ringods mentioned this issue Feb 22, 2020

Allow any Node to behave as a Cluster #34

Open

jonioni mentioned this issue Apr 5, 2020

Flexible Cluster #91

Open

mingrammer added comp/cluster Issue of cluster component status/need-to-review Need to review proposal Suggestion proposed and removed question Further information is requested labels May 1, 2020

mingrammer mentioned this issue May 15, 2020

Is it possible to draw connecting line between clusters? #128

Open

mingrammer pinned this issue May 17, 2020

meshuga mentioned this issue Jun 5, 2020

Enhance Route Tables reporting Cloud-Architects/cloudiscovery#24

Closed

steschuser mentioned this issue Sep 7, 2020

Arrow from one cluster to another #261

Open

andreujuanc mentioned this issue Oct 12, 2020

set the position of a cluster / node #249

Open

This was referenced Nov 5, 2020

Allow using different graphviz engines #193

Open

Spacing Between Clusters #183

Open

clayms mentioned this issue Jan 11, 2021

Edge labels not placed with line #433

Open

clayms mentioned this issue Jan 31, 2021

Is there a way to add clusters together with edges? #452

Open

clayms mentioned this issue Apr 5, 2022

Connecting two cluster #670

Open

clayms mentioned this issue Jun 13, 2022

unable to draw Linked Clusters #707

Closed

clayms mentioned this issue Dec 9, 2022

Can you nest and reference other diagrams? #436

Open

jtprogru mentioned this issue Nov 21, 2023

[Snyk] Upgrade minimist from 1.2.5 to 1.2.8 jtprogru/diagrams#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Is there a way to have arrows to and from Clusters #17

Is there a way to have arrows to and from Clusters #17

alexandrnikitin commented Feb 13, 2020

mingrammer commented Feb 14, 2020 •

edited

Loading

jbpratt commented Feb 15, 2020

mingrammer commented Feb 16, 2020 •

edited

Loading

kontrafiktion commented Feb 16, 2020 •

edited

Loading

jbpratt commented Feb 16, 2020

ringods commented Feb 22, 2020

IaroslavR commented Feb 26, 2020

ringods commented Feb 26, 2020

IaroslavR commented Feb 27, 2020

Karreg commented Apr 21, 2020

juv commented May 1, 2020 •

edited

Loading

mingrammer commented May 1, 2020

meshuga commented Jun 5, 2020

jaywarfield commented Oct 1, 2020

kjh235 commented Oct 8, 2020 •

edited

Loading

clayms commented Nov 5, 2020 •

edited

Loading

Karreg commented Nov 5, 2020

clayms commented Nov 5, 2020

clayms commented Nov 7, 2020 •

edited

Loading

clayms commented Nov 8, 2020 •

edited

Loading

geoffreywiseman commented Dec 30, 2020

clayms commented Dec 31, 2020

geoffreywiseman commented Dec 31, 2020

clayms commented Jan 1, 2021

bkmeneguello commented Jan 6, 2021

geoffreywiseman commented Jan 6, 2021

chadfurman commented Jun 10, 2021 •

edited

Loading

deeplook commented Oct 24, 2023

Is there a way to have arrows to and from Clusters #17

Is there a way to have arrows to and from Clusters #17

Comments

alexandrnikitin commented Feb 13, 2020

mingrammer commented Feb 14, 2020 • edited Loading

jbpratt commented Feb 15, 2020

mingrammer commented Feb 16, 2020 • edited Loading

kontrafiktion commented Feb 16, 2020 • edited Loading

jbpratt commented Feb 16, 2020

ringods commented Feb 22, 2020

IaroslavR commented Feb 26, 2020

ringods commented Feb 26, 2020

IaroslavR commented Feb 27, 2020

Karreg commented Apr 21, 2020

juv commented May 1, 2020 • edited Loading

mingrammer commented May 1, 2020

meshuga commented Jun 5, 2020

jaywarfield commented Oct 1, 2020

kjh235 commented Oct 8, 2020 • edited Loading

clayms commented Nov 5, 2020 • edited Loading

Karreg commented Nov 5, 2020

clayms commented Nov 5, 2020

clayms commented Nov 7, 2020 • edited Loading

clayms commented Nov 8, 2020 • edited Loading

geoffreywiseman commented Dec 30, 2020

clayms commented Dec 31, 2020

geoffreywiseman commented Dec 31, 2020

clayms commented Jan 1, 2021

bkmeneguello commented Jan 6, 2021

geoffreywiseman commented Jan 6, 2021

chadfurman commented Jun 10, 2021 • edited Loading

deeplook commented Oct 24, 2023

mingrammer commented Feb 14, 2020 •

edited

Loading

mingrammer commented Feb 16, 2020 •

edited

Loading

kontrafiktion commented Feb 16, 2020 •

edited

Loading

juv commented May 1, 2020 •

edited

Loading

kjh235 commented Oct 8, 2020 •

edited

Loading

clayms commented Nov 5, 2020 •

edited

Loading

clayms commented Nov 7, 2020 •

edited

Loading

clayms commented Nov 8, 2020 •

edited

Loading

chadfurman commented Jun 10, 2021 •

edited

Loading