Group nodes within a bounding container

[TomPoulton]

Apologies if this is already possible and I've just missed it, but I'm using dagre-d3 to draw our network diagrams (servers and load-balancers) within our provisioning tool so that we can spin up a new environment based on the resources in the diagram.

Dagre-d3 is awesome so far, I've been able to get all the functionality I needed without too much effort, the only thing that's missing is that I'd like to group my servers by subnet (or AWS availability zone, I haven't decided yet). I guess this is just a special case of bounding boxes around a group of nodes where the boxes don't overlap.

If this isn't already a feature that I've missed, is it something that I could do using d3 directly? Any pointers in the right direction would be great!

Thanks

To group the nodes you probably want clustering support (dagrejs/dagre#13). It turns out to be a fair amount of work to add this support - the ticket references some of the papers that describe how to add it. Unfortunately, I haven't had a lot of dedicated time to work on it.

[TomPoulton]

Haha yeah I figured it might require a little bit of effort to get it to work! No worries, is there some way of faking it in the meantime, maybe by adding the nodes to the graph in order and then manually drawing a rect around them? It wouldn't do "collision detection" or anything fancy like that, but it would serve as a visual aid for now!?

I just re-added forster's constrained ordering algorithm, which gets your part of the way to your goal. With this ordering algorithm all nodes in a subgraph are grouped contiguously in each layer. Furthermore, the ordering of subgraphs are maintained across layers, so that you can't have subgraph A before subgraph B in layer 1 and then subgraph B before subgraph A in layer 2.

This doesn't get you all of the way though. In addition to the above you want all of the nodes to be grouped into a rectangular region across layers. This will likely not be available until I finish support for clusters.

[alienrobotwizard]

Any update here? I little background. I'm currently updating lipstick (https://github.com/Netflix/Lipstick) to work with dagre instead of relying on graphviz. Things are going great so far: (https://github.com/thedatachef/Lipstick/blob/dagre/lipstick-server/web-app/tossboss-graph-view/js/tossboss-graph-renderer.js) with one real issue. Grouping nodes for a map-reduce job inside a rectangle. If you look (https://github.com/thedatachef/Lipstick/blob/dagre/lipstick-server/web-app/tossboss-graph-view/js/tossboss-graph-renderer.js#L102-L155) you'll see that I add the rectangles manually after the graph layout using d3. Unfortunately, this only works some of the time. Occasionally dagre puts nodes from one subgraph in the same region as another making my rectangles nest.

Now, I'm using a CDigraph for the graph and Digraphs for the subgraphs. Is there something else I should be doing?

Here's some screen shots:

With graphviz (what I want, sans graphviz itself and the ugly paths...):

And with dagre (what I've got so far):

Thanks!

[alienrobotwizard]

Here's a block that illustrates things simpler. http://bl.ocks.org/thedatachef/raw/9818190/

Just looking for a workaround or something. If necessary I can spend some time on the renderer code if I knew where to start. Thanks!

You're doing everything right, it's just that clustering support isn't done yet. I have a set of patches on a local branch that gets us closer to clustering support, but I haven't had time to work on them. The main issue seems to be that the position phase algorithm, which has been generally been problematic, doesn't seem to play well with clustering.

I'll take a look through the patches I have and if they aren't too embarrassing (they're proof-of-concept code) I'll clean them up and post them to a branch on github this weekend.

[alienrobotwizard]

That's good to hear. I've hacked together a workaround that (at least for this gnarly graph: http://bl.ocks.org/thedatachef/raw/9841692/) appears to work. The idea is that, when laying out nodes, group them by cluster. Then, in the post processing, select the clusters, get their bboxes, and run the layout again using just the clusters as nodes. Of course the details are the beast here: https://gist.github.com/thedatachef/9841692

I'm happy to work on the more general solution when you post your branch. Thanks!

[alienrobotwizard]

Ok, one last comment for now. Theres a problem in my hack related to the minLen variable that the dagre layout uses. That is, if I treat the clusters/groups as nodes and run the layout it sets the number of ranks for an edge to 1, which means the length of the intercluster edges is way too long. If I could set the minLen to a fractional value I think I could go home happy. Thoughts?

minLen itself has to be a whole number because it is used to set up ranks. You could change the rankSep for the inter-cluster edges, which would reduce their length. Take a look at https://github.com/cpettitt/dagre#configuring-the-layout for details.

I've pushed most of what I have for the clustering support to the "cluster" branch for dagre. It doesn't work as it is and there are at least three places that probably need attention:

1. I suspect something was not right in the way the border segments were added because I have some hacky code that was working on that.
2. I believe the positioning algorithm was not yielding straight lines for segments where it should. At some point I want to replace it with a slower but more reliable algorithm - the algorithm I'm using is known to have problems, some of which I've worked around already.
3. IIRC, I don't use any heuristics right now to figure out if an edge belongs inside or outside of a cluster. I think it always gets placed outside of all clusters right now.

Actually, 1 above may have been me trying to hack around the positioning algorithm (2 above). I would suspect the position algorithm first.

[alienrobotwizard]

Interesting about minLen. In my hacky example rankSep seems to work fairly well only if I set it to a negative number. However, if I try that with the lipstick graph there's cluster overlap again. Ah well.

I'll take a look at the cluster branch. Not sure how helpful I'll actually be since I'll have to read up on the algorithms you're using (and graph layout in general...) so I'm not totally lost. However, this is the last bit preventing me from totally replacing graphviz in lipstick so it's worth the effort on my part. I'll post any real progress here.

Latest code on the clusters branch addresses the bug with addBorderSegment and fixes some issues in ordering for clusters. Two obvious issues remain:

1. We really need to get edges in the right clusters. Right now dummy nodes for edges are always assigned to the root cluster, which means edges from one node to another in the same cluster will go out of the cluster and then back in. The Sander paper has 2 strategies for how to handle this.
2. I think I am finally at the point where the position algorithm is borked. Fixing issue 1 may inherently fix this for a number of graphs.

The code on the clusters branch is prototype-grade and needs some doc + tests before I push it to master. This will probably happen after 1 is addressed.