Tensorflow warning: The graph couldn't be sorted in topological order?

Question

When I run tensorflow training (with custom defined graph, closed source), it outputs the warning:

2018-10-03 14:29:24.352895: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:666] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.

What does it mean? What could likely cause this problem and how to avoid it?

Update: For the record, in my case, tensorflow still works correctly despite this warning. So I think it just means there are more loops in the computation graph than what tensorflow expects, not necessarily infinite loop. I fixed this by avoid using manual loop in code, but instead using tensor manipulation (stack, concat, slice, reshape...)

Answer 1

A topological ordering of a directed graph is an ordering of its vertices in such a way that whenever there is an edge from vertex u to vertex v, vertex u comes before vertex v in the ordering.

This kind of ordering is possible for every directed acyclic graph but not for arbitrary graphs. There is most likely some kind of cycle in your graph that prevents the sorting algorithm from succeeding. So, the way to go is searching for the cycle and removing it in some way.

As an example, consider a very small graph with two vertices, u and v, and two edges, u -> v and v -> u. There is no way to sort u and v in accordance with the requirements given above.

Without further information on your code it is hard to say exactly what is going on.