What is #tfop in Swift Tensorflow and where is it defined?

Question

I'm browsing the swift tensorflow code, and stumbled upon instances of

var result = #tfop("Mul", a, b)

#tfop is well explained in the doc here, in the sense of 'what it does' but I'm also interested in what is actually is from a language standpoint, or as a function implementation.

What does #tfop represent, beside a handle to the computation graph? why the '#'? Where can I find tfop implementation if I want to? (I browsed the code, but no luck, although I can't guarantee that I didn't miss anything).

Answer 1

per Chris Lattner:

#tfop is a “well known” representation used for tensor operations. It is an internal implementation detail of our stack that isn’t meant to be user visible, and is likely to change over time.

In Swift, "#foo(bar: 42)” is the general syntax used for “macro like” and “compiler magic” operations. For example C things like FILE are spelled as #file in swift: https://github.com/apple/swift-evolution/blob/master/proposals/0034-disambiguating-line.md

And the “#line 42” syntax used by the C preprocesser is represented with arguments like this: #sourceLocation(file: "foo", line: 42)

In the case of #tfop specifically, this is represented in the Swift AST as an ObjectLiteralExpr, which is the normal AST node for this sort of thing: https://github.com/google/swift/blob/tensorflow/include/swift/AST/Expr.h#L1097

We use special lowering magic to turn it into a SIL builtin instruction in SILGen, which are prefixed with "__tfop_" https://github.com/google/swift/blob/tensorflow/lib/SILGen/SILGenExpr.cpp#L3009

I’d like to move away from using builtin instructions for this, and introduce a first-class sil instruction instead, that’s tracked by: https://github.com/google/swift/issues/16

These instructions are specially recognized by the partitioning pass of GPE: https://github.com/google/swift/blob/tensorflow/lib/SILOptimizer/Mandatory/TFUtilities.cpp#L715

source here