Is metaprogramming a subset of reflection?

Question

I used to think that metaprogramming involved modifying the program, and (as do some answers to What is reflection and why is it useful? ) that reflection merely consisted of introspection of a program. However, the reflection tag wiki says

Reflection is the process by which a program can observe and modify its own structure and behavior at runtime.

Reflection is the process by which a program can perform introspection. This introspection usually involves the ability to observe and modify its own structure and behavior at runtime. From theoretical perspective reflection relates to the fact that program instructions are stored as data. The distinction between program code and data is a matter of how the information is treated. Hence programs can treat their own code as data and observe or modify them.

[emphasis added]

And the description for metaprogramming is

Metaprogramming is writing programs that write or manipulate other programs as their data.

Metaprogramming is useful because it can save programmers valuable time. Some languages have support to metaprogram themselves and this allows to create code with great expressive power.

(I assume that "write" doesn't mean writing source code to a file, because that'd be code generation.)

Would this make metaprogramming merely a subset of reflection?

Or are the terms distinct because some programming languages are metaprogrammed by another language, in which case metaprogramming occurs but not reflection? (There was a single uncited sentence claiming this in the metaprogramming Wikipedia article)

Or do the terms "reflection" and "metaprogramming" get used differently depending on what programming language the person is using?

Answer 1

No. Rather, reflection provides facilities that are a subset of what metaprogramming can do.

Metaprogramming is "programs which write programs". This includes programs that read the text of programs (arguably including themselves but that is rather rare), analyze that code, and make changes. Yes, it includes writing source text to files. Code generation is a special case of metaprogramming.

Reflection as I understand it is the ability for a program to inquire about its own structure. In virtually every system I have seen in which reflection is possible (with the really exceptional case of Lisp and equivalent variants), the reflection machinery provided only a limited means of introspection. Java and C# will let you find out the names of classes and methods, but you cannot ask these systems for the content of a method, statement or local declaration. Nor can you ask most such reflective langauges to actually change their structure, that is, you can't add new classes or fields using the reflection facilities. Most langauges (e.g., C++) have basically no built-in ability to "reflect". While the reflection utilities built into langauges can be useful, they tend to be idiosyncratic with respect to what the language designers/compiler builders decided to keep around at runtime.

You end up with a much more powerful "reflection" capability if you step outside the language and the set of restrictions the langauge designers built into it. A really good metaprogramming system has access to the entire program structure, and thus can answer arbitrary questions about the program structure (modulo Turing limitations).

As an example, our DMS Software Reengineering Toolkit is a program transformation tool that has complete access to the abstract syntax tree of the program and many other facts derived by the various DMS language front ends. So DMS can "reflect" (inspect/analyze/reason) rather arbitrarily about the language which it is processing. And it can do so for C, COBOL, Java, C# and C++; for many of these langauges, it can not only provide access to the AST, but access to symbol table information and various forms of control and data flow, which no reflection facilities I've ever seen offer you.

Additionally, a program transformation tool like DMS can modify the code based on the "reflection" to generate new code, optimize, restructure, instrument, ... The variety of effects achievable this way is surprisingly broad.

[Since DMS is implemented as set of DSLs, it in fact can and does reason ("reflect") about its own code. We use DMS to synthesize large parts of itself from its DSLs, including code generation with some pretty interesting optimizations, including working parallelization.]