Menu
I am writing an analyzer which needs an abstract syntax tree (AST) or control flow graph (CFG) of Rust code. It seems impossible for me to do this without implementing a parser by myself.
I've noticed some crates such as syn and quote, but they don't work without using procedural macros and creating a totally unnecessary project structure. I've found that there's a crate called syntex_syntax which meets my requirements, but it is no longer maintained and panics when some code with newer syntax is given.
Is there any way of parsing Rust code directly: read from an arbitrary external *.rs file and parse it using syn or quote just like syntex_syntax did?
982
Rust has excellent support for macros. Macros enable you to write code that writes other code, which is known as metaprogramming. Macros provide functionality similar to functions but without the runtime cost. There is some compile-time cost, however, since macros are expanded during compile time.
Procedural macros allow you to run code at compile time that operates over Rust syntax, both consuming and producing Rust syntax. You can sort of think of procedural macros as functions from an AST to another AST. Procedural macros must be defined in a crate with the crate type of proc-macro .
The most widely used form of macros in Rust is the declarative macro. These are also sometimes referred to as “macros by example,” “ macro_rules! macros,” or just plain “macros.” At their core, declarative macros allow you to write something similar to a Rust match expression.
Rust provides a powerful macro system that allows metaprogramming. As you've seen in previous chapters, macros look like functions, except that their name ends with a bang ! , but instead of generating a function call, macros are expanded into source code that gets compiled with the rest of the program.
Procedural macros, on the other hand, allow you to operate on the abstract syntax tree (AST) of the Rust code it is given. As far as function signatures go, a proc macro is a function from a TokenStream (or two) to another TokenStream, where the output replaces the macro invocation.
Text formatting, presented by macros like println or format ( println is a declarative macro which expands to a procedural macro format_args_nl included in rustc ). Note that forbidden symbols will not be allowed inside a procedural macro. To sum up, a procedural macro can contain only the tokens that are already allowed in Rust.
Now we can use fold_item_fn to inject print statements in our parsed code. This code example is from the syn examples repo, which is an excellent resource to learn about procedural macros. Custom derive macros in Rust allow auto implement traits. These macros enable you to implement traits using # [derive (Trait)].
The trace_vars macro takes the name of the variable it needs to trace and injects a print statement each time the value of the input variable i.e a changes. It tracks the value of input variables. First, parse the code to which the attribute-like macro attaches. syn provides an inbuilt parser for Rust function syntax.
syn is a Rust parser and is not only for procedural macros. Take a look at the "functions" section of the documentation. There you will find these functions (as of syn 0.15):
fn parse<T: Parse>(tokens: proc_macro::TokenStream) -> Result<T>: this is what you would use in a procedural macro.fn parse2<T: Parse>(tokens: proc_macro2::TokenStream) -> Result<T>: the same, but with the TokenStream from the proc_macro2 crate.fn parse_str<T: Parse>(s: &str) -> Result<T>: parsing from a simple string. No TokenStreams required.fn parse_file(content: &str) -> Result<File>: Very similar to parse_str, but some convenient differences. See docs for more information.You can use parse_str or parse_file to parse Rust code from outside of procedural macros.
A few additional notes:
quote is not needed in your case. This crate is just used to easily create TokenStreams; it's not required for parsing.proc_macro2 outside of a procedural macro, too!syntex_syntax is indeed deprecated and shouldn't be used anymore. Just thinking about how it was used makes me shudder :P
200
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
