20. Macros¶
Legality Rules
20:1 A macro is a custom definition that extends Rust by defining callable syntactic transformations. The effects of a macro are realized through macro invocations or attribute use. Macros come in two distinct forms:
20:2 Declarative macros define rules for recognizing syntactic patterns and generating direct syntax.
20:3 Procedural macros define augmented functions that operate on and return a stream of lexical elements.
20:4 Tokens are a subset of lexical elements consumed by macros.
20.1. Declarative Macros¶
Syntax
MacroRulesDeclaration
::= macro_rules !Name
MacroRulesDefinition
MacroRulesDefinition
::= (MacroRuleList
) ; | [MacroRuleList
] ; | {MacroRuleList
}MacroRuleList
::=MacroRule
(;MacroRule
)* ;?MacroRule
::=MacroMatcher
=>MacroTranscriber
MacroMatcher
::= (MacroMatch
* ) | [MacroMatch
* ] | {MacroMatch
* }MacroTranscriber
::=DelimitedTokenTree
MacroMatch
::=MacroMatcher
|MacroMatchToken
|MacroMetavariableMatch
|MacroRepetitionMatch
20.1:1
A MacroMatchToken
is any lexical element in category
LexicalElement
, except punctuation $
and category Delimiter
.
Legality Rules
20.1:2 A declarative macro is a macro that associates a name with a set of syntactic transformation macro rules.
20.1:3 A macro rule is a construct that consists of a macro matcher and a macro transcriber.
20.1:4 A macro matcher is a construct that describes a syntactic pattern that a macro must match.
20.1:5 A macro match is the most basic form of a satisfied macro matcher.
20.1:6 A macro transcriber is a construct that describes the replacement syntax of a macro.
20.1:7 A declarative macro is invoked using a macro invocation.
Examples
macro_rules! answer_to_life {
() => { 42 };
}
20.1.1. Metavariables¶
Syntax
MacroMetavariableMatch
::= $MacroMetavariable
:MacroFragmentSpecifier
MacroMetavariable
::=Keyword
|NonKeywordIdentifier
MacroFragmentSpecifier
::= block | expr | expr_2021 | ident | item | lifetime | literal | meta | pat | pat_param | path | stmt | tt | ty | visMacroMetavariableIndication
::= $MacroMetavariable
Legality Rules
20.1.1:1 Declarative macros employ metavariables to match a token of a particular kind and bind its value to a name for use during macro transcription.
20.1.1:2 A metavariable is a macro match that describes a variable.
20.1.1:3 A metavariable is visible in the macro transcriber of the macro rule of the macro matcher it is declared in.
20.1.1:4 A fragment specifier is a construct that indicates the type of a metavariable.
20.1.1:5 Fragment specifier kinds impose the following fragment specifier restrictions on the tokens that follow them:
20.1.1:6
expr
andexpr_2021
shall only be followed by=>
,,
, or;
.20.1.1:7
pat
shall only be followed by=>
,,
,=
,|
,if
, orin
.20.1.1:8
path
shall only be followed by=>
,,
,=
,|
,;
,:
,>
,>>
,[
,{
,as
,where
, or a metavariable with theblock
fragment specifier kind.20.1.1:9
pat_param
shall only be followed by=>
,,
,=
,|
,if
, orin
.20.1.1:10
stmt
shall only be followed by=>
,,
, or;
.20.1.1:11
ty
shall only be followed by=>
,,
,=
,|
,;
,:
,>
,>>
,[
,{
,as
,where
, or a metavariable with theblock
fragment specifier kind.20.1.1:12
vis
shall only be followed by,
, an identifier except forpriv
, any token that may begin aTypeSpecification
, or a metavariable with theident
,ty
orblock
fragment specifier kind.20.1.1:13 Any other kind may be followed by any token.
20.1.1:14 A metavariable indication is a construct that indicates a metavariable.
Examples
macro_rules! square {
($e:expr) => { $e * $e };
}
20.1.2. Repetition¶
Syntax
MacroRepetitionMatch
::= $ (MacroRepetitionMatchContent
)MacroRepetitionSeparator
?MacroRepetitionOperator
MacroRepetitionMatchContent
::=MacroMatch
*MacroRepetitionTranscriber
::= $ (TokenTree
* )MacroRepetitionSeparator
?MacroRepetitionOperator
MacroRepetitionOperator
::= + | * | ?
20.1.2:1
A MacroRepetitionSeparator
is any lexical element in category
LexicalElement
, except punctuation +
, *
, ?
, and category
Delimiter
.
Legality Rules
20.1.2:2 A macro repetition in matching allows for a syntactic pattern to be matched zero or multiple times during macro matching.
20.1.2:3 A macro repetition in transcription allows for a syntactic pattern to be transcribed zero or multiple times during macro transcription.
20.1.2:4 A macro repetition is either a macro repetition in matching or a macro repetition in transcription.
20.1.2:5 A repetition operator is a construct that indicates the number of times a macro repetition in matching or a macro repetition in transcription can be repeated.
20.1.2:6 The effects of a repetition operator are as follows:
20.1.2:7
*
- Zero or more repetitions.20.1.2:8
+
- One or more repetitions.20.1.2:9
?
- Zero or one repetition.
20.1.2:10 A macro repetition has the following additional restrictions:
20.1.2:11 If the macro repetition has a separator, the separator shall be allowed by the
MacroRepetitionMatchContent
‘s fragment specifier restrictions.20.1.2:12 If the repetition operator is
*
or+
, then the possible beginnings of theMacroRepetitionMatchContent
shall be allowed by itsMacroRepetitionMatchContent
‘s fragment specifier restrictions.20.1.2:13 If the repetition operator is
?
or*
, then the succeedingMacroMatch
must be allowed by the precedingMacroMatch
‘s fragment specifier restrictions.20.1.2:14 The possible beginnings of the
MacroRepetitionMatchContent
must be allowed by the precedingMacroMatch
‘s fragment specifier restrictions.20.1.2:15 The succeeding
MacroMatch
must be allowed by the possible endings of theMacroRepetitionMatchContent
‘s fragment specifier restrictions.
Examples
macro_rules! generate_pairs {
( $( $first:ident )* ; $( &second:ident )* )
=>
{ $( $first, $second )* };
}
20.2. Procedural Macros¶
Legality Rules
20.2:1 A procedural macro is a macro that encapsulates syntactic transformations in a function. Procedural macros consume one or more streams of tokens and produce a stream of tokens.
20.2:2
Procedural macros shall be defined in a crate subject to
attribute crate_type
where the type is proc-macro
.
20.2:3 A macro implementation function is the function that encapsulates the syntactic transformations of a procedural macro.
20.2:4 A macro implementation function enters the name of the procedural macro into the macro namespace.
20.2.1. Function-like Macros¶
Legality Rules
20.2.1:1 A function-like macro is a procedural macro that consumes a stream of tokens and produces a stream of tokens.
20.2.1:2 The macro implementation function of a function-like macro shall be subject to the following restrictions:
20.2.1:3 It shall be subject to attribute
proc_macro
,20.2.1:4 It shall be subject to visibility modifier
pub
,20.2.1:5 It shall lack function qualifiers,
20.2.1:6 It shall lack generic parameters,
20.2.1:7 It shall have a single function parameter whose type specification indicates type
proc_macro::TokenStream
,20.2.1:8 It shall have a return type whose type specification indicates type
proc_macro::TokenStream
.
20.2.1:9 A function-like macro is invoked using a macro invocation.
20.2.1:10
The sole parameter of the macro implementation function captures the
token stream produced from the DelimitedTokenTree
of the
macro invocation, excluding outer Delimiters
.
Examples
#[proc_macro]
pub fn make_answer_to_life(_items: TokenStream) -> TokenStream {
"fn answer_to_life() -> u32 { 42 }".parse().unwrap()
}
20.2.2. Derive Macros¶
Legality Rules
20.2.2:1 A derive macro is a procedural macro that consumes a stream of tokens and produces a stream of tokens. Derive macros are used to construct new syntax for abstract data types.
20.2.2:2 The macro implementation function of a derive macro shall be subject to the following restrictions:
20.2.2:3 It shall be subject to attribute
proc_macro_derive
,20.2.2:4 It shall be subject to visibility modifier
pub
,20.2.2:5 It shall lack function qualifiers,
20.2.2:6 It shall lack generic parameters,
20.2.2:7 It shall have a single function parameter whose type specification indicates type
proc_macro::TokenStream
,20.2.2:8 It shall have a return type whose type specification indicates type
proc_macro::TokenStream
.
20.2.2:9
A derive macro is invoked using attribute derive
.
20.2.2:10
The sole parameter of the macro implementation function captures
the token stream produced from the related EnumDeclaration
,
StructDeclaration
, or UnionDeclaration
.
20.2.2:11 A derive macro adds all its declared derive helper attributes into the derive helper attribute scope of the abstract data type the attribute is attached to.
20.2.2:12
A derive helper attribute is an inert attribute that acts as a
hint to attribute derive
.
Examples
#[proc_macro_derive(Answer)]
pub fn derive_answer_to_life(_items: TokenStream) -> TokenStream {
"fn answer_to_life() -> u32 { 42 }".parse().unwrap()
}
20.2.3. Attribute Macros¶
Legality Rules
20.2.3:1 An attribute macro is a procedural macro that consumes two streams of tokens to produce a single stream of tokens, and defines a new outer attribute that can be attached to items. Attribute macros are used to replace items with other items.
20.2.3:2 The macro implementation function of an attribute macro shall be subject to the following restrictions:
20.2.3:3 It shall be subject to attribute
proc_macro_attribute
,20.2.3:4 It shall be subject to visibility modifier
pub
,20.2.3:5 It shall lack function qualifiers,
20.2.3:6 It shall lack generic parameters,
20.2.3:7 It shall have two function parameters whose type specifications indicate type
proc_macro::TokenStream
,20.2.3:8 It shall have a return type whose type specification indicates type
proc_macro::TokenStream
.
20.2.3:9 An attribute macro is invoked using an attribute of the form
20.2.3:10
#[SimplePath]
, or20.2.3:11
#[SimplePath DelimitedTokenTree]
20.2.3:12
The first function parameter of the macro implementation function
captures the token stream produced from the DelimitedTokenTree
of the invoking attribute, excluding outer Delimiters
. If no
DelimitedTokenTree
is provided, then the token stream is considered
empty.
20.2.3:13 The second function parameter of the macro implementation function captures the token stream produced from the related item, including all outer attributes that apply to that item.
Examples
#[proc_macro_attribute]
pub fn output_and_return_item
(attr: TokenStream, item: TokenStream) -> TokenStream
{
println!("attr: \"{}\"", attr.to_string());
println!("item: \"{}\"", item.to_string());
item
}
20.3. Macro Invocation¶
Syntax
MacroInvocation
::=SimplePath
!DelimitedTokenTree
DelimitedTokenTree
::= (TokenTree
* ) | [TokenTree
* ] | {TokenTree
* }TokenTree
::=DelimitedTokenTree
|NonDelimitedToken
TerminatedMacroInvocation
::=SimplePath
! (TokenTree
* ) ; |SimplePath
! [TokenTree
* ] ; |SimplePath
! {TokenTree
* }
20.3:1
A NonDelimitedToken
is any lexical element in category
LexicalElement
, except category Delimiter
.
Legality Rules
20.3:2 A macro invocation is a call of a declarative macro or function-like macro that is expanded statically and replaced with the result of the macro.
20.3:3 A terminated macro invocation is a macro invocation that may be used as a statement.
Examples
20.3:4
See Paragraph 20.1. for the declaration of answer_to_life
.
answer_to_life!();
20.3:5
See Paragraph 20.1.1. for the declaration of square
.
square!(5);
20.3:6
See Paragraph 20.1.2. for the declaration of generate_pairs
.
generate_pairs!(1, 2, 3; 9, 8, 7);
20.3:7
See Paragraph 20.2.1. for the declaration of
make_answer_to_life
.
make_answer_to_life!();
20.3:8
See Paragraph 20.2.2. for the declaration of Answer
.
#[derive(Answer)]
struct derive_macro_invoker;
20.3:9
See Paragraph 20.2.3. for the declaration of
output_and_return_item
.
#[output_and_return_item]
fn attribute_macro_invoker() {}
20.4. Macro Expansion¶
Legality Rules
20.4:1 Macro expansion is the process of statically executing a macro invocation and replacing it with the produced output of the macro invocation.
20.4:2 Macro expansion of declarative macros proceeds as follows:
20.4:3 The
TokenTree
of the macro invocation has all outer block docs and outer line docs contained within replaced by their equivalent attributedoc
representation.20.4:4 The
TokenTree
of the macro invocation is matched against the macro rules of the resolved macro by considering individual macro matchers. It is a static error if no macro matcher is satisfied.20.4:5 The macro transcriber of the satisfied macro rule produces its result, with all metavariable indications resolved. It is a static error if the macro transcriber fails to produce its result.
20.4:6 The macro invocation is replaced with the result of the macro transcriber. It is a static error if the result cannot be parsed according to the expected expansion syntax of the context where the macro invocation resides. The expected expansion syntax is as follows:
20.4:7 If the macro invocation appears as part of an associated item, an item within an external block, or another macro invocation, the output is required to constitute zero or more items.
20.4:8 If the macro invocation appears as part of an expression-without-block, the output is required to constitute an expression.
20.4:9 If the macro invocation appears as part of a pattern-without-range, the output is required to constitute a pattern.
20.4:10 If the macro invocation appears as part of a statement, the output is required to constitute zero or more statements.
20.4:11 If the macro invocation appears as part of a type specification without bounds, the output is required to constitute a type.
20.4:12 Macro expansion of function-like macros proceeds as follows:
20.4:13 The
TokenTree
of the macro invocation has all outer block docs and outer line docs contained within replaced by their equivalent attributedoc
representation.20.4:14 The
TokenTree
of the macro invocation is transformed into a correspondingproc_macro::TokenStream
.20.4:15 The macro implementation function is called with the
proc_macro::TokenStream
as its sole argument. It is a static error if the macro implementation function call fails.20.4:16 The macro invocation is replaced with the returned
proc_macro::TokenStream
of the macro implementation function call. It is a static error if the result can not be parsed according to the expected expansion syntax of the context where the macro invocation resides. The expected expansion syntax is as follows:20.4:17 If the macro invocation appears as part of an associated item, an item within an external block, or another macro invocation, the output is required to constitute zero or more items.
20.4:18 If the macro invocation appears as part of an expression-without-block, the output is required to constitute an expression.
20.4:19 If the macro invocation appears as part of a pattern-without-range, the output is required to constitute zero or more patterns.
20.4:20 If the macro invocation appears as part of a statement, the output is required to constitute zero or more statements.
20.4:21 If the macro invocation appears as part of a type specification without bounds, the output is required to constitute a type.
20.4:22 Macro expansion of derive macros proceeds as follows:
20.4:23 The item subject to the derive macro has all outer block docs and outer line docs contained within replaced by their equivalent attribute
doc
representation.20.4:24 The item subject to the derive macro is transformed into a corresponding
proc_macro::TokenStream
without the invokingderive
attribute as well as any precedingderive
attributes.20.4:25 The macro implementation function is called with the
proc_macro::TokenStream
as its sole argument. It is a static error if the macro implementation function call fails.20.4:26 The returned
proc_macro::TokenStream
of the macro implementation function call is appended to the enclosing block expression or module where the relatedEnumDeclaration
,StructDeclaration
, orUnionDeclaration
resides. It is a static error if the outputproc_macro::TokenStream
does not constitute zero or more items.
20.4:27 Macro expansion of attribute macros proceeds as follows:
20.4:28 The
DelimitedTokenTree
of the invoking attribute macro is transformed into a correspondingproc_macro::TokenStream
without the outerDelimiters
. If noDelimitedTokenTree
is provided, and emptyproc_macro::TokenStream
is used. Thisproc_macro::TokenStream
constitutes the first function parameter of the macro implementation function.20.4:29 The item subject to the attribute macro has all outer block docs and outer line docs contained within replaced by their equivalent attribute
doc
representation.20.4:30 The item subject to the attribute macro is transformed into a corresponding
proc_macro::TokenStream
without the invoking attribute. Thisproc_macro::TokenStream
constitutes the second function parameter of the macro implementation function.20.4:31 The macro implementation function is called with the two
proc_macro::TokenStreams
as the two arguments. It is a static error if the macro implementation function call fails.20.4:32 The item subject to the attribute macro is replaced with the returned
proc_macro::TokenStream
of the macro implementation function call. It is a static error if the outputproc_macro::TokenStream
does not constitute zero or more items.
20.4.1. Macro Matching¶
Legality Rules
20.4.1:1 Macro matching is the process of performing rule matching and token matching.
20.4.1.1. Rule Matching¶
Legality Rules
20.4.1.1:1
Rule matching is the process of consuming a TokenTree
in an attempt
to fully satisfy the macro matcher of a macro rule that belongs to a
resolved declarative macro.
20.4.1.1:2 Rule matching proceeds as follows:
20.4.1.1:3 The macro matchers of all macro rules that belong to a resolved macro are tried against the
TokenTree
of the macro invocation, in declarative order. In the event of a static error, no further attempts at selecting a subsequent macro matcher are made.20.4.1.1:4 The macro match of a candidate macro matcher is tried against the
TokenTree
of the macro invocation by matching individual tokens, in left-to-right order. Matching does not employ lookahead. It is a static error if matching a candidate macro matcher is ambiguous. Matching does not employ backtracking. It is a static error if matching a candidate macro matcher fails while parsing into a metavariable and having consumed at least one token while parsing the metavariable.20.4.1.1:5 It is a static error if no macro matcher is selected.
20.4.1.2. Token Matching¶
Legality Rules
20.4.1.2:1
Token matching is the process of consuming a TokenTree
in an attempt
to fully satisfy a macro match of a selected macro matcher that
belongs to a resolved declarative macro.
20.4.1.2:2 Token matching proceeds as follows:
20.4.1.2:3
The outer Delimiters
of a macro matcher match any outer
Delimiters
in the macro invocation.
20.4.1.2:4 A metavariable in a macro matcher is matched against a sequence of tokens in the macro invocation based on its fragment specifier:
20.4.1.2:5 Fragment specifier block requires a block expression.
20.4.1.2:6 Fragment specifier expr and expr_2021 requires an expression that is not an underscore expression or const block expression.
20.4.1.2:7 Fragment specifier ident requires a pure identifier.
20.4.1.2:8 Fragment specifier item requires an item.
20.4.1.2:9 Fragment specifier lifetime requires character sequence 0x27 0x5F (apostrophe, low line), or character 0x27 (apostrophe) followed by an identifier.
20.4.1.2:10 Fragment specifier literal requires optional character 0x2D (hyphen-minus), followed by a literal expression.
20.4.1.2:11 Fragment specifier meta requires an attribute content.
20.4.1.2:12 Fragment specifier pat requires a pattern.
20.4.1.2:13 Fragment specifier pat_param requires a pattern-without-alternation.
20.4.1.2:14 Fragment specifier path requires a type path.
20.4.1.2:15 Fragment specifier stmt requires a statement without trailing character 0x3B (semicolon), excluding items that require character 0x3B (semicolon).
20.4.1.2:16 Fragment specifier tt requires a
TokenTree
.20.4.1.2:17 Fragment specifier ty requires a type specification.
20.4.1.2:18 Fragment specifier vis requires a possibly empty visibility modifier.
20.4.1.2:19 Once a metavariable is matched, the matching sequence of tokens is bound to that metavariable.
20.4.1.2:20
Repetition in a macro matcher is matched based on how many times the
pattern appears consecutively optionally separated by a separator in
the TokenTree
of the macro invocation, as follows:
20.4.1.2:21 If the repeated pattern includes a separator, then the separator must be able to follow the repeated pattern.
20.4.1.2:22 If the repeated pattern can appear multiple times, then the repeated pattern must be able to follow itself.
20.4.1.2:23 If the repeated pattern can appear zero times, then the preceding pattern must be able to follow the succeeding pattern.
20.4.1.2:24 The repeated pattern must be able to follow the preceding pattern.
20.4.1.2:25 The succeeding pattern must be able to follow the repeated pattern.
20.4.1.2:26 A repetition index is a monotonically increasing number that is initialized to zero, and incremented by one.
20.4.1.2:27 Once a metavariable is matched, the matching sequence of tokens is treated as follows:
20.4.1.2:28 The matching sequence of tokens is stored in an ordered collection at the current repetition index.
20.4.1.2:29 The current repetition index is incremented by one.
20.4.1.2:30 Each matched metavariable in a macro repetition in matching is bound separately, where the matches are stored in an ordered collection.
20.4.1.2:31
Any other token in a macro matcher is matched literally against the
TokenTree
of the macro invocation.
20.4.1.2:32
It is a static error if the TokenTree
of the macro invocation contains
leftover tokens after macro matching.
20.4.2. Macro Transcription¶
Legality Rules
20.4.2:1 Macro transcription is the process of producing the expansion of a declarative macro.
20.4.2:2 Macro transcription proceeds as follows:
20.4.2:3
Every metavariable indication found in the DelimitedTokenTree
of the
macro transcriber that belongs to a matched macro rule is replaced by
the matched sequence of tokens of the metavariable.
20.4.2:4 Unresolved metavariable indications are kept as tokens in the output verbatim.
20.4.2:5
Every macro repetition in transcription found in the
DelimitedTokenTree
of the macro transcriber shall be transcribed by
repeatedly transcribing the tokens inside of it.
20.4.2:6 The number of transcription repetitions for a macro repetition in transcription shall depend on its repetition operator, as follows:
20.4.2:7 A repetition operator denoted by
+
shall require one or more repetitions.20.4.2:8 A repetition operator denoted by
*
shall require zero or more repetitions.20.4.2:9 A repetition operator denoted by
?
shall require zero or one repetition.
20.4.2:10 A metavariable indication that is matched inside of a macro repetition shall not be used outside of a macro repetition in transcription.
20.4.2:11 A metavariable indication shall be used in a macro repetition in transcription of the same nesting depth as its corresponding metavariable appears in the macro matcher.
20.4.2:12 A metavariable indication within a macro repetition in transcription shall repeat the same number of times in its matching macro repetition if the macro repetition occurs at the same nesting depth.
20.4.2:13 Multiple transcribed metavariable indications in the same macro repetition in transcription shall repeat the same number of times.
20.4.2:14 When transcribing a metavariable indication in a macro repetition in transcription, the metavariable indication is replaced with the matched sequence of tokens of the corresponding iteration of the repetition metavariable taken from the ordered collection.
20.4.2:15 A metavariable indication in a macro repetition in transcription shall be transcribed to the matched tokens in order, as follows:
macro_rules! foo {
( $($expr:expr)* ) => {
$( $expr ; )*
// $expr is an error
};
( $( $( $expr:expr )* )* ) => {
$($($expr)*)*
}
}
foo! {
0
1
2
}
20.4.2:16
yields 0;1;2;
20.4.2:17
Given a macro invocation with N
metavariable arguments, a
macro of the form
macro_rules! m {
( $(param: expr)* ) => {
$( $param )*
}
}
20.4.2:18 is equivalent to a macro of the form
macro_rules! m {
( $param_1: expr $param_2: expr ... $param_N: expr) => {
$param_1 $param_2 ... $param_N
}
}
20.4.2:19
where the metavariable of the macro repetition in matching are
repeated N
times, and the metavariable indications of the
macro repetition in transcription are repeated N
times. Invoking such
a macro relates the first metavariable argument of the
macro invocation with the first metavariable of the
macro repetition in matching, the second metavariable argument with
the second metavariable, and so on.
20.5. Hygiene¶
20.5:1 Hygiene is a property of macros and identifiers that appear within them, which aims to eliminate the syntactic interference between a macro and its environment.
Legality Rules
20.5:2 Hygiene is categorized as follows:
20.5:3 Definition site hygiene, which resolves to a
MacroRulesDeclaration
site. Identifiers with definition site hygiene cannot reference the environment of theMacroRulesDeclaration
, cannot be referenced by the environment of aMacroInvocation
, and are considered hygienic.20.5:4 Call site hygiene, which resolves to a
MacroInvocation
site. Identifiers with call site hygiene can reference the environment of theMacroRulesDeclaration
, can reference the environment of theMacroInvocation
, and are considered unhygienic.20.5:5 Mixed site hygiene, which resolves to a
MacroRulesDeclaration
site for labels, variables, and the$crate
metavariable, and to theMacroInvocation
site otherwise, and is considered partially hygienic.
20.5:6 Every macro has associated hygiene that depends on its kind:
20.5:7 Declarative macros have mixed site hygiene.
20.5:8 Procedural macros have call site hygiene and mixed site hygiene depending on the implementation of the procedural macro.
20.5:9
The metavariable $crate
in a declarative macro‘s expansion refers
to the crate the declarative macro was declared in.