Numbers and values

Typically, a lexical analyzer returns a value to its caller indicating which token has been found. Within an action, this is done by writing a C return statement, which returns the appropriate value:

digit	[[:digit:]]
letter	[[:lower:]]
integer	{digit}+
name		{letter}({letter}│{digit})*
%%
"goto"    { return GOTO; }
{integer} { return INTEGER; }
{name}    { lookup(yytext); return NAME; }

In many cases, the lexical analyzer must supply other information to its caller. Within a compiler, for example, when an identifier is recognized, both a pointer to a symbol table entry and the token number NAME must be returned; however, the C return statement can return only a single value. yacc solves this problem by having the lexical analyzer set an external yylval to the token value, and return the token number. This mechanism can be used by lex programs when used with yacc; otherwise, you can define another interface. For example:

{name} { yylval = lookup(yytext); return(NAME); }

In the absence of a return statement, the lexical analyzer does not return to its caller, but looks instead for another token. This is typically used when a comment sequence has been discovered and discarded, or when the purpose of the lex program is to change a set of tokens into some other set of strings.

To summarize, the token number is set by the action with a return statement, and the token value is set by assigning this value to the external value yylval. An action need not return.