Re: [Boost-users] [wave] bug report for wave

From: Hartmut Kaiser (hartmut.kaiser_at_[hidden])
Date: 2005-11-20 20:37:14

Daryle Walker wrote:

> > Andreas Sæbjørnsen wrote:
> [SNIP]
> >> I believe the problem lies in that the lexer does not 
> recognize the 
> >> '$'  as a valid  character  within the name of macro definition.
> > 
> > Accordingly to the Standard the '$' character is _not_ part of the 
> > basic source character set (see 2.2.1 [lex.charset]). For 
> this reason 
> > it won't get recognized as the part of a identifier.
> 
> Nope.  Right answer, wrong reason.  An identifier does _not_ 
> have to be made solely of basic source characters (look in 
> section 2.10 of the standard).  A universal character name 
> can also be part of an identifier.  Not all UCNs are allowed 
> in identifiers; the real problem is that the '$' character is 
> _also_ in the excluded set of UCNs.  (The list of allowed 
> UCNs is in Annex E of the standard.)

Thanks for clarifying this. But since '$' is in the exluded set of UCNs it
boils down to the same effect in the end (at least for Wave - see below). 

> [SNIP]
> > What certainly could be done additionally is to add the '$' 
> character 
> > to the valid basic source character set to allow identifiers 
> > conatining a '$', but this weakens the Standards 
> conformance of Wave. Any suggestions?
> 
> Maybe it can be moved to the valid UCN list instead.  This 
> assumes that Wave is currently capable of UCN processing.  

It is. And it checks for UCN validity, but only as long these are specified
as \uxxxx or \Uxxxxxxxxx. (Wave leaves the actual translation into the
execution character set to the compiler which processes the preprocessed
Wave output - it acts solely on the character level).

> Either way, you should make this optional, and disabled by 
> default, to allow Standards conformance when needed.

I've added the '$' to the basic source character set and it is allowed to be
part of an identifier name now. I made this optional (configurable at
compile time). Currently its on by default ('$' is recognised), but this
arguable.

> Other posts in this thread suggest that everyone out there 
> accepts '$' as an identifier character.  Was that 
> intentional?  I've never heard of anyone using '$' within an 
> identifier until now.  Maybe we should submit bug reports for 
> everyone on their lexers.

This is probably done since '$' is valid for most assembler languages and
the mentioned compilers all have some assembler inline directives.

Regards Hartmut