Problems with wchar_t support in Intel C++ (was: Re: [boost] Re: lexical_cast<>() and character traits)

From: Terje Slettebø (tslettebo_at_[hidden])
Date: 2002-07-27 15:43:09

>From: "Gennaro Prota" <gennaro_prota_at_[hidden]>

I changed the subject line to something more appropriate, as this may have
interest for others, as well.

>>On Fri, 26 Jul 2002 16:18:23 +0200, Terje Slettebø
>><tslettebo_at_[hidden]> wrote:

>>>>This leads to the stupid question of the month: why not Source const?
>>>
>>>Well, what would it solve? Perhaps if the output operator is a const
member
>>>function, and lexical_cast is passed a non-const object?
>>
>>I'm still wondering about this one?

>Ah! I can't remember what I was exactly thinking. It is very likely
>that I was wrong.

Ok. Actually, it either passes by value, or by const reference, and in the
case of pass by value, it shouldn't matter whether or not the formal
parameter value is declared const. The output stream operators of
basic_ostream (which handles the built-in types) are also not declared
const.

>>Good idea. I tried it too, now. By the way, it shows that
>>NO_INTRINSIC_WCHAR_T is defined for Intel C++, even though it's able to
have
>>it (and I used the /Zc:wchar_t option). I guess config.hpp considers the
>>default MSVC mode of Intel C++. However, this means that the output of the
>>above prologue could be slightly misleading in this respect.

>It depends on what "no intrinsic wchar_t means" :-)

Well, it should mean that it's a separate type, distinguished from e.g.
unsigned short. However, I see below here that after all, this is far from
the case, for Intel C++.

>I have also looked at the msdn to see if .NET has a predefined macro
>to test for the /Zc:wchar_t switch and... well, there isn't! Because
>there's a macro that is defined also if a typedef unsigned short
>wchar_t is used!

>---
>"_WCHAR_T_DEFINED - Defined when wchar_t is defined. Typically,
>wchar_t is defined when you use /Zc:wchar_t or when typedef unsigned
>short wchar_t; is executed in code"
>---

Well, wchar_t is defined, alright, but in the case of the typedef, it's not
an intrinsic type.

I see that boost/config/compiler/intel.hpp does it like this:

#  ifndef _NATIVE_WCHAR_T_DEFINED
#     define BOOST_NO_INTRINSIC_WCHAR_T
#  endif

Where does "_NATIVE_WCHAR_T_DEFINED" come from? I haven't found it in the
Intel C++ docs, at least. It's not defined anywhere else in the library,
either.

Unfortunately, from the Intel C++ docs, there appears to be no way to detect
whether or not the /Zc:wchar_t switch is set. If this is the case, perhaps
it's time for another feature request from Intel? If someone else who are
familiar with Boost config could confirm this, I could bring it up with
Intel.

Surely, they should be interested in that their compiler works more smoothly
with Boost, and that you may take advantage of features, such as this. If
wchar_t support works properly, that is.

>>>The real questions are why VC++ doesn't choose (b) and why Intel C++
>>>with the switch calls (a).
>>
>>I think the reason may be quite simple. Intel C++ has (the possibility of)
>>intrinsic wchar_t. Therefore, when it's turned on, it calls the
>>char-version, and when it's turned off, it becomes a synonym of unsigned
>>short, like VC++, and therefore calls the other function.

>I find it quite horrible. It doesn't have an intrinsic wchar_t, even
>with the switch. It only acts as if it had in some situations. Which
>situations is very difficult to say.

>Also, I expected "microsoft emulation" without the switch and
>"compliance" with the /Zc:wchar_t (or with some other switch, I don't
>know what the /Zc:wchar_t does with .NET). Instead, _maybe_ there's no
>way to make it compliant (of course I've tried invoking it from the
>command line, after editing the icl.cfg file).

>Type identity is absolutely fundamental for the language and for
>program semantics. The fact the Intel with the switch gives true with
>this

>typeid(wchar_t) == typeid(unsigned short)
>means that wchar_t and unsigned short _are_ the same type

That's odd. I wasn't aware of that. That's more serious than that the name
is reported as the same.

However, this compiles:

void f(unsigned short) {}
void f(wchar_t) {}

Which lead me to believe it treated them as separate types, or it would have
given an error about duplicate definition. However, digging a little deeper
(inspired by you doing the same below here), shows that its support for
wchar_t as intrinsic type is partial, at best, as you point out:

#include <iostream>

void f(unsigned short)
{
std::cout << "unsigned short\n";
}

void f(wchar_t)
{
std::cout << "wchar_t\n";
}

int main()
{
unsigned short s=0;
wchar_t c=L'A';

f(s);
f(c);
}

This prints:

unsigned short
unsigned short

(!)

What the heck is going on here? I want my money back. :) (Just kidding!)

By the way, testing the above on 7.0 pre-beta gives the following...:

unsigned short
wchar_t

They _have_ known about the problem! They just haven't bothered to mention
it in the docs. Not good.

Also, the typid test you give below here, works on that version... It shows
that they are indeed different types. I also noticed now that "std::cout <<
typeid(wchar_t).name()" gives the correct "wchar_t". So the problem wasn't
the name, in 6.0, but the fact that the types were actually treated more or
less as the same. Somebody at Intel has a few things to answer to, here. :)

I'll send a bug-report for 6.0 about this, also asking why the docs don't
mention that the wchar_t support is hardly existent in that version. At
least Microsoft admits that their compiler doesn't have support for wchar_t.

Ok, so so far, we have the following wchar_t "support" for 6.0:

- It lets you define overloaded functions with unsigned short and wchar_t,
showing that they are somewhat treated differently.
- typeid shows that they are the same type.
- Overload resolution shows the same.

Apparently, the problems are deeper than I thought. I've earlier submitted a
feature request, to change the reported name from "unsigned short" to
"wchar_t", because that's something that is implementation-defined. However,
as they give the same typeid, this shows why they report as the same name,
as well.

> Also
>
>    typeid(L'1') == typeid (unsigned short)
>    typeid(L'1') == typeid (wchar_t)
>
>both yield true. OTOH, the example about overloading (or what kind of
>'over-stuff' is it now) tells that they are different, and also that
>the 'best match' for the call
>
> dummy (L'1')
>
>is void dummy (unsigned short)!

Yeah. Same as I found above, here.

>I'm sorry to be a little harsh but I really spent (wasted) a lot of
>time trying to rationalize the behaviour of the compiler.

You're not alone. :) we've both wondered about this, trying to get some
sense out of it. So no apology needed. :)

>I expect
>such things from MSVC but I was really astonished from Intel

Me too. When they first provide an intrinsic wchar_t, I expected them to go
all the way, and not give some pseudo-wchar_t. Or, as it appears,
undocumented partial support.

>>This of course means that you can't output an unsigned short value,
without
>>having it output as a character. In essence, VC++ "hijacks" the unsigned
>>short type, and uses it as if it was wchar_t, to compensate for compiler
>>deficiencies. For this reason, lexical_cast works even if it has no
>>intrinsic wchar_t.

>Well, fortunately all these things don't affect lexical_cast :-)
>My question was an aside in the context of this thread (that's why I
>put it in the post-scriptum).
>
>This doesn't prevent all the issue to be quite puzzling, though.

Yeah.

>>>Try for instance this trivial program (use the switch for icl):
>>>
>>>#include <sstream>
>>>
>>>int main(int argc, char* argv[])
>>>{
>>>using namespace std;
>>>
>>>basic_stringstream<unsigned short> str;
>>>str << L'1';
>>>
>>>return 0;
>>>}
>>
>>When I try this on Intel C++, with the switch, I get errors. It compiles
on
>>VC++, as expected

>Eheheh :-) To me, it didn't compile with VC++, but only because I
>forgot the Zc:wchar_t switch in the Project Settings dialog (try to
>believe...)

Yeah, that's another odd thing with VC++: If you leave the /Zc:wchar_t
switch there, it says in the output that it ignores it, but it gives a ton
of bogus errors, showing that it does _not_ ignore it. :)

>>Anyhow, I hope Intel didn't ruin a so good product with these kinds of
>>microsoft-related conundrums :-(
>
>No. As you can see, Intel C++ works fine with the switch. It's only without
>it that it emulates VC++.
>
>>What about the example I gave in my previous post and
>>typeid(wchar_t).name() on Intel C++ 7.0?
>
>It still gives "unsigned short". However, they are allowed to do this, as
>the result is implementation dependent. Still, it's not particularly
>helpful, so I'll mention this in an issue report to Intel.

Hm. Apparently, I didn't set the compiler to 7.0, because I found now that
it gives the correct "wchar_t".

>>>I'm happy that you find all those pedantic comments useful! Ehmm...
>>>Terje, are you sure? :-)
>>
>>Yes. :) And it's not pedantic. Your feedback is most valuable, as it has
let
>>me remove bugs in the version, and possible problems, such as the '\0' /
>>L'\0' issue.
>>
>>This is not pedantic. This is precision. And that's a virtue. By the way,
>>you've got credit as contributor in the lexical_cast file, as you may have
>>seen.

>Well, I'm a bit embarrassed because I think my comment were marginal.
>But only because frankly I didn't see anything wrong in the
>fundamental parts.

Well, that's a good thing for me. :) It showed I've done my "homework".
Having an extensive test suite helps a lot. That I didn't check that things
that were supposed to throw exceptions really did that, meant that one bug
got through (empty string to character), but that's now fixed, with a test
to ensure it. A nice thing about such unit tests is that you fix bugs once.
Rather than risking that you may accidentally undo the fix another time, or
use a version without the fix. In that case, the unit test will fail.

In any case, you're almost the only one with have replied to comments and
suggestions, to this version, although there's been a few others, too, like
Gennadiy, and I have yet to get around to do the timing tests. I guess it
was most important to first get it correct, before considering possible
optimisations. So it's only fitting that you get credit for it.

Regards,

Terje