Re: [boost] Boost.Test - misc observations

From: Gennadiy Rozental (gennadiy.rozental_at_[hidden])
Date: 2006-06-14 02:50:30

"Robert Ramey" <ramey_at_[hidden]> wrote in message 
news:e6mrml$ick$1_at_sea.gmane.org...
> Gennadiy Rozental wrote:
>> "Robert Ramey" <ramey_at_[hidden]> wrote in message
>> news:e6etq0$7he$1_at_sea.gmane.org...
>>> Dear Gennadiy Rozental,
>>
>> Hmm, no "sir" - well for the fellow booster plain Gennadiy should
>> work ;o)
>
> aaaa - unfortunately I can't figure out how to pronounce this so I can't 
> remember how it's spelled.  I didn't want to say "Mr. Rosenthal" given 
> that we're on such intimate terms, so I just copied your full name.  OTOH 
> I spent many years outside the U.S. after being living in Califonia until 
> my twenties.  When I came back to the U.S (now twenty years ago) is was 
> struck by the "instant familiarity" of american language and customs.  It 
> stll seems odd to me to hear - "My name is Brittany - I'll be your waiter 
> this evening.  The first time time this happened, I stood up shook her 
> hand, introduced my wife (she's from germany) and said "Hello, My name is 
> Robert and this is my wife Annegret and we'll be your customers". 
> Unfortunately,

LOL

> this provoked consturnation among the staff rather the the hilarity I had 
> anticipated. It wasn't totally lost - my wife thought it was hysterically 
> funny.  No real point here - I'm still hoping to find someone who can 
> laugh at my humor.
>
>> Hi, Robert
>>
>> If you care for my opinion, it would not be a wise move, unless you
>> don't case about the quality of your tests of course. Current state
>> of the Unit Test Framework (UTF) allows me to state that it's a
>> preferred solution in all possible use scenarios (simplest one-liner, 
>> simple single
>> function test with several assertions or complex multilevel test case
>> structure) from all possible angles:
>>
>> * usability
>> * functionality
>> * compilation time
>
> Hmmm - and universality?

I am not sure what you mean by term universality. Do you mean portability?

>> We discussed this couple time and I thought I was clear.
>> Ok. Lets imagine two years from now boost decides to drop msvc 6.5
>> from regression testing.
>
>>From my personal perspective, it could be dropped right now as I happen to
> test that on my own machine whenever I make a change.  Same goes for BCB 
> 5.51 and 5.64

IMO you underestimate crucial importance of testing. More on this below.

>> Would you still insist on maintaining this
>> configuration? Even though none of the boost components you depend on
>> are going to be tested on it and accordingly will start to fail
>> sooner or later?
>
>> My hope is that your answer is no. In this case I
>> could most solemnly swear that Boost.Test will work on all platforms
>> boost perform regression testing on and you have nothing to worry
>> about.
>
> ***
> Hmmm maybe this is the crux of the difference.  I don't see Boost Test as 
> primarily a means running boost regression testing.

No do I.

> Of course its that, but
> I see it as much more.  I see it as the "Tool of choice" for users making 
> C++ unit tests.  which is not quite the same thing.

Right

> The boost regression testing environment is sort of special - it 
> presupposes installation of bjam and building boost libraries etc.  This 
> is more of a "marketing" view point. Some boost libraries benefit by being 
> almost universal - others not.  Boost Test is one of the former.

I am not sure I understand: how any library may not benefit from being 
universal/portable?

> ***
> So, I have a different take on this.
>
> Suppose I make library A.  As library author I can decide which platforms 
> (in addition to the mythical totally conforming compiler and standard 
> library) it will run on. This isn't a decision made by boost or boost 
> policy.  And of course its going to vary by library.

And from that it follows that your decision with platforms your library 
works on depend on which platforms all the your library's dependencies works 
on. Formally: the set of platforms supported by your library could not 
exceed intersection of all respective sets for your dependencies. In more 
simple terms: if the library you depend on doesn't work on platform P 
regardless of you willingness to support this platform it couldn't be done. 
But how do we define formally: "Library A works on platform P". My 
understanding could be expressed in a following definition:

Library A works on platform P iff:
1. All libraries A depend on works on platform P
2. Library A could be build on platform P (if offline library is necessary)
3. All required unit tests for library A pass on platform P.

Item 3 in above definition is crucial IMO. From above definition it follows 
that if dependency D of library A is not tested on platform P - library A 
does *not* work on platform P.

> BOOST_STATIC_ASSERT can and should be made almost universal.

You may think so. I may think so. Even 10000 happy users may think so. But 
if we don't test  BOOST_STATIC_ASSERT on platform P, no library that depend 
on it could claim it works on that platform. Another important point is: 
nothing is universal. Do you mean to suggest that BOOST_STATIC_ASSERT should 
be written in a such a way that it will work on any C++ compilers currently 
used by at least single person as a development tool?

> Boost Lambda cannot be without crippling its usefulness.
>
> So library B is implemented and running nicely on the platforms the author 
> has decided its appropropriate to support.  Library B depends upon Library 
> A.   Library C comes along and doesn't support all the platforms that 
> Library B does - but that's no problem as Library A doesn't depend on B. 
> The author of Library A adds some new functionality.  Still no problem 
> since he left in the old code.  Now Library A makes some changes which 
> break the original API for some platforms.

I am not sure how all above is relevant. While Library A is tested on 
platform P library author is not allowed to make any changes that break 
existing API. End of story (some exceptions possible, but that is out of 
scope for this discussion). But! As soon as library A stops testing on 
platform P it (according to definition above) doesn't work on platform P 
anymore and author is free to make any changes affecting API specific for 
that platform, including completely removing it from source code.

>  Of course that is where the problem arises.  But why is it necessary for 
> the author of Library A to do this? What does it cost him to just leave to 
> original code in?  The code is already written and tested.

These points are incorrect on many levels IMO. Here just couple of them:

1. The code that is not being tested will stop working sooner or later
  a. All global changes you apply to your code will affect platform P 
specific areas either (unless you want to give up global substitution and 
make routine case by case modification)
  b. You make changes in areas around that does not work with code in 
platform P specific sections
2. Source code clarity decreasing exponentially depending number of 
workarounds applied. I may not be that bad with one. But try to decipher 
code with 10 different workarounds for different platforms applied within 
close vicinity.
3. Workaround potentially complicate both your design and implementation, 
when without particular workaround your solution could be much 
neater/clearer For example vc 6.5 does not support PS and PO. Now your code 
has whole extra level of indirection to workaround this.
4. You couldn't introduce in your code any dependencies that does not work 
on platform P - which would make workaround completely useless.

> As an example, the serialization library still works with BCB 5.51 and VC 
> 6.x.  Its not that I invest any effort to do this - Its just that it costs 
> me nothing to leave this facility available. A good example is spirit. 
> Spirit has moved on but left sprite 1.6x around - this solution has worked 
> well for me.

Yes, indeed, spirit is a good example. Instead of keeping tons of 
workarounds in current code they dropped obsolete platforms and moved on. 
Old version that support these platforms is still available. Funny thing is 
that the same is true for *all* boost libraries. Whether it's Boost.Test, 
Boost.Serialization or any other. Boost 1.33 contains version that works on 
all platforms that we are trying to obsolete. So now, once we stop testing 
on these platforms I do not see any reason to keep workarounds for them in 
latest codebase.

> I don't dispute that UTF is better.  Its just that sometimes I want to 
> make a test and I can't build the boost libraries without taking a big 
> detour into bjam country.

You could always employ included version of UTF, that does not require bjam 
involvement.

Regards,

Gennadiy