Re: [boost] Boost.Tests: Unit test defines own main() function, why?

From: Jens Seidel (jensseidel_at_[hidden])
Date: 2007-12-21 08:23:09

Hi Gennadiy,

first of all thanks for your comments. Some stuff is now a little bit
more understandable other stuff not.

I hope you didn't considered my mails as complaints, I just tried to
express that some parts of Boost.Test need some improvement. To be
honest I think that your library is a very important one and well done.
I will definitively look into it more carefully (reading the code/the
docs multiple days) as it is impossible to understand the current
implemention compared to the old outdated documenation with current
resources in let's say less than one day.

Please note that I'm also willing to resolve these issues and will
try some improvements to the documenation. I have to confess that
I did not write much documenation myself but I'm (mostly as translator)
involved in many Open Source documents such as Debian Reference, Debian
Website and other documents (HOWTOs, manpages, ...). I know XML and SGML
based tools really well (and maintain these partely myself) so I do not
expect problems with Boost documentation system.

I don't have much time to explain all my issues with Boost.Test now in
detail (I still have to buy some presents :-) but I will at least
answer your questions:

On Fri, Dec 21, 2007 at 02:27:31AM -0500, Gennadiy Rozental wrote:
> "Jens Seidel" <jensseidel_at_[hidden]> wrote in message 
> news:20071219090237.GA13691_at_imkf-pc073.imkf.tu-freiberg.de...
> > And why are static and shared libraries handled differently? Why not
> > just removing main also from static libraries?

Let me again note that I think that the words "static" and "shared" do
not match here as both libraries are indeed different ones (which share
a lot of code). This *needs* to be fixed! The current situation is
clearly NOT acceptable (no joke!). 
 
> 1. For backward compartibility

Here you assume that most users do not use the shared library. OK, let's
assume this for now.

> 2. For enhanced usebility in many cases. For people producing a lot of unit 
> tests adding extra function main()
> implementation is unnnecessary burdan (not mentioning that they will have to 
> actually learn how implement it properly.

Confusing users with all kind of combinations of BOOST_TEST_DYN_LINK,
BOOST_TEST_MAIN, BOOST_TEST_ALTERNATIVE_INIT_API, usage or not usage
of main and the init method is OK?

I'm sure many combination of these flags are just incompatible to each
other. Defining a one line main() method (containing also test module
initialization API code) instead of using macros would simplify a lot
in code and documentaion (but would require a new API :-().

> 3. In many (if not the most) cases function main() handling is hidden 
> completely from the users and usage and both usage variants behave pretty 
> much the same. The only difference is presence of an extra macro 
> BOOST_TEST_DYN_LINK.

Maybe. But I was never largely involved into Boost.Test and followed the
first time mainly the normal starting instructions from the old docs
(which are very easy!) and extented some parts later as needed
(test_tools::predicate_result stuff, ...). So I consider my code the
normal use case and you think it is very exotic as nearly no part of the
documentation and your mails adress my usage?
 
> You comming to this from *nix prospecive.

Right.

> On NT dll and lib are two quite 
> different ball games. Boost.Test opted to be consystent in between 
> platforms.

But why not select the best of both systems? (To be honest I do not know
anything with could be used from your propritary world.)
You did it the opposite way.
 
> > Did you also consider using a stable ABI (application binary interface, 
> > would allow a
> > binary to be used with different library version without need to
> > recompile)
> 
> I sugest I write Boost.Test using C? In Boost.Test  a lot of staff happends 

The first "I" should probably be "You".

> in headers. We might find it difficult.

No, one can also use ABIs in C++. Included code is part of the binary if
it comes from headers and fixed even if you move the binary to another
system where different shared libraries may be installed. ABI affects
exported symbols in shared libraries only. As most Boost libs are indeed
header based ABI does not affect these.
 
> > or API (application programming interface, no need to change
> > the source code to be able to compile against a new version of a
> > library) (at least some time in the future)
> 
> I do not plan any changes that should break backward compartibility. 1.34.1 
> did break it for very small number of use cases, which where never 
> officially supported in a first place (plus I believe 1 tool was found 
> unsafe and the interface was changes).

A stable API is a very good think. But if the current interface is
unusable (and also impossible to document) it may be useful.

All Boost libraries need to pass an initial review. Is it possible to
rewrite libs after it completely (as you did????) or would it make sense
to ask the community if changes are worth to be done?
 
> > which would make
> > incompatibilities between shared and static libraries impossible and
> > simplifies using different versions of Boost a lot? Whenever I try
> > another Boost version existing code fails to compile or link ...
> 
> I am sure we will be able to figure it out.

Would be great!

> As far as i know there is no problem compiling/linking Boost.Test.

I did not wrote about problems compiling Boost.Test but about using it!
I prefer of course to use Boost from the packages provided on my system.
Even if Boost isn't installed by default it is possible to select it for
installation during seconds on each modern system.

> > The problem is that the example in the official documentation (which is
> > (or should be) on www.boost.org not somewhere outside) just contains:
> > "the Unit Test Framework is responsible for supplying function main()"
> >
> > Even if you consider that to be outdated (again: Why do you not update
> > it??????)

This seems to be one of the most important questions: Why do you not
replace http://www.boost.org/libs/test/doc/index.html with
http://www.patmedia.net/~rogeeff/html/index.html?
Does the content need to be transfered from HTML first or what steps are
missing?

> > you should assume that most code arround does not provide
> > main() and fails that's why starting with recent versions.
> 
> Only those who employ shared library and manual registration had an issue. 

Which is the default case in the old documentation. I also see some
advantages:

* No usage of just another macro
* It's easier to uncomment a test (only a single line needs a comment,
  the compiler can still check the syntax of the not used test
* I know the order of the execution of the tests for sure and can test
  basic stuff at the beginning

> > How could I define my own main function which is compatible with shared
> > and dynamic libraries? unit_test_main has different signatures!?
> 
> 1. shared and dynamic libs are comparitble ;)

I assume you forgot "not" here. Otherwise you refer to code rewriting
(automatic registration, ...)?

> 2. Don't define init function at all.

I need an init function. I use a logger and set some parts to verbose
file output and compare as another test the generated output with
expected one. I would nevertheless like to do this directly in main()
without need for another init function.

> 3. The test runner  function signature is now the same for both static and 
> dyanmic library (see svn). I am yet to update the updated docs ;)
 
Good.

> Yes, the version on boost.org is very old. Docs writting take as much time 
> (if not more) as actual development.

I fully agree. But why not using the best of both? Remove outdated
pages on www.boost.org with your new docu *now*. Why do you wait?
 
> > Again: Why are
> > static and shared libraries handled differently? It shouldn't matter!
> 
> It does on windows.

But it does not have to differ! Most Open Source code from Linux can be
compiled on Windows as well. Do you think part of the porting is to
change interfaces just for fun? It is not necessary!

Jens