Re: [Boost-build] is it worth the effort?

Subject: Re: [Boost-build] is it worth the effort?
From: PJB (darthpjb_at_[hidden])
Date: 2016-01-24 13:16:15

As a user, I've not a huge amount to contribute - but professionally I
found myself in a very similar position (with a Win32/OSX stack - No MPI
though - Mostly due to home grown cross platform implementations).

*The Warnings:*
The number of warnings boost generates can be absurd, especially with
python (that to my knowledge still has an issue regarding Static-Linking).
When including my boost In a project, I generally wrap #pragma warning
(disable : xxxx) around includes of boost to address this issue. However I
agree that security issues may be a concern in that regard; However as far
as building goes - I can see swiftly the problem, profiling that many
warnings - never-mind hunting down the root causes would be a huge project.
I don't expect any major action in this regard, as much as I deem it
important.
Not to mention I'm pretty sure many (if not most) are actually supposed to
be there.

*Property Sheets:*
In my companies scenario, this is the easiest problem to solve. *My company
policy includes a section on the correct placement of boost. *Boost is
always placed relative to our version-control repositories, to be exact,
two levels higher in the file system.
Remember that boost's location on the system is one the user can/should
define.
Further, we use static linking for both boost and the MS-Runtimes (though
the latter isn't an option for certain security related applications).

In specific cases where a project has been "fixed" on a previous version of
boost, we've even been known to put all the boost headers and libraries for
that project in the project Repository.
IE: as far as I'm concerned, fixing the location of boost is up to the
user, am I wrong?

In summary:
There are too many warnings - but I doubt most will be fixed anytime soon.
You should decide where boost is on your system / development stack.

As a final note however - I'm not really sure where your E-mail is going?
Are you recommending something? Querying about your setup? etc?

On 24 January 2016 at 11:58, Terry Lyons <tlyons_at_[hidden]> wrote:

> Hello,
>
>
>
> I am new to this list – so apologies if I break conventions or if this
> mailing is long. I work primarily in well resourced windows environments –
> but often my production code (which tends to use large amounts of resource)
> needs to cross the divide and run partly or fully on linux servers. I use
> boost, MPI, python, iostreams, …
>
>
>
> So I try to build a full blown boost including MPI – across all versions
> from msvc-9.0 to msvc-14.0 and ideally for the intel compilers up to
> version 16 aswell. Although I am challenged enough that intel has to wait.
> My challenge is to make it all work in a way that a new compiler version,
> or boost version does not waste huge amounts of time. There seem so many
> issues that I lose confidence.
>
>
>
> The multiple toolset requirement is essential and for example CPython 2.7
> uses the vs2008 toolset, and cuda does not currently work on toolsets after
> vs2013.
>
>
>
> *MPI*
>
>
>
> is a challenge in two ways: although Microsoft have issued several
> versions of their MPI stack all recent ones set environmental variables to
> locate their parts, the mpi.jam needs updating for this. However, Microsoft
> have messed up with their macros around depreciated parts, and so in
> msvc-9.0 compilation that includes the mpi headers one needs a definition
> of MSMPI_NO_DEPRECATE_20. One can insert this in the mpi.jam and then one
> is in good shape. I have modified the mpi.jam to use the paths of the
> current mpi stacks and for the definition as follows:
>
>
>
> if ! $(mpicxx) && [ os.on-windows ]
>
>   {
>
> …
>
>       options = …
>
>                 <toolset>msvc:<define>_SECURE_SCL=0
>
>                <toolset>msvc:<define>MSMPI_NO_DEPRECATE_20
>
>               ;
>
> …
>
>   }
>
>
>
> However, I am worried by the toolset>msvc:<define>_SECURE_SCL=0 which is
> in the distributed mpi.jam as I don’t think this is a good move. It makes
> for binary inconsistency between parts of the boost library build using
> these assumptions (mpi) and those that are not. Personally I would delete
> it. Either it is global to all of boost or it is not and there are
> arguments in both directions. It should not be set in the boost mpi.jam. I
> have not changed it though.
>
>
>
> *BOOST-MPI*
>
>
>
> Then breaks properly since the graph package has some missing includes of
> std::list etc. in mpi_process_group.?pp that create compliation errors.
> Fixing these is actually very simple and produces error free boost building.
>
>
>
> *Files that generate warnings*
>
>
>
> Then one can see where the remaining troubles lie!! And a small number of
> files produce a huge number of warnings. Personally, I do not think they
> can be ignored – and the number of files causing trouble can and really,
> for the sake of the cross platform nature of boost, get fixed. Many are
> 32/64 bit mismatch errors. I feel this is almost always sloppy/histroical
> coding, and needs to be fixed to stop DOS vulnerabilities. size_t and int
> are different types, used for different sorts of things – they should not
> be converted blindly – if the developer is confident it is OK then the
> warnings should be surpressed. If not then they should be fixed. A lot more
> are about dll interfaces.
>
>
>
> *The following and only the following files produced compilation warnings
> in the win 64 msvc-14.0 setup (before I build python or iostreams)*.
>
>
>
> boost/archive/iterators/mb_from_wchar.hpp
>
> boost/context/execution_context.ipp
>
> boost/coroutine/detail/coroutine_context.hpp
>
> boost/graph/distributed/detail/mpi_process_group.ipp
>
> boost/locale/generic_codecvt.hpp
>
> boost/mpi/communicator.hpp
>
> boost/mpi/detail/binary_buffer_iprimitive.hpp
>
> boost/mpi/detail/binary_buffer_oprimitive.hpp
>
> boost/mpi/detail/content_oarchive.hpp
>
> boost/mpi/detail/mpi_datatype_primitive.hpp
>
> boost/mpi/detail/packed_iprimitive.hpp
>
> boost/mpi/detail/packed_oprimitive.hpp
>
> boost/mpi/exception.hpp
>
> boost/mpi/group.hpp
>
> boost/mpi/packed_iarchive.hpp
>
> boost/mpi/packed_oarchive.hpp
>
> boost/mpi/request.hpp
>
> boost/mpi/skeleton_and_content.hpp
>
> boost/signals/connection.hpp
>
> boost/signals/detail/named_slot_map.hpp
>
> boost/signals/detail/signal_base.hpp
>
> boost/signals/slot.hpp
>
> boost/signals/trackable.hpp
>
> boost/spirit/home/support/terminal.hpp
>
>
>
> In the attached file, I include the longer file with all warnings, ordered
> by file, and without repetition. To a user it is formidable, but in fact it
> is pretty manageable. Surely it is worth the battle.
>
>
>
> I include the two files I modified to allow compilation of mpi graph
> components and a version of mpi.jam that works with msmpi v7 (
> https://www.microsoft.com/en-us/download/details.aspx?id=49926 and you
> need to install both parts) but which has not been simplified to take
> account of the Microsoft definitions for MPI:
>
>
>
> C:\Users\Terry>set msmpi
>
> MSMPI_BIN=C:\Program Files\Microsoft MPI\Bin\
>
> MSMPI_INC=C:\Program Files (x86)\Microsoft SDKs\MPI\Include\
>
> MSMPI_LIB32=C:\Program Files (x86)\Microsoft SDKs\MPI\Lib\x86\
>
> MSMPI_LIB64=C:\Program Files (x86)\Microsoft SDKs\MPI\Lib\x64\
>
>
>
> And clearly should be.
>
>
>
> To use boost successfully and easily in visual studio, one needs props
> files, to set includes, paths etc. Along with setting environmental
> variables such as boost root, props files are an integral part of using
> third party libraries with visual studio. The boost build process should
> generate one. For example one can find one that is reasonable at
> https://social.msdn.microsoft.com/Forums/vstudio/en-US/7c719bcb-2880-46a4-8698-01789e4841ae/updating-the-path-for-the-application-while-debugging-using-props-files?forum=visualstudiogeneral
>  (see below).
>
>
>
> However, it can only be done if there is standardisation about the
> locations of the various libraries relative to platform etc. and given the
> flexibility of boost build on this there is a problem. For what it is
> worth, I fix on %BOOST_ROOT%\win32\lib and %BOOST_ROOT%\x64\lib for all the
> binaries. This fixing is important for use in projects, macros etc.. The
> props file then provide an easy way forward.
>
>
>
> Finally, I have created with a colleague, a script which actually succeeds
> in building all components (including the mpi, python, iostreams,
> vs2008-15) error free and over pretty generic configurations. produced the
> files of warnings attached.  But it is not hardened enough for circulation.
>
>
>
> Great to have some feedback. How does one create a robust, correct, and
> joined up windows friendly and usable build of boost? I have no doubt this
> is a challenge for boost as it involved many projects and individuals in a
> coordinated push. But surely it is worth it.
>
>
>
> Terry
>
>
>
> ·        boost.props from
>
> ********************************
>
> <?xml version="1.0" encoding="utf-8"?>
> < Project ToolsVersion="4.0" xmlns="
> http://schemas.microsoft.com/developer/msbuild/2003">
>   <ImportGroup Label="PropertySheets" />
>   <PropertyGroup Label="UserMacros">
>     <BOOST_BIN>$(BOOST_ROOT)\$(Platform)\lib</BOOST_BIN>
>     <BOOST_LIB>$(BOOST_BIN)</BOOST_LIB>
>     <BOOST_INC>$(BOOST_ROOT)</BOOST_INC>
>     <Path>$(BOOST_BIN);$(Path)</Path>
>     <LocalDebuggerEnvironment>PATH=$(Path)</LocalDebuggerEnvironment>
>   </PropertyGroup>
>   <PropertyGroup />
>   <PropertyGroup>
>     <LibraryPath>$(BOOST_LIB);$(LibraryPath)</LibraryPath>
>   </PropertyGroup>
>   <PropertyGroup>
>     <IncludePath>$(BOOST_INC);$(IncludePath)</IncludePath>
>   </PropertyGroup>
>   <PropertyGroup>
>     <SourcePath>$(BOOST_INC);$(SourcePath)</SourcePath>
>   </PropertyGroup>
>   <ItemDefinitionGroup>
>   </ItemDefinitionGroup>
>   <ItemGroup>
>     <BuildMacro Include="BOOST_BIN">
>       <Value>$(BOOST_BIN)</Value>
>     </BuildMacro>
>     <BuildMacro Include="BOOST_LIB">
>       <Value>$(BOOST_LIB)</Value>
>     </BuildMacro>
>     <BuildMacro Include="BOOST_INC">
>       <Value>$(BOOST_INC)</Value>
>     </BuildMacro>
>     <BuildMacro Include="Path">
>       <Value>$(Path)</Value>
>     </BuildMacro>
>     <BuildMacro Include="LocalDebuggerEnvironment">
>       <Value>$(LocalDebuggerEnvironment)</Value>
>     </BuildMacro>
>   </ItemGroup>
> < /Project>
>
> **********************************************************************
>
>
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost-build
>
>