Lighweight header-only version of Boost.Filesystem?
I use Boost.Filesystem in a C++ "script": I use C++ instead of Python or shell script in order to be portable across operating systems. That script is used in bootstrapping the build system of my C++ project, and people have complained that they need to build boost.filesystem just to be able to bootstrap the project. This is made worse by the fact than the rest of my C++ project requires very recent versions of Boost headers and the fact that the ABI of filesystem has changed relatively recently. I had instances of people having problems because they were linking old versions of the filesystem library but using recent filesystem headers. They had a lot of trouble compiling the correct versions of filesystem themselves. I would also like to ship the script at installation time. To do that I statically link filesystem to reduce dependencies. My pretty trivial program, however, is more than 650kB in release mode after stripping (2.2 MB before stripping). All it does is walk a directory recursively and create a few files. I think this is too big. Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories The first point being the most important.
On 17/08/2011 14:56, Mathias Gaunard wrote:
I use Boost.Filesystem in a C++ "script": I use C++ instead of Python or shell script in order to be portable across operating systems.
That script is used in bootstrapping the build system of my C++ project, and people have complained that they need to build boost.filesystem just to be able to bootstrap the project.
For information, I ended up writing my own abstraction over the filesystem system functions. I still think it would be useful to have a minimal header-only part in boost.filesystem itself though.
On 8/19/2011 9:41 AM, Mathias Gaunard wrote:
On 17/08/2011 14:56, Mathias Gaunard wrote:
I use Boost.Filesystem in a C++ "script": I use C++ instead of Python or shell script in order to be portable across operating systems.
That script is used in bootstrapping the build system of my C++ project, and people have complained that they need to build boost.filesystem just to be able to bootstrap the project.
For information, I ended up writing my own abstraction over the filesystem system functions.
I still think it would be useful to have a minimal header-only part in boost.filesystem itself though.
Do you mean you made your own abstraction over the OS functions? What is the difference in size? Have you considered runtime-link=shared? And how is Python not portable across operating systems? :) The problem with a "minimal" subset is defining what "minimal" is. I don't know what causes the code "bloat" in boost::filesystem, but I suspect it's the template code which makes it relatively easy to deal with wide character paths the same way as narrow paths. Is there a way to profile code size to figure out what functions and classes are creating the most object code? -Matt
On 19/08/2011 17:04, Matthew Chambers wrote:
Do you mean you made your own abstraction over the OS functions?
I implemented something similar to the directory_iterator and other functions from filesystem. My interface is not exactly the same since I didn't bother with the path class, and made its members free functions on strings instead.
What is the difference in size?
My executable went from 2.2 MB to 63 kB (unstripped). But then I've also removed program_options at the same time. I think that with just filesystem I went down to 300 kB or less. This definitely needs more testing and "profiling".
Have you considered runtime-link=shared?
The numbers I gave were with the standard C and C++ libraries linked dynamically.
And how is Python not portable across operating systems? :)
I cannot rely on it being installed everywhere. I can rely on being able to run an executable with no dependencies though.
On Wed, Aug 17, 2011 at 8:56 AM, Mathias Gaunard < mathias.gaunard@ens-lyon.org> wrote:
I.....>Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories
The first point being the most important.
A header-only version would be a nice feature to offer. But it may be more time consuming to implement than it would appear. Vicente Botet tried to do the same with Boost.System, and ran into problems. It seems to me that there should be some general recipe to create header-only libraries with identical functionality to the compiled library version. Has anyone figured out what that recipe is? --Beman
2011/8/20 Beman Dawes <bdawes@acm.org>
On Wed, Aug 17, 2011 at 8:56 AM, Mathias Gaunard < mathias.gaunard@ens-lyon.org> wrote:
I.....>Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories
The first point being the most important.
A header-only version would be a nice feature to offer.
But it may be more time consuming to implement than it would appear. Vicente Botet tried to do the same with Boost.System, and ran into problems.
It seems to me that there should be some general recipe to create header-only libraries with identical functionality to the compiled library version. Has anyone figured out what that recipe is?
templated & tag-dispatched? (tag for platform or something...)
Le 20/08/11 18:07, TONGARI a écrit :
2011/8/20 Beman Dawes<bdawes@acm.org>
On Wed, Aug 17, 2011 at 8:56 AM, Mathias Gaunard< mathias.gaunard@ens-lyon.org> wrote:
I.....>Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories
The first point being the most important.
A header-only version would be a nice feature to offer.
But it may be more time consuming to implement than it would appear. Vicente Botet tried to do the same with Boost.System, and ran into problems. I reached to get a header-only version of Boost.System. The main issues raised by Beman were that we need to have a consensus on how all the header only libraries behave under cygwin respect to the WIN not WIN interface. IMO when we use cygwin we shouldn't use the Windows API, Bemans prefered to use it as it was available. If Beman is interested we can resume this task and see how several libraries can use the detail/win header only interface. It seems to me that there should be some general recipe to create header-only libraries with identical functionality to the compiled library version. Has anyone figured out what that recipe is?
I gues I have reached it for Boost.Chrono.
Best, Vicente
on Sat Aug 20 2011, "Vicente J. Botet Escriba" <vicente.botet-AT-wanadoo.fr> wrote:
Le 20/08/11 18:07, TONGARI a écrit :
2011/8/20 Beman Dawes<bdawes@acm.org>
On Wed, Aug 17, 2011 at 8:56 AM, Mathias Gaunard< mathias.gaunard@ens-lyon.org> wrote:
I.....>Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories
The first point being the most important.
A header-only version would be a nice feature to offer.
But it may be more time consuming to implement than it would appear. Vicente Botet tried to do the same with Boost.System, and ran into problems. I reached to get a header-only version of Boost.System. The main issues raised by Beman were that we need to have a consensus on how all the header only libraries behave under cygwin respect to the WIN not WIN interface. IMO when we use cygwin we shouldn't use the Windows API, Bemans prefered to use it as it was available.
What does "under cygwin" mean, exactly? It seems to me like a no-brainer that when the target OS is Cygwin, we should use the cygwin POSIX APIs, and when the target OS is Windows (as it might be when using cygwin GCC with the -mno-cygwin flag), we should use the Windows APIs. Is there anything more to this decision than that? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Le 24/08/11 04:41, Dave Abrahams a écrit :
on Sat Aug 20 2011, "Vicente J. Botet Escriba"<vicente.botet-AT-wanadoo.fr> wrote:
Le 20/08/11 18:07, TONGARI a écrit :
2011/8/20 Beman Dawes<bdawes@acm.org>
On Wed, Aug 17, 2011 at 8:56 AM, Mathias Gaunard< mathias.gaunard@ens-lyon.org> wrote:
I.....>Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories
The first point being the most important.
A header-only version would be a nice feature to offer.
But it may be more time consuming to implement than it would appear. Vicente Botet tried to do the same with Boost.System, and ran into problems. I reached to get a header-only version of Boost.System. The main issues raised by Beman were that we need to have a consensus on how all the header only libraries behave under cygwin respect to the WIN not WIN interface. IMO when we use cygwin we shouldn't use the Windows API, Bemans prefered to use it as it was available. What does "under cygwin" mean, exactly?
It seems to me like a no-brainer that when the target OS is Cygwin, we should use the cygwin POSIX APIs, and when the target OS is Windows (as it might be when using cygwin GCC with the -mno-cygwin flag), we should use the Windows APIs. Is there anything more to this decision than that?
Hi, thanks for the precisions. I guess you are right. I don't remember exactly the source of the problem, but when I tried to use Boost.Chrono and Boost.System inlined both libraries where not using the same interface. I will need to restore the context and see what was going wrong (Or find in this ML) Best, Vicente
Vicente Botet wrote:
Le 24/08/11 04:41, Dave Abrahams a écrit :
on Sat Aug 20 2011, "Vicente J. Botet Escriba"<vicente.botet-AT-wanadoo.fr> wrote:
Vicente Botet tried to do the same with Boost.System, and ran into problems. I reached to get a header-only version of Boost.System. The main issues raised by Beman were that we need to have a consensus on how all the header only libraries behave under cygwin respect to the WIN not WIN interface. IMO when we use cygwin we shouldn't use the Windows API, Bemans prefered to use it as it was available. What does "under cygwin" mean, exactly?
It seems to me like a no-brainer that when the target OS is Cygwin, we should use the cygwin POSIX APIs, and when the target OS is Windows (as it might be when using cygwin GCC with the -mno-cygwin flag), we should use the Windows APIs. Is there anything more to this decision than that?
I don't remember exactly the source of the problem, but when I tried to use Boost.Chrono and Boost.System inlined both libraries where not using the same interface.
Hi, I found where the problem was. Boost.System defines the global macro BOOST_WINDOWS_API when __CYGWIN__ is defined. # if defined(_WIN32) || defined(__CYGWIN__) // Windows default, including MinGW and Cygwin # define BOOST_WINDOWS_API # else # define BOOST_POSIX_API # endif This couldn't work if other Boost libraries defines also BOOST_POSIX_API in the same conditions. Best, Vicente -- View this message in context: http://boost.2283326.n4.nabble.com/Lighweight-header-only-version-of-Boost-F... Sent from the Boost - Dev mailing list archive at Nabble.com.
On Sat, Aug 20, 2011 at 6:38 AM, Beman Dawes <bdawes@acm.org> wrote:
On Wed, Aug 17, 2011 at 8:56 AM, Mathias Gaunard < mathias.gaunard@ens-lyon.org> wrote:
I.....>Therefore, I would really like it if there was a lightweight header-only version of Boost.Filesystem. I don't need the full extent of features; I just need to have the ability to do the following portably: - read all files of a directory - create directories - remove files and directories - rename files and directories
The first point being the most important.
A header-only version would be a nice feature to offer.
But it may be more time consuming to implement than it would appear. Vicente Botet tried to do the same with Boost.System, and ran into problems.
It seems to me that there should be some general recipe to create header-only libraries with identical functionality to the compiled library version. Has anyone figured out what that recipe is?
Here's something easy: new includes that have the implementation. Library authors don't need to worry about ODR issues -- let the user include it in only a single source file. I.e. my project would have a boost.cpp that consists of: #include <boost/filesystem.cpp> #include <boost/system.cpp> etc. -- Cory Nelson http://int64.org
On 21/08/2011 00:28, Cory Nelson wrote:
Here's something easy: new includes that have the implementation. Library authors don't need to worry about ODR issues -- let the user include it in only a single source file. I.e. my project would have a boost.cpp that consists of:
#include<boost/filesystem.cpp> #include<boost/system.cpp>
There are many reasons why that doesn't work. - The files are not in the include path of a typical Boost installation - The functions are not inline, and are even usually dllexport - Source files have no header guards, don't need to care about masking details, and can even use "using namespace".
participants (8)
-
Beman Dawes -
Cory Nelson -
Dave Abrahams -
Mathias Gaunard -
Matthew Chambers -
TONGARI -
Vicente Botet -
Vicente J. Botet Escriba