Relocating boost to a different namespace?
I have heard unconfirmed rumors that someone has a script that takes Boost code and moves it into a different top-level namespace. Does such a script really exist? The goal is to be able to automatically munge Boost code so that several versions can be used in parallel without conflicts. If anybody is currently doing this, I'd love to hear from them. -- Eric Niebler Boost Consulting www.boost-consulting.com
To use multiple versions in parallel wouldn't you also need to mangle the include guard symbol names to allow each version to independently remember which headers have been processed? Eric Niebler wrote:
I have heard unconfirmed rumors that someone has a script that takes Boost code and moves it into a different top-level namespace. Does such a script really exist?
The goal is to be able to automatically munge Boost code so that several versions can be used in parallel without conflicts. If anybody is currently doing this, I'd love to hear from them.
Glenn Schrader wrote:
Eric Niebler wrote:
I have heard unconfirmed rumors that someone has a script that takes Boost code and moves it into a different top-level namespace. Does such a script really exist?
The goal is to be able to automatically munge Boost code so that several versions can be used in parallel without conflicts. If anybody is currently doing this, I'd love to hear from them.
To use multiple versions in parallel wouldn't you also need to mangle the include guard symbol names to allow each version to independently remember which headers have been processed?
Yes, and all macro names would need to change, too. -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
I have heard unconfirmed rumors that someone has a script that takes Boost code and moves it into a different top-level namespace. Does such a script really exist?
The goal is to be able to automatically munge Boost code so that several versions can be used in parallel without conflicts. If anybody is currently doing this, I'd love to hear from them. To use multiple versions in parallel wouldn't you also need to mangle the include guard symbol names to allow each version to independently remember which headers have been processed?
Yes, and all macro names would need to change, too.
Not only this. Consider that some Boost file contains this: #include "boost/config.hpp" #include "boost/detail/workaround.hpp" // ... The physical meaning of "boost/..." is resolved thanks to the appropriate -I compiler option. If you want to have several versions in parallel, then I presume that you want to #include them together from your project - in which case the meaning (resolution) of "boost/..." would need to be different in each version, so that there are physically multiple header trees. This cannot be done with multiple -I options. This means that not only namespace names and macros, but also #include paths in Boost files should change to use separate path prefixes, like: #include "boost_X/config.hpp" #include "boost_X/detail/workaround.hpp" // ... Hint: Vera++ can do this. I already have transformation scripts that do all three tricks. They will be released together with one important bugfix. -- Maciej Sobczak * www.msobczak.com * www.inspirel.com
Maciej Sobczak wrote:
Not only this.
Consider that some Boost file contains this:
#include "boost/config.hpp" #include "boost/detail/workaround.hpp" // ...
The physical meaning of "boost/..." is resolved thanks to the appropriate -I compiler option. If you want to have several versions in parallel, then I presume that you want to #include them together from your project - in which case the meaning (resolution) of "boost/..." would need to be different in each version, so that there are physically multiple header trees. This cannot be done with multiple -I options.
This means that not only namespace names and macros, but also #include paths in Boost files should change to use separate path prefixes, like:
#include "boost_X/config.hpp" #include "boost_X/detail/workaround.hpp" // ...
My bcp code does this as well. In fact I did some experimentation with support of replacing boost with nested namespace ( boost --> myprod::boost )and there are code to handle this giving: #include "myprod/boost/config.hpp" #include "myprod/boost/detail/workaround.hpp" or whatever you specify. However the tricky part for nested namespace support was never solved. In principle you need something that can match end braces for the boost namespace in the code. I experimented with xpressive for this, but I have a feeling there are much more to it than nested brace matching. I would expect there to be unbalanced braces due to preprosessor conditionals for one thing. -- bjorn
Eric Niebler wrote:
Glenn Schrader wrote:
Eric Niebler wrote:
I have heard unconfirmed rumors that someone has a script that takes Boost code and moves it into a different top-level namespace. Does such a script really exist?
The goal is to be able to automatically munge Boost code so that several versions can be used in parallel without conflicts. If anybody is currently doing this, I'd love to hear from them.
To use multiple versions in parallel wouldn't you also need to mangle the include guard symbol names to allow each version to independently remember which headers have been processed?
Yes, and all macro names would need to change, too.
My modified version of bcp does all of the above as it copies a subset or all of boost into a new location. It has never been used in production, however I am convinced it is useful or usable if you like to call it that. As it has not been used in production some more testing is needed. My once urgent need faded of completly unrelated resons. I have posted a bcp patch to the list at least once: http://www.nabble.com/Re%3A-Using-multiple-versions-of-Boost-p10706158.html I did not find it on gmane or anything giving the attachments in any usable form. So I can dig it out, zip down the bcp directory and repost it if there is interest. By the way, does anybody know if bcp -cvs work correctly after the move to svn? -- Bjorn
Bjørn Roald wrote:
My modified version of bcp does all of the above as it copies a subset or all of boost into a new location. It has never been used in production, however I am convinced it is useful or usable if you like to call it that. As it has not been used in production some more testing is needed. My once urgent need faded of completly unrelated resons.
I have posted a bcp patch to the list at least once:
http://www.nabble.com/Re%3A-Using-multiple-versions-of-Boost-p10706158.html
Ah, I think I must have missed those patches :-(
I did not find it on gmane or anything giving the attachments in any usable form. So I can dig it out, zip down the bcp directory and repost it if there is interest.
By the way, does anybody know if bcp -cvs work correctly after the move to svn?
No, I'm afraid not. It's on my TODO list, but SVN uses a binary format for it's "entries" files, so it's not so easy to figure out what is and is not under version control. John.
John Maddock wrote:
Bjørn Roald wrote:
My modified version of bcp does all of the above as it copies a subset or all of boost into a new location. It has never been used in production, however I am convinced it is useful or usable if you like to call it that. As it has not been used in production some more testing is needed. My once urgent need faded of completly unrelated resons.
I have posted a bcp patch to the list at least once:
http://www.nabble.com/Re%3A-Using-multiple-versions-of-Boost-p10706158.html
Ah, I think I must have missed those patches :-(
Ok, I remerged toward svn trunk and posted in reply to Erik.
By the way, does anybody know if bcp -cvs work correctly after the move to svn?
No, I'm afraid not. It's on my TODO list, but SVN uses a binary format for it's "entries" files, so it's not so easy to figure out what is and is not under version control.
Hm, if it is binary, would using a svn library be acceptable? Maybe we could do it trough a dynamically loadable plug-in so it would not be required for the basic bcp tool. Anyways, I guess licensing may be an issue, but this is not a boost library, it is a tool. -- Bjørn
Bjørn Roald wrote:
Hm, if it is binary, would using a svn library be acceptable? Maybe we could do it trough a dynamically loadable plug-in so it would not be required for the basic bcp tool. Anyways, I guess licensing may be an issue, but this is not a boost library, it is a tool.
Indeed, I just haven't had the time to figure out what's required yet :-( John.
John Maddock wrote:
Bjørn Roald wrote:
Hm, if it is binary, would using a svn library be acceptable? Maybe we could do it trough a dynamically loadable plug-in so it would not be required for the basic bcp tool. Anyways, I guess licensing may be an issue, but this is not a boost library, it is a tool.
Indeed, I just haven't had the time to figure out what's required yet :-(
I looked at the svn library sources a bit. I concluded that this is relatively simple text parsing. There is one important point to note. The format of the entries file has changes from XML to a simpler text format. Older clients may still use the old format and that will not work. New clients will convert the files automatically as they are used the first time. Hence I decided to ignore the XML format. I wrote a function that I added to scan_cvs_path.cpp, see attachment. It fill the m_cvs_path map with data as it is done in the original cvs version. The only tricky part was getting to the mime info, which are in separate property files, to determine if files are text or binary. Also you have to determine whether various mime types map to binary or text mode. In my function, if a file does not have mime property, it is assumed to be text. If it has mime property starting with anything other than "text/" then it is assumed to be binary. I tested in my version of bcp and it seems to work fine. I added an additional --svn flagg for the command line and the other obvious control logic changes. Maybe people are using bcp automatically for various boost releases, in that case we could make --cvs and --svn aliases, and automatically check for entries files in CVS or .svn directories respectively to check which mode to set. I can provide a patch, but my code is kind of tangled with my other changes for the replace namespace stuff, so did not post that now. If you want me post a patch with or without the namespace stuff, please tell me. Note. I struggled with multi line regexp for tokenizer matching, urrr.. I gave up and did it old fashion plain and simple. I think my logic may be cleaned up and made simpler with less nested if statements by using more clever regexp. However, as it is now, it works. :-) -- Bjørn
Bjørn Roald wrote:
I looked at the svn library sources a bit. I concluded that this is relatively simple text parsing. There is one important point to note. The format of the entries file has changes from XML to a simpler text format. Older clients may still use the old format and that will not work. New clients will convert the files automatically as they are used the first time. Hence I decided to ignore the XML format.
I wrote a function that I added to scan_cvs_path.cpp, see attachment. It fill the m_cvs_path map with data as it is done in the original cvs version. The only tricky part was getting to the mime info, which are in separate property files, to determine if files are text or binary. Also you have to determine whether various mime types map to binary or text mode.
In my function, if a file does not have mime property, it is assumed to be text. If it has mime property starting with anything other than "text/" then it is assumed to be binary. I tested in my version of bcp and it seems to work fine. I added an additional --svn flagg for the command line and the other obvious control logic changes. Maybe people are using bcp automatically for various boost releases, in that case we could make --cvs and --svn aliases, and automatically check for entries files in CVS or .svn directories respectively to check which mode to set.
I can provide a patch, but my code is kind of tangled with my other changes for the replace namespace stuff, so did not post that now. If you want me post a patch with or without the namespace stuff, please tell me.
Note. I struggled with multi line regexp for tokenizer matching, urrr.. I gave up and did it old fashion plain and simple. I think my logic may be cleaned up and made simpler with less nested if statements by using more clever regexp. However, as it is now, it works. :-)
Yup, I've applied your SVN code to SVN HEAD: I made a minor change to the properties regex as some entries had a blank space at the start of the line: not sure if this is a bug in our property settings or what, but they seem to work OK on the server, so I'm assuming we should allow for this :-) Many thanks, John.
John Maddock wrote:
Bjørn Roald wrote:
I looked at the svn library sources a bit. I concluded that this is relatively simple text parsing. <snip>
Yup, I've applied your SVN code to SVN HEAD
I don't see any change to either bcp's usage message or its documentation. What are the new features and how do I use them? -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
John Maddock wrote:
Bjørn Roald wrote:
I looked at the svn library sources a bit. I concluded that this is relatively simple text parsing. <snip>
Yup, I've applied your SVN code to SVN HEAD
I don't see any change to either bcp's usage message or its documentation. What are the new features and how do I use them?
Just support for --svn which does what --cvs used to do. I haven't looked at the namespace renaming code yet :-( John.
Eric Niebler wrote:
John Maddock wrote:
Bjørn Roald wrote:
I looked at the svn library sources a bit. I concluded that this is relatively simple text parsing.
<snip>
Yup, I've applied your SVN code to SVN HEAD
I don't see any change to either bcp's usage message or its documentation. What are the new features and how do I use them?
If you expected the namespace renaming code you need to pull in the patch I posted a few days ago. To be honest it would be great if you where able to try this out so I can get your opinion on the approach before John spend a lot of time on it. I am willing to help fixing any sharp edges you may find. -- Bjørn
on Tue Oct 16 2007, "John Maddock" <john-AT-johnmaddock.co.uk> wrote:
Bjørn Roald wrote:
My modified version of bcp does all of the above as it copies a subset or all of boost into a new location. It has never been used in production, however I am convinced it is useful or usable if you like to call it that. As it has not been used in production some more testing is needed. My once urgent need faded of completly unrelated resons.
I have posted a bcp patch to the list at least once:
http://www.nabble.com/Re%3A-Using-multiple-versions-of-Boost-p10706158.html
Ah, I think I must have missed those patches :-(
I did not find it on gmane or anything giving the attachments in any usable form. So I can dig it out, zip down the bcp directory and repost it if there is interest.
By the way, does anybody know if bcp -cvs work correctly after the move to svn?
No, I'm afraid not. It's on my TODO list, but SVN uses a binary format for it's "entries" files, so it's not so easy to figure out what is and is not under version control.
Huh, looks plaintext to me. -- Dave Abrahams Boost Consulting http://www.boost-consulting.com
David Abrahams wrote:
on Tue Oct 16 2007, "John Maddock" <john-AT-johnmaddock.co.uk> wrote:
Bjørn Roald wrote:
By the way, does anybody know if bcp -cvs work correctly after the move to svn?
No, I'm afraid not. It's on my TODO list, but SVN uses a binary format for it's "entries" files, so it's not so easy to figure out what is and is not under version control.
Huh, looks plaintext to me.
Looks like plain text to me as well. I googled a little late last night but did not come up with much other than discussions whether it was UTF-8 or not. It was pointed out by several postings that it should be UTF-8 encoded even after the format had changed from XML. When I get some time I will check libsvn sources for the truth ;-) -- Bjorn
Bjørn Roald wrote:
Eric Niebler wrote:
Glenn Schrader wrote:
Eric Niebler wrote:
I have heard unconfirmed rumors that someone has a script that takes Boost code and moves it into a different top-level namespace. Does such a script really exist?
The goal is to be able to automatically munge Boost code so that several versions can be used in parallel without conflicts. If anybody is currently doing this, I'd love to hear from them.
To use multiple versions in parallel wouldn't you also need to mangle the include guard symbol names to allow each version to independently remember which headers have been processed?
Yes, and all macro names would need to change, too.
My modified version of bcp does all of the above as it copies a subset or all of boost into a new location. It has never been used in production, however I am convinced it is useful or usable if you like to call it that. As it has not been used in production some more testing is needed. My once urgent need faded of completly unrelated resons.
I have posted a bcp patch to the list at least once:
http://www.nabble.com/Re%3A-Using-multiple-versions-of-Boost-p10706158.html
I did not find it on gmane or anything giving the attachments in any usable form. So I can dig it out, zip down the bcp directory and repost it if there is interest.
Sounds very interesting! Yes, there is interest. Please send the patches around again. Vera++ also sounds good too. I'll look at both. -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
Bjørn Roald wrote:
Sounds very interesting! Yes, there is interest. Please send the patches around again.
I have merged with the latest svn trunk. Are there any guidelines for producing a patch with svn? Attached is a patch generated with plain 'svn diff' command in the trunk directory. In addition I attached a file with some notes. I have just merged the code, compiled bcp and done a simple inspection of some of the generated files. If you are interested in how I tested earlier, I basically ran the status directory jamfile on all the converted code. I.e. including all also automatically converted regression tests. At one point I was down to less than 10 failures that was not in the boost sources I converted from. These seemed mostly to be related to workarounds in Jamfiles to Boost.Build v1 limitations. A good place to start reading a list thread on this from 2005 is here: http://aspn.activestate.com/ASPN/Mail/Message/boost/2837479 Also worth noting, I think, is that I introduce the --diff-only command for bcp. This enable reduction in rebuilds as only generated source files which are different from existing files in destination causes an overwrite. My basic idea is to facilitate maintenance of boost in context of the standard boost namespace. Using bcp to generate boost sources in different namespace is just a build step. Boost SCM should be in boost svn only if that is feasable. Note also that the --cvs option does not work in svn, so there are currently a limitation in the usefulness of bcp. By the way, is the prefered way to post this on the mailinglist or in trac or wiki? John, if you considder applying the patch, you should look at some of the diagnostics cout and some of the TODOs marked. I would be willing to help solving issues including the support for svn if this become official bcp features.
Vera++ also sounds good too. I'll look at both.
Vera++ may also provide a good path to supporting replacing boost with nested namespaces as I think it may be capable of reliably locating end braces of the original boost namespace. To approaches comes to mind. Use Vera++ to modify code directly or use it to enforce tagging of ending braces like this: } // end namespace boost In which case the rest would be trivial in bcp. -- bjorn This file contains notes regarding enhansements to BOOST_ROOT/tools/bcp to support replacement of the boost namespace in sourcecode copied from the boost distrubution. ==================== HOWTO ====================== A new option: --namespace=ns set a namespace that will replace 'boost' is added to the bcp tool. This allow the user to specify a namespace to use as replacement for boost in exported source code. example use:
cd ~/src/boost mkdir -p /tmp/testbcp/mylib tools/bcp/run/bcp -cvs -namespace=mylib filesystem /tmp/testbcp/mylib
later you can use
tools/bcp/run/bcp -cvs --namespace=spoost --diff-only filesystem /tmp/testbcp/mylib
to only do the changes You can add boost-build.jam Jamfile Jamrules to the module-list on the bcp command line to get the top level build files needed to build the exported code. If you change the following 2 lines in the exported Jamfile [ glob-tree $(BOOST_ROOT)/boost/compatibility/cpp_c_headers : c* ] [ glob-tree $(BOOST_ROOT)/boost : *.hpp *.ipp *.h *.inc ] to [ glob-tree $(BOOST_ROOT)/mylib/compatibility/cpp_c_headers : c* ] [ glob-tree $(BOOST_ROOT)/mylib : *.hpp *.ipp *.h *.inc ] you can actually build the exported libraries with bjam. NOTE: see limitation regarding Boost.Build ================== TODO ========================= 1. Support replacing namespace boost with nested namespace, e.g. namespace boost ====> namespace mylib::boost or namespace boost ====> namespace mylib::stuff::platform 2. Better Boost.Build and Jamfile support in export =============== LIMITATIONS ====================== 1. Only replacement of boost namespace with single level namespace is suported. No replacement with nested namespace supported jet. 2. Boost.Build export not suported properly. Buildfiles are still configured for the "boost" name after export. E.g. libriaries built contain the string "boost" in their filenames. Index: tools/bcp/file_types.cpp =================================================================== --- tools/bcp/file_types.cpp (revision 40093) +++ tools/bcp/file_types.cpp (working copy) @@ -9,6 +9,7 @@ * void bcp_implementation::is_source_file(const fs::path& p) * void bcp_implementation::is_html_file(const fs::path& p) * void bcp_implementation::is_binary_file(const fs::path& p) + * void bcp_implementation::is_bjam_file(const fs::path& p) */ #include "bcp_imp.hpp" @@ -19,8 +20,8 @@ static const boost::regex e( ".*\\." "(?:" - "c|cxx|h|hxx|inc|.?pp|yy?" - ")", + "c|cxx|h|hxx|inl|inc|.?pp|yy?" + ")", boost::regex::perl | boost::regex::icase ); return boost::regex_match(p.string(), e); @@ -50,8 +51,23 @@ "c|cxx|cpp|h|hxx|hpp|inc|html?|css|mak|in" ")" "|" - "(Jamfile|makefile|configure)", + "(Jamfile|Jamfile\\.v2|Jamrules|Jambase|makefile|configure)", boost::regex::perl | boost::regex::icase); return !boost::regex_match(p.leaf(), e); } + + +bool bcp_implementation::is_bjam_file(const fs::path& p) +{ + static const boost::regex e( + ".*\\." + "(?:" + "jam" + ")" + "|" + "(Jamfile|Jamfile\\.v2|Jamrules|Jambase)", + boost::regex::perl | boost::regex::icase); + return boost::regex_match(p.leaf(), e); + +} Index: tools/bcp/add_path.cpp =================================================================== --- tools/bcp/add_path.cpp (revision 40093) +++ tools/bcp/add_path.cpp (working copy) @@ -123,6 +123,11 @@ // or we'll get an error: if(s.compare(0, 2, "./") == 0) s.erase(0, 2); + // + // if the name contain .html# , remove the html bookmark + // or we'll get warnings of none existing files + if(s.find(".html#") != std::string::npos) + s.erase(s.find(".html#")+5); if(s.find(':') == std::string::npos) { // only concatonate if it's a relative path Index: tools/bcp/bcp_imp.cpp =================================================================== --- tools/bcp/bcp_imp.cpp (revision 40093) +++ tools/bcp/bcp_imp.cpp (working copy) @@ -18,7 +18,7 @@ #include <string> bcp_implementation::bcp_implementation() - : m_list_mode(false), m_list_summary_mode(false), m_license_mode(false), m_cvs_mode(false), m_unix_lines(false), m_scan_mode(false), m_bsl_convert_mode(false), m_bsl_summary_mode(false) + : m_list_mode(false), m_list_summary_mode(false), m_license_mode(false), m_cvs_mode(false), m_unix_lines(false), m_diff_only_mode(false), m_scan_mode(false), m_bsl_convert_mode(false), m_bsl_summary_mode(false) { } @@ -71,6 +71,11 @@ m_unix_lines = true; } +void bcp_implementation::enable_diff_only_mode() +{ + m_diff_only_mode = true; +} + void bcp_implementation::set_boost_path(const char* p) { m_boost_path = fs::path(p, fs::native); Index: tools/bcp/bcp.hpp =================================================================== --- tools/bcp/bcp.hpp (revision 40093) +++ tools/bcp/bcp.hpp (working copy) @@ -21,11 +21,13 @@ virtual void enable_summary_list_mode() = 0; virtual void enable_cvs_mode() = 0; virtual void enable_unix_lines() = 0; + virtual void enable_diff_only_mode() = 0; virtual void enable_scan_mode() = 0; virtual void enable_license_mode() = 0; virtual void enable_bsl_convert_mode() = 0; virtual void enable_bsl_summary_mode() = 0; virtual void set_boost_path(const char* p) = 0; + virtual void set_replacement_namespace(const char* ns) = 0; virtual void set_destination(const char* p) = 0; virtual void add_module(const char* p) = 0; Index: tools/bcp/scan_cvs_path.cpp =================================================================== --- tools/bcp/scan_cvs_path.cpp (revision 40093) +++ tools/bcp/scan_cvs_path.cpp (working copy) @@ -57,7 +57,6 @@ ++i; m_cvs_paths[file] = binary; } - } } } Index: tools/bcp/Jamfile.v2 =================================================================== --- tools/bcp/Jamfile.v2 (revision 40093) +++ tools/bcp/Jamfile.v2 (working copy) @@ -7,6 +7,7 @@ add_path.cpp bcp_imp.cpp copy_path.cpp file_types.cpp fileview.cpp main.cpp path_operations.cpp scan_cvs_path.cpp licence_info.cpp scan_licence.cpp output_licence_info.cpp + replace_namespace.cpp /boost/filesystem//boost_filesystem /boost/regex//boost_regex /boost/test//boost_prg_exec_monitor Index: tools/bcp/bcp_imp.hpp =================================================================== --- tools/bcp/bcp_imp.hpp (revision 40093) +++ tools/bcp/bcp_imp.hpp (working copy) @@ -52,11 +52,13 @@ void enable_summary_list_mode(); void enable_cvs_mode(); void enable_unix_lines(); + void enable_diff_only_mode(); void enable_scan_mode(); void enable_license_mode(); void enable_bsl_convert_mode(); void enable_bsl_summary_mode(); void set_boost_path(const char* p); + void set_replacement_namespace(const char* ns); void set_destination(const char* p); void add_module(const char* p); @@ -68,10 +70,15 @@ void add_directory(const fs::path& p); void add_file(const fs::path& p); void copy_path(const fs::path& p); + void replace_namespace(const fs::path& p); + fs::path destination_path(const fs::path p, std::string seperator = "/"); void add_file_dependencies(const fs::path& p, bool scanfile); bool is_source_file(const fs::path& p); bool is_html_file(const fs::path& p); bool is_binary_file(const fs::path& p); + bool is_bjam_file(const fs::path& p); + bool diff_only_copy( const std::string& new_file, + fs::path destination_path ); void add_dependent_lib(const std::string& libname, const fs::path& p); void create_path(const fs::path& p); // license code: @@ -84,6 +91,8 @@ bool m_license_mode; // generate license information for files listed bool m_cvs_mode; // check cvs for files bool m_unix_lines; // fix line endings + bool m_diff_only_mode; // diff only mode + bool m_replace_namespace; // replace the boost namespace bool m_scan_mode; // scan non-boost files. bool m_bsl_convert_mode; // try to convert to the BSL bool m_bsl_summary_mode; // summarise BSL issues only Index: tools/bcp/main.cpp =================================================================== --- tools/bcp/main.cpp (revision 40093) +++ tools/bcp/main.cpp (working copy) @@ -33,10 +33,13 @@ "\n" "Options:\n" " --boost=path sets the location of the boost tree to path\n" - " --scan treat the module list as a list of (possibly non-boost)\n" + " --namespace=ns set a namespace that will replace 'boost'\n" + " --scan treat the module list as a list of (possibly non-boost)\n" " files to scan for boost dependencies\n" " --cvs only copy files under cvs version control\n" " --unix-lines make sure that all copied files use Unix style line endings\n" + " --diff-only only overwrite files when new file differs from\n" + " existing original. May save you rebuilds.\n" "\n" "module-list: a list of boost files or library names to copy\n" "html-file: the name of a html file to which the report will be written\n" @@ -123,12 +126,21 @@ { papp->enable_unix_lines(); } + else if(0 == std::strcmp("--diff-only", argv[i])) + { + papp->enable_diff_only_mode(); + } else if(0 == std::strncmp("--boost=", argv[i], 8)) { papp->set_boost_path(argv[i] + 8); } + else if(0 == std::strncmp("--namespace=", argv[i], 12)) + { + papp->set_replacement_namespace(argv[i] + 12); + } else if(argv[i][0] == '-') { + std::cout << "wrong argument: " << argv[i] << std::endl; show_usage(); return 1; } Index: tools/bcp/copy_path.cpp =================================================================== --- tools/bcp/copy_path.cpp (revision 40093) +++ tools/bcp/copy_path.cpp (working copy) @@ -20,35 +20,50 @@ void bcp_implementation::copy_path(const fs::path& p) { assert(!fs::is_directory(m_boost_path / p)); - if(fs::exists(m_dest_path / p)) - { - std::cout << "Copying (and overwriting) file: " << p.string() << "\n"; - fs::remove(m_dest_path / p); - } - else - std::cout << "Copying file: " << p.string() << "\n"; + fs::path dest_p = destination_path( p ); + // // create the path to the new file if it doesn't already exist: // - create_path(p.branch_path()); + create_path(dest_p.branch_path()); // // do text based copy if requested: // - if(m_unix_lines && !is_binary_file(p)) + if(m_replace_namespace && (is_source_file(p) || is_bjam_file(p))) { - std::ifstream is((m_boost_path / p).native_file_string().c_str()); - std::istreambuf_iterator<char> isi(is); - std::istreambuf_iterator<char> end; - - std::ofstream os((m_dest_path / p).native_file_string().c_str(), std::ios_base::binary | std::ios_base::out); - std::ostreambuf_iterator<char> osi(os); - - std::copy(isi, end, osi); + replace_namespace(p); } else { - // binary copy: - fs::copy_file(m_boost_path / p, m_dest_path / p); + + if( fs::exists(m_dest_path / dest_p)) + { + // TODO: need some support for diff mode here, for now + // diff_mode is only supported within replace_namespace(p); + + std::cout << "Copying (and overwriting) file: " << p.string() << "\n"; + fs::remove(m_dest_path / dest_p ); + } + else + std::cout << "Copying file: " << p.string() << "\n"; + + if(m_unix_lines && !is_binary_file(p)) + { + std::ifstream is((m_boost_path / p).native_file_string().c_str()); + std::istreambuf_iterator<char> isi(is); + std::istreambuf_iterator<char> end; + + std::ofstream os( ( m_dest_path / dest_p ).native_file_string().c_str(), + std::ios_base::binary | std::ios_base::out); + std::ostreambuf_iterator<char> osi(os); + + std::copy(isi, end, osi); + } + else + { + // binary copy: + fs::copy_file(m_boost_path / p, m_dest_path / dest_p); + } } } Index: tools/bcp/replace_namespace.cpp =================================================================== --- tools/bcp/replace_namespace.cpp (revision 0) +++ tools/bcp/replace_namespace.cpp (revision 0) @@ -0,0 +1,437 @@ +/* + * + * Copyright (c) 2005 Bjorn Roald + * Use, modification and distribution is subject to the + * Boost Software License, Version 1.0. (See accompanying file + * LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) + * + * This file implements the following: + * void bcp_implementation::destination_path(const fs::path& p) + * void bcp_implementation::copy_path_and_replace_namespace(const fs::path& p) + */ + +#include "bcp_imp.hpp" +#include <boost/filesystem/operations.hpp> +#include <boost/algorithm/string.hpp> +#include <boost/regex.hpp> +#include <boost/tokenizer.hpp> +#include <fstream> +#include <sstream> +#include <streambuf> +#include <iterator> +#include <algorithm> +#include <set> +#include <iostream> + + +namespace +{ + + std::string expression_text; + std::string format_string; + std::string jamfile_expression_text; + std::string jamfile_format_string; + std::string header_text; + std::string footer_text; + std::string replacement_ns; + std::string format_replacement_ns; + std::string underscored_ns; + std::string slashed_ns; + std::string dashed_ns; + boost::regex expression_cpp; + boost::regex expression_jam; + + // some default data + const char* header_text_default = + "The following file was processed and modified with the bcp tool\n" + "to change namespace and paths to include files. The intention \n" + "of this processing is to allow non-boost library developers \n" + "to use the boost libararies internally in their product without \n" + "risking potential conflicts with the use of other versions \n" + "of boost together with their product."; + + const char* footer_text_default = " ** end of bcp generated file **"; + + const int LL = 80; + + std::string make_comment(const fs::path& p, + const std::string& comment) + { +// std::cout << "in make_comment()\n"; + using std::string; + string newline = "\n"; // TODO: need to handle configurable dos/unix/mac differences + string block_prefix; + string block_top_border; + string block_bottom_border; + string line_prefix; + string block_postfix; + string escape_this_string; + string escaped_string; + + // c++ style + static const boost::regex e_cpp( + ".*\\." + "(?:" + "cxx|cpp|h|hxx|hpp|inc" + ")", + boost::regex::perl | boost::regex::icase); + if ( boost::regex_match(p.leaf(), e_cpp) ) + { + block_top_border = "//" + std::string( LL-4, '-' ); + line_prefix = "// "; + block_bottom_border = block_top_border; + } + // shell style + else + { + static const boost::regex e_shell( + ".*\\." + "(?:" + "ch|in|jam|mk" + ")|" + "(makefile|Jamfile|Jamrules|configure)", + boost::regex::perl | boost::regex::icase); + if ( boost::regex_match(p.leaf(), e_shell) ) + { + block_top_border += "#" + std::string( LL-4, '=' ); + line_prefix = "# "; + block_bottom_border = block_top_border; + } + else + { + // c style + static const boost::regex e_c( + ".*\\." + "(?:" + "c|css" + ")", + boost::regex::perl | boost::regex::icase); + if ( boost::regex_match(p.leaf(), e_c) ) + { + block_prefix = "/* "; + line_prefix = " * "; + block_postfix = " */"; + block_top_border += "/" + std::string( LL-4, '*' ); + block_bottom_border += " " + std::string( LL-4, '*' ) + + '/'; + escape_this_string = "\\*/"; + escaped_string = ""; + } + } + } + + // if we don't know how, we silently do not generate the comment + if ( block_prefix == block_postfix && + block_postfix == line_prefix && + line_prefix.size() == 0 ) + return string(); + + // fix any illegal internal character sequences + string escaped_comment; + if ( escape_this_string.size() > 0 ) + escaped_comment = boost::regex_replace( + comment, + boost::regex(escape_this_string), + escaped_string ); + else + escaped_comment = comment; + + string block; + + // generate single line comment + if(escaped_comment.find('\n') == string::npos) + { + if ( block_prefix.size() == 0 ) + block += line_prefix + escaped_comment; + else + block += block_prefix + escaped_comment + block_postfix; + } + else // generate multi line + { + block = block_top_border + newline + line_prefix; + for ( unsigned int i = 0 ; i < escaped_comment.size() ; ++i ) + { + if ( escaped_comment[i] == '\n' ) + block += newline + line_prefix; + else + block += escaped_comment[i]; + } + block += newline + block_bottom_border + newline; + + } + +// std::cout << "end make_comment()\n"; + return block; + } + + // list of directories to ignore, ie. leave files unchanged + std::set<fs::path, path_less> leave_dirs; + + // list of files to ignore, ie. leave unchanged + std::set<fs::path, path_less> leave_files; + + +} + + + +bool bcp_implementation::diff_only_copy( const std::string& new_file, + fs::path destination_path ) +{ + bool diff_found = false; + fs::path tmp_path = destination_path.string() + ".bcptmp"; + + { + // write temp file scope + fs::remove( tmp_path ); + std::ofstream tmp_fs(tmp_path.native_file_string().c_str()); + tmp_fs << new_file; // write to tmp destination file + } // close file at end of scope + + if ( fs::exists( destination_path ) && + fs::file_size( destination_path ) == fs::file_size( tmp_path ) ) + { + std::ifstream dest( destination_path.string().c_str() ); + std::ifstream tmp( tmp_path.string().c_str() ); + + diff_found = !std::equal(std::istream_iterator<char>( tmp ), + std::istream_iterator<char>(), + std::istream_iterator<char>( dest ) ); + } + else + diff_found = true; + + if ( diff_found ) // move tmp to destination + { + fs::remove( destination_path ); + fs::rename( tmp_path, destination_path ); + } + else + fs::remove( tmp_path ); + return diff_found; +} + + +void bcp_implementation::set_replacement_namespace(const char* ns) +{ + replacement_ns = ns; + m_replace_namespace = true; + + // set default data, TODO: improve so header and footer can be configured + header_text = header_text_default; + footer_text = footer_text_default; + + boost::regex re("::"); + underscored_ns = boost::regex_replace(replacement_ns, re, "_"); + slashed_ns = boost::regex_replace(replacement_ns, re, "/"); + dashed_ns = boost::regex_replace(replacement_ns, re, "-"); + + // escape hell follows, format string for replacing :: need to be "\:\:" + // to avoid colons being interpetet as else in conditional format, + // see http://www.boost.org/libs/regex/doc/format_boost_syntax.html + format_replacement_ns = boost::regex_replace(replacement_ns, re, "\\\\:\\\\:"); + + + std::string macrofied_ns( underscored_ns ); + boost::to_upper( macrofied_ns ); + + std::cout << "replacement_ns = " << replacement_ns << std::endl + << "format_replacement_ns = " << format_replacement_ns << std::endl + << "underscored_ns = " << underscored_ns << std::endl + << "slashed_ns = " << slashed_ns << std::endl + << "dashed_ns = " << dashed_ns << std::endl + << "macrofied_ns = " << macrofied_ns << std::endl; + + + expression_text = + // don't replace referances: index 1 + "(www.boost.org)|" + // replace namespace: index 2 + "(boost)|" + // macro defs: index 3 + "(BOOST)" + ; + + expression_cpp = boost::regex(expression_text); + + format_string = format_string + + "(?1www.boost.org)" + + "(?2" + format_replacement_ns + ")" + + "(?3" + macrofied_ns + ")"; + + std::cout << "expression_text = " << expression_text << std::endl; + std::cout << "format_string = " << format_string << std::endl; + + jamfile_expression_text = + // don't replace referances: index 1 + "(www.boost.org)|" + // macro defs: index 2 + "(BOOST)|" + // paths to include dir: index 3 + "(/boost/)" + ; + + expression_jam = boost::regex(jamfile_expression_text); + + jamfile_format_string = jamfile_format_string + + "(?1www.boost.org)" + + "(?2" + macrofied_ns + ")" + + "(?3/" + slashed_ns + "/)"; + + +// leave_dirs.insert( "libs/wave/test/testwave/testfiles"); + leave_files.insert( "libs/wave/test/testwave/testfiles/t_9_014.cpp" ); + leave_files.insert( "libs/wave/test/testwave/testfiles/t_9_015.cpp" ); + leave_files.insert( "libs/wave/test/testwave/testfiles/t_1_001.cpp" ); + leave_files.insert( "libs/wave/test/testwave/testfiles/t_1_002.cpp" ); + leave_files.insert( "libs/wave/test/testwave/testfiles/t_1_003.cpp" ); + leave_files.insert( "libs/wave/test/testwave/testfiles/t_1_004.cpp" ); +} + +fs::path bcp_implementation::destination_path(fs::path p, + std::string seperator) +{ +// std::cout << "in destination_path\n"; + using std::string; + if ( ! m_replace_namespace ) + return p; + fs::path result; + + // check for directory named "boost" in in path by + // by checking elements in path and rebuilding + // the path with replacement + for ( fs::path::iterator el = p.begin(); el != p.end(); ++el ) + { + string element; + if ( *el == "boost" ) // directory named boost + { + element = slashed_ns; + } + else if ( el->find( "boost" ) != string::npos && + *el != "boost-build.jam" && + *el != "boost-base.jam" ) + { + // string "boost" is part of a directory/file name + // need to determine how to replace it, use underscore for now + element = boost::regex_replace( *el, + boost::regex( "boost" ), + underscored_ns ); + } + else + { + element = *el; + } + + result /= element; + } + + return result; +} + +// intended to be called from inside copy_path() method +void bcp_implementation::replace_namespace(const fs::path& p) +{ +// std::cout << "in replace_namespace\n"; + assert(m_replace_namespace && !is_binary_file(p)); + fs::path dest_p = destination_path( p ); + try + { +// std::cout << "Processing file " +// << ( m_boost_path / p).native_file_string().c_str() +// << std::endl; + std::ifstream fs(( m_boost_path / p).native_file_string().c_str()); + std::string in; + in.reserve(fs.rdbuf()->in_avail()); + char c; + while(fs.get(c)) + { + if(in.capacity() == in.size()) + in.reserve(in.capacity() * 3); + in.append(1, c); + } + + bool leave_it_unchanged = leave_dirs.find( p.branch_path() ) != leave_dirs.end() || + leave_files.find( p ) != leave_files.end(); + + std::stringstream os; + if ( ! leave_it_unchanged ) + { + std::string file_hdr = header_text; + file_hdr += "namespace 'boost' has been replaced with namespace '" + + replacement_ns + "'.\n" + + "original file: BOOST_PATH/" + p.string(); + + + os << make_comment( p, file_hdr ); + + std::ostream_iterator<char, char> out(os); + if ( is_bjam_file( p ) ) + { + boost::regex_replace(out, + in.begin(), + in.end(), + expression_jam, + jamfile_format_string, + boost::match_default | boost::format_all); + } + else + { + boost::regex_replace(out, + in.begin(), + in.end(), + expression_cpp, + format_string, + boost::match_default | boost::format_all); + } + os << make_comment( p, footer_text_default ) << std::endl; + + } + else + { + os << in; + } + + fs::path dest = m_dest_path / dest_p; + + // diff against existing destination file + // then write only if there is a difference, purpose + // of this is to avoid excessive rebuilds when maintaining + // boost code and testing in a bcp generated source tree + + if ( m_diff_only_mode ) + { + if ( diff_only_copy( os.str(), dest ) ) + { + std::cout << "Writing diff:\t" << dest.string() << std::endl; +// fs::remove( dest ); +// std::ofstream ofs(dest.native_file_string().c_str()); +// ofs << os.str(); // write to destination file + } + else + { + std::cout << "Identical:\t" << dest.string() << std::endl; + } + } + else + { + std::cout << "Writing: \t" << dest.string() << std::endl; + fs::remove( dest ); + std::ofstream ofs(dest.native_file_string().c_str()); + ofs << os.str(); // write to destination file + } + + } + catch(std::exception& e) + { + std::cout << "error: replace_namespace: " << e.what() << "\n"; + exit(1); + } + catch(...) + { + std::cout << "error: replace_namespace : unexpected exception \n"; + exit(1); + } + +} + +
Bjørn Roald wrote:
Eric Niebler wrote:
Vera++ also sounds good too. I'll look at both.
Vera++ may also provide a good path to supporting replacing boost with nested namespaces as I think it may be capable of reliably locating end braces of the original boost namespace. To approaches comes to mind. Use Vera++ to modify code directly or use it to enforce tagging of ending braces like this:
} // end namespace boost
In which case the rest would be trivial in bcp.
Also, Vera++ could be useful to fix potential issues such as include guards without BOOST in symbols: #ifndef PROGRAM_OPTIONS_VP_2003_05_19 #define PROGRAM_OPTIONS_VP_2003_05_19 should be #ifndef BOOST_PROGRAM_OPTIONS_VP_2003_05_19 #define BOOST_PROGRAM_OPTIONS_VP_2003_05_19 allowing the bcp translation to replace it with: #ifndef MYLIB_BOOST_PROGRAM_OPTIONS_VP_2003_05_19 #define MYLIB_BOOST_PROGRAM_OPTIONS_VP_2003_05_19 which would be much safer in allowing proper side-by-side headers from two versions of boost. -- Bjørn
Bjørn Roald wrote:
Eric Niebler wrote:
Bjørn Roald wrote: Sounds very interesting! Yes, there is interest. Please send the patches around again.
I have merged with the latest svn trunk. Are there any guidelines for producing a patch with svn?
Attached is a patch generated with plain 'svn diff' command in the trunk directory.
<snip> I've attached an updated patch for these changes against HEAD, for anybody else who is interested. (This patch adds the ability to move boost into a different top-level namespace, rename macros, etc.) -- Eric Niebler Boost Consulting www.boost-consulting.com Index: tools/bcp/file_types.cpp =================================================================== --- tools/bcp/file_types.cpp (revision 40295) +++ tools/bcp/file_types.cpp (working copy) @@ -9,6 +9,7 @@ * void bcp_implementation::is_source_file(const fs::path& p) * void bcp_implementation::is_html_file(const fs::path& p) * void bcp_implementation::is_binary_file(const fs::path& p) + * void bcp_implementation::is_bjam_file(const fs::path& p) */ #include "bcp_imp.hpp" @@ -19,8 +20,8 @@ static const boost::regex e( ".*\\." "(?:" - "c|cxx|h|hxx|inc|.?pp|yy?" - ")", + "c|cxx|h|hxx|inl|inc|.?pp|yy?" + ")", boost::regex::perl | boost::regex::icase ); return boost::regex_match(p.string(), e); @@ -50,8 +51,23 @@ "c|cxx|cpp|h|hxx|hpp|inc|html?|css|mak|in" ")" "|" - "(Jamfile|makefile|configure)", + "(Jamfile|Jamfile\\.v2|Jamrules|Jambase|makefile|configure)", boost::regex::perl | boost::regex::icase); return !boost::regex_match(p.leaf(), e); } + + +bool bcp_implementation::is_bjam_file(const fs::path& p) +{ + static const boost::regex e( + ".*\\." + "(?:" + "jam" + ")" + "|" + "(Jamfile|Jamfile\\.v2|Jamrules|Jambase)", + boost::regex::perl | boost::regex::icase); + return boost::regex_match(p.leaf(), e); + +} Index: tools/bcp/add_path.cpp =================================================================== --- tools/bcp/add_path.cpp (revision 40295) +++ tools/bcp/add_path.cpp (working copy) @@ -125,6 +125,11 @@ // or we'll get an error: if(s.compare(0, 2, "./") == 0) s.erase(0, 2); + // + // if the name contain .html# , remove the html bookmark + // or we'll get warnings of none existing files + if(s.find(".html#") != std::string::npos) + s.erase(s.find(".html#")+5); if(s.find(':') == std::string::npos) { // only concatonate if it's a relative path Index: tools/bcp/bcp_imp.cpp =================================================================== --- tools/bcp/bcp_imp.cpp (revision 40295) +++ tools/bcp/bcp_imp.cpp (working copy) @@ -18,7 +18,17 @@ #include <string> bcp_implementation::bcp_implementation() - : m_list_mode(false), m_list_summary_mode(false), m_license_mode(false), m_cvs_mode(false), m_svn_mode(false), m_unix_lines(false), m_scan_mode(false), m_bsl_convert_mode(false), m_bsl_summary_mode(false) + : m_list_mode(false) + , m_list_summary_mode(false) + , m_license_mode(false) + , m_cvs_mode(false) + , m_svn_mode(false) + , m_unix_lines(false) + , m_diff_only_mode(false) + , m_replace_namespace(false) + , m_scan_mode(false) + , m_bsl_convert_mode(false) + , m_bsl_summary_mode(false) { } @@ -76,6 +86,11 @@ m_unix_lines = true; } +void bcp_implementation::enable_diff_only_mode() +{ + m_diff_only_mode = true; +} + void bcp_implementation::set_boost_path(const char* p) { m_boost_path = fs::path(p, fs::native); Index: tools/bcp/bcp.hpp =================================================================== --- tools/bcp/bcp.hpp (revision 40295) +++ tools/bcp/bcp.hpp (working copy) @@ -22,11 +22,13 @@ virtual void enable_cvs_mode() = 0; virtual void enable_svn_mode() = 0; virtual void enable_unix_lines() = 0; + virtual void enable_diff_only_mode() = 0; virtual void enable_scan_mode() = 0; virtual void enable_license_mode() = 0; virtual void enable_bsl_convert_mode() = 0; virtual void enable_bsl_summary_mode() = 0; virtual void set_boost_path(const char* p) = 0; + virtual void set_replacement_namespace(const char* ns) = 0; virtual void set_destination(const char* p) = 0; virtual void add_module(const char* p) = 0; Index: tools/bcp/scan_cvs_path.cpp =================================================================== --- tools/bcp/scan_cvs_path.cpp (revision 40295) +++ tools/bcp/scan_cvs_path.cpp (working copy) @@ -59,7 +59,6 @@ ++i; m_cvs_paths[file] = binary; } - } } } Index: tools/bcp/Jamfile.v2 =================================================================== --- tools/bcp/Jamfile.v2 (revision 40295) +++ tools/bcp/Jamfile.v2 (working copy) @@ -7,6 +7,7 @@ add_path.cpp bcp_imp.cpp copy_path.cpp file_types.cpp fileview.cpp main.cpp path_operations.cpp scan_cvs_path.cpp licence_info.cpp scan_licence.cpp output_licence_info.cpp + replace_namespace.cpp /boost/filesystem//boost_filesystem /boost/regex//boost_regex /boost/test//boost_prg_exec_monitor Index: tools/bcp/bcp_imp.hpp =================================================================== --- tools/bcp/bcp_imp.hpp (revision 40295) +++ tools/bcp/bcp_imp.hpp (working copy) @@ -53,11 +53,13 @@ void enable_cvs_mode(); void enable_svn_mode(); void enable_unix_lines(); + void enable_diff_only_mode(); void enable_scan_mode(); void enable_license_mode(); void enable_bsl_convert_mode(); void enable_bsl_summary_mode(); void set_boost_path(const char* p); + void set_replacement_namespace(const char* ns); void set_destination(const char* p); void add_module(const char* p); @@ -70,10 +72,15 @@ void add_directory(const fs::path& p); void add_file(const fs::path& p); void copy_path(const fs::path& p); + void replace_namespace(const fs::path& p); + fs::path destination_path(const fs::path p, std::string seperator = "/"); void add_file_dependencies(const fs::path& p, bool scanfile); bool is_source_file(const fs::path& p); bool is_html_file(const fs::path& p); bool is_binary_file(const fs::path& p); + bool is_bjam_file(const fs::path& p); + bool diff_only_copy( const std::string& new_file, + fs::path destination_path ); void add_dependent_lib(const std::string& libname, const fs::path& p); void create_path(const fs::path& p); // license code: @@ -87,6 +94,8 @@ bool m_cvs_mode; // check cvs for files bool m_svn_mode; // check svn for files bool m_unix_lines; // fix line endings + bool m_diff_only_mode; // diff only mode + bool m_replace_namespace; // replace the boost namespace bool m_scan_mode; // scan non-boost files. bool m_bsl_convert_mode; // try to convert to the BSL bool m_bsl_summary_mode; // summarise BSL issues only Index: tools/bcp/main.cpp =================================================================== --- tools/bcp/main.cpp (revision 40295) +++ tools/bcp/main.cpp (working copy) @@ -33,10 +33,13 @@ "\n" "Options:\n" " --boost=path sets the location of the boost tree to path\n" - " --scan treat the module list as a list of (possibly non-boost)\n" + " --namespace=ns set a namespace that will replace 'boost'\n" + " --scan treat the module list as a list of (possibly non-boost)\n" " files to scan for boost dependencies\n" " --cvs only copy files under cvs version control\n" " --unix-lines make sure that all copied files use Unix style line endings\n" + " --diff-only only overwrite files when new file differs from\n" + " existing original. May save you rebuilds.\n" "\n" "module-list: a list of boost files or library names to copy\n" "html-file: the name of a html file to which the report will be written\n" @@ -127,12 +130,21 @@ { papp->enable_unix_lines(); } + else if(0 == std::strcmp("--diff-only", argv[i])) + { + papp->enable_diff_only_mode(); + } else if(0 == std::strncmp("--boost=", argv[i], 8)) { papp->set_boost_path(argv[i] + 8); } + else if(0 == std::strncmp("--namespace=", argv[i], 12)) + { + papp->set_replacement_namespace(argv[i] + 12); + } else if(argv[i][0] == '-') { + std::cout << "wrong argument: " << argv[i] << std::endl; show_usage(); return 1; } Index: tools/bcp/copy_path.cpp =================================================================== --- tools/bcp/copy_path.cpp (revision 40295) +++ tools/bcp/copy_path.cpp (working copy) @@ -20,35 +20,50 @@ void bcp_implementation::copy_path(const fs::path& p) { assert(!fs::is_directory(m_boost_path / p)); - if(fs::exists(m_dest_path / p)) - { - std::cout << "Copying (and overwriting) file: " << p.string() << "\n"; - fs::remove(m_dest_path / p); - } - else - std::cout << "Copying file: " << p.string() << "\n"; + fs::path dest_p = destination_path( p ); + // // create the path to the new file if it doesn't already exist: // - create_path(p.branch_path()); + create_path(dest_p.branch_path()); // // do text based copy if requested: // - if(m_unix_lines && !is_binary_file(p)) + if(m_replace_namespace && (is_source_file(p) || is_bjam_file(p))) { - std::ifstream is((m_boost_path / p).native_file_string().c_str()); - std::istreambuf_iterator<char> isi(is); - std::istreambuf_iterator<char> end; - - std::ofstream os((m_dest_path / p).native_file_string().c_str(), std::ios_base::binary | std::ios_base::out); - std::ostreambuf_iterator<char> osi(os); - - std::copy(isi, end, osi); + replace_namespace(p); } else { - // binary copy: - fs::copy_file(m_boost_path / p, m_dest_path / p); + + if( fs::exists(m_dest_path / dest_p)) + { + // TODO: need some support for diff mode here, for now + // diff_mode is only supported within replace_namespace(p); + + std::cout << "Copying (and overwriting) file: " << p.string() << "\n"; + fs::remove(m_dest_path / dest_p ); + } + else + std::cout << "Copying file: " << p.string() << "\n"; + + if(m_unix_lines && !is_binary_file(p)) + { + std::ifstream is((m_boost_path / p).native_file_string().c_str()); + std::istreambuf_iterator<char> isi(is); + std::istreambuf_iterator<char> end; + + std::ofstream os( ( m_dest_path / dest_p ).native_file_string().c_str(), + std::ios_base::binary | std::ios_base::out); + std::ostreambuf_iterator<char> osi(os); + + std::copy(isi, end, osi); + } + else + { + // binary copy: + fs::copy_file(m_boost_path / p, m_dest_path / dest_p); + } } }
I'm playing around with Bjørn's --namespace option and it seems to work well. I've run into one problem with bcp, though, having to do with regex. The docs say this:
bcp boost/regex.hpp /foo
Copies boost/regex.hpp and all dependencies including the regex source code (in libs/regex/src) and build files (in libs/regex/build) to /foo. Does not copy the regex documentation, test, or example code.
But this doesn't copy all the files needed to build regex. For instance, it doesn't copy boost/regex/concepts.hpp, which is required by the regex makefiles. I don't really understand the bcp source code, but I've discovered that if I add the following "special" dependency to add_path.cpp, it works: std::pair<fs::path, fs::path>("boost/regex.hpp", "boost/regex"), Is this is the right fix? -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
I'm playing around with Bjørn's --namespace option and it seems to work well. cool, :-)
I've run into one problem with bcp, though, having to do with regex. The docs say this:
bcp boost/regex.hpp /foo
Copies boost/regex.hpp and all dependencies including the regex source code (in libs/regex/src) and build files (in libs/regex/build) to /foo. Does not copy the regex documentation, test, or example code.
But this doesn't copy all the files needed to build regex. For instance, it doesn't copy boost/regex/concepts.hpp, which is required by the regex makefiles.
I don't really understand the bcp source code, but I've discovered that if I add the following "special" dependency to add_path.cpp, it works:
std::pair<fs::path, fs::path>("boost/regex.hpp", "boost/regex"),
Is this is the right fix?
I am not sure, but I think tis basically assure everything in boost/regex directory is copied if you need boost/regex.hpp. I.e. not only those files found by dependency scanning of #include. So if there are stuff like: # include BOOST_REGEX_USER_CONFIG which I found in boost/regex/config.hpp which is included from boost/regex.hpp you got it... So I think your fix is Ok, -- but John will be the judge. -- Bjørn
Eric Niebler wrote:
I'm playing around with Bjørn's --namespace option and it seems to work well. I've run into one problem with bcp, though, having to do with regex. The docs say this:
bcp boost/regex.hpp /foo
Copies boost/regex.hpp and all dependencies including the regex source code (in libs/regex/src) and build files (in libs/regex/build) to /foo. Does not copy the regex documentation, test, or example code.
But this doesn't copy all the files needed to build regex. For instance, it doesn't copy boost/regex/concepts.hpp, which is required by the regex makefiles.
I don't really understand the bcp source code, but I've discovered that if I add the following "special" dependency to add_path.cpp, it works:
std::pair<fs::path, fs::path>("boost/regex.hpp", "boost/regex"),
Is this is the right fix?
Not really, it's a bug in the Makefiles - or rather the shell script that creates them - they shouldn't depend upon the concepts.hpp header as that's not required to build the regex lib, only to test it. Hopefully I have this fixed in SVN now. John.
John Maddock wrote:
Eric Niebler wrote:
I've run into one problem with bcp, though, having to do with regex. The docs say this:
bcp boost/regex.hpp /foo
Copies boost/regex.hpp and all dependencies including the regex source code (in libs/regex/src) and build files (in libs/regex/build) to /foo. Does not copy the regex documentation, test, or example code. But this doesn't copy all the files needed to build regex. For instance, it doesn't copy boost/regex/concepts.hpp, which is required by the regex makefiles.
it's a bug in the Makefiles - or rather the shell script that creates them - they shouldn't depend upon the concepts.hpp header as that's not required to build the regex lib, only to test it. Hopefully I have this fixed in SVN now.
OK, we're one step closer. :-) Now it complains that boost/regex/v4/regex_cstring.hpp is missing. John, if after using bcp as above it should be possible to build the regex lib with the makefiles, could you pls give it a shot on your end and see if it works? Thanks. -- Eric Niebler Boost Consulting www.boost-consulting.com
Eric Niebler wrote:
OK, we're one step closer. :-) Now it complains that boost/regex/v4/regex_cstring.hpp is missing.
John, if after using bcp as above it should be possible to build the regex lib with the makefiles, could you pls give it a shot on your end and see if it works? Thanks.
OK, I think I really do have this right now :-) There were some "dead" header files lurking around that are no longer used: now removed. Please update and try again. Regards, John.
John Maddock wrote:
Eric Niebler wrote:
OK, we're one step closer. :-) Now it complains that boost/regex/v4/regex_cstring.hpp is missing.
OK, I think I really do have this right now :-) There were some "dead" header files lurking around that are no longer used: now removed. Please update and try again.
Works, thanks. -- Eric Niebler Boost Consulting www.boost-consulting.com
participants (6)
-
Bjørn Roald -
David Abrahams -
Eric Niebler -
Glenn Schrader -
John Maddock -
Maciej Sobczak