August 2013 - Boost-users - lists.boost.org

Re: [Boost-users] Boost.Asio, 0MQ, and Windows
by Damien Kick 13 Aug '13

13 Aug '13

On Aug 11, 2013, at 4:32 PM, Damien Kick <dkick(a)shoretel.com<mailto:dkick@shoretel.com>> wrote: Yeah, so far I've been having very good luck with Asio and 0MQ on POSIX. But I'm still stuck on Windows :( The code that I was able to get to compile is not happy at runtime. int main() { zmq::context_t zenv(1); zmq::socket_t zocket(zenv, ZMQ_PUSH); SOCKET zfd; std::size_t zfd_size = sizeof(zfd); zocket.getsockopt(ZMQ_FD, &zfd, &zfd_size); boost::asio::windows::stream_handle::native_handle_type zfd_native(&zfd); boost::asio::io_service ios; boost::asio::windows::stream_handle io(ios, zfd_native); } Stepping line-by-line in the debugger for Visual Studio 2010, when I get to the line attempting to create the windows::stream_handle, I get Unhandled exception at 0x77b815de in test-zee-zmq-socket.exe: Microsoft C++ exception: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::system::system_error> > at memory location 0x00c1fc7c.. And so there obviously must be more to creating a windows::stream_handle than this but I'm still quite in the dark as to what that might be. The stack trace I'm getting doesn't help me much, either, but I am also am quite ignorant about most things Windows. ntdll.dll!77b815de() [Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll] ntdll.dll!77b815de() ntdll.dll!77b7014e() KernelBase.dll!7608c41f() > msvcr100d.dll!_nh_malloc_dbg_impl(unsigned int nSize=8, int nhFlag=0, int nBlockUse=1, const char * szFileName=0x00000000, int nLine=0, int * errno_tmp=0x00c1fb0c) Line 239 + 0x19 bytes C++ msvcr100d.dll!_nh_malloc_dbg(unsigned int nSize=8, int nhFlag=0, int nBlockUse=1, const char * szFileName=0x00000000, int nLine=0) Line 302 + 0x1d bytes C++ msvcr100d.dll!malloc(unsigned int nSize=12712804) Line 56 + 0x15 bytes C++ 00c1fb64() 00cc8220() I've also tried int main() { zmq::context_t zenv(1); zmq::socket_t zocket(zenv, ZMQ_PUSH); SOCKET zfd; std::size_t zfd_size = sizeof(zfd); zocket.getsockopt(ZMQ_FD, &zfd, &zfd_size); boost::asio::io_service ios; boost::asio::windows::stream_handle io(ios, GetStdHandle(zfd)); } Which also compiles but then I'm still getting the same exception at runtime. And so apparently a SOCKET is not the same as a HANDLE (?), even though <http://msdn.microsoft.com/en-us/library/windows/desktop/ms741476(v=vs.85).a…> tells me that The WPUCreateSocketHandle function creates a new socket handle. Syntax C++ SOCKET WPUCreateSocketHandle( _In_ DWORD dwCatalogEntryId, _In_ DWORD_PTR dwContext, _Out_ LPINT lpErrno ); <sigh/> It's so annoying because I have this feeling that there is probably just one stupid little conversion function that I need to be calling but it continues to elude me :( ________________________________ This e-mail and any attachments are confidential. If it is not intended for you, please notify the sender, and please erase and ignore the contents.

2 3

Boost.Asio, 0MQ, and Windows
by Damien Kick 13 Aug '13

13 Aug '13

I have been following the excellent advice on using 0MQ and Boost.Asio on a POSIX-ish platform (MacOS) at <http://preview.tinyurl.com/jw4krek>. Unfortunately, though, I am writing code which needs to work on both POSIX-ish and Windows. I'm not having much luck, so far, with my attempts to translate this advice to the Windows side. I can't seem to figure out how to match the type returned by getsockopt(ZMQ_FD) on Windows and what I think should be the Windows equivalents of boost::asio::posix::stream_descriptor. Here is what I've tried: zmq::context_t zenv(1); zmq::socket_t zocket(zenv, ZMQ_PUSH); SOCKET zfd; std::size_t zfd_size = sizeof(zfd); zocket.getsockopt(ZMQ_FD, &zfd, &zfd_size); boost::asio::io_service ios; boost::asio::windows::stream_handle io(ios, zfd); And the error I get when I try to compile it. 2>Z:\zee-zmq-socket.cpp(14): error C2664: \ 'boost::asio::windows::basic_stream_handle<>::basic_stream_handle(\ boost::asio::io_service &,void *const &)' : cannot convert parameter \ 2 from 'SOCKET' to 'void *const &' 2> Reason: cannot convert from 'SOCKET' to 'void *const ' 2> Conversion from integral type to pointer type requires \ reinterpret_cast, C-style cast or function-style cast So far, my Google-fu has been weak; I can only seem to find examples which deal with Boost.Asio and 0MQ on POSIX-ish. I'm not sure what I need to do to correctly get from a SOCKET to a `void * const`. The closest advice that I've been able to find is about combining 0MQ and libcurl: <http://preview.tinyurl.com/lbffx64>. And this advice also makes me nervous that my compilation problems are only the tip of a much nastier iceberg hiding under the surface. In particular, > The reason why there is a assign, but no release-function for > boost::asio::ip::tcp::sockets is that once a socket is "bound" to a > completion port you cannot "unbound" it or "bound" it to another > completion-port. And so I'm worried there might be other gotchas about which I might not know and which would not even be so polite as to manifest themselves as compile-time errors. Thank you in advance for any help. ________________________________ This e-mail and any attachments are confidential. If it is not intended for you, please notify the sender, and please erase and ignore the contents.

2 5

Iostreams with two filters throws exception
by NuSkooler 12 Aug '13

12 Aug '13

I've been attempting to use Boost.Iostreams with two filters to compress + encrypt ("out") and decrypt + uncompress ("in"). This works for some files, while others throw an exception during the "in" phase. If I remove either of the filters for out/in (e.g. remove the compress or the encrypt) everything works fine -- the issue only occurs with two filters chained. >From what I understand in the documentation, this should be supported, but I must be missing something. Below is the relevant portions of my code: Input & output cipher filters (note: I'm using Boost.Crypto, and boost::crypto::rc4_cipher specifically) template <typename StreamCipherT> class StreamCipherFilterBase { public: typedef char char_type; StreamCipherFilterBase(const std::string& key) { m_cipher.set_key(key.data(), static_cast<unsigned int>(key.length())); } protected: char m_buffer[1024 * 2]; // 2k internal buffer size StreamCipherT m_cipher; }; template <typename StreamCipherT> class StreamCipherEncryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_output_filter_tag category; StreamCipherEncryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Sink> std::streamsize write(Sink& snk, const char* s, std::streamsize n) { std::streamsize written = 0; std::streamsize remain = n; std::streamsize eat; while(remain) { eat = remain < sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer) ? remain : sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer); StreamCipherFilterBase<StreamCipherT>::m_cipher.encrypt( s, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); written += boost::iostreams::write(snk, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); remain -= eat; s += eat; } return written; } }; template <typename StreamCipherT> class StreamCipherDecryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_input_filter_tag category; StreamCipherDecryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Source> std::streamsize read(Source& src, char* s, std::streamsize n) { const std::streamsize read = boost::iostreams::read(src, s, n); if(EOF == read) { return EOF; } StreamCipherFilterBase<StreamCipherT>::m_cipher.decrypt(s, s, read); return read; } }; And compress + encrypt ("out") & decrypt + uncompress ("in"): (error checking removed for this post) // // Write out data compressing and encrypting as we go // boost::filesystem::ifstream inf(inFile.Get(), std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(outFile.Get(), std::ios_base::binary | std::ios_base::out); StreamCipherEncryptFilter<CipherT> outCipherFilter(key); boost::iostreams::filtering_ostreambuf out; out.push(boost::iostreams::zlib_compressor()); // compress out.push(outCipherFilter); // then encrypt out.push(outf); // write to file boost::iostreams::copy(inf, out); // // Write out decrypted and uncompressed data: // boost::filesystem::ifstream inf(inFile.Get(), std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(outFile.Get(), std::ios_base::binary | std::ios_base::out); StreamCipherDecryptFilter<CipherT> inCipherFilter(key); boost::iostreams::filtering_istreambuf in; in.push(boost::iostreams::zlib_decompressor()); in.push(inCipherFilter); in.push(inf); boost::iostreams::copy(in, outf); // <--- exception is thrown here for many files; some succeed with input md5 = output md5. Can anyone help? Bryan Ashby

2 2

Re: [Boost-users] [Interprocess] Page default
by Kelly, Dan 12 Aug '13

12 Aug '13

To elaborate on Ion's remarks, ALL virtual memory in user space in windows is backed by the page file, and ALL file I/O is memory mapped, the only difference being virtual memory is backed by the page file rather than a specific named file in the file system. Your inter process memory is just as likely to be in physical memory as any other data. So most likely this is not the cause of your performance issue, unless memory is so tight that your entire process working set is continually thrashing. Dan -----Original Message----- From: Ion Gaztañaga [igaztanaga(a)gmail.com<mailto:igaztanaga@gmail.com>] Sent: Wednesday, July 24, 2013 06:48 PM Eastern Standard Time To: boost-users(a)lists.boost.org Subject: Re: [Boost-users] [Interprocess] Page default El 18/07/2013 14:45, Oodini escribió: > Hello, > > I use Boost.Interprocess on Windows 2003 Server. > > I am experiencing performances troubles. I am trying to understand > what happens, and arrived on Microsoft web pages. > > I seems that on Windows, shared memory is implemented only through > mapped files, and that only a portion of the shared memory is > physically in the RAM (this limited number of pages is called > "working set"). > > The Windows API provides some functions for this working set : > http://msdn.microsoft.com/en-us/library/windows/desktop/cc441804%28v=vs.85%… > > I would have liked Interprocess wraps the functions > GetProcessMemoryInfo() and SetProcessWorkingSetSize(), so that the > full shared memory is put in RAM. Inteprocess can't do that because it's a portable library and POSIX systems don't know what a working set is. You can perfectly call those functions yourself as you don't need any internal Interprocess data to call them. Although shared memory is never guaranteed to be in RAM, you don't any any guarantee with your "process memory" (if memory pressure is detected the OS can swap you heap and/or executable pages to the swap file to free some RAM for other processes). Best, Ion _______________________________________________ Boost-users mailing list Boost-users(a)lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users ________________________________ IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.

3 3

[zlib] Compression intended use
by Michael Powell 11 Aug '13

11 Aug '13

Hello, This is probably out of scope with boost's intended usage pattern, but I'll ask anyhow. I've got a package of output files that I want to bundle, tarball, gzip, or however I can manage to compress them. I see there is the zlib usage, but methinks that is more for in-memory compression. Not sure exactly how that translates to folders-/files-oriented compression I am thinking of. Thank you... Regards, Michael Powell

3 2

Re: [Boost-users] Iostreams with two filters throws exception (Ken Appleby)
by NuSkooler 10 Aug '13

10 Aug '13

Below is a little app that illustrates the issue (Can also be found here: http://pastebin.com/k8jRN64R) I've removed most error handling/etc. for illustration purposes. The program expects a "input.txt" to be present in the same directory it's run from. Note again that this works *most* of the time, but depending on the input file, will crash. Also I forgot to mention before that I'm using Clang on OS X 8.4 and Boost v1.53. Any help is greatly appreciated! // Boost.Crypto #include <boost/crypto/rc4_cipher.hpp> // Boost #include <boost/filesystem/fstream.hpp> #include <boost/iostreams/filter/zlib.hpp> #include <boost/iostreams/copy.hpp> #include <boost/iostreams/filtering_stream.hpp> template <typename StreamCipherT> class StreamCipherFilterBase { public: typedef char char_type; StreamCipherFilterBase(const std::string& key) { m_cipher.set_key(key.data(), static_cast<unsigned int>(key.length())); } protected: char m_buffer[1024 * 2]; // 2k internal buffer size StreamCipherT m_cipher; }; template <typename StreamCipherT> class StreamCipherEncryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_output_filter_tag category; StreamCipherEncryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Sink> std::streamsize write(Sink& snk, const char* s, std::streamsize n) { std::streamsize written = 0; std::streamsize remain = n; std::streamsize eat; while(remain) { eat = remain < sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer) ? remain : sizeof(StreamCipherFilterBase<StreamCipherT>::m_buffer); StreamCipherFilterBase<StreamCipherT>::m_cipher.encrypt( s, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); written += boost::iostreams::write(snk, StreamCipherFilterBase<StreamCipherT>::m_buffer, eat); remain -= eat; s += eat; } return written; } }; template <typename StreamCipherT> class StreamCipherDecryptFilter : public StreamCipherFilterBase<StreamCipherT> { public: typedef boost::iostreams::multichar_input_filter_tag category; StreamCipherDecryptFilter(const std::string& key) : StreamCipherFilterBase<StreamCipherT>(key) { } template<typename Source> std::streamsize read(Source& src, char* s, std::streamsize n) { const std::streamsize read = boost::iostreams::read(src, s, n); if(EOF == read) { return EOF; } StreamCipherFilterBase<StreamCipherT>::m_cipher.decrypt(s, s, read); return read; } }; const std::string KEY = "Kitteh"; typedef boost::crypto::rc4_cipher CipherT; void Encrypt( const std::string& in, const std::string& out) { // error handling omitted boost::filesystem::ifstream inf(in, std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(out, std::ios_base::binary | std::ios_base::out); StreamCipherEncryptFilter<CipherT> outCipherFilter(KEY); boost::iostreams::filtering_ostreambuf outs; outs.push(boost::iostreams::zlib_compressor()); // compress outs.push(outCipherFilter); // then encrypt outs.push(outf); // write to file boost::iostreams::copy(inf, outs); } void Decrypt( const std::string& in, const std::string& out) { // error handling omitted boost::filesystem::ifstream inf(in, std::ios_base::binary | std::ios_base::in); boost::filesystem::ofstream outf(out, std::ios_base::binary | std::ios_base::out); StreamCipherDecryptFilter<CipherT> inCipherFilter(KEY); boost::iostreams::filtering_istreambuf ins; ins.push(boost::iostreams::zlib_decompressor()); ins.push(inCipherFilter); ins.push(inf); boost::iostreams::copy(ins, outf); // <-- exception thrown here with some files } int main() { // compress+encrypt input.txt -> output.encrypt Encrypt("input.txt", "output.encrypt"); // decrypt+decompress output.encrypt -> input.read.txt Decrypt("output.encrypt", "input.read.txt"); } ------------------------------ > > Message: 5 > Date: Thu, 08 Aug 2013 17:52:50 +0100 > From: Ken Appleby <ken.appleby(a)btopenworld.com> > To: boost-users(a)lists.boost.org > Subject: Re: [Boost-users] Iostreams with two filters throws exception > Message-ID: <5203CCE2.4060200(a)btopenworld.com> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > > You are a lot more likely to get a response if you post a complete > example. Just a single file with a main() would make it a lot easier to > help you. You're not far from that with the code you posted. > >

2 1

Boost.Variant broke in 1.54?
by gast128 09 Aug '13

09 Aug '13

Dear all, I get a compilation error with Boost.Variant (which compiles fine on previous releases) with vstudio 2010 when a function returns a const Boost.Variant, e.g.: typedef boost::variant<int, double> Variant; const Variant GetVar() { return Variant(); } void TestVariantDnc() { const Variant var = GetVar(); } This gives error C2666 ('boost::variant<T0_,T1>::convert_construct' : 3 overloads have similar conversions'). Shall a file a ticket or do I something wrong?

4 8

Releasing Boost Dependency Analyzer 1.1
by Jens Weller 09 Aug '13

09 Aug '13

Hello boost community, a short notice that I just have released the version 1.1 of my dependency analyzing tool for boost. For now I have only managed to get a proper Windows Version, Linux Version will follow later. New Features: Configuration - the tool remembers the path to bcp now - select the source for libraries - maintainers.txt - libraries.xml - 2 filter options - filter libraries by name - filter edges by type, for now its header, source/lib, test and documentation. Export: - Export CSV File which has all libraries and the number of dependencies (sorted) libraries.xml - this is now a optional source for the library names Some feedback: libraries.xml as a lib name source is nice, but not quite perfect. I've seen different results from maintainers.txt and libraries.xml. As maintainers.txt also libraries.xml contains duplicates, some of its library names don't fit the expectation of bcp, so I applied some fixes for this in my program. The Dependency filter is yet not perfect, especially in detection. More details about this version and the download for it at my blog: http://www.meetingcpp.com/index.php/br/items/releasing-boost-dependency-ana… kind regards, Jens Weller

1 0

Compiling Boost 1.54.0 for the iOS simulator (6.1) on OS X 10.8.4
by Dan Leach 09 Aug '13

09 Aug '13

Hello, I am new to the Boost mailing list so please let me know if this is not the appropriate place to discuss my problem. I have successfully managed to compile several of the Boost libraries (regex, filesystem, thread) and create a framework for use with OS X, iOS, and the iOS simulator using this script: https://github.com/wuhao5/boost/blob/188e905626dbd522f65950102ed0c8ce77cb28… on Peter Goodliffe's work here: http://goodliffe.blogspot.com/2010/09/building-boost-framework-for-ios-ipho… Boost 1.53.0 using clang and linking against libc++. The script fails when compiling for the iOS simulator with Boost 1.54.0 with the following error:/bin/sh: line 1: -ftemplate-depth-128: command not found. I did quite a bit of digging and here is what I know: * In boost.sh, the iOS simulator bjam and compiler options include "-arch i386" * The release notes for Boost 1.54.0 state that "Boost no longer supports the 80386 target CPU" * The Boost 1.53.0 libraries mentioned above appear to compile fine for OS X, iOS, and the iOS simulator using the same script * The Boost 1.54.0 libraries appear to compile fine for OS X and iOS itself using the same script Here is what I think I know: * When passing "-d13" to bjam (to generate debug information), it appears that the variable "CONFIG_COMMAND" is not being set when building for the iOS simulator. For the other configurations, this variable is set to the path to clang and includes the compiler options. For the simulator, it is empty which I believe leads to the error above as when bjam attempts to run the compiler using this variable, it instead runs some appended options which begin with the template depth. Therefore, the CONFIG_COMMAND variable does not appear to be being set properly causing the failure. I do not know why it's not being set as I don't know enough about the Boost build system to sift through the rest of the output generated with the "-d13" flag. I suspect an incompatibility with the "-arch i386" option was introduced with 1.54.0. I've tried changing this to "-arch i486" and "-arch x86_32" but the same problem occurs. Has anyone else had success with this? I'd like to use Boost.Log which is why I'm even attempting this with 1.54. Thanks!

1 1

[program_options] Parsing config files
by Michael Powell 08 Aug '13

08 Aug '13

Hello, Need a little help grasping the nature of parse_config_file. The example: http://www.boost.org/doc/libs/1_54_0/doc/html/program_options/tutorial.html… also seems a bit lacking: says that there are two calls to store, but I only see one. Maybe the example has not been updated? What I need to know is, if we specify some options, do we specify the same option names in a config file? So that if we have an option on the command like like: --my-option-name 1234, then the corresponding line in the config file would go: my-option-name=1234 ? Thank you. Regards, Michael Powell

2 9