- Boost - lists.boost.org

Secret handles and descriptors: Enemies of reuse,
by Simon Meiklejohn 14 Sep '04

14 Sep '04

>From my perspective developing interactive telephony applications this problem is an example of a more general one. Heres the outline of a solution that I've been using In general a program may need to integrate a variety of libraries which are sources of information/events. Often it's the nature of these libraries that they either own a thread which calls back into application code, or require the user to call a potentially blocking function that returns the data/event. Then again there are the libraries which explicity make available o/s objects that can be waited on. Its frequently the case that no single 'raw' demultiplexor system can interface to all of these at once. (I guess that's the reason for this thread). To me the underlying issue is one of threading. Heres a list of typical issues that arise in my apps. - my networking library wants to call me back in its own thread - ditto for my multi-media timer - my telephony library wants to signal me by setting an event - so I'd better dedicate a thread to it. That thread will also call me back. - my application objects are having events fired at them from all sides from multiple threads and therefore need to use Mutex's, but events from below (telephony) keep on deadlocking with events from above (network). - my COM interface wants to issue connection point callbacks to me in a the app main thread via the app message pump. - I cant call out on my COM interface on my Network library's callback thread because its not marshaled for that kind of stuff. - So I put the network messages in a queue - now I have to send a windows message to the main thread, where its ok to call COM. This isn't necessary on my unix version, so I have ugly platform specific code in the middle of my app. - My smart network library parses my messages into complex data types before I get to see them. So getting them out of the queue in the main thread involves some kind of dispatch on type. This is expensive and hard to maintain. STEPPING BACK A BIT While it may seem that many of my problems stem from COM, I think it stems from an unfortunate need to implement the application's threading policy at all levels of the code - library and application code. Also, the hoops I have to jump through to get COM working wont be at all applicable if I shift to unix or move to a non-COM interface. What I really need is the ability to define a threading policy at some high level point in my code (e.g. main), have library code that defers to that policy. I've called this a Thread Redirector policy Looking at the Network lib to COM path as an example: - the network lib receives bytes from the network in its own thread. When it has a complete message (but before it starts decoding the message) it calls the Thread Redirector asking for a callback in the appropriate thread. - in main() we've set things up so that when Networking asks for a callback it will be called back in the main thread via a windows message. (On Unix I substitute a pthreads based callback.) - On windows the callback occurs in the main thread. The Network library starts decoding the message. It determines it's a message type XXX and calls back into the App code MyAppObj.DeliverMessageXXX( MsgXXXType& XXX ). - the app can now call into its COM API like - myCOMIF->DoSomething( XXX.param) - without first queueing the message. I can define it so that all my libs deliver their events to the application objects on the same thread. This relieves the thread deadlock issues. I can go into more detail if required - but that's the flavour of it. Simon Meiklejohn

2 1

IOStreams formal review start
by Jeff Garland 14 Sep '04

14 Sep '04

All - Today (August 28th, 2004) is the start of the formal review of the Iostreams library by Jonathan Turkanis. I will be serving as review manager. Note that this is a somewhat unusual situation in that we have several libraries that overlap in the same area, so comments related to the MoreIo overlap are needed. As usual, please state in review comments how you reviewed the library and whether the you think the library should be accepted into Boost. Further guidelines for writing reviews can be found on the website at: http://www.boost.org/more/formal_review_process.htm#Comments ********************************************** Library Synopsis ********************************************** The Iostreams Library serves two main purposes: * To allow the easy creation of standard C++ stream and stream buffer classes for new data sources and sinks. * To provide a convenient interface for defining i/o filters and attaching them to standard streams and stream buffers. The library focuses on freeing users from writing boiler plate code and allowing them instead to create highly reusable components. In addition to providing an abstract framework the library provides a number of concrete filters, sources and sinks which serve as example applications of the library but are also useful in their own right. These include components for accessing memory-mapped files, for file access via operating system file descriptors, for code conversion, for text filtering with regular expressions, for line-ending conversion and for compression and decompression in the zlib, gzip and bzip2 formats. The latest package can be found at the following locations: http://home.comcast.net/~jturkanis/iostreams/ Thanks, Jeff

14 88

meta-comm page not found
by troy d.straszheim 13 Sep '04

13 Sep '04

I was hunting around for something to do. If I click on any of the failed tests for gcc-3.4.1 on this page: http://tinyurl.com/3uxpm, you get a "page not found" error, on, for instance, the following page: http://www.meta-comm.com/engineering/boost-regression/developer/output/ bin-boost-libs-test-test-token_iterator_test-test-gcc-3-4-1-linux- debug.html ? -t

2 1

[ANN] asio - yet another networking library!
by Christopher Kohlhoff 13 Sep '04

13 Sep '04

Hi all, In light of some of the recent discussion about sockets (e.g. the "behond IOStreams" thread), I'd like to take this opportunity to announce "asio". asio will be going live in a commercial system in a little over a month, and I think it's now at a point where it is suitable for use by a wider audience. I would like to think that asio has potential as the basis of a boostified networking library, or that at least it can inject some ideas into the discussion. asio is a cross-platform library for doing network programming, using a "modern" c++ style. It is designed to use an asynchronous I/O model for virtually all operations, even when the OS lacks native asynchronous I/O. The library is intended to ease the development of scalable network applications through the use of a proactive event-driven architecture. You can download asio from: http://tenermerx.com/programming/cpp/asio/asio-0.1.12.tar.gz I have also put the generated documentation online at: http://tenermerx.com/programming/cpp/asio/asio-0.1.12/doc/html/ asio supports Win32 and Linux, and has been infrequently tested on Solaris. It is free for any use. Here is a sample usage of asio showing how to receive data on a socket: char buf[BUFSIZ]; void handle_receive(asio::error& e, size_t bytes_recvd) { if (!e) { /*...*/ } } int start_receiving(asio::socket& s) { s.async_recv(buf, BUFSIZ, boost::bind( handle_receive, asio::arg::error, asio::arg::bytes_recvd)); } I have been working on asio for well over a year and, as I said, it will be going into production in a commercial system in a little over a month. For commercial reasons I can't give too much detail, but I can say that asio has been used to develop a custom HTTP proxy that supports hundreds of concurrent connections. The inspiration for asio was the work of Alex Libman in developing a "portable Proactor framework" for ACE (see http://www.terabit.com.au) This is a Proactor that can use select or a similar synchronous event demultiplexer if no more appropriate OS mechanism is available. Regards, Chris

5 10

Secret handles and descriptors: Enemies of reuse, conspirators with frameworks.
by Aaron W. LaFramboise 13 Sep '04

13 Sep '04

Boost now contains several classes that provide abstraction for operating system resources that are logically _waitable_. A waitable resource is any resource for which an operation on it might not complete immediately. Examples include threads and processes, files and the file system, and sockets and pipes. A notification of some activity on a waitable resource is often called an _event_. In modern software, it is often necessary to wait for events asynchronously. Quite often threading is a good solution for this, but quite often it isn't. In these latter cases, it is necessary to have some sort of _demultiplexor_ entity that handles the system-specific mechanics of waiting for events. In both UNIX and Windows, to wait for events relating to a resource, one must have a handle or a descriptor associated with that resource. In general, this must be provided by the class that maintains the abstraction itself. If a class managing a waitable resource, through encapsulation, allows no way for a demultiplexor to access the handle needed for waiting, waiting is impossible. Despite having all the functionality that a programmer may require, this class is useless to the programmer because it can not be demultiplexed. This is unacceptable. To solve this design deficiency, a particular Boost demultiplexor class might be created. Boost classes that are associated with waitable resources might provide the needed handles to the demultiplexor—perhaps using _friend_—and so make asynchronous waiting possible. Unfortunately, there is a problem with this approach. Resources managed by classes outside of Boost would, in general, be incompatible with the demultiplexor. In addition, demultiplexors other than the one provided by Boost—that might be able to handle these classes incompatible with the Boost demultiplexor—would likely be unusable. The nature of typical demultiplexor implementations is that they monopolize control flow (for reasons that are usually unavoidable), and may only cooperate within the same program by using separate processes or threads, which may be inappropriate if there is no other logical reason for such separation. The affect is that waitable resources provided by libraries other than Boost are useless to the Boost programmer, as they are fundamentally incompatible. This is the mark of a conniving, conspiratorial framework, which lures its victims in with attractive features, and then locks them in its dungeon by preventing them from using other libraries. This is also unacceptable. It is the unfortunate truth that until the C++ world decides on universal resource concepts there will always need to be some piece of code that knows the interfaces of both the waitable resource and the demultiplexor. However, there is no need for this dependent code to be part of either the demultiplexor or the resource. This code might be an adapter for the Boost thread class that allows it to be compatible with a particular third party demultiplexor class. In consideration of the these issues, one may conclude that there must be a mechanism in Boost classes to allow arbitrary code to access the encapsulated handle or descriptor that describes the underlying resource for purposes of implementing demultiplexing. While this may, at first, seem a potential violation of encapsulation, it might also be seen as proper application of the Open-Closed Principle. One simple mechanism for achieving this is presented: class unix_thread_handle { protected: pthread_t handle() { return hand; } private: pthread_t hand; }; class thread : public unix_thread_handle, public boost::noncopyable { // ... }; Demultiplexor adaptors are permitted to access the underlying handles by deriving from a system-specific type and copying from an existing resource. Such access is subject to the lifetime of the origin object and documented constraints about the meaning of the handle. A demultiplexor should also be designed to be compatible. The demultiplexor must provide some mechanism to allow code outside of Boost to obtain control flow when particular system events occur. In addition, interoperability should be maintained not only with foreign waitable resource classes, but with other demultiplexors, to the maximum extent possible. For example, on a system where an event is delivered through a chain of handlers (such as Windows hooks or UNIX sigaction), the demultiplexor must be careful to pass on the event when appropriate. In light of recent interest and development relating waiting and demultiplexing, it is especially important that the above points are considered. Writing a naïve demultiplexor that will forever plague future programmers who use third party libraries is a temptation that should be strenuously ignored. There are already plenty of frameworks that have implemented this unfortunate xenophobia. If Boost is able to offer an interface that makes its waitable resources and future demultiplexor openly compatible with the rest of the world, it will continue to be popular long after other libraries have outlived their usefulness. Any thoughtful consideration and comments would be greatly appreciated. If there is some interest in these things for Boost, I’d like to prepare some proof-of-concept alterations for waitable parts of Boost. I also have designed a cross-platform demultiplexor class, but unfortunately it is not in a state that is immediately ready to be shared with Boost. However, if there was interest, I would consider polishing it up. Aaron W. LaFramboise

5 14

RE: [boost] Formal review of "Output Formatters" library begins today
by Reece Dunn 13 Sep '04

13 Sep '04

Paul A Bristow wrote: >| [mailto:boost-bounces@lists.boost.org] On Behalf Of Reece Dunn >| > One thing I would REALLY REALLY like is a Boost 'Standard' way of >| >colouring and indenting code similar to Visual Studio IDE - >| though I don't >| >feel their colour scheme is quite as good mine! >| >| What do you think about the new Boost.quickdoc? The docs for >| quickbook are >| available at http://tinyurl.com/64el7, and the source is at >| http://tinyurl.com/4tdjj. Thanks to Eric and Joel for this great tool. > >I don't like the greyed background to anything. I don't think it helps at >all. >It makes things take longer to print out if you need a paper copy - when >the >going gets really tough. >Nor do the boxes around really help much either, but they at least don't >hinder. > >The coloring isn't strong enough for my taste - using blue for keywords is >a >great help in reading, IIUC, this can all be controlled via the CSS, e.g.: .keyword{ color: red; font-weight: bold; } >My view is that color is a massive hit in helping people to read code, >especially if it can be applied uniformly, something that Boost could >'impose'. This is true and would be where QuickDoc --> BoostBook --> HTML+CSS pays off. I am not sure about how the styling is controlled in XSL:FO+PDF, though. QuickDoc for syntax highlighting the C/C++ code; BoostBook for a consistent documentation format and CSS for consistent styling. Regards, Reece _________________________________________________________________ Want to block unwanted pop-ups? Download the free MSN Toolbar now! http://toolbar.msn.co.uk/

1 0

[1.32 Release] Branch-for-release date
by Aleksey Gurtovoy 13 Sep '04

13 Sep '04

Hi All, With the new MPL in the main trunk and the updated MPL documentation coming later this week, we've finally come to the point when it's feasible to talk about the branch-for-release date. Given that there is still some work on regressions to be done, I'd like to aim for the end of the week, let's say Friday, September 17th. Does that sound OK to everybody? I'm assuming that at this point all library authors/maintainers are finished with the tasks they wanted to perform before the release, minor regression fixes aside. If not, please scream loudly! -- Aleksey Gurtovoy MetaCommunications Engineering

7 8

RE: [boost] Re: Compression usinig IOStreams library
by Ben Hutchings 13 Sep '04

13 Sep '04

George M. Garner Jr. <gmgarner(a)erols.com> wrote: [Jonathan Turkanis wrote:] > > Dumb question - is Winzip supposed to be able to read the zlib > > format? I usually use it to decompress .zip or tar.gz archives. > > Well I made that assumption because Winzip and XP both maintain > associations for the .z extension. This is from the Winzip help > file: <snip> The ".Z" extension is conventionally used for files compressed with the Unix "compress" program, which uses LZW. zlib and gzip were created to provide a patent-free alternative to it. They use the same compression algorithm as PKZip and WinZip but do not use the PKZip file format. I doubt that WinZip has any support for raw compressed files.

2 1

[1.32 Release] Fixing long filenames
by Aleksey Gurtovoy 13 Sep '04

13 Sep '04

One thing that we really have to take care of before branching is long filenames. Could maintainers of the following libraries please fix the corresponding problems listed in detail in http://www.boost.org/regression-logs/inspection_report.html? archive (serialization) build date_time graph iterator preprocessor property_map python serialization spirit test utility/enable_if Thanks! -- Aleksey Gurtovoy MetaCommunications Engineering

7 8

Re: [boost] Re: Formal review of "OutputFormatters"library beginstoday
by Reece Dunn 13 Sep '04

13 Sep '04

John Torjo wrote: >>That is basically what I do already: >>wrappedfmt() is associated with openclose_formatter; >>pairfmt() is associated with formatter (using default_nary_traits) >>containerfmt() is associated with formatter (using >>default_sequence_traits) >>etc. > >Yes, and that is great! >But you hold them in the formatob object. I have this relationship: formatob isa FormatObject isa formatter usesa DefaultTraits FormatObject hasa FormatObject >What I'm saying is that (overridable) defaults for each of the above can >also be kept in the stream object itself. I understand what you are saying, I am just looking at how to implement it. >>so you can theoretically use any string type to store the default >>delimeter values. How do we do the equivalent with ios_base::pword/iword? > >That's your job - to come up with a technique for this ;) :) >Basically, the library's underlying concept doesn't have to change. This is >just an extra goodie you can provide. Sure. >>openclose_formatter/formatter set their delimeter values to the values >>provided by the default traits object passed to them. You can then >>override these defaults by calling format(). How do you tell that the >>default has been overrided? > >It's certainly possible. Internally, in the delim object, you can keep the >data like this: > >template<class value_type> struct val_keeper { > value_type val; > bool has_been_set; > //getter and setter - I assume you get the point >}; This might be easier if instead of formatter holding DelimeterType objects for each delimeter, have it store an io::delimeter< CharT > object and have that responsible for access, thus: template< typename CharT > class delimeter { private: const CharT * value; bool is_set; public: const CharT * get() const{ return( is_set ? value : ios_base::pword -> value()); } void set( const CharT * v ){ value = v; is_set = true; } delimeter(): value( "" ), is_set( false ){} }; and template< typename CharT > class formatter { public: delimeter< CharT > separator; // separator is now a property! }; >>The main questions are: >>[1] how do you store different DelimeterTypes in this mechanism? > >Which brings us to ;) : >I think one type of delimeter is enough (I assume you wholeheartidly >disagree with this). I don't wholeheartedly disagree... I am open to persuasion ;) >This type should be the std::basic_string<underlying_char_type>(). This would still mean that you have one per underlying_char_type, so the same problem exists although it is now in a smaller problem domain. My main reason against using std::basic_string as the storage type is performance (excessive copying of the delimeter values). Also, what about managed strings in C++/CLI? Don't they require a String^ type, or am I missing something? Regards, Reece _________________________________________________________________ Get ready for school! Find articles, homework help and more in the Back to School Guide! http://special.msn.com/network/04backtoschool.armx

2 1