- Boost - lists.boost.org

Re: Sockets: proposal for a library design
by Don G 16 Mar '05

16 Mar '05

Hi Scott, > Networking is inherently asynchronous. Initiation and acceptance > of connections and I/O may be disguised as synchronous but other > activity (e.g. "network down") cannot. Network APIs will often > defer such errors until the next reporting opportunity (e.g. next > I/O request) but this is a band-aid that is sometimes enough and > at other times downright misleading. I assume that something > going by the name of Boost.Net would be targeting a higher > sophistication. Amen. It took me some time to get used to the work around that one should always try to have a pending read in order to detect problems. In some cases that works, but it depends on the protocol whether or not you can read at the right time. > Proposing that all network communication is asynchronous does > not preclude any synchronous coding. As discussed in other > threads (i.e. RPC) a "low-level", asynchronous library may be > the "core" of a Boost.net while synchronous services may be > built on top. While this will work, in my experience there are efficiencies to be had if the synchronous methods aren't forced to create/store callbacks that simply signal condition variables. Of course, the gains will depend on the cost of the async mechanism in question. > I think an asynchronous, core Boost.Net should expose fundamental > network services. In the case of TCP, that would be the reliable, > async 2-way transfer of data blocks. Achieving a clean and > portable library at this level would be achievement enough. What, no UDP? :) > Dragging I/O streams into the "network discussion" at this > point is almost premature; it moves the focus up the network > stack. Again this doesn't preclude those who need to from > integrating with I/O streams at a later point. I agree. > $0.01 Hey! That was at least $0.03! Best, Don __________________________________ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs

1 0

Re: Sockets: proposal for a library design
by Don G 16 Mar '05

16 Mar '05

Hi Boris, My responses are below... >> I would suggest that a proposed Boost.Net library should focus >> on networking in a more abstract manner, and not be merely a >> socket encapsulation. IMHO, sockets are a terrible API and >> should be hidden as fast as possible. A socket wrapper class >> is appropriate, but I would not go to any trouble to change >> the socket API. Simply wrap the resources and provide portable >> access to the underlying system. The real end-user part of the >> Net library should not talk about sockets (unless one provides >> a method like get_native to return the socket handle, and that >> method is defined as non-portable). > > This reminds me of the layers requirement from > http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl? > BoostSocket/SocketRequirements. > I agree that it would be nice to have some easy-to-use high-level > classes for concepts that are used in network applications again > and again. Yes; sockets are just a means to that end and not the end in themselves. As such, they need not receive much special design consideration because that is not where you want users to interact with the library. A robust socket layer may only encourage undesirable use. The socket layer that has involved in my own code consists of a single Socket class with: a destructor for cleanup, methods that handle the mapping of non-portable EFOOBAR codes to portable exception types or return values, const-correctness, default arguments in a few places, etc.. The network layer simply uses sockets to implement the abstraction, and is the only direct user of sockets. > I agree again. There shouldn't be any of the four I/O models > omitted. We will see as we proceed if supporting various I/O > models is easy to implement and to use at runtime (especially > switching between I/O models). The I/O model that troubles me is the select/poll model. That is the mechanism I have used to implement the other three and it is not readily apparent how to expose it to the user as well. I may be surprised, but I think most folks would not mind seeing that model go away. The async I/O model with callbacks (essentially made from inside the select/poll/whatever loop) would provide a robust replacement for that tedious body of code. >> As others have also suggested, I would agree that the net >> facilities not be tied up into I/O streams. It would be >> better to provide the network facilities and then layer >> on I/O streams for those that want it. > > Several people dislike the idea of a network library built > on I/O streams. What are the reasons for this? I can only speak for myself, but I just don't like I/O streams. They seem to have a lot of clunk and little value, IMHO. They may work for text protocols, but in my use of network communications, they don't feel right. For example, the output would probably need consistent flushing do get data out of the buffer and on to the wire. > Is this a request for a more low-level access to socket > functions like read(), write(), send() and recv()? Again, I don't view socket access as the goal, but essentially yes: access to the "primitives" is what I would want. For example, I would want to be able to differentiate EOF from ECONNRESET(?) from ETIMEOUT, etc.. > The only reason I see why I/O streams should not be used is > the lack of support for an asynchronous I/O model. The > network library would have to add functions for asynchronous > support Yes, the network layer has to define what asynchronous I/O looks like. Again, I think a function<> with an appropriate signature is the answer. If the users needs those calls to be queued/marshalled, that is not the problem of the network layer. > until one day I/O streams support an asynchronous > I/O model by default (btw, anyone working on this?). I don't see how I/O streams could be asynchronous because async file I/O is not portable. For example Windows 95/98/Me don't have it. I suspect that many other systems are similar. Even though Windows NT (and 2000, XP, 2K Server) have async file I/O, it is not something that select() can handle as it can on *nix. >> Usage examples: > > Thanks again! When the socket class hierarchy is more > stable I hope to move upwards to high-level classes > representing concepts from the network world you used > in your examples. Most welcome. If I recall, the actual number of calls to socket API's made in my entire code base was so few (probably around 10-20) that even the little time I had spent on the socket layer was probably overkill. >> One thing I think is essential when using a blocking model >> is cancellation. For example, I have a reset() method on >> Acceptor, Address, Datagram and Stream that can be called >> at any time to cause blocking methods to terminate with a >> cancel exception (obviously this must be from another >> thread). Ironically, these reset() methods meant I could >> not use blocking socket calls to implement my blocking >> methods - recv() cannot be canceled once it blocks (at >> least not portably)! > > I added this to the Wiki but I am not sure if a network > library should support it. If you have two threads in your > application and one thread is halted because of a blocking > call it might have been better to use a > nonblocking/asynchronous I/O model? Perhaps, but some produces are just best designed as synchronous. They can go along merrily producing data and waiting for it to be consumed before producing more. Eventually they will block, and if they cannot be canceled (forced to unblock), you have a problem. IMO, any mechanism that can block indefinitely really needs some way to cancel. This should not be confused with thread cancellation. In this situation, once a stream is canceled/reset, it is almost completely unusable (most methods will throw the cancel exception). Best, Don __________________________________ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs

1 0

boost::bind now with limited operator support
by Peter Dimov 15 Mar '05

15 Mar '05

boost::bind now supports operator! and the relational operators ==, !=, <, <=, >, >=. For the latter, the left operand must be a bind expression, the right operand can be a value, a placeholder, or another bind expression. Not exactly a Lambda, but good enough for many common tasks. -- Peter Dimov http://www.pdimov.com

1 0

Re: [boost] Re: [review] hash functions
by brangdon＠cix.compulink.co.uk 15 Mar '05

15 Mar '05

In-Reply-To: <d11prg$uvt$1(a)sea.gmane.org> daniel(a)calamity.org.uk (Daniel James) wrote (abridged): > > Firstly, I think we have some agreement that the signature: > > > > size_t hash_range( It first, It last ); > > > > is wrong > > I didn't get the impression of any agreement about that. > > > and there should be some way to provide an initial seed. > > Although, I agree with this. That's all I meant. I knew there wasn't much agreement over whether side effects are bad :-) My review was a lot more conservative than my comments in discussion. > > We have some agreement that using reinterpret_cast would be better. > > Yes, and this will be fixed. When I wrote it seemed there were still unresolved issues. In fact, I don't believe anyone has yet posted an acceptable hash_value for pointers. Presumably we want to shift out bits which are always 0 for alignment reasons. > It is proposed that the standard explicitly specifies the hash > functions (and I've been criticized for not doing that) but part > of the purpose of this library is to get real world experience > on what the specification should be. And change it, if necessary. OK. > I do intend to spend more time looking into alternatives in the future > (and will look into adding a constant to hash_combine very soon). OK. Although I suggested it, I'm not sure it's the best fix, as compared with using non-zero initial seeds. const size_t default_seed = 0x9e3779b9; // Or whatever. size_t hash_range( It first, It last, seed=default_seed ); A signature like that provides a hook to introduce variation between similar types if the author desires, and also allows ranges to be hashed as subranges with the output of one subrange feeding into the next. The default value allows the issue to be safely ignored by naive users. So it's a good idea anyway, and since it also fixes the problem with zeros maybe we don't need to fix that twice. This does leave the issue of other places where hash_combine is called. It would be sufficient if we could be sure that all used a non-zero initial seed, eg: size_t hash_value( std::pair<int,int> p ) { size_t hash = default_seed; combine_hash( hash, p.first ); combine_hash( hash, p.second ); return hash; } and the documentation would encourage that, but we can't really be sure of it. Another approach is to change all the primitive functions. For example: size_t hash_value( int i ) { return static_cast<unsigned int>(i) + default_seed; } Introducing the non-zero constant here is surely as efficient as in combine_hash, if not more so. Admittedly it doesn't eliminate the zero, it just moves it around, but it should massively reduce its dynamic frequency. (Actually it will eliminate the zero where it matters most, ie for types like bool, char and short which are smaller than size_t.) > Did you see Peter's original proposal? Yes, I had followed most of the links. > > For example, the proposal has no specialisation for chars at all. I > > can't tell if that is an oversight, or whether it is a design > > aim [...] > I don't think it's either. Peter? Peter has now said it is a design aim. That even you weren't sure illustrates my point :-) > When hash_range is called with a seed argument, it would be equivalent > to calling hash_combine for pair<int, int>. If I understand the principles being advocated here, pair< pair<int,int>, pair<int,int> > should have the same hash_value as int[4]. How do you achieve that without giving hash_value( pair ) an initial seed? ----- I find myself increasingly drawn to the idea of a hasher object. The main signature should be like: template <typename T> Hasher &hash( Hasher &hasher, const T &t ); with implementations looking like, eg: template <typename First, typename Second> Hasher &hash( Hasher &hasher, const pair<First,Second> &p ) { hash( hasher, first ); hash( hasher, second ); return hash; } Then hash_value, if we still need it, would be like: template <typename T> size_t hash_value( T &t ) { Hasher hasher; hash( hasher, t ); hasher.finalise(); return hasher.value(); } but would rarely be called directly. This solves the problem of the default value, allows us to store intermediate results bigger than size_t (which may be important for cryptographic-strength hashing), and the finalise() step provides a hook for the final mixing/avalanching which some algorithms use (eg Paul Hsieh's). > Perhaps it would be better spelt as: > > template <class It> > void hash_combine(std::size_t& seed, It begin, It end); Yes, I considered that. I wasn't sure whether overloading the name would help or hinder - hash_range and hash_combine are not substitutable for each other. > Well, we did discus the type of review before it started, and nobody > raised any objections. I don't object to quick reviews in principle (although if someone can only contribute, eg, on Sunday mornings, a 5-day period is going to leave them out completely). > > I am not an expert on hashing, but I have implemented my own hashing > > library. I have encountered practical problems due to different > > objects having the same hash values. > > Is this in C++? Yes. The main problem was zeros. My hash primitives produced zeros and my hash_combine turned zeros into more zeros. Just as with the initial boost proposal. > I would have thought that type safety would have helped avoid > such problems. This was specifically an issue with optional or alternate types. For example, I had something similar to: size_t MyType::hash_value() const { return use_a ? hash_value(a) : hash_value(b); } where a and b could be different types. We can avoid that mistake in library classes like boost:variant, but that doesn't prevent it recurring in user code. When I am dealing with types without much intrinsic variation, eg bools, I found it important to exploit such variation as I did have as much as possible. Hashing false and 0 to different values helped. > Thank you for reviewing the library and bringing up several good points. My pleasure. -- Dave Harris, Nottingham, UK

3 3

boost::filesystem in Windows 95
by Jorge Lodos 15 Mar '05

15 Mar '05

Hi We have a code base for Windows that compiles with common settings for all Windows platforms. When using boost::filesystem, this is no longer possible, due to the fact that Windows implementation in operations_posix_windows.cpp uses the API function GetFileAttributesExA, not available in Windows 95. GetFileAttributesExA could be replaced by FindFirstFile, which gives the same information and is available in all Windows platforms. This API is used in two different places, BOOST_FILESYSTEM_DECL bool _is_empty( const path & ph ) implementation and BOOST_FILESYSTEM_DECL boost::intmax_t file_size( const path & ph ) implementation. Using FindFirstFile instead of GetFileAttributesExA we would get: BOOST_FILESYSTEM_DECL bool _is_empty( const path & ph ) { # ifdef BOOST_POSIX ........ # else WIN32_FIND_DATA ; HANDLE hf = ::FindFirstFileA(ph.string().c_str(), &ffd); if (hf == INVALID_HANDLE_VALUE) boost::throw_exception( filesystem_error( "boost::filesystem::is_empty", ph, fs::detail::system_error_code() ) ); CloseHandle(hf); return ( ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY ) ? is_empty_directory( ph ) :( !ffd.nFileSizeHigh && !ffd.nFileSizeLow ); # endif } BOOST_FILESYSTEM_DECL boost::intmax_t file_size( const path & ph ) { # ifdef BOOST_POSIX ....... # else // by now, intmax_t is 64-bits on all Windows compilers WIN32_FIND_DATA ; HANDLE hf = ::FindFirstFileA(ph.string().c_str(), &ffd); if (hf == INVALID_HANDLE_VALUE) boost::throw_exception( filesystem_error( "boost::filesystem::file_size", ph, fs::detail::system_error_code() ) ); CloseHandle(hf); if ( (ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) !=0 ) boost::throw_exception( filesystem_error( "boost::filesystem::file_size", ph, "invalid: is a directory", is_directory_error ) ); return (static_cast<boost::intmax_t>(ffd.nFileSizeHigh) << (sizeof(ffd.nFileSizeLow)*8)) + ffd.nFileSizeLow; # endif } Is there any chance this could be solved in boost::filesystem? Thank you for boost :-) Best regards Jorge

2 1

Re: [boost] [review] hash functions
by brangdon＠cix.compulink.co.uk 15 Mar '05

15 Mar '05

In-Reply-To: <001201c527e4$7405efa0$6501a8c0@pdimov2> pdimov(a)mmltd.net (Peter Dimov) wrote (abridged): > > In my experience, if you want two different things to hash the same, > > you need to write their hash functions with that in mind. It is a > > special case which justifies not using the default hash function. > > Maybe, but this is not an argument for making the hash values different. True. Arguments for that have been given elsewhere. One argument is to maximise variation (and for types like bool it is desperately needed). Another is to allow treating of 32-bit values as bytes. For example, size_t hash_range( const int *first, const int *last ) { return hash_range( static_cast<const unsigned char *>(first), static_cast<const unsigned char *>(last) ); } may be a good implementation on some platforms. It does 4 times as many mixing steps (assuming sizeof(int) == 4) as the proposed algorithm. It is, I think, how the designer of the proposed hash_combine algorithm expected it to be used. However, it yields hash values for int * different to the values for char *, so with the current design aims we can't use it. > > But doesn't this mean that hash_value( pair<int,int> ) also needs a > > seed argument, if you want it to match hash_range( int[2] ) ? > > No. The seed version of hash_range is used when you want to compute the > hash function of a sequence, but you don't have access to the whole > sequence at once. It's goal is not to provide a way to change the > initial seed, but to make it possible to continue from an intermediate > seed. But suppose I have pair< pair<int,int>, pair<int,int> > ? Shouldn't that yield the same hash value as int[4] ? -- Dave Harris, Nottingham, UK

5 5

ANN: review of FSM library starts
by Pavel Vozenilek 15 Mar '05

15 Mar '05

Review of Finite State Machine library written by Andreas Huber starts today, February 21, and lasts 10 days. The library can be found at: http://boost-sandbox.sf.net/fsm.zip (781 kB). FSM is feature rich library to design finite state machines without need for external generator. It is accompanied by extensive documentation. Here are some questions you might want to answer in your review: * What is your evaluation of the documentation? How easy (or hard) it is to understand library features? What can be improved? * What is your evaluation of the design? What features are supported better by alternative FSMs? What could be added (or removed) from the library? * The library documentation contains few not yet solved issues (name, separating the library into two parts, exception handling). What is you opinion here? * What is your evaluation of the implementation? Are there parts of code too obscure or duplicating exiting Boost functionality? Can something be factored out to standalone library or among general utilities? * Are there performance bottlenecks? Does the library design fit requirements of real-time systems? How is it useable for embedded systems? Is the documentation clear on performance tradeoffs? * What is your evaluation of the potential usefulness of the library? Can you compare this FSM to other implementations? * Did you try to use the library? With what compiler? Did you have any problems? Do you have tips for better support of older compilers? What are possible portability problems? * How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? * Are you knowledgeable about the problem domain? And finally, every reviewer should answer this question: * Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. Pavel Vozenilek FSM review manager

7 19

FSM Review
by Jody Hagins 15 Mar '05

15 Mar '05

I am sorry this is late. I made the mistake of introducing my kids to snow skiing. Now they can not get enough of it. Thus, we were gone again until Wednesday (yes, I let them skip a few days of school to go skiing). * What is your evaluation of the documentation? How easy (or hard) it is to understand library features? What can be improved? I think the documentation is relatively complete. However, I am not exactly thrilled by the requirement on the reader to understand UML diagrams. On the other hand, this library is intended to represent UML in C++ (at least to my understanding), so I'm not sure what else can be done. A link to a basic tutorial on UML state machine diagrams would be appropriate in the "UML to boost::fsm Mapping Summary." I know you link to the 730 page OMG document on UML, but I'd also like to see something that describes the basic UML for state charts. * What is your evaluation of the design? What features are supported better by alternative FSMs? What could be added (or removed) from the library? I think the requirements dictate the design. Specifically, the desire to mimic UML statecharts in C++. As stated before, the design brings with it some performance tradeoffs that will make this library unusable to some. * The library documentation contains few not yet solved issues (name, separating the library into two parts, exception handling). What is you opinion here? I think it is obvious that the library should be separated, though I do not think this issue itself should be a stumbling block for acceptance. Exception handling is very important, and I do not buy the rationale that allowing destructors to throw is not a problem in practice. I also do not think I really understand "two stage exit." * What is your evaluation of the implementation? Are there parts of code too obscure or duplicating exiting Boost functionality? Can something be factored out to standalone library or among general utilities? I did not look too much at the implementation, so I can not comment much here. * Are there performance bottlenecks? Does the library design fit requirements of real-time systems? How is it useable for embedded systems? Is the documentation clear on performance tradeoffs? I think we have to define real-time. Traditionally, real-time systems have hard completion time requirements (e.g., the XYZ event must be processed in under 50ms). Even though I have performance issues with this library, I think it can still be utilized in many real-time situations. Andreas has stated several times that the dispatch time will be small compared to the handling time. However, I find this argument a bit lacking, especially for hard real-time requirements, where each cycle taken for dispatch is stolen from the reaction handler. My current application requirements do not fit the traditional definition of real-time. However, my SMs need to be as fast as possible, because in a real sense, my code is competing with other code. Without going into too much detail, when an "opportunity" is available, it is only available to the first one to see it, respond, and take that opportunity. Thus, my real-time requirement is that I have to be faster than anyone else playing my game. I think using this library would put me at a disadvantage in that particular space. However, I do have some applications in which this library could be used successfully (if I were inclined to learn enough UML to make sense of all the details). The documentation is clear enough about the performance issues. In fact, it may be too clear, as some people seem to see problems that may not exist in most applications. One last thing on performance, since it has been a hot topic in relation to this library. I do not think it has sufficient performance to warrant use in all my applications, and I assume the same for many others. However, it does have sufficient performance to be used in a large number of applications. If the docuemntation is clear (which it is), then users can decide when/how it can be used. I could make similar performance claims agains almost every boost library (and I have made them on this list about shared_ptr and signals). However, that does not mean we should eliminate those libraries from boost, nor does it mean that they should never have been accepted (but I would like to see a larger commitment to address performance issues in all currently accepted libraries). I would, however, like to see a separate submission which has a focus on performance, and I would really like to see the performance issue of this library revisited, especially with input from some of the folks here who claim expertise in that area. * What is your evaluation of the potential usefulness of the library? Can you compare this FSM to other implementations? I have used sevaral versions of SMC, solutions from Samek's "Practical Statecharts in C/C++," and my own hand-crafted stuff. I think SMC is the easiest to use, though it also offers the least flexibility. With moderate use, I think this library could become the standard tool for implementing state machines, with the caveat of every tool: it may need to be replaced if it is too slow. However, I also think that this tool requires a more structured design of your state machine than ad-hoc methods. Thus, if speed were an issue, translating this code to something else would not be too difficult. One thing I have not seen in other reviews is a comparison to GUI state machine tools. I know this is a library, with very different requirements. However, having virtually no experience with GUI tools that build FSMs, I'd be interested to see how this code stacks up with code generated by such tools. Specifically, I have avoided them because I want more control than they allow (or at least allowed when I first looked at them). Does anyone have experiences to offer in this regard? * Did you try to use the library? With what compiler? Did you have any problems? Do you have tips for better support of older compilers? What are possible portability problems? I used it on linux, with gcc. I am probably the last person in this community to offer porting advice to older compilers. * How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I read all the documentation, and played with a few examples. Specifically, I tried to implement a subset of a current state machine to compare ease of use and performance. This library seemed more difficult to use, but I attribute most of that to inexperience. I am convinced that with some moderate level of experience, I could write state machines more easily with this tool. * Are you knowledgeable about the problem domain? I would not consider myself an expert, by any stretch of imagination. However, I have done my fair share of work coding state machines. I have used a small number of tools, and I have crafted my own, so I am at least comfortable with the problems represented by such a library. And finally, every reviewer should answer this question: * Do you think the library should be accepted as a Boost library? Be sure to say this explicitly so that your other comments don't obscure your overall opinion. Yes, with two conditions. 1. The name should definitely be changed. I am not sure, however, what a new name should be. I tend to favor one of Andreas' alternatives: umlfsm. 2. I also think there should be some commitment to revisit the performance issues. I am not sure how this works, but I'd be satisfied with a commitment from Andreas, possibly accompanied by a request to the Boost community for assitance. I am not certain that an acceptable solution can be found, but I think the issue at least needs to be addressed again, specifically with input from the Boost community.

12 37

[Boost-bugs] [ boost-Feature Requests-1163952 ] handle double '/' in the boost::filesystem
by SourceForge.net 15 Mar '05

15 Mar '05

Feature Requests item #1163952, was opened at 2005-03-15 22:17 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=357586&aid=1163952&group_… Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Serg Repalov (rsrus) Assigned to: Nobody/Anonymous (nobody) Summary: handle double '/' in the boost::filesystem Initial Comment: Hi. Sometimes I make some path by hand. For example std::string str = "./"; //... some code .... std::string str2 = str + "/path/to/file"; The result is ".//path/to/file". If I try to make path from it: boost::filesystem::path fs_path2(str2, fs::native); I got exception. Is it possible to correct function path::m_path_append to handle this situation (double/triple/... separator in the middle of string)? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=357586&aid=1163952&group_… ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Boost-bugs mailing list Boost-bugs(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/boost-bugs

1 0

Re: [boost] [FSM library] review result: accepted with fewconditions
by Iain Hanson 15 Mar '05

15 Mar '05

On Tue, 2005-03-15 at 19:10 +0100, Pavel Vozenilek wrote: > FSM library written by Andreas Huber has been > accepted into Boost, with few conditions > and few recomendations. > > > Thanks, Andreas, for your work and stamina > to get thus far! > Yes and congratulations to Andreas. > > The documentation is now very dense, splitting > it into smaller parts may help. More hyperlinking > would also help a lot. > Then could there also be a version suitable for printing structured pretty much as it is now please. /ikh

2 1