March 2005 - Boost - lists.boost.org

Re: [boost] [review] hash functions
by brangdon＠cix.compulink.co.uk 16 Mar '05

16 Mar '05

In-Reply-To: <01b201c52987$e0bbb950$6801a8c0@pdimov> pdimov(a)mmltd.net (Peter Dimov) wrote (abridged): > If you _know_ how the default hash function will perform on your key > set, you can make the decision of whether to use it or not, and if the > hash function is standard, you do not have to re-evaluate your decision > each time you change platforms. I do appreciate the benefits of stability. What isn't clear is whether it trumps hash quality. This is a judgement call, in that it partly depends on which issue is more important to you. Ultimately if you want stability you can get it by using your own hasher. If you want the best quality the platform can deliver, then that's intrinsically non-portable; it should be provided by the vendor but in a standard way. We could both be made happy if boost provided 2 hashers as standard, one for stability and one for quality. However, I don't think we want to go that route at this stage, and anyway there can only be one default. This is /not/ a deal-breaker for me. I won't reject the proposal because it favours stability over hash quality. -- Dave Harris, Nottingham, UK

2 1

Re: [boost] Re: [review] hash functions
by brangdon＠cix.compulink.co.uk 16 Mar '05

16 Mar '05

In-Reply-To: <01c301c52989$d50a2f40$6801a8c0@pdimov> pdimov(a)mmltd.net (Peter Dimov) wrote (abridged): > I think that it is better to fix the zero trap in hash_combine, because > this does not depend on users remembering to initialize the seed to > default_seed, instead of zero. Did you see my suggestion of changing the primitive hash_value functions instead? That also does not depend on users remembering to initialise the seed. size_t hash_value( int v ) { return size_t(v) + default_seed; } And similarly for long, etc. Is there a reason to reject this? > 2. Add a hash_range overload that takes a seed. > > 2a. void hash_range( size_t & seed, It first, It last ); > 2b. size_t hash_range( size_t seed, It first, It last ); > 2c. size_t hash_range( It first, It last, size_t seed ); > > which are listed in order of my preference. So we're talking about a family of 3 functions: 2a: void hash_combine( size_t &seed, T value ); void hash_range( size_t &seed, It first, It last ); size_t hash_range( It first, It last ); 2b: void hash_combine( size_t &seed, T value ); size_t hash_range( size_t seed, It first, It last ); size_t hash_range( It first, It last ); 2c: void hash_combine( size_t &seed, T value ); size_t hash_range( It first, It last, size_t seed=0 ); I think 2b is very error-prone, because of the lack of consistency between how hash_combine and the seeded version of hash_range return their results. 2a is less error-prone, but it seems needlessly confusing to have the two versions of hash_range return their results in different ways. In general there's a lot of similarity between hash_combine and hash_range which those versions fail to bring out. I'd rather be more consistent, eg: (a) size_t hash_combine( size_t seed, T value ); size_t hash_range( size_t seed, It first, It last ); or: (b) void hash_combine( size_t &seed, T value ); void hash_range( size_t &seed, It first, It last ); (Perhaps with overloads to catch passing a non-size_t first argument.) I don't see much value in adding a hash_range which doesn't take a seed. Especially with (a), because we can easily pass 0 explicitly. -- Dave Harris, Nottingham, UK

2 1

Re: [boost] Re: [review] hash functions
by brangdon＠cix.compulink.co.uk 16 Mar '05

16 Mar '05

In-Reply-To: <019d01c52985$a04f54f0$6801a8c0@pdimov> pdimov(a)mmltd.net (Peter Dimov) wrote (abridged): > You also need equal_ptr<MyType>. Indeed. > Also, it is easy to break the above unordered_set my modifying some > of the MyTypes. Yes, that's something to be aware of. I use the approach on types which are immutable, so it doesn't arise. > size_t Derived::hash_value() const > { > size_t seed = my_unique_value; > > hash_combine( seed, Base::hash_value() ); > hash_combine( seed, m_value ); > > return seed; > } That leaves the problem of assigning a different my_unique_value to each subclass, bearing in mind that different subclasses may be written by different people, teams or organisations. Without some kind of system people may just copy and paste code from one class to another and not change the value. -- Dave Harris, Nottingham, UK

2 1

Re: [boost] Re: [review] hash functions
by brangdon＠cix.compulink.co.uk 16 Mar '05

16 Mar '05

In-Reply-To: <d17nlh$8o3$1(a)sea.gmane.org> daniel(a)calamity.org.uk (Daniel James) wrote (abridged): > I'm glad that I'm not the only person who's worried about alignment. > But this is a problems for a cross-platform library. Either I'll have > to introduce a load of macros to tweak it on various platforms (I'll > need help there) or just accept it. We can probably find something which gives reasonable results across a wide range of platforms. Shuffling the bits in a hash_combine-like manner would help. Eg something like: template <typename T> size_t hash_value( const T *p ) { size_t value = reinterpret_cast<size_t>(p); return value + value / sizeof(T); } If there's an easy way to turn sizeof(T) into a power of 2, use it. In many cases it will already be a power of 2, and I suspect the performance difference isn't crucial anyway. Alternatively we could just use: return value + (value >> 3); which ought to be reasonable for most small objects. It may also be reasonable for large objects that are heap-allocated, depending on how the heap is implemented. Anyway, I think it is better to do something like this than to just return value. -- Dave Harris, Nottingham, UK

1 0

Re: [boost] [review] hash functions
by brangdon＠cix.compulink.co.uk 16 Mar '05

16 Mar '05

In-Reply-To: <989aceac05031508512fb3bd43(a)mail.gmail.com> (reply-to:to:subject:in-reply-to:mime-version:content-type:content-transfe r-encoding:references; b=R/dgT5RcmFTM4l6wo1FsmJV0WmZRxxS4UwfDvrZfxiBXu3i07NjqdqXmY0k93EfeA2rzuv0U dyun6iZo4Mn86mrySznULAY97Xn/zgqN3tU49cmIB/EtvwNWQzyHhnrH/iKu8pfDQoo6swyqX4 AJ8ih0MJXLu4umRrhBw7YSD2E=) wrote (abridged): > > However, it yields hash values for int * different to the values for > > char *, so with the current design aims we can't use it. > > It also has the undesirable side-effect of making the hash value > depend on the machine endianness. The same set of integers would hash > to different values on (say) Sparc and Intel machines. True - another variant of the stability issue. -- Dave Harris, Nottingham, UK

1 0

Re: [boost] Re: Re: [review] hash functions
by brangdon＠cix.compulink.co.uk 16 Mar '05

16 Mar '05

In-Reply-To: <d174ak$vp3$1(a)sea.gmane.org> nesotto(a)cs.auc.dk (Thorsten Ottosen) wrote (abridged): > are you saying we don't want to hash a Base* as a void*, but use > object properties instead? Sometimes. It's a reasonable thing to do. It's what my own app actually does. It shouldn't be the default, of course. Mainly I'm saying that equality tests sometimes need to be polymorphic, and when they are the corresponding hash_value needs to be as well. So sometimes we will be comparing hash values from different types. (Other people here have claimed that never happens.) -- Dave Harris, Nottingham, UK

1 0

No tests on como?
by Peter Dimov 16 Mar '05

16 Mar '05

We don't seem to be running any tests on Comeau C++, why? Not using the "reference compiler" for our testing doesn't make much sense. Has the "reference compiler" changed without me noticing? -- Peter Dimov http://www.pdimov.com

8 23

How to add examples ot the regression testing?
by Gennadiy Rozental 16 Mar '05

16 Mar '05

I want to make sure that all examples are successfully compilable. What is the best way to do this? Gennadiy

3 3

Re: Sockets: proposal for a library design
by Don G 16 Mar '05

16 Mar '05

Hi Scott, > Networking is inherently asynchronous. Initiation and acceptance > of connections and I/O may be disguised as synchronous but other > activity (e.g. "network down") cannot. Network APIs will often > defer such errors until the next reporting opportunity (e.g. next > I/O request) but this is a band-aid that is sometimes enough and > at other times downright misleading. I assume that something > going by the name of Boost.Net would be targeting a higher > sophistication. Amen. It took me some time to get used to the work around that one should always try to have a pending read in order to detect problems. In some cases that works, but it depends on the protocol whether or not you can read at the right time. > Proposing that all network communication is asynchronous does > not preclude any synchronous coding. As discussed in other > threads (i.e. RPC) a "low-level", asynchronous library may be > the "core" of a Boost.net while synchronous services may be > built on top. While this will work, in my experience there are efficiencies to be had if the synchronous methods aren't forced to create/store callbacks that simply signal condition variables. Of course, the gains will depend on the cost of the async mechanism in question. > I think an asynchronous, core Boost.Net should expose fundamental > network services. In the case of TCP, that would be the reliable, > async 2-way transfer of data blocks. Achieving a clean and > portable library at this level would be achievement enough. What, no UDP? :) > Dragging I/O streams into the "network discussion" at this > point is almost premature; it moves the focus up the network > stack. Again this doesn't preclude those who need to from > integrating with I/O streams at a later point. I agree. > $0.01 Hey! That was at least $0.03! Best, Don __________________________________ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs

1 0

Re: Sockets: proposal for a library design
by Don G 16 Mar '05

16 Mar '05

Hi Boris, My responses are below... >> I would suggest that a proposed Boost.Net library should focus >> on networking in a more abstract manner, and not be merely a >> socket encapsulation. IMHO, sockets are a terrible API and >> should be hidden as fast as possible. A socket wrapper class >> is appropriate, but I would not go to any trouble to change >> the socket API. Simply wrap the resources and provide portable >> access to the underlying system. The real end-user part of the >> Net library should not talk about sockets (unless one provides >> a method like get_native to return the socket handle, and that >> method is defined as non-portable). > > This reminds me of the layers requirement from > http://www.crystalclearsoftware.com/cgi-bin/boost_wiki/wiki.pl? > BoostSocket/SocketRequirements. > I agree that it would be nice to have some easy-to-use high-level > classes for concepts that are used in network applications again > and again. Yes; sockets are just a means to that end and not the end in themselves. As such, they need not receive much special design consideration because that is not where you want users to interact with the library. A robust socket layer may only encourage undesirable use. The socket layer that has involved in my own code consists of a single Socket class with: a destructor for cleanup, methods that handle the mapping of non-portable EFOOBAR codes to portable exception types or return values, const-correctness, default arguments in a few places, etc.. The network layer simply uses sockets to implement the abstraction, and is the only direct user of sockets. > I agree again. There shouldn't be any of the four I/O models > omitted. We will see as we proceed if supporting various I/O > models is easy to implement and to use at runtime (especially > switching between I/O models). The I/O model that troubles me is the select/poll model. That is the mechanism I have used to implement the other three and it is not readily apparent how to expose it to the user as well. I may be surprised, but I think most folks would not mind seeing that model go away. The async I/O model with callbacks (essentially made from inside the select/poll/whatever loop) would provide a robust replacement for that tedious body of code. >> As others have also suggested, I would agree that the net >> facilities not be tied up into I/O streams. It would be >> better to provide the network facilities and then layer >> on I/O streams for those that want it. > > Several people dislike the idea of a network library built > on I/O streams. What are the reasons for this? I can only speak for myself, but I just don't like I/O streams. They seem to have a lot of clunk and little value, IMHO. They may work for text protocols, but in my use of network communications, they don't feel right. For example, the output would probably need consistent flushing do get data out of the buffer and on to the wire. > Is this a request for a more low-level access to socket > functions like read(), write(), send() and recv()? Again, I don't view socket access as the goal, but essentially yes: access to the "primitives" is what I would want. For example, I would want to be able to differentiate EOF from ECONNRESET(?) from ETIMEOUT, etc.. > The only reason I see why I/O streams should not be used is > the lack of support for an asynchronous I/O model. The > network library would have to add functions for asynchronous > support Yes, the network layer has to define what asynchronous I/O looks like. Again, I think a function<> with an appropriate signature is the answer. If the users needs those calls to be queued/marshalled, that is not the problem of the network layer. > until one day I/O streams support an asynchronous > I/O model by default (btw, anyone working on this?). I don't see how I/O streams could be asynchronous because async file I/O is not portable. For example Windows 95/98/Me don't have it. I suspect that many other systems are similar. Even though Windows NT (and 2000, XP, 2K Server) have async file I/O, it is not something that select() can handle as it can on *nix. >> Usage examples: > > Thanks again! When the socket class hierarchy is more > stable I hope to move upwards to high-level classes > representing concepts from the network world you used > in your examples. Most welcome. If I recall, the actual number of calls to socket API's made in my entire code base was so few (probably around 10-20) that even the little time I had spent on the socket layer was probably overkill. >> One thing I think is essential when using a blocking model >> is cancellation. For example, I have a reset() method on >> Acceptor, Address, Datagram and Stream that can be called >> at any time to cause blocking methods to terminate with a >> cancel exception (obviously this must be from another >> thread). Ironically, these reset() methods meant I could >> not use blocking socket calls to implement my blocking >> methods - recv() cannot be canceled once it blocks (at >> least not portably)! > > I added this to the Wiki but I am not sure if a network > library should support it. If you have two threads in your > application and one thread is halted because of a blocking > call it might have been better to use a > nonblocking/asynchronous I/O model? Perhaps, but some produces are just best designed as synchronous. They can go along merrily producing data and waiting for it to be consumed before producing more. Eventually they will block, and if they cannot be canceled (forced to unblock), you have a problem. IMO, any mechanism that can block indefinitely really needs some way to cancel. This should not be confused with thread cancellation. In this situation, once a stream is canceled/reset, it is almost completely unusable (most methods will throw the cancel exception). Best, Don __________________________________ Do you Yahoo!? Make Yahoo! your home page http://www.yahoo.com/r/hs

1 0