[boost] Re: Yet Another Network Library

From: Don G (dongryphon_at_[hidden])
Date: 2005-04-24 15:21:19

Hi Giovanni,

First off, thanks for the posting. I downloaded and will try to
understand your library as time permits. After a brief look, my head
is still spinning trying to understand the central concepts (my
limitation, not your code<g>). If I understand correctly, the library
is service/server side and not client-side at present, right?

> These are the main features/peculiarities of my
> library:
>
> - There is no socket concept because i don't really
> think it is natural, at least not for a C++ programmer.
> I use the acceptor, connector and stream concepts.

I think we have similar opinions about sockets ;) just followed
different paths from there.

> - User is not required to reference streams by
> pointer, streams are stack allocated or are simply
> members of another object. Internally they have a
> smart pointer to an implementaion handle. Consider
> them stack-based proxies.  The acceptor and the
> connector return the handle that is asigned to the
> stream.

So is this choice just for user simplification? Internally, the user
is still holding a pointer, right? What are the copy semantics of the
objects held by the user? This is where things can be tricky any way
you go. Either the object is copyable and confusion can come via
aliasing, or they aren't which is probably better in this case, but
could possibly cause some idioms to not work (like "stream s =
my_clever_stream_creator()"). My preference was to use shared_ptr<>
as the semantics are well understood and objects can layer easily in
obvious ways, but the cost is "->" vs "." syntax.

> - The preferred way to do input output is to use
> standard-like algorithms (i.e. copy) with buffered
> stream adaptors and specialized input/output
> iterators.  I believe that an efficient library can
> be written this way and be very C++-user-friendly.
> Classic read/write are still available, but their
> semantics might be surprising.

I agree that this is the right approach for many users and protocols,
but most network programmers (including myself<g>) need access to the
primitive behaviors. They won't find them surprising unless the
wrapping violates expectations coming from sockets-like programming.

> - All classes are concrete, no polymorphism is used
> (i.e. no virtuals). Polimorphic behaviour must
> currently be achieved with some external mean (i.e
> using the external polymorphism pattern.  I think
> that the boost::IDL library would be great).

Here is where we are at different ends of the spectrum :). I didn't
see any SSL code, so I can only imagine how the http code would
handle SSL vs. non-SSL stream underneath. Ideally, this should not
require two template instantiations like http<stream> and
http<ssl_stream>, for example.

In the end, I thought templates had little to offer at this level.
Parameterizing protocols by stream type seems (IMHO) to buy nothing
in particular except the removal of virtual at the expense of the
user having to specify <kind_of_stream> and _lots_ of extra code
generation. The app should be able to layer objects as it sees fit
and run-time polymorphism is (again, IMHO) the right solution to that
problem.

> - Errors can be reported both with exceptions and
> with error codes. Exceptions are used by default
> unless error callbacks are passed. This seems to
> work quite well.  Internally only error codes are
> used and exceptions are thrown only at the most
> external abstracion layer.

This is a good idea, and very similar to what I have done as well. At
least for async. What is the behavior of blocking read in the face of
error? Is the user callback made inside read? If so, what does read()
return?

> I will probably add status bits a-la iostreams.

What kind of bits? I can see eof and fail and those cannot be
cleared. Others?

> - File streams.  The library actually try to be
> a generalized i/o framework, and file streams are
> provided for completeness.

At an abstract level, they are very similar and should behave in a
similar way. I haven't tried to tackle that part because it is an
area where there is already something in place, albeit not async, and
I didn't want to try to integrate into iostream (not my cup of tea).

> - The library can be extended simply by creating
> new handles.  In addition to TCP streams there are
> Unix streams (come almost for free :-) and file
> streams. SSL/TLS was present but did get broken
> some time ago and didn't have the time to fix it.

I would be most curious to know how SSL fit in your library and how
other layers interact with or are shielded from it.

> - Input/Output buffer.
[some good stuff was here<g>]

>From the little I've read through the code, it looks like this is a
layer above the raw stream impl. I think that is exactly the right
way to go. :)

> Missing (definitelly not complete list):
>
> The library is fully sinchronous for now.  I'm
> still considering how to add asynch support. I
> think i will implement it in the buffered adaptor 
> I/O is done asynchronously to the internal
> buffers that can grow as much as it is necessary.
> Timeouts are definitelly a must-have.

Agreed on async and timeout. Can sync calls be manually/explicitly
canceled? In my experience (and opinion<g>), a reader/writer MT
design needs cancel semantics. Without it, such an app cannot be
responsive to outside stimuli.

> Final notes:
>
> I've have seen that the current consens is to
> encode the the stream type in the address, so
> to allow a dynamic behaviour: the actual
> transport is selected only at runtime, based
> on the address string. I think this is a bad
> decision (i considered doing it while
> implementing my library) and this is why:

I am more and more convinced that this is not the right approach for
the library core, but from different reasons (see other posts). It
could be offered as a stand-alone library for an app that has the
need for this, but I think it is most likely a trivial map problem
(plus a little text manipulation).

> - C++ is a static language, let's leave these
> niceties to more dynamic languages.

I think C++ is quite dynamic (not in the java script way<g>) and
should exercise that power where appropriate :) It pains me to see
Java servers everywhere. C++ can and should have all the HTTP, SSL
and server stuff and be as easy to develop servlet-like things. One
does not need reflection, dynamic loading whatnot to play well in
that space. One does need standard (or at least defacto) libraries.
Without them, effort is fragmented and disjoint.

Which is why I joined boost. :)

> I have found myself weeks hunting a bug in the
> http code because i thought that it would be
> cool if i could access http properties using a
> map instead of proper accessor functions. I spent
> weeks hunting persistent connection bug. It was
> a simple typo inside a http property string that
> would have been caught immediately by the
> compiler if i were using functions or costants [1].

I agree that strings literals are not the right way to interact with
a library in general. Addresses seem near the border though because
they are user facing typically. So while I am not in favor of this
approach at this layer, I can see some merit to the general idea.

> - It mimics standard library usage.  You cannot
> open the input stream instead of a file by using
> the "input:" file name. (well under Unix you
> might actually do it, but i don't think this was
> the intention of the standard library authors
> and it is not portable any way).

Devils advocate: but in the same sense, the std lib supports mounts,
chroot, NFS, Samba file shares, etc.. Anything that can be mapped to
an OS file entity.

Things are more murky in network programming: credentials, auth,
keys, security in general as you suggest next.

> - It is extremely insecure. In a network library
> security must be paramount. If the transport type
> were encoded in the address, it would be much
> harder to validate externally received addresses.

Good point. Validation is one thing, but meeting expectations of the
software is another. In some cases, just any transport may not be
appropriate and hence should be validated. This can be done from the
string form, of course, but it presents a wider interface.

> A similar argument can be made for the port numbers.
> It is better to keep these things separated. The
> library user can create its own indexed factory 
> collection if it really needs to.

Not sure I see the connection to ports, but I agree with the rest
here. This is something that can easily be done at a higher level.
The trade-off is that libraries won't accept the same indirection in
the address, which leaves all the mapping up to the app (not just the
configuration of the mapping).

> Sorry for the long post, just tryin' to be usefull :-).

Don't be sorry. I am sure I've written longer posts and it was
helpful.

Best regards,
Don

                
__________________________________ 
Do you Yahoo!? 
Yahoo! Small Business - Try our new resources site!
http://smallbusiness.yahoo.com/resources/