Re: [boost] [ann] Urdl - a library for downloading web content

Subject: Re: [boost] [ann] Urdl - a library for downloading web content
From: Jose (jmalv04_at_[hidden])
Date: 2009-06-18 05:36:43

Hi Chris,

Glad to see you're back!

On Wed, Jun 17, 2009 at 2:24 AM, Christopher Kohlhoff<chris_at_[hidden]> wrote:
> Hi all,
>
> I have just released a new Boost.Asio-based library, Urdl, which can be
> used for accessing and downloading web content.
>
>  <http://think-async.com/Urdl>
>  <http://sourceforge.net/projects/urdl>
>
> It currently has limited support for the protocols "http", "https" and
> "file". It provides an easy-to-use extension to standard C++ iostreams
> and an asynchronous interface for use with Boost.Asio.

My feedback:

* The #1 feature should be supporting http 1.1 well (also https). Many
libaries provide a http 1.0 implementation but fail short of
supporting the many options http provides (I know this is a huge
undertaking!) . At this point I don't see the value of supporting file
(or planning to support ftp). In this case, I don't understand the
value of future support for runtime polymorphism (can you explain how
user-supplied backends would work ?)

* It would be great to clarify why you based the design on a buffered
stream (below are my perceived pros/cons):
Pros
- easy to add support for new protocols with read_until
- easier header parsing
Cons
- increased implementation complexitiy with istreambuf
- Maybe small performance penalty

* In detail/http_read_stream.hpp

open_coro chains the whole sequence from connecting to
request/response. I think it would be better to split it in two
coroutines (one for opening the connection and another for sending a
request and getting a reply). This would make it easier to later
implement keep-alive.

* A newbie question:

In open_coro you don't use Stream as a template parameter. Is this
because Stream is a reference and you don't need to enforce any
concept ? (I am trying to learn from your great coding practices :) )

Thank you for sharing your library !

regards
jose