Re: [beast] help with http::dynamic_body and http::buffer_body

From: Richard Hodges (hodges.r_at_[hidden])
Date: 2021-04-26 14:29:58

Interesting.

Would you mind sharing your code (perhaps in a git repo) so that I can
reproduce, debug and advise?

You can email me direct on hodges.r_at_[hidden]

R

On Mon, 26 Apr 2021 at 16:26, Klebsch, Mario via Boost-users <
boost-users_at_[hidden]> wrote:

> Hello,
>
>
>
> After  using boost::asio for several years I thought, I might give beast a
> try. I need to write a http server processing a POST request with an upload
> of a big file. So I thought, I could start with
> libs/beast/example/http/server/async. I soon found the function
> handle_request() and that I have to add something like
>
>
>
> if (req.method() == http::verb::post && req.target()=="/xxx" )
>
>
>
> to catch the http POST transaction. But since I have to process the upload
> of a large file, I thought that using a http::string_body might not be
> the best choice. I found several other body types in the documentation.  I
> thought
>
> buffer_body ought to be fine, since the documentation says: ‘This allows
> for serialization of body data coming from external sources, and
> incremental parsing of message body content using a fixed size buffer.’.
>
>
>
> incremental processing of the input stream is what I need. But now, I have
> no clue, how to change the example to use a buffer_body.
>
>
>
> I changed http::request<http::string_body> req_ to http::request<http::buffer_body>
> req_. But now I must process the request message in chunks.
>
>
>
> How do I get my callback invoked for each chunk?
>
> Without additional changes, I get a read error: need buffer.
>
>
>
> But the documentation on buffer_body does not help me. Buffer_body does
> not seem to have any methods, that I can call.
>
> It does have a member type reader, that is documented as
> implementation-defined. So there is nothing I can do with this reader. ☹
>
> There is a member type value_type, that does have data, size and more
> mebers, but I have no idea, how I could et hold of a variable of this type
> containing a part of my message body.
>
>
>
> The error message might indicate, that I need to provide a fixed size
> buffer to the buffer_body, but I have no idea, how to do this.
>
>
>
> I tried to add the following code to do_read():
>
>
>
>              req_.body().data = my_buffer;
>
>              req_.body().size = sizeof(my_buffer);
>
>
>
> It compiles, so req_body() seems to be the way to get hold on a
> http::buffer_body::value_type, but simply initializing the two variables
> does not seem to be sufficient. I also tried to set req_.body().more to
> false or to true, but it didn’t change anything.
>
>
>
> I then thought, I might try a dynamic_body. This would store the entire
> upload in memory, but perhaps I get more familiar with the beat framework,
> when I try to use this body.
>
>
>
> And in deed, I managed to write some code, that shows all request headers
> and the request body on stdout. To get the body, I had to copy the buffer
> contents into a std::string:
>
>
>
>              std::string body{
> boost::asio::buffers_begin(req.body().data()),
>
>                                boost::asio::buffers_end(req.body().data())
> };
>
>
>
> Since large files have to be handled by my code, this additional copy of
> the data is to be avoided. So I tried to iterate the body using the
> iterators:
>
>
>
>              const auto begin =
> boost::asio::buffers_begin(req.body().data());
>
>              const auto end   =
> boost::asio::buffers_end(req.body().data());
>
>              for (auto it = begin; it != end; it++)
>
>                     putchar(*it);
>
>
>
> While it runs fine on the windows box, the loop crashes on linux in the 512
> th iteration in it++.
>
>
>
> Program received signal SIGSEGV, Segmentation fault.
>
> 0x0000555555592432 in
> boost::intrusive::list_node_traits<void*>::get_previous (n=@0x7fffffffc118:
> 0x1ff) at ../boost/boost/intrusive/detail/list_node.hpp:54
>
> 54         {  return n->prev_;  }
>
> (gdb) print n
>
> $1 = (const boost::intrusive::list_node_traits<void*>::node_ptr &)
> @0x7fffffffc118: 0x1ff
>
> (gdb)
>
>
>
> Here is the call stack:
>
> #0  0x0000555555592432 in
> boost::intrusive::list_node_traits<void*>::get_previous (n=@0x7fffffffc118:
> 0x1ff) at ../boost/boost/intrusive/detail/list_node.hpp:54
>
> #1
> boost::intrusive::list_iterator<boost::intrusive::bhtraits<boost::beast::basic_multi_buffer<std::allocator<char>
> >::element, boost::intrusive::list_node_traits<void*>,
> (boost::intrusive::link_mode_type)0, boost::intrusive::dft_tag, 1u>,
> true>::operator-- (this=0x7fffffffc118) at
> ../boost/boost/intrusive/detail/list_iterator.hpp:107
>
> #2
> std::__advance<boost::intrusive::list_iterator<boost::intrusive::bhtraits<boost::beast::basic_multi_buffer<std::allocator<char>
> >::element, boost::intrusive::list_node_traits<void*>,
> (boost::intrusive::link_mode_type)0, boost::intrusive::dft_tag, 1u>, true>,
> long> (__i=..., __n=0) at
> /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/g++-v10/bits/stl_iterator_base_funcs.h:169
>
> #3  0x000055555558daf0 in
> std::advance<boost::intrusive::list_iterator<boost::intrusive::bhtraits<boost::beast::basic_multi_buffer<std::allocator<char>
> >::element, boost::intrusive::list_node_traits<void*>,
> (boost::intrusive::link_mode_type)0, boost::intrusive::dft_tag, 1u>, true>,
> long> (__i=..., __n=-1) at
> /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/g++-v10/bits/stl_iterator_base_funcs.h:206
>
> #4  0x0000555555589254 in
> std::prev<boost::intrusive::list_iterator<boost::intrusive::bhtraits<boost::beast::basic_multi_buffer<std::allocator<char>
> >::element, boost::intrusive::list_node_traits<void*>,
> (boost::intrusive::link_mode_type)0, boost::intrusive::dft_tag, 1u>, true>
> > (__x=..., __n=1) at
> /usr/lib/gcc/x86_64-pc-linux-gnu/10.2.0/include/g++-v10/bits/stl_iterator_base_funcs.h:230
>
> #5  0x0000555555589608 in
> boost::beast::basic_multi_buffer<std::allocator<char>
> >::subrange<true>::const_iterator::operator* (this=0x7fffffffc4f8) at
> ../boost/boost/beast/core/impl/multi_buffer.hpp:384
>
> #6  0x0000555555589b4b in
> boost::asio::buffers_iterator<boost::beast::basic_multi_buffer<std::allocator<char>
> >::subrange<true>, char>::increment (this=0x7fffffffc4d0)
>
>     at ../boost/boost/asio/buffers_iterator.hpp:370
>
> #7  0x0000555555584894 in
> boost::asio::buffers_iterator<boost::beast::basic_multi_buffer<std::allocator<char>
> >::subrange<true>, char>::operator++ (this=0x7fffffffc4d0)
>
>     at ../boost/boost/asio/buffers_iterator.hpp:229
>
> #8  0x000055555557e7c0 in
> boost::asio::buffers_iterator<boost::beast::basic_multi_buffer<std::allocator<char>
> >::subrange<true>, char>::operator++ (this=0x7fffffffc4d0)
>
>     at ../boost/boost/asio/buffers_iterator.hpp:237
>
> #9  0x0000555555579ba8 in
> handle_request<boost::beast::http::basic_dynamic_body<boost::beast::basic_multi_buffer<std::allocator<char>
> > >, std::allocator<char>, session::send_lambda&> (doc_root=...,
>
>     req=..., send=...) at t.cpp:164
>
>
>
> I hope anybody can give my a hint on how to use this beast.
>
>
>
> Right now it feels that it might be easier to write the http server using
> plain boost::asio.
>
>
>
>
>
>
> *Klebsch Mario *Funktion | R&D
>
>
>
>
>
> Tel: [phone]
> Raum: Braunschweig, E20
>
>
>
> Diese E-Mail und die an sie angehängten Dateien sind ausschließlich für
> Personen oder Institutionen bestimmt, deren Namen oben aufgeführt sind. Sie
> können Informationen enthalten, die durch das Berufsgeheimnis geschützt
> sind und deren Weitergabe strengstens untersagt ist. Jede elektronische
> Nachricht kann geändert werden. ACTIA lehnt jede Verantwortung für diese
> Nachricht ab. Der Inhalt dieser Nachricht stellt keine Verpflichtung
> seitens unseres Unternehmens dar. Wenn Sie kein Empfänger sind, weisen wir
> Sie darauf hin, dass das Lesen, Vervielfältigen oder Verteilen strengstens
> untersagt ist. Wir bitten Sie daher, uns umgehend über diesen Brief zu
> informieren und diese Nachricht sowie eventuell beigefügte Unterlagen aus
> Ihrem Postfach zu löschen. Danke.
>
> This e-mail and the files attached to it are intended exclusively for
> persons or institutions whose names are listed above. They may contain
> information that is protected by professional secrecy and the disclosure of
> which is strictly prohibited. Any electronic message can be modified. ACTIA
> declines all responsibility for this message. The content of this message
> does not represent a commitment on the part of our company. If you are not
> a recipient, we would like to point out that reading, copying or
> distribution is strictly prohibited. We therefore ask you to inform us
> immediately about this letter and to delete this message and any
> accompanying documents from your mailbox. Thank you.
>
>
>
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> https://lists.boost.org/mailman/listinfo.cgi/boost-users
>