Re: C++ Deflate (zlib-like) library

From: Domen Vrankar (domen.vrankar_at_[hidden])
Date: 2020-03-11 06:32:51

On Wed, Mar 11, 2020 at 7:12 AM Domen Vrankar <domen.vrankar_at_[hidden]>
wrote:

> aOn Tue, Mar 10, 2020 at 5:49 PM Vinnie Falco <vinnie.falco_at_[hidden]>
> wrote:
>
>> On Tue, Mar 10, 2020 at 1:31 AM Domen Vrankar <domen.vrankar_at_[hidden]>
>> wrote:
>> > You compress/decompress in chunks so all you need is a const std::span
>> for input and std::span for output and call decompress on a for loop.
>>
>> You need to know:
>>
>>     1. how much input was consumed
>>     2. how much output was produced
>>
>
> Now I understand what you mean. You got me confused as your compress
> function wasn't receiving any state (like z_stream in zlib) so I assumed
> that you meant to compress the entire buffer in one go.
>
>
>> > ...input iterator pair/range for input/output...
>> > ...std::istream_iterator...
>> > ...boost.spirit x3 parser...
>> > ...read->decompress->parse/process
>> > ...
>> > Whit iterators/ranges described above you wouldn't
>> > need to return std::size_t so return can be used for
>> > error codes instead.
>>
>> It isn't clear what these things mean without seeing a function
>> signature or class declaration.
>>
>
> Since you need to hold state I was talking about something like this:
>
> struct Reader // should be implemented for reading from vector, file
> stream... some provided as convinience, others user defined so an extension
> point
> {
>     std::span<std::byte> read(); // remains stable untill next call to
> read, at that point it is auto consumed
>     bool eof();
> };
>
> template<typename InReader, typename WriteBuffer>
> class Deflate_range
> {
> public:
>     Deflate_range(InReader& in, WriteBuffer& write);
>

Sry... I meant to write std::span<std::byte> instead of WriteBuffer
template but completely forgot while composing the mail... It's a temporary
buffer used by deflate process to be used for iterator buffering. Just
replace it while reading ;)

>
>     class Deflate_iterator{...};
>
>     Deflate_iterator begin(); // can be called only once
>     Deflate_iterator end();
>
> private:
>     InReader& in_;
>     WriteBuffer& write_;
>     z_stream zs_; // holds the compression/encryption/... state
> };
>
> Polymorphic_deflate_range compress(std::istream_iterator<std::byte> begin,
> std::istream_iterator<std::byte> end); // it packs iterators into a Reader
> (an overload of compress coud also take in streams directly - it's just a
> convinience function)
>
> std::istringstream str{"1234567890"};
> auto range = compress(std::istream_iterator<std::byte>{str},
> std::istream_iterator<std::byte>{});
>
> bool ok = boost::spirit::x3::phrase_parse(range.begin(), range.end(), ...);
>
> I haven't used std::ranges yet so perhaps such a range should be
> implemented differently.
>
> Then I got confused by your compress API so size_t my return comment was
> just me being confused and reading/replying to mails on my phone...
>
> Hope this clarifies things a bit.
>
> I'm guessing that I could use such thing in my toy game engine - not
> certain but perhaps I won't need the whole libarchive functionality
> there... To early to say.
>
> Regards,
> Domen
>
>
>>
>> Thanks
>>
>