Re: [boost] compact_optional -- prompting interest

Subject: Re: [boost] compact_optional -- prompting interest
From: Gonzalo BG (gonzalobg88_at_[hidden])
Date: 2015-09-28 06:45:28

Hi Andrzej,

I've been using compact optional extensively for almost 2 months already
and found it very useful so I'd like to see an improved version of it in
Boost.

My use case was an application in which the main data structor is a
vector<optional<int>> (with ~O(10^10) elements), and the application
expends most of the time processing it. Compared to compact_optional,
Boost.Optional requires 2x more memory, and reduces the memory bandwidth by
2x. Using a vector<int> + boost::dynamic_bitset is infinitely better than
using Boost.Optional, but complicates the code significantly since there
are some values that the ints in that vector cannot never take. Arguably
compact_optional is a perfect fit for my use case.

Things that I liked using the library so far:

1. being able to customize the sentinel type via policies: sometimes I have
a signed integer that can only be positive and want to use -1 as a
sentinel, and some times I want to use std::numeric_limits<...>::max(), or
some other value.

Andrey Semashev wrote:
> 5. A suggestion: add evp_zero and evp_empty policies. The first uses
literal zero as the special value and can be used with numeric (integer and
fp) and pointer types. The second uses a default constructed value as the
magic value and a member empty() function to test for magic value. This
could be useful with containers, strings and ranges.

I have mixed feelings about these since someone using compact optional
should explicitly know what sentinel value they are using (e.g. for things
like hashing, serialization, ...). So I would rather like it to strive for
making this as explicit and less magical as possible, in particular for
integers and fp types. The empty policy seems very convenient though so
maybe the library should include some convenient policies somewhere (this
would also make for good examples of how to write policies).

2. Tags: this allowed me to strongly type my interfaces very easily, and
caught a lot of errors.

3. unsafe_raw_value together with a way to query the sentinel value, which
I needed to implement e.g. `std::hash` for compact_optional.

The library is WIP, but I also had some points of friction using it:

4. Operators: I want my compact_optional<int> to work "as an int" but fail
fast (assert, throw, terminate) on any operation involving the empty state,
so I had to implement my own operators for compact optional. Still, other
applications have different needs, and one might want the operators to
propagate the empty value, to throw an exception instead of calling assert,
or to no exist at all and result in a compiler error.

Since some operators must be implemented as member functions, there should
be a way to customize this using a policy.

Other points:

5. Being part of Boost.Optional:

I think compact_optional must be part of Boost.Optional. Stateful policies
would allow Boost.Optional to be implemented on top of compact_optional,
which would allow everybody to experiment with better storage for their own
types. For example sizeof(optional<T*>) can be made to be sizeof(T*), and
the same could be done for e.g. optional<variant<...>>. Right now
experimenting with this is hard because Boost.Optional interface doesn't
allow to customize any of this. But basing its implementation on top of
compact_optional could change that.

6. Name

I really don't mind about compact_optional since 99% of the time one is
going to use it through an alias anyways (specifying the type, policy, and
tag all the time is just too painful).

If Boost.Optional is reimplemented on tope of it, one could honor
std::string by calling this basic_optional instead.

Vladimir Batov wrote:
> To me having allocated -1 or std::max<int> (or whatever) to represent
no-value, i.e. "no-int" feels hackish. If, as a developer, I advertise
"int" to the user, IMO the user is entitled to expect the full "int" range
available. If, as a developer, I do not provide the full range, the proper
way (IMO) would be to introduce a new explicit type to represent the
supported subset (via enum or proper sensibly-named type -- time, speed,
temperature, etc.) In other words, if I cannot provide "int", I should not
be saying that I do.

Using the library the best thing that I can say is that I'm providing a
"compact_optional<int, some_policy, struct this_is_my_temperature_type_tag>
".

Vladimir Batov wrote:
> I do have the need to be real frugal when I store data to the disk. But
for that purpose I have to be far more economical and size-specific than
compact_optional.

I use compact_optional to store data on disk and since
sizeof(compact_optional<T>) == sizeof(T) I haven't ran into issues yet. How
can one be more economical than that?