Re: A question for folks here

From: me_at_[hidden]
Date: 2024-07-10 22:04:51

Hi all,

My name is Kenneth Reitz, and I got involved with Boost when Vinnie Falco asked me to join the C++ Alliance to work on the new boost website. 

You might recognize my name from the Python community, where I’m known for making Requests, Pipenv, and other tools. 

It’s nice to meet everyone!

—
Kenneth Reitz

>   1. A question for folks here (David Sankel)
>   2. Re: A secure string library? (Andrey Semashev)
>   3. Re: A secure string library? (Andrey Semashev)
>   4. Re: A question for folks here (Richard Hodges)
>   5. Re: A secure string library? (Daniela Engert)
>   6. Re: A secure string library? (Andrey Semashev)
>   7. Re: A secure string library? (Christian Mazakas)
>   8. Re: A question for folks here (Matt Borland)
>   9. Re: A question for folks here (Christopher Kormanyos)
>  10. Re: A question for folks here (Louis Tatta)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Wed, 10 Jul 2024 10:47:50 -0400
> From: David Sankel <camior_at_[hidden]>
> To: Boost <boost_at_[hidden]>
> Subject: [boost] A question for folks here
> Message-ID:
> 	<CAFYdOO+m95fKjxnKY3SY_eHC-X=PLs12BteaONG+QpS6fTCxcQ_at_[hidden]>
> Content-Type: text/plain; charset="UTF-8"
> 
> For those who got involved in Boost within the last couple years, how did
> you hear about boost? What attracted you to it?
> 
> For those who have been around for a while, what keeps you here? Why do you
> stay engaged?
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Wed, 10 Jul 2024 18:26:20 +0300
> From: Andrey Semashev <andrey.semashev_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A secure string library?
> Message-ID: <6265cabd-c0b9-405e-98f4-8a47683e5472_at_[hidden]>
> Content-Type: text/plain; charset=UTF-8
> 
> On 7/10/24 14:59, Peter Dimov via Boost wrote:
>> Ruben Perez wrote:
>>> There is OPENSSL_Cleanse, which does a similar job. I found it surprisingly
>>> simple - I wonder if this is just enough or there are any corner cases where this
>>> doesn't work:
>>> https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85b5b6/crypto/mem_clr.c#L22
>> 
>> That's an interesting approach. I can't offhand think of a reason why it wouldn't
>> work.
> 
> The compiler might convert OPENSSL_cleanse to something like this:
> 
>  void OPENSSL_cleanse(void *ptr, size_t len)
>  {
>    memset_t func = memset_func;
>    if (func != memset)
>      func(ptr, 0, len);
>    else
>      memset(ptr, 0, len);
>  }
> 
> The purpose here is that, if memset_func is actually memset most of the
> time, it can further optimize the call to memset, including to
> completely remove it, depending on the call context. The
> (well-predictable) branch is typically cheaper than an indirect call. I
> think I've seen compilers do something along those lines as a result of
> call devirtualization, especially with IPO and LTCG.
> 
> I'm not saying that's what actually happens in OpenSSL, just that
> something like this is possible. I think, a dummy asm statement is more
> reliable and more efficient.
> 
>  void secure_cleanse(void *ptr, size_t len)
>  {
>    memset(ptr, 0, len); // a normal memset, optimizations are welcome
>    __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
>  }
> 
> You can even make that function inline and it'll work, and in an optimal
> way, too.
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Wed, 10 Jul 2024 18:37:00 +0300
> From: Andrey Semashev <andrey.semashev_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A secure string library?
> Message-ID: <57326885-13ab-4a73-ac5a-5aa6357e799c_at_[hidden]>
> Content-Type: text/plain; charset=UTF-8
> 
> On 7/10/24 18:26, Andrey Semashev wrote:
>> On 7/10/24 14:59, Peter Dimov via Boost wrote:
>>> Ruben Perez wrote:
>>>> There is OPENSSL_Cleanse, which does a similar job. I found it surprisingly
>>>> simple - I wonder if this is just enough or there are any corner cases where this
>>>> doesn't work:
>>>> https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85b5b6/crypto/mem_clr.c#L22
>>> 
>>> That's an interesting approach. I can't offhand think of a reason why it wouldn't
>>> work.
>> 
>> The compiler might convert OPENSSL_cleanse to something like this:
>> 
>>  void OPENSSL_cleanse(void *ptr, size_t len)
>>  {
>>    memset_t func = memset_func;
>>    if (func != memset)
>>      func(ptr, 0, len);
>>    else
>>      memset(ptr, 0, len);
>>  }
>> 
>> The purpose here is that, if memset_func is actually memset most of the
>> time, it can further optimize the call to memset, including to
>> completely remove it, depending on the call context. The
>> (well-predictable) branch is typically cheaper than an indirect call. I
>> think I've seen compilers do something along those lines as a result of
>> call devirtualization, especially with IPO and LTCG.
>> 
>> I'm not saying that's what actually happens in OpenSSL, just that
>> something like this is possible. I think, a dummy asm statement is more
>> reliable and more efficient.
>> 
>>  void secure_cleanse(void *ptr, size_t len)
>>  {
>>    memset(ptr, 0, len); // a normal memset, optimizations are welcome
>>    __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
>>  }
>> 
>> You can even make that function inline and it'll work, and in an optimal
>> way, too.
> 
> And for compilers that don't support __asm__, I think you could replace
> it with:
> 
>  std::atomic_signal_fence(std::memory_order::acq_rel);
> 
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Wed, 10 Jul 2024 18:01:03 +0200
> From: Richard Hodges <hodges.r_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A question for folks here
> Message-ID:
> 	<CALvx3hYSO6svxz_aEw_LoaraweTguqW+jJScTyfymZAveOrXZg_at_[hidden]>
> Content-Type: text/plain; charset="UTF-8"
> 
> On Wed, 10 Jul 2024 at 16:48, David Sankel via Boost <boost_at_[hidden]>
> wrote:
> 
>> For those who got involved in Boost within the last couple years, how did
>> you hear about boost? What attracted you to it?
>> 
> 
> I've been using boost as an essential force multiplier of C++ since version
> 1.35 when Asio appeared, courtesy of Chris Kohlhoff.
> 
> More recently in 2017, I needed an asynchronous websocket library for C++
> and lo-and-behold, boost had one, courtesy of Vinnie Falco.
> 
> 
>> 
>> For those who have been around for a while, what keeps you here? Why do you
>> stay engaged?
>> 
> 
> After finishing a contract in which I used Boost.Beast et.al. in production
> code, I took up an offer by the C++ Alliance to become a maintainer of it.
> A bit of a "Victor Kiam" moment, if you will.
> 
> While working for the C++ Alliance I was also involved in the design and
> first cut of Boost.Json and had some input into Boost.Url, another two very
> keenly needed libraries sponsored by the C++ Alliance.
> I am no longer working for the C++ Alliance, but maintain a warm
> relationship with the organisation.
> 
> I am currently using Boost.Cobalt in a financial trading software suite.
> 
> I stay engaged because:
> - I don't want to have to switch languages - C++ is perfectly fine for me.
> - Boost polyfills all the essential functionality that the children running
> WG21 continue to refuse to provide.
> - It's convenient to have one library that provides (almost) every tool I
> need.
> 
> My very great thanks to the original authors and contributors, and more
> recently to the C++ Alliance, which as far as I can see is the only major
> sponsor of Boost.
> Without this organisation's involvement, I believe the libraries would have
> been allowed to fall into disrepair a long time ago.
> 
> I don't think I have met more than one team in the financial industry which
> uses C++ but which does not use Boost.
> 
> On a separate note, I was deeply disappointed by your recent public
> outbursts, David.
> 
> Sincerely,
> 
> Richard Hodges
> A C++phile
> ex maintainer of Boost.Beast
> contributor to Boost.Json, Boost.Url
> Advisor to Boost.MySQL, Boost.Cobalt
> 
> 
>> 
>> _______________________________________________
>> Unsubscribe & other changes:
>> http://lists.boost.org/mailman/listinfo.cgi/boost
>> 
> 
> 
> ------------------------------
> 
> Message: 5
> Date: Wed, 10 Jul 2024 18:13:41 +0200
> From: Daniela Engert <dani_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A secure string library?
> Message-ID: <98140e71-ef1e-431a-a089-e24f56e390bd_at_[hidden]>
> Content-Type: text/plain; charset=UTF-8; format=flowed
> 
> Am 10.07.2024 um 17:37 schrieb Andrey Semashev via Boost:
>> On 7/10/24 18:26, Andrey Semashev wrote:
>>> On 7/10/24 14:59, Peter Dimov via Boost wrote:
>>>> Ruben Perez wrote:
>>>>> There is OPENSSL_Cleanse, which does a similar job. I found it surprisingly
>>>>> simple - I wonder if this is just enough or there are any corner cases where this
>>>>> doesn't work:
>>>>> https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85b5b6/crypto/mem_clr.c#L22
>>>> That's an interesting approach. I can't offhand think of a reason why it wouldn't
>>>> work.
>>> The compiler might convert OPENSSL_cleanse to something like this:
>>> 
>>>   void OPENSSL_cleanse(void *ptr, size_t len)
>>>   {
>>>     memset_t func = memset_func;
>>>     if (func != memset)
>>>       func(ptr, 0, len);
>>>     else
>>>       memset(ptr, 0, len);
>>>   }
>>> 
>>> The purpose here is that, if memset_func is actually memset most of the
>>> time, it can further optimize the call to memset, including to
>>> completely remove it, depending on the call context. The
>>> (well-predictable) branch is typically cheaper than an indirect call. I
>>> think I've seen compilers do something along those lines as a result of
>>> call devirtualization, especially with IPO and LTCG.
>>> 
>>> I'm not saying that's what actually happens in OpenSSL, just that
>>> something like this is possible. I think, a dummy asm statement is more
>>> reliable and more efficient.
>>> 
>>>   void secure_cleanse(void *ptr, size_t len)
>>>   {
>>>     memset(ptr, 0, len); // a normal memset, optimizations are welcome
>>>     __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
>>>   }
>>> 
>>> You can even make that function inline and it'll work, and in an optimal
>>> way, too.
>> And for compilers that don't support __asm__, I think you could replace
>> it with:
>> 
>>   std::atomic_signal_fence(std::memory_order::acq_rel);
>> 
> I think Peter is actually right in his assessment:
> 
> While 'memset_func' is TU-local, it is also 'volatile'. This implies 
> every look at it might reveal a different content. Therefore the 
> necessity for non-constness and dynamic initialization. We humans see 
> that the variable can't change its value other than in the 
> initialization. The compiler can't reason that because said properties 
> are the only ones known during the compilation of 'OPENSSL_cleanse()'. 
> It can't perform unbounded look-ahead until the end of the TU like we do.
> 
> Dani
> 
> -- 
> PGP/GPG: 2CCB 3ECB 0954 5CD3 B0DB 6AA0 BA03 56A1 2C4638C5
> 
> ------------------------------
> 
> Message: 6
> Date: Wed, 10 Jul 2024 19:24:48 +0300
> From: Andrey Semashev <andrey.semashev_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A secure string library?
> Message-ID: <905dd5f7-5d29-48ab-87aa-6aa0a23e60c6_at_[hidden]>
> Content-Type: text/plain; charset=UTF-8
> 
> On 7/10/24 19:13, Daniela Engert via Boost wrote:
>> Am 10.07.2024 um 17:37 schrieb Andrey Semashev via Boost:
>>> On 7/10/24 18:26, Andrey Semashev wrote:
>>>> On 7/10/24 14:59, Peter Dimov via Boost wrote:
>>>>> Ruben Perez wrote:
>>>>>> There is OPENSSL_Cleanse, which does a similar job. I found it
>>>>>> surprisingly
>>>>>> simple - I wonder if this is just enough or there are any corner
>>>>>> cases where this
>>>>>> doesn't work:
>>>>>> https://github.com/openssl/openssl/blob/b544047c99c4a7413f793afe82ab1c165f85b5b6/crypto/mem_clr.c#L22
>>>>> That's an interesting approach. I can't offhand think of a reason
>>>>> why it wouldn't
>>>>> work.
>>>> The compiler might convert OPENSSL_cleanse to something like this:
>>>> 
>>>> ?? void OPENSSL_cleanse(void *ptr, size_t len)
>>>> ?? {
>>>> ???? memset_t func = memset_func;
>>>> ???? if (func != memset)
>>>> ?????? func(ptr, 0, len);
>>>> ???? else
>>>> ?????? memset(ptr, 0, len);
>>>> ?? }
>>>> 
>>>> The purpose here is that, if memset_func is actually memset most of the
>>>> time, it can further optimize the call to memset, including to
>>>> completely remove it, depending on the call context. The
>>>> (well-predictable) branch is typically cheaper than an indirect call. I
>>>> think I've seen compilers do something along those lines as a result of
>>>> call devirtualization, especially with IPO and LTCG.
>>>> 
>>>> I'm not saying that's what actually happens in OpenSSL, just that
>>>> something like this is possible. I think, a dummy asm statement is more
>>>> reliable and more efficient.
>>>> 
>>>> ?? void secure_cleanse(void *ptr, size_t len)
>>>> ?? {
>>>> ???? memset(ptr, 0, len); // a normal memset, optimizations are welcome
>>>> ???? __asm__ __volatile__ ("" : : "r" (ptr), "r" (len) : "memory");
>>>> ?? }
>>>> 
>>>> You can even make that function inline and it'll work, and in an optimal
>>>> way, too.
>>> And for compilers that don't support __asm__, I think you could replace
>>> it with:
>>> 
>>> ?? std::atomic_signal_fence(std::memory_order::acq_rel);
>>> 
>> I think Peter is actually right in his assessment:
>> 
>> While 'memset_func' is TU-local, it is also 'volatile'. This implies
>> every look at it might reveal a different content. Therefore the
>> necessity for non-constness and dynamic initialization. We humans see
>> that the variable can't change its value other than in the
>> initialization. The compiler can't reason that because said properties
>> are the only ones known during the compilation of 'OPENSSL_cleanse()'.
>> It can't perform unbounded look-ahead until the end of the TU like we do.
> 
> With volatile, the compiler is not allowed to optimize away or reorder
> loads and stores of the variable. There's no restriction on what the
> compiler is allowed to do with the loaded value.
> 
> 
> 
> ------------------------------
> 
> Message: 7
> Date: Wed, 10 Jul 2024 10:10:43 -0700
> From: Christian Mazakas <christian.mazakas_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A secure string library?
> Message-ID:
> 	<CAHf7xWueD5=ekMC178pikzH2iuhAtyzC6v5UgPBSoj0NMy3y9Q_at_[hidden]>
> Content-Type: text/plain; charset="UTF-8"
> 
>> Andrey Semashev via Boost:
>> With volatile, the compiler is not allowed to optimize away or reorder
>> loads and stores of the variable. There's no restriction on what the
>> compiler is allowed to do with the loaded value.
> 
> It could insert a branch against `memset` but that seems really only like a
> theoretical
> optimization.
> 
> This is an example where the volatile preserves the secure_erase() call:
> https://godbolt.org/z/854jfarb9
> 
> Kind of neat trying to get the compiler to elide this stuff. You were
> right, Andrey, you really need
> LTO and LTCG and the like to get this kind of behavior. Super neat stuff.
> 
> Idk, seems like whatever OpenSSL is doing is Sound Enough in practice.
> 
> I'd be curious to know if it's actually sound for a compiler to make that
> kind of optimization or at least
> what are the limits on how far a compiler can actually go to ensure
> soundness. If you know those limits,
> you can probably make this pattern work reliably.
> 
> - Christian
> 
> 
> ------------------------------
> 
> Message: 8
> Date: Wed, 10 Jul 2024 17:57:57 +0000
> From: Matt Borland <matt_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A question for folks here
> Message-ID:
> 	<LQ-gmVceAUPyCRzsmP5gN1Z4NJiwrRtJ-yJY5Q6Ne-3SnJ_omC84hltKP9AbS3x-TVlz35Amln3jW0CBdG0nnzAioM53fxXobq0GO3zYb2E=@mattborland.com>
> 	
> Content-Type: text/plain; charset="utf-8"
> 
> 
>> For those who got involved in Boost within the last couple years, how did
>> you hear about boost?
> 
> I heard about it from a university professor as a freshman undergraduate student in 2011. Since C++11 was just coming about, and the speed of updating toolchains in academia is slow so we were stuck in the C++03 days on old RHEL systems.
> 
>> What attracted you to it?
> 
> I was studying aeronautical engineering so having pre-made libraries like Boost.Math, Boost.Multiprecision, and ODEint were useful to me. I could focus on the problem at hand rather than having to roll my own special functions, solvers, big numbers, constants, etc.
> 
>> For those who have been around for a while, what keeps you here? Why do you stay engaged?
> 
> I started to get involved with development during the COVID lockdowns because like many people I found myself with quite a bit of extra free time. What hooked me was learning from and working with extremely competent programmers and mathematicians like John Maddock, Chris Kormanyos, and Nick Thompson. I was hired by the C++ Alliance at the beginning of 2023 and they have sponsored my development since. This allows me to work on new (https://github.com/cppalliance/decimal), and existing libraries (charconv, math, multiprecision, random, etc) in a way that I was never able to on a volunteer basis with a full time job. The community is large, knowledgeable, and helpful so I continue to enjoy working on Boost. 
> 
> 
> I think it would be beneficial to the developer community for the Boost Foundation board members to also answer these questions. Frankly some even require introductions with their answers. There are members of the board like Peter, Glen, and Zach that are Titans of Boost, and those like Kristen who I had not heard of until her post about wanting to cut ties with the C++ Alliance. Having unknown members of the Boost Foundation runs counter to their mission statement whereby the board intends to achieve its mission by "fostering community engagement, [and] nurturing leaders..."[1].
> 
> Matt
> 
> [1] https://sites.google.com/boost.org/boost-foundation/home
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: publickey - matt_at_[hidden] - 0xC1382EAD.asc
> Type: application/pgp-keys
> Size: 652 bytes
> Desc: not available
> URL: <https://lists.boost.org/boost/attachments/20240710/0b9a9928/attachment.bin>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 249 bytes
> Desc: OpenPGP digital signature
> URL: <https://lists.boost.org/boost/attachments/20240710/0b9a9928/attachment.sig>
> 
> ------------------------------
> 
> Message: 9
> Date: Wed, 10 Jul 2024 18:14:21 +0000 (UTC)
> From: Christopher Kormanyos <e_float_at_[hidden]>
> To: Boost <boost_at_[hidden]>
> Subject: Re: [boost] A question for folks here
> Message-ID: <1213258294.1810576.1720635261713_at_[hidden]>
> Content-Type: text/plain; charset=UTF-8
> 
>> who have been around for a while, what
>> keeps you here? Why do you stay engaged?
> 
> A good reason to stay engaged is to maintain
> your own libraries. Then you might make
> some new ones or review some new ones.
> It grows over time.
> 
> But how do you get there?
> 
> In the beginning I found Boost to be a
> place of C++ competence. This was in those
> dark times when it seemed like C++ would
> just fail (curiously enough with no replacement).
> Sometime around the turn of the
> century, I started using Boost for stuff like
> noncopyable and iterator_facade, spirit,
> math, gil, boost.python and more.
> 
> I'll never forget the symbiosis between Boost
> and C++11, the standard that literally saved
> the language itself, granting it maturity and
> longevity. I think without Booost, C++ might
> be a lot worse off, maybe never even gaining
> traction.
> 
> For this reason alone and for many more
> (such as finding like minds to play with)
> I stay.
> 
> Let's party
> 
> Chris
> 
>    On Wednesday, July 10, 2024 at 04:48:37 PM GMT+2, David Sankel via Boost <boost_at_[hidden]> wrote:  
> 
> For those who got involved in Boost within the last couple years, how did
> you hear about boost? What attracted you to it?
> 
> For those who have been around for a while, what keeps you here? Why do you
> stay engaged?
> 
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
> 
> 
> ------------------------------
> 
> Message: 10
> Date: Wed, 10 Jul 2024 12:33:14 -0700
> From: Louis Tatta <louis_at_[hidden]>
> To: boost_at_[hidden]
> Subject: Re: [boost] A question for folks here
> Message-ID: <A8EEF4DC-F152-472B-B281-1B5CF3D6DCA1_at_[hidden]>
> Content-Type: text/plain; charset=utf-8
> 
> I got involved with Boost when Vinnie tapped me to run the non-technical side of the C++ Alliance. Being part of a project like Boost, which strives to make open source software available, is incredibly rewarding. The dedication of the community and the impact our work has on developers worldwide keep me motivated and passionate about what we do.
> 
>> On Jul 10, 2024, at 7:48?AM, David Sankel via Boost <boost_at_[hidden]> wrote:
>> 
>> ?For those who got involved in Boost within the last couple years, how did
>> you hear about boost? What attracted you to it?
>> 
>> For those who have been around for a while, what keeps you here? Why do you
>> stay engaged?
>> 
>> _______________________________________________
>> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
> 
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> Unsubscribe &amp; other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
> 
> 
> ------------------------------
> 
> End of Boost Digest, Vol 7278, Issue 3
> **************************************