- Boost - lists.boost.org

[hash2] Review
by Julien Blanc 13 Dec '24

13 Dec '24

his is my review of the proposed boost::hash2 library # What is your evaluation of the design? The base idea, separating the hash algorithm from the actual hashing of objects, is a brilliant one. The library consist on two parts: * a collection of hashing algorithm, some of them of crypto quality * a c++ objects hashing facility Given this design, i would not have expected from the library to provide a thorough collection of hash algorithm. But it's a good think to get them. I would have expected, however, to see some guidance about how to integrate an external hashing library. Actually it is unfortunate that this part is missing, because it seems a lot more harder than it should be. Several design decisions could be improved in my opinion: * the requirements on the HashAlgorithm concept are too strict, and have no rationale in the documentation * copy constructibility and copy assignment are unusual across current hash implementations, as it has been noted by other reviewers. I can understand a rationale for copy construct, copy assignment looks very weird to me. The only rationale i can think of is resetting the internal state to a defaulted state, which could be a specific operation. * creation with seed (either as uint64 or as an array) as well as default construction is required. It seems odd. The documentation, for example, strongly advice agains using siphash without a seed. I would prefer that it is a compile time error to use siphash without a seed / with an incorrectly sized seed. I've no problem adding a key/seed to algorithms that do not mandate them, though. But, since the library itself does not create the HashAlgorithm themselve (there is one notable exception to that that), it should not put additional requirements beyond what it uses. The HashAlgorithm concept could be: struct HashAlgorithm { using result_type = /*integral or array-like*/; static constexpr int block_size = /*...*/; // optional void update( void const* data, std::size_t n ); result_type result(); }; Interestingly enough, most of object hashing works if you provide it with such an algorithm (not copyable and not default constructible). I failed to see where default construction is used outside tests, for example. * the usage of void const* + size in interface is controversial, as several people would prefer a span<>. There are good reasons for not using span, and this is not something that should be an issue, as this should only impact HashAlgorithm writers. It is in my opinion an unfortunate design choice that this internal interface is leaked to the end user. The functionality is already provided by hash_append_range, which could be the advertised way of hashing bytes via that library (ie, when mixing hashing of object and hashing of bytes, eg when implementing custom hashing for a type). * constexpr use mandates usage of unsigned char const*. So it could be the primary supported interface over void const*, to avoid interface duplication. * the requirement on result extension is also uncommon. While i understand the rationale, it should be noted that it is likely to cause some language interoperability issues, as it is not necessarily specified by the algorithm. Implementations in other languages are unlikely to provide them, making the feature a niche one, unless the underlying algorithm specification provides it. * most of the functionality is provided by the hash_append function, and the hash_append_xxx helpers. This is a neat design. * The Flavor template argument allows to customize hashing of objects, to get platform-independants hashes. This is a good idea, but it comes with some problems: * the flavor is repeated over each hash_append call. I don't think it is a realistic use case that the flavor changes over time. If i were to integrate the library in our code base, the first thing i'll do is write a wrapper to get rid of that flavor argument at every call site. * the default flavor uses std::uint64_t for size. It seems odd. If i'm not asking for a specific flavor, i'm not asking the hash to be consistent accross different systems. So std::size_t should be used there instead. This looked to me like a pay for what you don't use situation, so i ran a quick near worst-case benchmark on our arm platform (hashing a million times a std::vector of the three integers 1-2-3), to find out that it makes a the code using size_t runs 15% faster. # What is your evaluation of the implementation? The implementation looks good. Code is clean, very readable. A minor remark in hash_append_unordered_range : m should be declared as std::size_t (or std::uint64_t). The current typename std::iterator_traits<It>::difference_type m = 0; is wrong. I'm not at ease with hash_append_unordered_range. The fact that it reduces the hash to 64 bits, simply making a sum of the element hashes, and then hashing that, does not look right. If the goal is simply to avoid unintended collisions, it's probably fine. If the goal is to preserve against an attacker mindfully trying to provoke collisions, it may do the trick,although intuitively xoring the whole results would be better at that job than summing truncated results. If it's to preserve cryptographic properties of the hash function, then it looks wrong. Another remark is about size serialization. The library relies on std::tuple_size to identify container-like structs, but not include the size in the hash (or is it because of is_contiguously_hashable?). This leads to unintuitive results, such as: std::array<int, 4> a{1,2,3,4}; std::span<int, 4> s(a); Hash h1,h2,h3,h4; hash_append(h1, {}, a); hash_append_range(h2, {}, begin(a), end(a)); assert(h1.result() == h2.result()); // TRUE hash_append(h3, {}, s); hash_append_range(h4, {}, begin(s), end(s)); assert(h3.result() == h4.result()); // FAILS I'm not sure there's a general solution to this issue, though. But since std::array models a container, not including its size in the digest may not be the wisest choice (the doc talks about plain arrays, not about std::array, so i don't know if it's a bug or a feature) # What is your evaluation of the documentation? The order of the documentation is odd. I would expect it to first explain me how i use the library with the provided algorithm to hash c++ data or bytes, but it first goes into deep details about how for example how result() can be called multiple times, or what requirements are set on custom hash algorithms. A description of the scope of the library would be welcomed. It both provides hashing for use by containers, and cryptographic quality hashing. These two seems at first unrelated, and the library lies in a not clearly defined in-between. It has been explain during the review, and the arguments given are convincing, and should find their place in the documentation. The motivation for machine architecture independant hashes is lacking. Usually, the way to achieve this is to use a domain specific canonical form, because endianness or word size is unlikely to be the only variable at stake here. But overall, the documentation has all the relevant information. A tutorial / quick start would be a nice addition. I dislike the "single page style" documentation, and hope the tooling can be improved in this regard. # What is your evaluation of the potential usefulness of the library? Do you already use it in industry? I have a need for an easy to use sha1, that i can easily compile for embedded targets. Having that in boost actually lowers the number of external dependancies, so yes this is definitely useful. I currently don't have a need for the object hashing facility, but would definitely use the library it if the needs arise. # Did you try to use the library? With which compiler(s)? Did you have any problems? I tried with several flavors of gcc, cross-compiling for bare-metal cortex-m0 targets or embedded arm32 linux, as well as standard debian x64. Everything worked. # How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I did not review the algorithm implementations. I spent around 15 hours on studying the library doc and internals. # Are you knowledgeable about the problem domain? No specific knowledge of the domain. # What is your final recommendation I would recommend to CONDITIONALLY ACCEPT the library into boost, the conditions being: * demonstrate the usage of an existing external hash library into the boost::hash2 framework * remove default construction and too short key construction for algorithms where this is actually harmful (such as siphash). I don't have an issue with providing constructors with keys for algorithms that do not mandate it. But boost code should encourage correct usage of the algorithms it propose, not allow bad usage in the name of genericity. The following improvements should imho be done, but i don't base my conditional acceptance on them. Actually some of them can be done on top of the proposed library: * advertise hash_append_range as the recommended way to add untyped data to the hash algorithm (or define an additional hash_append_bytes to fulfill this role). * adds a tutorial / introduction in the doc, move the whole hashing byte sequences item after the hashing c++ objects item (if previous point is adressed, part of it would have to be included in the hashing c++ objects item). * makes copy construction requirement on hash algorithm a requirement only if hash_append_unordered_range is used. Hashing unordeder ranges is an open problem, some creative other solutions may be found, and i see no reasons why they would involve duplicating the algorithm state. Actually, if the hash is default constructible, then it is not needed. You need either one of default or copy constructible, not both. * use std::size_t instead of std::uint64_t for the default_flavor, as this is faster on some platforms. * provide wrappers that avoid repeating the flavor over and over. It is much more convenient for the user to write once using MyHash = hash2::FlavoredHash<hash2::default_flavor> and then MyHash::append(X) than repeating over and over the parameter. Finally, i want to thank Peter for developing and proposing this library for boost, and Matt for managing the review. Regards, Julien

3 3

Ruben's Hash2 review
by Ruben Perez 13 Dec '24

13 Dec '24

Hi all, This is my review of the proposed Boost.Hash2. Thanks Peter and Chris for submitting the library, and Matt for managing the review. - What is your evaluation of the design? Very good. This is how hashing C++ objects should work, as it properly separates concerns between types and hash functions. I think usability could be improved if the template class hash shown as an example was included as part of the library and introduced much earlier in the documentation. In general, the HashAlgorithm concept makes sense. I'm not convinced about the HashAlgorithm::result() interface, for two reasons: * It makes integration of third-party hashes more difficult, as already noted by other reviewers. * The naming suggests an accessor, rather than a modifier. I'd consider splitting it into two functions, maybe finalize_block() and result(). But I can also see the benefits of the approach taken. I miss some higher-level features when working with untyped byte sequences. While this doesn't seem the main value proposition of the library, it's listed as a valid use case. See below for more info. These are nice-to-haves rather than requirements, though. I was initially surprised to find out that the library was header-only, as I didn't know that compile-time hashing was something worth it. I was initially worried about compile-times. I built one of the examples with compile-time profiling (-ftime-trace) and the library is pretty lightweight, so I have no problems with the header-only design. Have you considered somehow supporting hashing for objects describable via Boost.Pfr? I had it requested by some users in MySQL even when I had built-in Boost.Describe support, so it may get requested here, too. - What is your evaluation of the implementation? The parts where I've peaked in are good. Some comments would have helped me, but in general I've been able to follow. The test suite looks pretty comprehensive, too. Some fuzz tests for the cryptographic algorithms would be appreciated (I can help if required). As other reviewers have commented, the algorithm for hashing unordered ranges can use some improvements to make it more secure. I'm no security expert and can't judge how important this is, but I'm pretty sure the author will solve this promptly. The implementation of cryptographic functions is currently slower than solutions like OpenSSL [4], but the author has manifested the possibility of closing this gap. It would be nice having HashAlgorithm modelled as a C++20 concept, although the required macro machinery required to perform the checking only if concepts are supported (like BOOST_ASIO_COMPLETION_TOKEN_FOR does) may be too much effort for the benefit it brings. Regarding Boost integration, a build.jam should likely be added, so the library integrates in the new modular Boost infrastructure (although I've been able to use the library fine without it). This can be done after acceptance. As a matter of curiosity: which compilers do not support __builtin_is_constant_evaluated? (as they will always get the slow memcpy implementation, even at runtime). - What is your evaluation of the documentation? The documentation is good, and explains rationales pretty well, but the order could be improved: * Most users will likely want to use the existing algorithms with their unordered containers, so this should be explained first. * The next block of users will likely want to use the existing cryptographic hash algorithms to untyped byte sequences, so this should be next. * The rest can be left as is. I miss the motivation for the flavor construct in the docs. If I understood correctly, it's only relevant if you plan to somehow serialize your hashes. Maybe an example could help. The library is also suitable for use cases that combine the C++ object hashing and untyped byte sequence hashing parts, as explained by the author here: https://lists.boost.org/Archives/boost/2024/12/258765.php. Such cases are the rationale behind having cryptographic functions within the same umbrella as non-cryptographic ones. I'd suggest adding these to the docs. In general, I dislike single-page docs (which unfortunately seem pretty popular in Boost), as they're difficult to search. I also miss linking (e.g. hash_append should contain a link to its description in the reference). It would also be beneficial to explain the library's relationship with boost::hash / Boost.ContainerHash. In general, making it more approachable for non-expert users would be good. - What is your evaluation of the potential usefulness of the library? Do you already use it in industry? Robust, extensible C++ object hashing is very useful. My immediate use case is related to hashing untyped byte sequences, though. MySQL/MariaDB authentication involve a challenge-response protocol that involves hashing the user-supplied password with a server-supplied challenge, using either SHA1 (sigh) or SHA256 [1]. I'm currently using OpenSSL, as I use it for TLS, too. This library would allow me to make the OpenSSL dependency optional (which I've been requested). I'm also planning on writing a Postgres protocol library. It will likely need to support SCRAM-SHA256 authentication, similar to what I've described. This library could be handy for this. - Did you try to use the library? With which compiler(s)? Did you have any problems? I've re-written the password hashing functionality in Boost.MySQL to use the proposed library [2]. Everything has been very smooth. The test suite passes for all the compilers I support [3], which is really good. I'd have liked some utilities on top of HashAlgorithm to make hashing byte sequences safer and easier. These utilities would *not* imply any change in the HashAlgorithm interface: * Adding a contiguous range of bytes to an algorithm currently supports only a pointer + size pair. This is the right choice for HashAlgorithm, but having a helper function on top of it would be great, as proposed by other reviewers. This would make my code go from: // password is a string_view hash2::sha1_160 hasher1; hasher1.update(password.data(), password.size()); auto password_sha1 = hasher1.result(); To: hash2::sha1_160 hasher1; hash2::hash_add_bytes(hasher1, password); auto password_sha1 = hasher1.result(); * Adding a function that computes the hash of an entire byte sequence, as a range, could be useful. OpenSSL has this as SHA1() and SHA256(). This would make my code go from: hash2::sha1_160 hasher1; hasher1.update(password.data(), password.size()); auto password_sha1 = hasher1.result(); To: auto password_sha1 = hash2::compute_hash(hash2::sha1_160(), password); These are just nice to have and trivially implemented in user-land, if required. The code is already good as is today. - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? I have dedicated around 8 hours to study the documentation, build the library and examples, integrate it into Boost.MySQL, update my CIs to run my tests with the library, and write this review. - Are you knowledgeable about the problem domain? I am a mere user of hash algorithms, as I usually write networking code. I don't know about their internals. I haven't authored associative containers, either. - Affiliation disclosure I am currently affiliated with the C++ Alliance. - Final decision My recommendation is to CONDITIONALLY ACCEPT the library into Boost. My only acceptance condition is adding the hash template class developed in the examples to the library. The benefit of having the library in Boost is big and overcomes any concerns I might have. Many thanks to both Peter and Matt. Kind regards, Ruben. [1] https://dev.mysql.com/doc/dev/mysql-server/8.4.3/page_caching_sha2_authenti… [2] https://github.com/boostorg/mysql/pull/389 [3] https://github.com/boostorg/mysql/blob/c438f26731e36c2db6457705ec5dbb9f7657… [4] https://lists.boost.org/Archives/boost/2024/12/258687.php

4 4

Re: [boost] [hash2] My Review
by Vinnie Falco 12 Dec '24

12 Dec '24

On Sat, Dec 7, 2024 at 5:47 AM Matt Borland via Boost <boost(a)lists.boost.org> wrote: > The review of Hash2 by Peter Dimov and Christian Mazakas begins today > Saturday, December 7th through Sunday December 15th, 2024. > > Code: https://github.com/pdimov/hash2 > Docs: https://pdimov.github.io/hash2/ Disclaimer: I am affiliated with The C++ Alliance and I am also a co-author of N3980 (the paper referenced by the library). This is my review of Hash2. Spoiler alert, my suggestion is to: ACCEPT While most of my review is dedicated to discussing some concerns I have about some aspects of the library, it is clear that Boost as a collection is better with Hash2 than without it. Also, Peter is going to do what Peter is going to do so why fight it. The uncontroversial parts are stated first, and after lies my assessment of some questionable design choices. > - What is your evaluation of the implementation? > I looked over some of the implementation to better understand how certain things work. The tests pass on CI and the test code looks comprehensive. Peter is not in the habit of producing anything less than high-quality implementations and this library is no exception. I don't know which parts are from Peter and which are from Christian and the quality appears high throughout. > - What is your evaluation of the documentation? > I think the documentation could be better in places. The named requirements exposition is too informal for my taste and talks about things that are not strictly relevant, such as zero-padding. The library alludes to cryptographic usage yet glosses over some details which surfaced during mailing list discussion (more on that below). It would be nice if there was some exposition up front explaining which groups of users the library is for, and how each group uses the library. > - What is your evaluation of the potential usefulness > of the library? Do you already use it in industry? > Not only is the library useful, but the C++ Standard is poorer for not having a modern hash solution. Competing approaches were developed by Google and N3980, then promptly abandoned. > - Did you try to use the library? With which compiler(s)? Did > you have any problems? > I tried to build a Visual Studio Solution from the CMakeLists.txt after cloning hash2 into my superproject, with this command line: cmake -G "Visual Studio 17 2022" -A x64 -B bin64 -DCMAKE_TOOLCHAIN_FILE="C:/vcpkg/scripts/buildsystems/vcpkg.cmake" -DVCPKG_CHAINLOAD_TOOLCHAIN_FILE="C:/users/vinnie/src/toolchains/msvc.cmake" It started downloading things from the internet (FetchContent I guess) and then failed. I was not able to generate a solution. I asked Peter about this and he said it was not supported. It would be nice if it was. > - How much effort did you put into your evaluation? A glance? A quick reading? In-depth study? > I focused primarily on the ways in which the library differs from the original paper N3980. I looked at the relevant documentation, interfaces, and implementation, and engaged the author on the mailing list. - Are you knowledgeable about the problem domain? Fairly knowledgeable. - What is your evaluation of the design? This is where I have concerns. The audience of users for Hash2 can be divided into roughly these groups: 1. Authors of types who just want to make objects of their type hashable without having to learn about hash algorithms cryptography, and users who want higher quality results when working with unordered containers. Hash2 is basically perfect for these two groups of users (which often overlap) although the documentation could be improved to speak more directly to them. 2. Users who wish to calculate a cryptographic digest for objects of Hash2-enabled types, or for untyped binary data. Hash2 is also almost perfect for these use cases, and as before the documentation could be more explicit about these cases. 3. Users who wish to adapt existing code to meet the HashAlgorithm requirements from Hash2. This is where I believe the library has some design problems. An unfortunate consequence of being right almost all the time, is that it becomes more unlikely to recognize when you are wrong. I will assume that the reader is at least somewhat familiar with N3980 and some of the arguments I made about the HashAlgorithm named requirements on the mailing list. Hash2 attempts to improve the quality of life for users by mandating additional requirements for the HashAlgorithm concept which may be difficult or impossible to meet for some existing libraries. For example, it requires that every HashAlgorithm support: * Seeded construction from std::uint64_t * Keyed construction from binary data * Copy construction * Digest extension from repeated calls to result() * Interleaving of calls to update() and result() * ...Future stuff that the author alludes to These issues above are already discussed extensively on the mailing list and I can't be bothered to spend hours putting it all together into a beautiful post the way that some of the other reviewers have done (Andrey Semashev and Samuel Neves come to mind). The burdensome additional named requirements of Hash2's HashAlgorithm ensure that no casual users will be authoring them. Or worse, they will get them wrong and the results will not have the same guarantees as the underlying algorithms. Right now there are approximately 1,649,127 implementations of cryptographic and non-cryptographic hash algorithms. I think it might be more prudent for Hash2 to offer a system of traits for identifying which algorithms are capable of what, such as CopyConstruction, digest extension, and so on, and give the user tools which only work in safe ways given the traits of a hash algorithm. This would make it easier to write HashAlgorithms. And as a consequence we might get many more of them rather than fewer. Realistically I don't expect the library to change, and a proliferation of user-created HashAlgorithms is probably conditional on the Boost library collection being much more popular than it currently is. This is partly why I have decided to just accept the library. And I could be wrong, and Peter's unwavering insistence that users get HashAlgorithms which all offer the same quality of service might turn out to be better. Time will tell. Thanks

2 1

[hash2] Difference from paper: HashAlgorithm interface vs. function object
by Andrey Semashev 12 Dec '24

12 Dec '24

Hi, The HashAlgorithm interface defined in Boost.Hash2 is notably different from the one described in N3980 (https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3980.html) Specifically, the paper defines HashAlgorithm as a function object, whose operator() is equivalent to Boost.Hash2's HashAlgorithm::update method. The function object also provides a conversion operator to result_type, which is equivalent to Boost.Hash2's HashAlgorithm::result method. Is this discrepancy intentional and are there plans (on the paper authors' or the library authors' side) to eliminate it? Personally, I don't think a function object is the right interface for a hash algorithm. Update and finalize are two distinct operations provided by hash algorithms and both should be performed explicitly by the user. I don't think a conversion operator is the right name for finalize. Conversion operators are normally used when one type can meaningfully "impersonate" another, and this is not the case here. The function object is also misusing result_type, as its operator() doesn't return it. The paper discusses that the fact that a hash algorithm is a function object can be used to wrap it in std::function, which can be useful with pimpl idiom and for the purpose of type erasure. I don't think this will actually work, at least not as simple as the paper describes it, because std::function won't allow to extract the computed hash digest (i.e. you won't be able to call the type conversion operator on the wrapped function object). I think, the user will have to write some wrapper classes anyway to retain access to the wrapped hash algorithm. However, this is still an interesting use case. Does Boost.Hash2 intend to provide utilities to address it? For example, Boost.Hash2 could provide a hash-algorithm-agnostic interface for feeding an algorithm with input data, and a wrapper for an external algorithm object that would implement this interface. Something along these lines: class polymorphic_hash_algorithm { public: polymorphic_hash_algorithm( polymorphic_hash_algorithm const&) = delete; polymorphic_hash_algorithm& operator=( polymorphic_hash_algorithm const&) = delete; virtual void update(const void* data, size_t size) = 0; protected: polymorphic_hash_algorithm() = default; ~polymorphic_hash_algorithm() = default; }; template< typename Hash > class polymorphic_hash_algorithm_wrapper : public polymorphic_hash_algorithm { public: using hash_type = Hash; using result_type = typename hash_type::result_type; static constexpr size_t block_size = hash_type::block_size; private: Hash& h; public: explicit polymorphic_hash_algorithm_wrapper(hash_type& h) noexcept : h(h) {} void update(const void* data, size_t size) override { h.update(data, size); } result_type result() { return h.result(); } }; This way, polymorphic_hash_algorithm could be used to abstract away the hash algorithm type while still be compatible with hash_append and friends. As a side note, this would be another use case where the hash algorithm passed to hash_append is non-copyable. And while copyability could be added, most likely it wouldn't be free.

6 11

The Master branch is now open for changes
by Marshall Clow 12 Dec '24

12 Dec '24

Thanks to everyone for their patience. The next release cycle will begin in late February. — The release managers.

1 0

[hash2] Question about cryptographic guarantees of hash_append_unordered_range
by Ivan Matek 12 Dec '24

12 Dec '24

Hi everyone, Implementation uses uint64_t in which it sums into hashes of particular elements(each computed with "fresh" hasher so ordering does not matter), then it finally hashes that sum and range size into original passed hasher. I see no problem with logic this since it cleverly exploits the fact that uint64_t overflow is defined and that sum is commutative. But I wonder if this preserves the cryptographic quality of original hash. For example if we use sha2 it seems to me that finding collisions becomes easier since sum is "only" 64 bits, and length is "only" 64 bits(or 32 on some systems), and may be even less if attacker knows unordered range was not super long, e.g. number of items was less than 10000. Does somebody know more about this particular domain and could point me to some resource to learn more? I know it was in the original proposal, but it is possible original proposal was not focusing only on cryptographically secure algorithms. I was unable to google any guarantees, maybe because idk what keywords to use. :)

9 17

[release] Boost 1.87.0
by Marshall Clow 12 Dec '24

12 Dec '24

Release 1.87.0 of the Boost C++ Libraries is now available. These open-source libraries work well with the C++ Standard Library, and are usable across a broad spectrum of applications. The Boost license encourages both commercial and non-commercial use. This release contains two new libraries: * Boost.Parser: A framework for building parsers, from Zach Laine. The release also contains numerous enhancements and bug fixes for existing libraries. For details, including download links, see <http://www.boost.org/users/news/version_1.87.0> You can also download directly from: <https://boostorg.jfrog.io/artifactory/main/release/1.87.0/> To install this release on your system, see <http://www.boost.org/doc/libs/release/more/getting_started/index.html> Thanks to everyone who participated in this release. -- The Boost Release Team Marshall Clow, Glen Fernandes, Ion Gaztañaga

2 2

[Hash2] Current list of items brought up during review
by Peter Dimov 11 Dec '24

11 Dec '24

This is what I've scribbled so far as a list of to-do items: - add hash_append_range taking a range - add a "provider" argument (providing the hash_append API) to tag_invoke - example of hashing boost::json::value - maybe change argument of tag_invoke to take the value by pointer, rather than by reference - maybe expose has_tag_invoke to users so they can specialize - make byte sequence seed constructor take `void const*` as well - change get_integral_result to take a Hash, instead of h.result(), so that it can invoke result() more than once - a user-accessible way to test whether a hash algorithm conforms to the requirements - add a variadic hash_append overload - hash_append_unordered_range: use more than 64 bits for intermediate hashes (uint64_t[4]); improve somehow https://lists.boost.org/Archives/boost/2024/12/258643.php - acknowledge https://github.com/HowardHinnant/hash_append - in documentation, focus more on the std::hash case, rather than on the md5sum case - in FNV-1a byte seed constructor, include the seed size - change block_size to std::size_t

2 1

Re: [boost] [Hash2] Result extension
by Peter Dimov 11 Dec '24

11 Dec '24

Vinnie Falco wrote: > Sigh.. copy/paste typo. > > /** Return a seeded HashAlgorithm > */ > template< class HashAlgorithm > > HashAlgorithm make_seeded( > unsigned char const* seed, std::size_t n ) > { > if constexpr(std::is_constructible<HashAlgorithm, > unsigned char const*, std::size_t>) > return HashAlgorithm(seed, n); > else > { > HashAlgorithm h; > hash_append(h, seed, n); You can in principle do something like that (except fix it to compile) but (much) better practices would be - hash.update( seed, n ) - encode `n` as 64 bit little endian in 8 bytes - hash.update these bytes - hash.update( "\x80", 1 ) - hash.update( "\x00", 1 ) - hash.update with as many zeroes as needed to reach a multiple of HashAlgorithm::block_size That's, incidentally, exactly what `update(p, n); result();` does. (What an amazing coincidence.) Note that reaching a multiple of block_size is important, because it ensures that the secret key you passed as the seed is not left in the internal buffer of the hash algorithm, visible in memory dumps. The test https://github.com/pdimov/hash2/blob/develop/test/plaintext_leak.cpp tries to catch these mistakes. That's why this line https://github.com/pdimov/hash2/blob/7a25f8518692b657e9272884519519fbaca2ec… is needed, for example. > return h; > } > } > > Thanks >

4 6

Github Actions CI updates - e.g. Node 20
by Alexander Grund 10 Dec '24

10 Dec '24

Dear maintainers, I made a bunch of PRs to fix the Github Actions CI for almost 40 repositories. Opening a PR in GitHub is manual work while I was able to apply similar changes over all those repositories using a script which commits to a single branch (and hence PR) per repo. Hence the PRs contain a few related changes that in combination allow the CI to pass again. To avoid repeating this in each PR I wanted to shortly explain the changes here: 1. macos-12 runners are dropped by github and jobs won't start anymore. at least macos-13 needs to be used 2. Ubuntu 23.* doesn't work anymore as e.g. some APT repositories don't exist anymore. Ubuntu 24.04 can be used instead 3. clang 17 (and partly 16) don't work in C++23/2b mode with the stdlibc++ (12) installed on ubuntu 24.04, clang 18+ and/or the other C++ standards can still be used 4. Support for Node 16 was recently fully dropped by Github Actions so the @v3 actions must be upgraded to @v4, which require Node 20. The workaround using $ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION has no effect anymore 5. The node 20 version available by default conflicts with the glibc version in the Ubuntu 16/18 container images used for testing older compilers. I addressed all of them with last one being the major one: Installing a compatible version inside a custom image to be used instead doesn't work because GHA mounts the node 20 installation from the host os. The solution I implemented downloads an unofficial node 20 built against the correct/older glibc and creates 2 mount points such that we can put that node onto the host runner and mount that instead of what github gives us using the idea from https://github.com/dixyes/ghactionsplay/blob/bf889ee2fed8dcd72cc8e32bfd4789… To avoid overloading the unofficial-builds.nodejs.org server we use a copy hosted at boost.io Download and extraction requires curl and xz-utils to be installed which I added where required. Additionally I noticed additional failures due to use of C++03 where the dependencies require C++11, or of C++0x using GCC < 4.9 where e.g. C++11 `alignof`, required by the Boost.SmartPtr dependency, is not available. For (only) those I removed the C++03 and/or GCC 4.7/4.8 jobs (which cannot succeed). I think combining those changes into a single PR avoids a lot of additional manual work by me and allows to verify that the changes are sufficient to make CI pass again. Keeping e.g. C++03 builds, that cannot succeed due to dependencies, makes verifying the success of the Node 20 fix harder/less clear If anyone prefers to separate those changes into multiple PRs please mention that in the relevant PR for me to keep track of, or cherry-pick the individual changes. However it might be sufficient to check the status of the CI for the individual commits to compare the state (success/failure) for the jobs fixed or removed by the next commit. I realize maintaining the CI is a PITA and hope this helps. Regards, Alex

6 8