Cryptography for Boost. Boost.Crypto3.

From: Mikhail Komarov (nemo_at_[hidden])
Date: 2020-09-02 00:50:10

Hello!

Some time ago I promised to show something about cryptography library architecture and implementation for Boost  (https://lists.boost.org/Archives/boost//2020/02/248205.php <https://lists.boost.org/Archives/boost//2020/02/248205.php>), so here I am. I’m actually glad my two innocent letters triggered such a conversation. That is what I wanted to see!

Please, read this letter very carefully, because some explaining needs to be done before you go to the code.

By the way. The code: https://github.com/NilFoundation/boost-crypto3 <https://github.com/NilFoundation/boost-crypto3>.

Warning. This letter contains mentioning of my non-commercial organizations I develop Boost.Crypto3 with. Don’t consider that as advertising. It is not supposed to be an advertisement.

So, what cryptography for Boost (and probably STL some time after) should look like?

1. It should definitely be simple to use and implemented with some generic programming spirit.

That is exactly what was done. Remember std::transform? 
Typical usage technique for all of the modules of a suite looks as follows.

Classic iterator pair with output iterator:

using namespace boost::crypto3;
std::string input = “Weird German 2 byte thing: Ã.“, out;
encode<codec::base64>(input.begin(), input.end(), std::inserter(out, out.end()));

Classic iterator pair with output value return:

using namespace boost::crypto3;
std::string input = “00112233445566778899aabbccddeeff”;
std::string key = “000102030405060708090a0b0c0d0e0f”;
std::string out = encrypt<block::rijndael<128, 128>>(input.begin(), input.end(), key.begin(), key.end());

C++20 Ranges (or Boost.Range) usage:

using namespace boost::crypto3;
std::vector<std::uint8_t> input = {0x27, 0x0f, 0xb1, 0x89, 0x82, 0x80, 0x0d, 0xa6, 0x40};
std::string out = encode<codec::base32>(input);
// or
encode<codec::base32>(input, out);

What about generic encoding/encryption/signing/hashing/%name_your_action% interface for classes?

You can definitely do that in case you define implicit conversion of your class to integral type. I’m thinking of introducing Inspectable concept which will require for the developer to have a function in his class which would return every class member which developer thinks needs to be encrypted/hashes/encoded/etc, converted to the particular algorithm’s format. (e.g. satisfying Integral concept for ciphers/codes/hashes).

2. It should contain only time-proven schemes implemented with well-known and proven techniques. 

Yes. Boost.Crypto3 (https://github.com/NilFoundation/boost-crypto3 <https://github.com/NilFoundation/boost-crypto3>) is derived from =nil; Crypto3 C++ Cryptography Suite (https://github.com/NilFoundation/crypto3 <https://github.com/NilFoundation/crypto3>), which my non-commercial foundation designed and developed for it’s projects.

A little bit of information on how exactly it was derived from the original suite you can get from the readme.md in both or repositories.

As far as you can see in the original repository, the suite contains some very novel schemes 
(e.g. Verifiable Delay Functions: https://github.com/nilfoundaton/crypto3-vdf.git <https://github.com/nilfoundaton/crypto3-vdf.git>) and it is going to contain more (threshold cryptography, zero-knowlege 
cryptography, cryptographic accumulators etc.). But Boost-ified version has to be a solid one. No novel
schemes. Only proven ones with proven implementation techniques. Which ones? This is what Boost community is up to decide.

So, the question for the community. What schemes should be included?

For now we only present small set of block ciphers, hashes and codecs listed in here https://crypto3.nil.foundation/projects/crypto3/db/d97/group__codec.html <https://crypto3.nil.foundation/projects/crypto3/db/d97/group__codec.html>, https://crypto3.nil.foundation/projects/crypto3/db/d97/group__hashes.html <https://crypto3.nil.foundation/projects/crypto3/db/d97/group__hashes.html>, https://crypto3.nil.foundation/projects/crypto3/d2/dba/group__block.html <https://crypto3.nil.foundation/projects/crypto3/d2/dba/group__block.html>. (Or you can find the list in the include directories of particular repositories: https://github.com/nilfoundation/crypto3-block.git <https://github.com/nilfoundation/crypto3-block.git>, https://github.com/nilfoundation/crypto3-codec <https://github.com/nilfoundation/crypto3-codec.git>.git <https://github.com/nilfoundation/crypto3-codec.git>, https://github.com/nilfoundation/crypto3-hash <https://github.com/nilfoundation/crypto3-hash.git>.git <https://github.com/nilfoundation/crypto3-hash.git>).

Peter (Dimov), I’m aware of your abandoned hash library implementation (https://github.com/pdimov/hash2 <https://github.com/pdimov/hash2>) based on Vinnie’s paper. I would really like to see it’s contents and concepts (probably) adapted to hash module of Boost.Crypto3. Would be a great synergy.

3. It should have a concept-based architecture.

Concept-based architecture supposes every reused type trait to be formalized, and standardized. This is what exactly has been done. In particular, concepts for contents of Boost.Crypto3 are listed in sufficient documentation chapters: https://crypto3.nil.foundation/projects/crypto3/df/d5d/block_ciphers_concepts.html <https://crypto3.nil.foundation/projects/crypto3/df/d5d/block_ciphers_concepts.html>, https://crypto3.nil.foundation/projects/crypto3/d7/da8/codec_concepts.html <https://crypto3.nil.foundation/projects/crypto3/d7/da8/codec_concepts.html>, https://crypto3.nil.foundation/projects/crypto3/d1/d1f/hashes_concepts.html <https://crypto3.nil.foundation/projects/crypto3/d1/d1f/hashes_concepts.html>.

Long story short. We standardize stateless policies containing all the params as constexpr ones used for functions (interface-functions like encrypt/decrypt) and actual algorithms.

We also figured out that most of cryptographic algorithms are about accumulation and only then processing. So we used Accumulator concept from Boost.Accumulator to accumulate (surprise surprise) data in a format, which is required by particular algorithm to be passed into.

You can get the idea of how I’ve merged the Accumulator concept in from architecture and implementation notes (for Block Ciphers) in here: https://crypto3.nil.foundation/projects/crypto3/db/dee/block_ciphers_impl.html <https://crypto3.nil.foundation/projects/crypto3/db/dee/block_ciphers_impl.html>. Pretty drafty as well, but better than nothing.

Yes, Hash concept should be made consent with currently existing Hash concept. This is in progress.

Vinnie (Falco), I remember, you’ve mentioned some more generic concepts like OneWayFunction (in here: https://lists.boost.org/Archives/boost//2020/02/248228.php <https://lists.boost.org/Archives/boost//2020/02/248228.php>). I agree, we should do that, but we simply had no time to put this fine idea to our architecture. We would really appreciate any help with that.

Cristopher (Kormanyos), I’m aware of your CRC catalog library (https://github.com/ckormanyos/crc-catalog <https://github.com/ckormanyos/crc-catalog>) I like it because of how in particular it is implemented. I would really like to merge it into our CRC implementation in Crypto3.Hash.

By the way. I wonder if we could sometime remove separate Boost.CRC library and use CRC from more generic Boost.Crypto3? Hypothetically. Just asking.

Yeah, we have checksums being implemented in Crypto3.Hash (and in Boost.Crypto3 as well), but they were simply not verified yet, so they are hanging in a PR (https://github.com/NilFoundation/crypto3-hash/pull/68 <https://github.com/NilFoundation/crypto3-hash/pull/68>).

4. Other modules? Roadmap?

Yes, there will be other modules being proposed in some time. Public Key cryptography in Boost? Yes. Based on enhanced for cryptographic purposes Boost.Multiprecision (https://github.com/NilFoundation/crypto3-multiprecision <https://github.com/NilFoundation/crypto3-multiprecision>). SSL based on Boost.Crypto3? Yes. Why not. We are already in progress.

Major updates are going to be performed with merging in the Boost-ified version of bunch of cross-dependent modules of the original suite. For example, public key cryptography would require enhancing Boost.Multiprecision (which is already in progress) and introducing finite fields library (https://github.com/NilFoundation/crypto3-algebra <https://github.com/NilFoundation/crypto3-algebra>).

By the way. The original =nil; Crypto3 suite is highly modular. Every cryptography field is emplaced in separate submodule. There are more than 30 of them now in total (counting closed ones)(yeah, it is huge). Would you like to keep this modularity (like introducing Boost.BlockCiphers, Boost.Hashes and Boost.Codec instead of Boost.Crypto3)? Or do we need to keep everything in header-only Boost.Crypto3? I would vote for the second option.

John (Maddock), do you understand now why I was attempting to get accepted  those strange PRs in Boost.Multiprecision for the last year?

Degski (Sorry, I’m not aware of your name), I remember you’ve mentioned some kind of compression library of yours. I would like to see the compression in the  suite. Do you think I could fork it and fit into our architecture?

5. Why do we think we can implement properly design and cryptography? Ask me :).

6. Conclusion.

In some time, this would definitely result in C++ Standardization Committee proposal. It is a long way, but I hope to bring it there.

What we would like to hear from Boost community is some guidance about what in particular from this field would be useful for Boost, proposals and suggestions about architecture and concepts. 

Let us standardize some cryptography together (starting with this).

Sincerely yours,

Mikhail Komarov
nemo_at_nil.foundation