randombit / botan

Cryptography Toolkit

Botan: Crypto and TLS for Modern C++

Botan (Japanese for peony flower) is a C++ cryptography library released under the permissive Simplified BSD license.

Botan's goal is to be the best option for cryptography in C++ by offering the tools necessary to implement a range of practical systems, such as TLS protocol, X.509 certificates, modern AEAD ciphers, PKCS#11 and TPM hardware support, password hashing, and post quantum crypto schemes. A Python binding is included, and several other language bindings are available. The library is accompanied by a featureful command line interface.

See the documentation for more information about included features.

Development is coordinated on GitHub and contributions are welcome. If you need help, please open an issue on GitHub.

If you think you have found a security issue, see the security page for contact information.

Releases

The latest release from the Botan3 release series is 3.4.0 (sig), released on 2024-04-08.

The latest release from the Botan2 release series is 2.19.4 (sig), released on 2024-02-20.

All releases are signed with a PGP key. See the release notes for what is new. Botan is also available through most distributions such as Fedora, Debian, Arch and Homebrew.

Find Enclosed

Transport Layer Security (TLS) Protocol

• TLS v1.2/v1.3, and DTLS v1.2
• Supported extensions include session tickets, SNI, ALPN, OCSP stapling, encrypt-then-mac CBC, and extended master secret.
• Supports authentication using certificates or preshared keys (PSK)
• Supports record encryption with ChaCha20Poly1305, AES/OCB, AES/GCM, AES/CCM, Camellia/GCM as well as legacy CBC ciphersuites.
• Key exchange using ECDH, FFDHE, or RSA

Public Key Infrastructure

• X.509v3 certificates and CRL creation and handling
• PKIX certificate path validation, including name constraints.
• OCSP request creation and response handling
• PKCS #10 certificate request generation and processing
• Access to Windows, macOS and Unix system certificate stores
• SQL database backed certificate store

Public Key Cryptography

• RSA signatures and encryption
• DH and ECDH key agreement
• Signature schemes ECDSA, DSA, Ed25519, Ed448, ECGDSA, ECKCDSA, SM2, GOST 34.10
• Post-quantum signature schemes Dilithium, SPHINCS+, and XMSS
• Post-quantum key agreement schemes McEliece, Kyber and FrodoKEM
• ElGamal encryption
• Padding schemes OAEP, PSS, PKCS #1 v1.5, X9.31

Ciphers, hashes, MACs, and checksums

• Authenticated cipher modes EAX, OCB, GCM, SIV, CCM, (X)ChaCha20Poly1305
• Cipher modes CTR, CBC, XTS, CFB, OFB
• Block ciphers AES, ARIA, Blowfish, Camellia, CAST-128, DES/3DES, IDEA, Lion, SEED, Serpent, SHACAL2, SM4, Threefish-512, Twofish
• Stream ciphers (X)ChaCha20, (X)Salsa20, SHAKE-128, RC4
• Hash functions SHA-1, SHA-2, SHA-3, MD5, RIPEMD-160, BLAKE2b, Skein-512, SM3, Streebog, Whirlpool
• Password hashing schemes PBKDF2, Argon2, Scrypt, bcrypt
• Authentication codes HMAC, CMAC, Poly1305, SipHash, GMAC, X9.19 DES-MAC
• Non-cryptographic checksums Adler32, CRC24, CRC32

Other Useful Things

• Full C++ PKCS #11 API wrapper
• Interfaces for TPM v1.2 device access
• Simple compression API wrapping zlib, bzip2, and lzma libraries
• RNG wrappers for system RNG and hardware RNGs
• HMAC_DRBG and entropy collection system for userspace RNGs
• SRP-6a password authenticated key exchange
• Key derivation functions including HKDF, KDF2, SP 800-108, SP 800-56A, SP 800-56C
• HOTP and TOTP algorithms
• Format preserving encryption scheme FE1
• Threshold secret sharing
• NIST key wrapping
• Boost.Asio compatible TLS client stream

botan's Issues

Missing dependency: module "pem" depends on "codec_filt"

We do an amalgamation that implicitly depends on the module "pem". When compiling, the object Base64_Decoder is missing, which is not in the module "base64" but in "codec_filt". The dependency "pem" depends on "codec_filt" is not set. Thus I had to manually add "codec_filt".

I am not very familiar with the module system, thus I don't feel confident to open a PR.

New FFI module uses funny flags&0 instead of flags&1

https://github.com/randombit/botan/blob/net.randombit.botan/src/lib/ffi/ffi.cpp#L396
in botan_cipher_init() so it seems that decryption isn't supported at all :)

and of course symbolic constant is preferred

More windows woes

This is something I have been seeing for a few weeks and just working around locally, but it doesn't seem to be going away.

4>botan-1.11.lib(botan_all.obj) : error LNK2005: "public: unsigned int __thiscall Botan::BigInt::bytes(void)const " (?bytes@BigInt@Botan@@QBEIXZ) already defined in cppsshd.lib(keys.obj)
4>botan-1.11.lib(botan_all.obj) : error LNK2005: "public: class std::vector<unsigned char,class std::allocator<unsigned char> > __thiscall Botan::PK_Signer::sign_message(unsigned char const * const,unsigned int,class Botan::RandomNumberGenerator &)" (?sign_message@PK_Signer@Botan@@QAE?AV?$vector@EV?$allocator@E@std@@@std@@QBEIAAVRandomNumberGenerator@2@@Z) already defined in cppsshd.lib(keys.obj)

It seems that this is related to using a static build on windows (again) and functions defined in the header that I actually use. It seems that the functions are defined in the library, and also pulled in to my exe from the .h file, and the two definitions collide at final link time. I work around it by just moving the definition of the functions in question to the cpp file.

warning: remote HEAD refers to nonexistent ref, unable to checkout.

It looks as though the default branch has been deleted:

$ git clone https://github.com/randombit/botan.git
Cloning into 'botan'...
remote: Reusing existing pack: 55631, done.
remote: Total 55631 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (55631/55631), 48.20 MiB | 6.55 MiB/s, done.
Resolving deltas: 100% (41697/41697), done.
Checking connectivity... done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.

VS2013::Botan app hangs forever, maybe related to AutoSeeded_RNG

I've tried the latest src from github, compiled fine on VS2013 with cpu=x64. But when trying to run a simple program that creates an RSA key, program hangs forever. Same thing happens trying to run botan.exe and botan_test.exe. Not sure if its an issue with AutoSeeded_RNG or what.

Code is pretty simple:

For 1.11.16

#include <fstream>
#include <iostream>
#include <string>
#include <vector>
#include <cstring>
#include <memory>

#include <botan/botan.h>
#include <botan/rsa.h>
#include <botan/x509cert.h>
#include <botan/x509self.h>
#include <botan/pem.h>
#include <botan/oids.h>

using namespace std;

int main()
{
    try
    {
        cout << "Hello Botan!" << endl;
    cout << "Generate key.." << endl;
    AutoSeeded_RNG rng;
        RSA_PrivateKey rsaPrivate(rng, 1024);

        // just testing some stuff
        vector<Botan::byte> publicBytes  = X509::BER_encode(rsaPrivate);
    vector<Botan::byte> privateBytes = PKCS8::BER_encode(rsaPrivate, rng, "passphrase");

    ofstream pub("public.pem");
    pub << X509::PEM_encode(rsaPrivate);

    getchar();

     }
     catch(std::exception& e)
     {
        cerr << "Error: " << e.what() << "\n";
     }

     return 0;
}

Too many dependencies on Linux

There are far too many dependencies on Linux that prevent this library from building on Windows. For example, in this file:

src\cmd\tls_client.cpp

There are socket library dependencies that require linux headers, whereas on Windows it should be using Winsock2.

This library has gone down the shitter apparently.

VS2013::Multi-threaded DEBUG DLL problem

I found that when trying to link as /MDd, application crashes with error:
"memory access violation occurred at address 0x00298000, while attempting to read inaccessible data"

I've built botan with --enable-debug option and see that the flag /MDd is added in the makefile, but for some reason applications crashes.

I can only run the app if linked with /MD. Thats weird and wrong, since I expect the library to be linked to with /MDd.

Any ideas?

rdrand causing amalgmation build problems

Using the following configure line:

./configure.py --disable-shared --disable-modules=mce,selftest,tls,ffi --enable-debug --via-amalgamation --disable-ssse3 --disable-sse2 --disable-avx2 --disable-aes-ni --disable-altivec

I get:

./libbotan-1.11.a(botan_all.o): In function `Intel_Rdrand':
/home/chrisd/sw/repo/botan/botan_all_internal.h:2454: undefined reference to `vtable for Botan::Intel_Rdrand'
collect2: error: ld returned 1 exit status
make: *** [botan] Error 1
make: *** Waiting for unfinished jobs....
./libbotan-1.11.a(botan_all.o): In function `Intel_Rdrand':
/home/chrisd/sw/repo/botan/botan_all_internal.h:2454: undefined reference to `vtable for Botan::Intel_Rdrand'
collect2: error: ld returned 1 exit status
make: *** [botan-test] Error 1

It appears that the botan_all_rdrand.cpp file is not compiled in, I have tried to play with the --single-amalgamation-file option, but it ends up with a bunch of errors like this:

In file included from botan_all.cpp:875:0:
/usr/lib/gcc/x86_64-linux-gnu/4.7/include/wmmintrin.h:34:3: error: #error "AES/PCLMUL instructions not enabled"
In file included from botan_all.cpp:1677:0:
/usr/lib/gcc/x86_64-linux-gnu/4.7/include/tmmintrin.h:31:3: error: #error "SSSE3 instruction set not enabled"
botan_all.cpp: In function ‘__m128i Botan::{anonymous}::aes_256_key_expansion(__m128i, __m128i)’:
botan_all.cpp:925:64: error: ‘_mm_aeskeygenassist_si128’ was not declared in this scope
botan_all.cpp: In member function ‘virtual void Botan::AES_128_NI::encrypt_n(const byte*, Botan::byte*, size_t) const’:
botan_all.cpp:1006:7: error: ‘_mm_aesenc_si128’ was not declared in this scope
botan_all.cpp:1007:7: error: ‘_mm_aesenc_si128’ was not declared in this scope

Just wanted to let you know, additionally I had to do add some extra #ifdefs to test_mceliece.cpp... but that is minor.

Segmentation fault in zero_mem()

I am getting a segmentation fault from zero_mem(). I have however not been successful locating the source of it.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff78d4bf0 in Botan::zero_mem(void*, unsigned long) () from /usr/local/lib/libbotan-1.11.so.14
(gdb) bt
#0  0x00007ffff78d4bf0 in Botan::zero_mem(void*, unsigned long) ()
   from /usr/local/lib/libbotan-1.11.so.14
#1  0x00007ffff7860630 in Botan::HMAC_RNG::~HMAC_RNG() () from /usr/local/lib/libbotan-1.11.so.14
#2  0x00000000004546f1 in operator() (this=<optimized out>, __ptr=<optimized out>)
    at /usr/include/c++/4.8/bits/unique_ptr.h:67
#3  ~unique_ptr (this=0x6a1cd8, __in_chrg=<optimized out>) at /usr/include/c++/4.8/bits/unique_ptr.h:184
#4  ~AutoSeeded_RNG (this=0x6a1cd0, __in_chrg=<optimized out>)
    at /usr/local/include/botan-1.11/botan/auto_rng.h:16
#5  Botan::AutoSeeded_RNG::~AutoSeeded_RNG (this=0x6a1cd0, __in_chrg=<optimized out>)
    at /usr/local/include/botan-1.11/botan/auto_rng.h:16

While searching for the error, I noticed some other errors in zero_mem.cpp. BOTAN_USE_VOLATILE_MEMSET should be changed to BOTAN_USE_VOLATILE_MEMSET_FOR_ZERO and p into ptr.

#elif defined(BOTAN_USE_VOLATILE_MEMSET_FOR_ZERO) && (BOTAN_USE_VOLATILE_MEMSET_FOR_ZERO == 1)
   static void* (*const volatile memset_ptr)(void*, int, size_t) = std::memset;
   (memset_ptr)(ptr, 0, n);
#else

D version of Botan

Just wanted to let you know that the D version of Botan v1.11.10 is finished now! I've had a lot of fun translating the architecture which I believe is the most modern and convenient of all, and the D community will benefit from such.

Pros:

• Faster compilation times: 7 seconds vs 70 seconds from a clean make
• No separate header files
• Speeds approx. equal: Powermod unit test at debug times shows this (release is 2x slower with DMD)
• Benefits from a community package manager similar to npm
• No pre-processor (all CTFE) and no python scripts (also CTFE but with the help of dub package manager)
• Backed by a garbage collector but uses custom allocators

Cons:

• Not GCC/LLVM ready (work in progress in GDC/LDC)
• Only available for x86 / x64
• Moving all new commits manually

I will maintain it myself as well, commit to commit, probably keep a 6 month delay but if anything urgent needs to be mirrored you can always ping me.

This was purely informational, no need to keep this open in the issue tracker.

MSVC x86 Unresolved External Symbol Error on __mm256_set_epi64x

However it compiles just fine on x64. My research indicates that this intrinsic is present in both archs (http://msdn.microsoft.com/en-us/library/hh977023.aspx), so I am wondering if a header is missing from the x86 amalgamation build or some other misconfiguration?

Error in CPUID with Visual C++ 2013

The CPUID X86_CPUID_SUBLEVEL macro function breaks in Visual C++ 2013 at line 192. Visual C++ requires the use of the __cpuidex() function to access sublevel information.

error C2660: '__cpuid' : function does not take 3 arguments

GCM and EAX modes do not support Threefish

I know that Threefish is a 512-bit block cipher in Botan, but is there any reason that the GCM and EAX modes can't be expanded to support it?

Make module dependencies clearer

Someone used --no-autoload and then was rightfully confused when the RNG didn't seed, because dev_random wasn't loaded. Add an option to configure that dumps the module dependency tree, and potentially merge some mutually dependent modules.

Visual C++ 2013 does not support noexcept

I have solved this problem for now with a sed call in my build script. But I thought it would be good to note it here. According to Microsoft the next release of VC++ will support noexcept.

sed -i 's/noexcept//g' ../TitaniumCore/Botan_WIN*

"nmake install" fails

Hi,

nmake install fails with following output:

nmake install

Microsoft (R) Program Maintenance Utility, Version 12.00.21005.1
Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.

        copy doc\manual\*.rst build\docs\manual
doc\manual\aead.rst
doc\manual\bigint.rst
doc\manual\building.rst
doc\manual\contents.rst
doc\manual\credentials_manager.rst
doc\manual\cryptobox.rst
doc\manual\ffi.rst
doc\manual\filters.rst
doc\manual\firststep.rst
doc\manual\fpe.rst
doc\manual\index.rst
doc\manual\kdf.rst
doc\manual\lowlevel.rst
doc\manual\ocsp.rst
doc\manual\passhash.rst
doc\manual\pbkdf.rst
doc\manual\pubkey.rst
doc\manual\python.rst
doc\manual\rng.rst
doc\manual\secmem.rst
doc\manual\srp.rst
doc\manual\tls.rst
doc\manual\versions.rst
doc\manual\x509.rst
       24 Datei(en) kopiert.
        src\scripts\install.py --destdir=c:\Botan --bindir=bin --libdir=lib --docdir=docs --includedir=include
  ERROR: Failure: [Errno 2] No such file or directory: 'botan-1.11.lib'
   INFO: Traceback (most recent call last):
  File "C:\devel\botan\src\scripts\install.py", line 204, in <module>
    sys.exit(main())
  File "C:\devel\botan\src\scripts\install.py", line 145, in main
    copy_file(static_lib, os.path.join(lib_dir, os.path.basename(static_lib)))
  File "C:\devel\botan\src\scripts\install.py", line 94, in copy_file
    shutil.copyfile(src, dst)
  File "C:\Python27\lib\shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'botan-1.11.lib'

NMAKE : fatal error U1077: "src\scripts\install.py": Rückgabe-Code "0x1"
Stop.

The issue here is that the library name is "botan.lib" on Windows and the install script assumes it to be "botan-1.11.lib".

I've already tried to adjust the libname in nmake.in but msvc is somehow ignoring everything after a dot. This results in an output of "botan-1.lib"

VS2013::Unresolved reference when using Win32_EntropySource version 1.10.9

Im on windows 7, VS2013. Everything seems to work fine, except when i try to use the Entropy source, Win32_EntropySource. I get an unresolved reference in the poll method. Not sure why, unless some particular library needs to be added to the linker.

---

Error 15 error LNK2001: unresolved external symbol "public: virtual void __cdecl Botan::Win32_EntropySource::poll(class Botan::Entropy_Accumulator &)" (?poll@Win32_EntropySource@Botan@@UEAAXAEAVEntropy_Accumulator@2@@z) C:\Fuentes\workspace\BotanTest\BotanTest\main.obj BotanTest

---

#include <botan/botan.h>
#include <botan/rsa.h>
#include <botan/x509cert.h>
#include <botan/x509self.h>
#include <botan/pem.h>
#include <botan/internal/es_win32.h>
#include <botan/oids.h>

using namespace std;
using namespace Botan;


int main()
{
    try
    {
    Botan::LibraryInitializer init;
        cout << "Hello Botan!" << endl;

    cout << "Generate key.." << endl;
    Botan::AutoSeeded_RNG rng;
        Botan::Win32_EntropySource entropySource; ==> unresolved external reference
        rng.add_entropy_source(&entropySource);

        Botan::RSA_PrivateKey rsaPrivate(rng, 1024);

        MemoryVector<Botan::byte> publicBytes  = Botan::X509::BER_encode(rsaPrivate);
    MemoryVector<Botan::byte> privateBytes = Botan::PKCS8::BER_encode(rsaPrivate, rng, "passphrase");

    ofstream pub("public.pem");
    pub << X509::PEM_encode(rsaPrivate);

    getchar();

     }
     catch(std::exception& e)
     {
        cerr << "Error: " << e.what() << "\n";
     }

     return 0;
}

Im compiling everything in x64, but the same errors occurs in 32bits . Here's the linking line:

---

/OUT:"S:\Fuentes\workspace\BotanTest\BotanTest\x64\Release\BotanTest.exe" /MANIFEST /LTCG /NXCOMPAT /PDB:"S:\Fuentes\workspace\BotanTest\BotanTest\x64\Release\BotanTest.pdb" /DYNAMICBASE "botan.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /DEBUG /MACHINE:X64 /OPT:REF /PGD:"S:\Fuentes\workspace\BotanTest\BotanTest\x64\Release\BotanTest.pgd" /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /ManifestFile:"x64\Release\BotanTest.exe.intermediate.manifest" /OPT:ICF /ERRORREPORT:PROMPT /NOLOGO /TLBID:1

---

Win32_Stats module does not include Windows.h in amalgamation build

This prevents the entropy source from becoming available.
Note that this will require the same fix as noted in issue #13

Unresolved reference when using Entropy

./botan-test crashes when statically linked

When I build the current master (1bf1490), botan-test crashes

$ ./botan-test
[1]    520 segmentation fault (core dumped)  ./botan-test

while the help output works

$ ./botan-test --help
Usage: ./botan-test [suite]
Suites: all (default), block, hash, bigint, rsa, ecdsa, ...

as well as running the tests individually

$ suites=(block modes aead ocb stream hash mac pbkdf kdf keywrap transform rngs passhash9 bcrypt cryptobox tss rfc6979 bigint rsa rw dsa nr dh dlies elgamal ecc_pointmul ecdsa gost_3410 curve25519 mceliece ecc_unit ecdsa_unit ecdh_unit pk_keygen cvc x509 nist_x509 tls); \
for suite in $suites; do echo "$suite"; ./botan-test "$suite"; done

Here some tests fail for different reasons but the app does not crash.

The same goes for Linux (g++), OS X (clang) and Windows (MSVC2013).

Can anyone reproduce this behaviour?

Cannot get_cipher() for some DES modes

Trying to use get_cipher() for ”DES/OFB/NoPadding” and ”DES/CFB/NoPadding” with Botan 1.11.14, but getting the following errors:

• Could not find any algorithm named "DES/OFB/NoPadding”
• Creating 'CFB_Encryption(DES,NoPadding)' failed: Could not read 'NoPadding' as decimal string

Algo_Registry doesn't really work when static linking.

I don't know if you already know this or not, or if you have a plan to make this work better, so I thought I would just let you know what is happening. Last night I decided to pull the latest from botan, and when I got through all the compile errors, it started running and I started seeing that a bunch of get___function were returning NULL. This was because the REGISTER__ macros were not being invoked for anything that I did not specifically refer to in my application. When static linking to a library the linker only pulls in files that are "required", so things that I do not specifically refer to do not get linked into my application, and as such the REGISTER_ macros do not run for those classes. To work around it I just declared an instance of anything that I was going to lookup. In the end it was getting a bit out of control so I decided to revert back to the commit I was pointing to previously, which still uses the old algorithm_factory.

Coverity Scan Integration

Would be nice to have Coverity Scan integration with the travis ci build as described here:

https://scan.coverity.com/travis_ci

Support setting alternate lib/include paths at configure time

Eg stuff like:

./configure.py --with-boost=/opt/boost_1_55

• look in /opt/boost_1_55/include and /opt/boost_1_55/lib resp

./configure.py --with-zlib=/usr --with-zlib-libdir=lib64

• where libdir is relative to the base dir

Or at least that's first thought. Should try to match autoconf conventions, assuming there are any.

CLMUL No Header Guard Found

Attempting to include the CLMUL module in an amalgamation build results in a No Header Guard Found Error.

No default constructor for OCSP::Response

To reproduce the following error you must attempt to build Botan with TLS enabled in an amalgamation build using Visual Studio 2013.

C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(294): error C2512: 'Botan::OCSP::Response' : no appropriate default constructor available (Botan_WIN32.cpp) [d:\Projects\Titanium\TitaniumCore\TitaniumCore.vcxproj]

As near as I can tell the offending line of code is the following function:
const X509_CRL* find_crls_for(const X509_Certificate& cert, const std::vector<Certificate_Store*>& certstores)

The line of code itself appears to be:
std::vectorstd::futureOCSP::Response ocsp_responses

Is a module missing from the amalgamation build? Or is this another VC++ "special"?

Note that of the three issues I reported today, this is the only true blocker. I've made the code compile in the other two cases with sed hacks.

The joys of python...

Your CI build is broken because of python 2 vs 3 differences, my build is also broken as a result of this:

http://stackoverflow.com/questions/10971033/backporting-python-3-openencoding-utf-8-to-python-2

OSX can not compile amalgamation using clang

The version I use is commit 7e6ac35 and
botan_all files are generated by "./configure.py --gen-amalgamation --cc=clang"

I got the following errors while compling with clang:
./xx/botan_all.h:13260:40: error: unknown type name 'McEliece_PrivateKey'
McEliece_Private_Operation(const McEliece_PrivateKey& mce_key);
^
../xx/botan_all.h:13270:3: error: unknown type name 'McEliece_PrivateKey'
McEliece_PrivateKey const& get_key() const { return m_priv_key; };
^
../xx/botan_all.h:13270:23: error: expected member name or ';' after declaration specifiers
McEliece_PrivateKey const& get_key() const { return m_priv_key; };

../xx/botan_all.h:13273:13: error: unknown type name 'McEliece_PrivateKey'
const McEliece_PrivateKey m_priv_key;
^
../xx/botan_all.h:13279:35: error: unknown type name 'McEliece_PublicKey'
McEliece_Public_Operation(const McEliece_PublicKey& public_key, u32bit code_length);
^
../xx/botan_all.h:13284:3: error: unknown type name 'McEliece_PublicKey'
McEliece_PublicKey const& get_key() const { return m_pub_key; };
^
../xx/botan_all.h:13284:22: error: expected member name or ';' after declaration specifiers
McEliece_PublicKey const& get_key() const { return m_pub_key; };

../xx/botan_all.h:13287:3: error: unknown type name 'McEliece_PublicKey'
  McEliece_PublicKey m_pub_key;
  ^
../xx/botan_all.h:20354:2: error: #endif without #if
#endif
 ^
../xx/botan_all.cpp:17170:30: error: unknown type name '__m256i'
inline void interleave_epi64(__m256i& X0, __m256i& X1)
                             ^
../xx/botan_all.cpp:17170:43: error: unknown type name '__m256i'
inline void interleave_epi64(__m256i& X0, __m256i& X1)
                                          ^
../xx/botan_all.cpp:17175:10: error: unknown type name '__m256i'
   const __m256i T0 = _mm256_unpacklo_epi64(X0, X1);
         ^
../xx/botan_all.cpp:17176:10: error: unknown type name '__m256i'
   const __m256i T1 = _mm256_unpackhi_epi64(X0, X1);
         ^
../xx/botan_all.cpp:17182:32: error: unknown type name '__m256i'
inline void deinterleave_epi64(__m256i& X0, __m256i& X1)
                               ^
../xx/botan_all.cpp:17182:45: error: unknown type name '__m256i'
inline void deinterleave_epi64(__m256i& X0, __m256i& X1)
                                            ^
../xx/botan_all.cpp:17184:10: error: unknown type name '__m256i'
   const __m256i T0 = _mm256_permute4x64_epi64(X0, _MM_SHUFFLE(3,1,2,0));
         ^
../xx/botan_all.cpp:17185:10: error: unknown type name '__m256i'
   const __m256i T1 = _mm256_permute4x64_epi64(X1, _MM_SHUFFLE(3,1,2,0));
         ^
../xx/botan_all.cpp:17198:10: error: unknown type name '__m256i'
   const __m256i ROTATE_1 = _mm256_set_epi64x(37,19,36,46);
         ^
../xx/botan_all.cpp:17198:29: error: use of undeclared identifier '_mm256_set_epi64x'
   const __m256i ROTATE_1 = _mm256_set_epi64x(37,19,36,46);
                            ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

It appears that the location of class McEliece_PrivateKey is wrong and AVX is accidentally enabled.

OSX build error in TLS client

There is a build error on OSX 10.10 clang, that I dont understand. Maybe someone has an idea:

src/cmd/tls_client.cpp:67:28: error: cast from 'char *' to 'struct in_addr *' increases required alignment from 1 to 4 [-Werror,-Wcast-align]
   socket_info.sin_addr = *(struct in_addr*)host_addr->h_addr_list[0];
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CryptoAPI_RNG Module Build Error

Including the cryptoapi_rng module in an amalgamation build causes numerous build errors after the point of inclusion. This is because windows.h has #define min and #define max directives that overrides std::min and std::max. Since Botan does not use the min or max defines, adding a #undef min and #undef max directly after the Windows header includes for that module should suffice to fix the problem.

Compiler error in donna128.h with msvc2013

        cl /MTd  /EHs /GR /Od /Zi /DDEBUG /W3 /wd4275 /wd4267 /Ibuild\include /nologo /c C:\Users\chrisd\software_devel\botan\src\lib\pubkey\curve25519\donna.cpp /Fobuild\obj\lib\chrisd_software_devel_botan_src_lib_pubkey_curve25519_donna.obj
donna.cpp
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(114) : error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(114) : error C2146: syntax error : missing ',' before identifier 'a'
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(116) : error C2065: 'a' : undeclared identifier
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(116) : error C2065: 'shift' : undeclared identifier
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(119) : error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(119) : error C2146: syntax error : missing ',' before identifier 'a'
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(122) : error C2065: 'a' : undeclared identifier
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(122) : error C2065: 's1' : undeclared identifier
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(122) : error C2065: 'b' : undeclared identifier
C:\Users\chrisd\software_devel\botan\build\include\botan/internal/donna128.h(122) : error C2065: 's2' : undeclared identifier
C:\Users\chrisd\software_devel\botan\src\lib\pubkey\curve25519\donna.cpp(325) : warning C4146: unary minus operator applied to unsigned type, result still unsigned
NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.EXE"' : return code '0x2'
Stop.

Two new functions were recently added, but they break the build for me, I just commented them out and not it builds fine. I suspect that is not the correct fix however.

Config script broken

Attempting to build the latest version results in:
ERROR: Unbound var 'botan_config' in template src/build-data/makefile/header.in

Windows/Python3 make install failed

 make install
cp doc/manual/*.rst build/docs/manual
src/scripts/install.py --destdir=D:/MinGW/msys/home/Symeon/botan_x64 --bindir=bin --libdir=lib --docdir=share/doc --includedir=include
  File "D:\MinGW\msys\home\Symeon\Botan-1.11.12\src\scripts\install.py", line 94
    exe_mode = 0777
                  ^
SyntaxError: invalid token
makefile:1360: recipe for target 'install' failed
make: *** [install] Error 1

I tried to comment line 94 and line 100, but still get failed.

Env: Windows (MinGW) + Python 3.4.2

Visual C++ 2013 balks at initializer-list in TLS - Botan::Channel

To reproduce the following error you must attempt to build Botan with TLS enabled in an amalgamation build using Visual Studio 2013.

d:\projects\titanium\titaniumcore\Botan_WIN32.h(5191): error C2664: 'std::map
Botan::u16bit,std::shared_ptr<Botan::TLS::Connection_Cipher_State,std::less<_
Kty>,std::allocator<std::pair<const _Kty,_Ty>>>::map(std::initializer_list<std:
:pair<const _Kty,_Ty>>,const std::less<_Kty> &,const std::allocator<std::pair<c
onst _Kty,_Ty>> &)' : cannot convert argument 1 from 'initializer-list' to 'con
st std::allocator<std::pair<const _Kty,Ty>> &' [d:\Projects\Titanium\TitaniumC
ore\TitaniumCore.vcxproj]
d:\projects\titanium\titaniumcore\Botan_WIN32.h(5193): error C2664: 'std::map
Botan::u16bit,std::shared_ptr<Botan::TLS::Connection_Cipher_State,std::less<
Kty>,std::allocator<std::pair<const _Kty,_Ty>>>::map(std::initializer_list<std:
:pair<const _Kty,_Ty>>,const std::less<_Kty> &,const std::allocator<std::pair<c
onst _Kty,_Ty>> &)' : cannot convert argument 1 from 'initializer-list' to 'con
st std::allocator<std::pair<const _Kty,_Ty>> &' [d:\Projects\Titanium\TitaniumC
ore\TitaniumCore.vcxproj]

I can fix this with sed by removing the initializer, but I do not know if that will change the behavior of the code itself.

The following are the sed commands that I run in my build script:
sed -i 's/std::map<u16bit, std::shared_ptr<Connection_Cipher_State>> m_write_cipher_states =/std::map<u16bit, std::shared_ptr<Connection_Cipher_State>> m_write_cipher_states;/g' ../TitaniumCore/Botan_WIN*
sed -i 's/std::map<u16bit, std::shared_ptr<Connection_Cipher_State>> m_read_cipher_states =/std::map<u16bit, std::shared_ptr<Connection_Cipher_State>> m_read_cipher_states;/g' ../TitaniumCore/Botan_WIN*
sed -i 's/{ { 0, nullptr } };//g' ../TitaniumCore/Botan_WIN*

Windows MinGW/GCC static build failed

python configure.py --prefix=$HOME/botan_x64 --disable-shared --os=windows --cpu=x86_64 --cc=gcc --disable-modules=zlib,boost,openssl,lzma,zlib,x509,rc2,rc5,rc6,tls,md2,md4,lion,threefish,http_util,mars,twofish,threefish_avx2

Then the last error log is

build/obj/app/asn1.obj:asn1.cpp:(.text+0x2bb): undefined reference to `__stack_chk_fail'
build/obj/app/asn1.obj:asn1.cpp:(.text.startup+0x3c9): undefined reference to `__stack_chk_fail'
d:/mingw/4.9.2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.9.2/../../../../x86_64-w64-mingw32/bin/ld.exe: build/obj/app/asn1.obj: bad reloc address 0x3c9 in section `.text.startup'

If I specify --os=mingw then almost all modules will be skipped because of dependency failure. So I have to use --os=windows which is also the default value if I don't specify that at all.

The reason that I disable some modules is because it'll use boost which is not installed on my machine despite that I didn't set --with-boost.

I have to note that even though I configured with --disable-shared, the Makefile still has a line like

LIB_LINK_CMD = $(CXX) -shared -fPIC -Wl,-soname,$(SONAME)

and I changed -shared to -static manually.

In addition to that, another obvious fault on Windows is to assume the suffix is .lib (MSVC). I'm afraid you need to distinguish GCC from MSVC and generate libbotan-1.11.a instead of botan-1.11.lib. This behaviour doesn't show up in 1.10 series. There also is another error regarding libs name. The line

LIB_LINKS_TO = -ladvapi32.lib -luser32.lib

is wrongly generated. The .lib suffix should be omitted in whichever case.

Visual Studio 2013 does not support __func__, but __FUNCTION__

The assertions use func but they should use FUNCTION for Visual Studio 2013 since VS2013 does not yet support func.

Visual Studio 2013 issues when iterator debugging is enabled

Hi, there are a few minor and a couple of major issues to do with Visual Studio 2013. My attached patch is against botan_all.cpp (sorry).

To summarize:

• noexcept is not supported
• Addressing past the end of a vector on purpose is "OK" but causes runtime errors with iterator debugging enabled
• Fancy-pants Initializer lists are wishful thinking

There are issues here to cover the other two and I believe they are resolved, however the addressing past the end of a vector causes the startup tests, as well as anything that uses OctetString to die needlessly when iterator debugging is enabled. The suggested patch is to use &.back() + 1 to indicate the intention instead of addressing past the end of the vector using operator[].

Library crashes when compiled with _GLIBCXX_DEBUG

In our project we use _GLIBCXX_DEBUG macro definition to enable additional safety checking in standard library.
But Botan and its unit tests compiled with this macro constantly crash.

Typical error (produced by botan-test):

/usr/include/c++/4.9/debug/vector:357:error: attempt to subscript container 
    with out-of-bounds index 140, but container only holds 140 elements.

Objects involved in the operation:
sequence "this" @ 0x0x7fff62ec6c90 {
  type = NSt7__debug6vectorIjN5Botan16secure_allocatorIjEEEE;
}

The problem is in widely-used among the code pattern:

algorithm(&vector[offset], length)

When offset >= vector.size(), operator[] will raise an error even if length == 0.
Since _GLIBCXX_DEBUG macro is intended to increase code stability and safety, I think that supporting it in cryptographic library like Botan would be beneficial.

PRNG_Unseeded exception on Computers without RDRAND.

Are there any other Entropy sources that I am missing or is this the same bug we were discussing earlier? RNG's work on my Haswell box, but it not on my Xeon E5620 test server.

PKCS #7 Support

Hi,

I tried to follow the discussion on attempting to digitally sign a pdf file, which commonly needs DER-encoded PKCS #7 object. I found two discussions 2008 and 2011.

Is there any leads on attempting such task with current botan version?

Thanks!

base64_encode does not work with zero-length input

base64_encode does allocate an output string using the following length calculation

(round_up<size_t>(input_length, 3) / 3) * 4

It turns out that round_up<size_t>(0, 3) is 3. From reading the comment, that is not the expected bahavior:

* Round up
* @param n an integer
* @param align_to the alignment boundary
* @return n rounded up to a multiple of align_to

If n = 0 and align-to = 3, than it must return 0 because 0 is a multiple of everything

If the function cannot be changed, base64_encode needs to treat the case input_length = 0 separately. I was able to fix the problem by calculating the output length before allocating the string:

size_t output_length = input_length != 0 ? (round_up<size_t>(input_length, 3) / 3) * 4 : 0;
std::string output(output_length, 0);

I could provide a pull request for the 2 possible fixes:

• Change round_up
• Add case distiction in base64_encode

AccessViolation Exception BlockCipher when deleting any Symmetric Algorithm/Mode on Windows

Stack Trace for EAX:

TitaniumNET.dll!std::default_deleteBotan::BlockCipher::operator()(Botan::BlockCipher * _Ptr) Line 1200 + 0x2e bytes C++
TitaniumNET.dll!std::unique_ptrBotan::BlockCipher,std::default_delete<Botan::BlockCipher >::~unique_ptrBotan::BlockCipher,std::default_delete<Botan::BlockCipher >() Line 1449 C++
TitaniumNET.dll!Botan::EAX_Mode::~EAX_Mode() + 0x73 bytes C++
TitaniumNET.dll!Botan::EAX_Decryption::~EAX_Decryption() + 0x28 bytes C++
TitaniumNET.dll!Botan::EAX_Decryption::`scalar deleting destructor'() + 0x2c bytes C++
TitaniumNET.dll!std::default_deleteBotan::EAX_Decryption::operator()(Botan::EAX_Decryption * _Ptr) Line 1200 + 0x31 bytes C++
TitaniumNET.dll!std::unique_ptrBotan::EAX_Decryption,std::default_delete<Botan::EAX_Decryption >::~unique_ptrBotan::EAX_Decryption,std::default_delete<Botan::EAX_Decryption >() Line 1449 C++

Stack Trace for GCM:

TitaniumNET.dll!std::default_deleteBotan::BlockCipher::operator()(Botan::BlockCipher * _Ptr) Line 1200 + 0x2e bytes C++
TitaniumNET.dll!std::unique_ptrBotan::BlockCipher,std::default_delete<Botan::BlockCipher >::~unique_ptrBotan::BlockCipher,std::default_delete<Botan::BlockCipher >() Line 1449 C++
TitaniumNET.dll!Botan::CTR_BE::~CTR_BE() + 0x51 bytes C++
TitaniumNET.dll!Botan::CTR_BE::scalar deleting destructor'() + 0x2c bytes C++ TitaniumNET.dll!std::default_delete<Botan::StreamCipher>::operator()(Botan::StreamCipher * _Ptr) Line 1200 + 0x30 bytes C++ TitaniumNET.dll!std::unique_ptr<Botan::StreamCipher,std::default_delete<Botan::StreamCipher> >::~unique_ptr<Botan::StreamCipher,std::default_delete<Botan::StreamCipher> >() Line 1449 C++ TitaniumNET.dll!Botan::GCM_Mode::~GCM_Mode() + 0x40 bytes C++ TitaniumNET.dll!Botan::GCM_Decryption::~GCM_Decryption() + 0x28 bytes C++ TitaniumNET.dll!Botan::GCM_Decryption::scalar deleting destructor'() + 0x2c bytes C++
TitaniumNET.dll!std::default_deleteBotan::GCM_Decryption::operator()(Botan::GCM_Decryption * _Ptr) Line 1200 + 0x31 bytes C++
TitaniumNET.dll!std::unique_ptrBotan::GCM_Decryption,std::default_delete<Botan::GCM_Decryption >::~unique_ptrBotan::GCM_Decryption,std::default_delete<Botan::GCM_Decryption >() Line 1449 C++

I don't have this problem with Hashes so it's probably not the .NET Interop code since those layers are functionally identical.

botan-config sometimes guesses wrong prefix

botan-config tries to find itself and guesses the prefix from that. This fails if it is reached via a symlink.

On Fedora systems after the UsrMove /bin is a symlink to /usr/bin. Now, if /bin is in the PATH, calling botan-config yields wrong results:

$ botan-config-1.10 --cflags --prefix
-I//include/botan-1.10
/

The issue came up in this Fedora ticket.

It can be fixed e.g. by inserting a readlink cmd:

--- /usr/bin/botan-config-1.10.orig 2014-10-06 07:35:30.295820646 +0200
+++ /usr/bin/botan-config-1.10  2014-10-06 07:36:29.483950371 +0200
@@ -1,7 +1,7 @@
 #!/bin/sh

 # For normal builds:
-guess_prefix=`dirname \`dirname $0\``
+guess_prefix=$(dirname $(dirname $(readlink -f $0)))
 includedir=include/botan-1.10
 libdir=lib64

However, this solution might still be non-portable, or completely wrong ;) - see for example the explanations in the Bash FAQ or the manifold discussions about corner cases on StackOverflow.

So in the end, a "hard-coded" (i.e. substituted during installation) value for the prefix might be the best solution.

DTLS-SRTP support

I would like to use Botan library within my C++ project which is a WebRTC server. WebRTC requires DTLS and the extension DTLS-SRTP defined in RFC 5764:

Such a extension is used to negotiate the cipher used for the SRTP session key and the key itself. The OpenSSL API is basically as follows:

// Once the DTLS connection is established:

uint8_t material[SRTP_MASTER_LENGTH * 2];
uint8_t localMasterKey[SRTP_MASTER_LENGTH];
uint8_t remoteMasterKey[SRTP_MASTER_LENGTH];
uint8_t *local_key, *local_salt, *remote_key, *remote_salt;

SSL_export_keying_material(ssl, material, SRTP_MASTER_LENGTH * 2,
"EXTRACTOR-dtls_srtp", 19, NULL, 0, 0);

if (dtls_setup == SETUP_ACTIVE) {
  local_key = material;
  remote_key = local_key + SRTP_MASTER_KEY_LENGTH;
  local_salt = remote_key + SRTP_MASTER_KEY_LENGTH;
  remote_salt = local_salt + SRTP_MASTER_SALT_LENGTH;
} else {
  remote_key = material;
  local_key = remote_key + SRTP_MASTER_KEY_LENGTH;
  remote_salt = local_key + SRTP_MASTER_KEY_LENGTH;
  local_salt = remote_salt + SRTP_MASTER_SALT_LENGTH;
}

// After this we can get the local and remote master keys for SRTP and
use them within libsrtp.

Given that Botan implements DTLS, may you please add support for this extension so Botan becomes another WebRTC capable DTLS library?

Unable to configure with amalgamation unless non-amalgmation performed first.

Ran into the #52 issues with static linking, and thus tried the amalgamation build.
On a fresh 1.11.15 tarball, configuration fails with:

  ERROR: [Errno 2] No such file or directory: 'build/build.h'

when the --via-amalgamation flag is used, unless the configuration script is first run without the --via-amalgamation flag present. Thereafter, it'll work.

Issue was present on both OS X and Linux platforms.

Grendel% ./configure.py --via-amalgamation --cc=clang --disable-shared --disable-aes-ni --disable-avx2
   INFO: Guessing target OS is darwin (use --os to set)
   INFO: Guessing target processor is a x86_64/x86_64 (use --cpu to set)
   INFO: Target is clang-darwin-x86_64-x86_64
   INFO: Disabling assembly code, cannot use in amalgamation
   INFO: Skipping, by request only - cvc
   INFO: Skipping, dependency failure - sessions_sqlite3
   INFO: Skipping, incompatible CPU - md4_x86_32 md5_x86_32 mp_x86_32 serpent_x86_32 sha1_x86_32 simd_altivec threefish_avx2
   INFO: Skipping, incompatible OS - asm_x86_32 asm_x86_64 beos_stats cryptoapi_rng dyn_load locking_allocator win32_stats
   INFO: Skipping, incompatible compiler - clmul mp_x86_32_msvc
   INFO: Skipping, loaded only if needed by dependency - mp_generic simd_scalar
   INFO: Skipping, requires external dependency - boost bzip2 lzma openssl sqlite3 zlib
   INFO: Skipping, uses assembly and --disable-asm set - sha1_x86_64
   INFO: Using MP module mp_x86_64
   INFO: Using SIMD module simd_sse2
   INFO: Loading modules adler32 aead aes aes_ni aes_ssse3 alloc aont asn1 auto_rng base base64 bcrypt benchmark bigint block blowfish camellia cascade cast cbc cbc_mac ccm cfb chacha chacha20poly1305 cmac codec_filt comb4p compression crc24 crc32 credentials cryptobox ctr curve25519 datastor des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecb ecc_key ecdh ecdsa egd elgamal eme_oaep eme_pkcs1 emsa1 emsa1_bsi emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gost_28147 gost_3410 gost_3411 has160 hash hash_id hex hkdf hmac hmac_drbg hmac_rng hres_timer http_util idea idea_sse2 if_algo kasumi kdf kdf1 kdf2 keccak keypair lion mac mars mce mceies md2 md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp mp_x86_64 noekeon noekeon_simd nr numbertheory ocb ofb oid_lookup openpgp par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pk_pad poly1305 prf_tls prf_x942 proc_walk pubkey rc2 rc4 rc5 rc6 rdrand rfc3394 rfc6979 rmd128 rmd160 rng rsa rw safer salsa20 seed serpent serpent_simd sessions_sql sha1 sha1_sse2 sha2_32 sha2_64 simd simd_sse2 siphash siv skein srp6 stream system_rng tea threefish tiger tls tss twofish unix_procs utils whirlpool x509 x919_mac x931_rng xtea xtea_simd xts
 NOTICE: Writing amalgamation header to botan_all.h
  ERROR: [Errno 2] No such file or directory: 'build/build.h'

Grendel% ./configure.py --cc=clang --disable-shared --disable-aes-ni --disable-avx2
   INFO: Guessing target OS is darwin (use --os to set)
   INFO: Guessing target processor is a x86_64/x86_64 (use --cpu to set)
   INFO: Target is clang-darwin-x86_64-x86_64
   INFO: Skipping, by request only - cvc
   INFO: Skipping, dependency failure - sessions_sqlite3 sha1_x86_64
   INFO: Skipping, incompatible CPU - md4_x86_32 md5_x86_32 mp_x86_32 serpent_x86_32 sha1_x86_32 simd_altivec threefish_avx2
   INFO: Skipping, incompatible OS - asm_x86_32 asm_x86_64 beos_stats cryptoapi_rng dyn_load locking_allocator win32_stats
   INFO: Skipping, incompatible compiler - clmul mp_x86_32_msvc
   INFO: Skipping, loaded only if needed by dependency - mp_generic simd_scalar
   INFO: Skipping, requires external dependency - boost bzip2 lzma openssl sqlite3 zlib
   INFO: Using MP module mp_x86_64
   INFO: Using SIMD module simd_sse2
   INFO: Loading modules adler32 aead aes aes_ni aes_ssse3 alloc aont asn1 auto_rng base base64 bcrypt benchmark bigint block blowfish camellia cascade cast cbc cbc_mac ccm cfb chacha chacha20poly1305 cmac codec_filt comb4p compression crc24 crc32 credentials cryptobox ctr curve25519 datastor des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecb ecc_key ecdh ecdsa egd elgamal eme_oaep eme_pkcs1 emsa1 emsa1_bsi emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gost_28147 gost_3410 gost_3411 has160 hash hash_id hex hkdf hmac hmac_drbg hmac_rng hres_timer http_util idea idea_sse2 if_algo kasumi kdf kdf1 kdf2 keccak keypair lion mac mars mce mceies md2 md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp mp_x86_64 noekeon noekeon_simd nr numbertheory ocb ofb oid_lookup openpgp par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pk_pad poly1305 prf_tls prf_x942 proc_walk pubkey rc2 rc4 rc5 rc6 rdrand rfc3394 rfc6979 rmd128 rmd160 rng rsa rw safer salsa20 seed serpent serpent_simd sessions_sql sha1 sha1_sse2 sha2_32 sha2_64 simd simd_sse2 siphash siv skein srp6 stream system_rng tea threefish tiger tls tss twofish unix_procs utils whirlpool x509 x919_mac x931_rng xtea xtea_simd xts
   INFO: Assuming CPU is little endian
   INFO: Assuming unaligned memory access works
   INFO: Using symlink to link files into build directory (use --link-method to change)
   INFO: Botan 1.11.15 (released dated 20150308) build setup is complete

Grendel% ./configure.py --via-amalgamation --cc=clang --disable-shared --disable-aes-ni --disable-avx2
   INFO: Guessing target OS is darwin (use --os to set)
   INFO: Guessing target processor is a x86_64/x86_64 (use --cpu to set)
   INFO: Target is clang-darwin-x86_64-x86_64
   INFO: Disabling assembly code, cannot use in amalgamation
   INFO: Skipping, by request only - cvc
   INFO: Skipping, dependency failure - sessions_sqlite3
   INFO: Skipping, incompatible CPU - md4_x86_32 md5_x86_32 mp_x86_32 serpent_x86_32 sha1_x86_32 simd_altivec threefish_avx2
   INFO: Skipping, incompatible OS - asm_x86_32 asm_x86_64 beos_stats cryptoapi_rng dyn_load locking_allocator win32_stats
   INFO: Skipping, incompatible compiler - clmul mp_x86_32_msvc
   INFO: Skipping, loaded only if needed by dependency - mp_generic simd_scalar
   INFO: Skipping, requires external dependency - boost bzip2 lzma openssl sqlite3 zlib
   INFO: Skipping, uses assembly and --disable-asm set - sha1_x86_64
   INFO: Using MP module mp_x86_64
   INFO: Using SIMD module simd_sse2
   INFO: Loading modules adler32 aead aes aes_ni aes_ssse3 alloc aont asn1 auto_rng base base64 bcrypt benchmark bigint block blowfish camellia cascade cast cbc cbc_mac ccm cfb chacha chacha20poly1305 cmac codec_filt comb4p compression crc24 crc32 credentials cryptobox ctr curve25519 datastor des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecb ecc_key ecdh ecdsa egd elgamal eme_oaep eme_pkcs1 emsa1 emsa1_bsi emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 entropy fd_unix ffi filters fpe_fe1 gcm gost_28147 gost_3410 gost_3411 has160 hash hash_id hex hkdf hmac hmac_drbg hmac_rng hres_timer http_util idea idea_sse2 if_algo kasumi kdf kdf1 kdf2 keccak keypair lion mac mars mce mceies md2 md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp mp_x86_64 noekeon noekeon_simd nr numbertheory ocb ofb oid_lookup openpgp par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pk_pad poly1305 prf_tls prf_x942 proc_walk pubkey rc2 rc4 rc5 rc6 rdrand rfc3394 rfc6979 rmd128 rmd160 rng rsa rw safer salsa20 seed serpent serpent_simd sessions_sql sha1 sha1_sse2 sha2_32 sha2_64 simd simd_sse2 siphash siv skein srp6 stream system_rng tea threefish tiger tls tss twofish unix_procs utils whirlpool x509 x919_mac x931_rng xtea xtea_simd xts
 NOTICE: Writing amalgamation header to botan_all.h
 NOTICE: Writing amalgamation source to botan_all.cpp
 NOTICE: Writing amalgamation source to botan_all_aesni.cpp
 NOTICE: Writing amalgamation source to botan_all_ssse3.cpp
 NOTICE: Writing amalgamation source to botan_all_rdrand.cpp
   INFO: Assuming CPU is little endian
   INFO: Assuming unaligned memory access works
   INFO: Using symlink to link files into build directory (use --link-method to change)
   INFO: Botan 1.11.15 (released dated 20150308) build setup is complete
Grendel%

[OS X] Amalgamation breaks memset_s

To use memset_s, one must first #define __STDC_WANT_LIB_EXT1__ 1 before including string.h. This is done in src/lib/utils/zero_mem.cpp, so its use of memset_s should work just fine if the file is compiled by itself.

However, there are other locations in Botan where string.h is included. When generating the amalgamation, these files might occur before the contents of zero_mem.cpp. For me, this is the case with src/lib/entropy/dev_random/dev_random.cpp, where string.h is included without first defining __STDC_WANT_LIB_EXT1__.

There are currently three places where <string.h> is included without first defining the flag:

• src/lib/entropy/dev_random/dev_random.cpp
• src/lib/pubkey/curve25519/donna.cpp
• src/lib/rng/system_rng/system_rng.cpp

Proposed solutions

Some possible solutions I could come up with:

1. Include the following code near the bottom of src/build-data/buildh.in:

   #if defined(BOTAN_TARGET_OS_HAS_MEMSET_S)
     #define __STDC_WANT_LIB_EXT1__ 1
   #endif
   

2. Copy the code from 1. to every place where string.h is included.

3. Define some header (for example botan_string.h) that contains the above code and #include <string.h>, in this order. Replace all other inclusions of string.h with botan_string.h.

Discussion of the solutions

1. This is not great for users of Botan because it defines the flag in botan_all.h, so that it might interact with unrelated client code.
2. This would only change botan_all.cpp, but introduce some repetion, which is bad for maintenance.
3. Probably the cleanest solution.

Any opinions on this issue? I'm willing to send in a pull request.

Using AES with CUDA

Considering the superior performance of CUDA algorithms and AES being a TLS bottleneck, are there any reasons why it wasn't implemented in botan?

source: http://www.dolbeau.name/dolbeau/crypto/

Including Serpent cipher causes amalgamation build to error

Including the Serpent cipher in an amalgamation build with the causes the following error to occur in configure.py.

ERROR: No header guard found in .\src\block\serpent\serpent_sbox.h

I suspect this is because the header guard names differ from the file name?

Can't generate amalgamation on OS X

Downloaded and tar xvzf-ed Botan-1.11.10. Inside the directory I tried to generate the amalgamation and got the following error:

$ ./configure.py --gen-amalgamation
  INFO: Guessing target OS is darwin (use --os to set)
  INFO: Guessing to use compiler gcc (use --cc to set)
  INFO: Guessing target processor is a x86_64/x86_64 (use --cpu to set)
  INFO: Target is gcc-darwin-x86_64-x86_64
  INFO: Disabling assembly code, cannot use in amalgamation
  INFO: Skipping, by request only - bzip2 cvc lzma openssl sqlite3 zlib
  INFO: Skipping, dependency failure - dyn_engine sessions_sqlite
  INFO: Skipping, incompatible CPU - md4_x86_32 md5_x86_32 mp_x86_32 serpent_x86_32     sha1_x86_32 simd_altivec
  INFO: Skipping, incompatible OS - asm_x86_32 asm_x86_64 beos_stats cryptoapi_rng dyn_load     locking_allocator win32_stats
  INFO: Skipping, incompatible compiler - mp_x86_32_msvc
  INFO: Skipping, loaded only if needed by dependency - asm_engine mp_generic simd_scalar
  INFO: Skipping, uses assembly and --disable-asm set - sha1_x86_64
  INFO: Using MP module mp_x86_64
  INFO: Using SIMD module simd_sse2
  INFO: Loading modules adler32 aead aead_filt aes aes_isa_eng aes_ni aes_ssse3 algo_base  algo_factory alloc aont asn1 auto_rng base64 bcrypt benchmark bigint block blowfish boost camellia cascade cast cbc cbc_mac ccm cfb chacha clmul cmac codec_filt comb4p compression core_engine crc24 crc32 credentials cryptobox cryptobox_psk ctr datastor des dev_random dh dl_algo dl_group dlies dsa eax ec_gfp ec_group ecb ecc_key ecdh ecdsa egd elgamal eme_oaep eme_pkcs1 emsa1 emsa1_bsi emsa_pkcs1 emsa_pssr emsa_raw emsa_x931 engine entropy fd_unix filters fpe_fe1 gcm gost_28147 gost_3410 gost_3411 has160 hash hash_id hex hkdf hmac hmac_drbg hmac_rng hres_timer http_util idea idea_sse2 if_algo kasumi kdf kdf1 kdf2 keccak keypair libstate lion mac mars mce md2 md4 md5 mdx_hash mgf1 misty1 mode_pad modes mp mp_x86_64 noekeon noekeon_simd nr numbertheory ocb ofb oid_lookup openpgp par_hash passhash9 pbes2 pbkdf pbkdf1 pbkdf2 pem pk_pad prf_ssl3 prf_tls prf_x942 proc_walk pubkey rc2 rc4 rc5 rc6 rdrand rfc3394 rfc6979 rmd128 rmd160 rng rsa rw safer salsa20 seed selftest serpent serpent_simd sha1 sha1_sse2 sha2_32 sha2_64 simd simd_engine simd_sse2 siv skein srp6 ssl3mac stream system_rng tea threefish threefish_avx2 tiger tls tss twofish unix_procs utils whirlpool x509 x919_mac x931_rng xtea xtea_simd xts
  INFO: Assuming CPU is little endian
  INFO: Assuming unaligned memory access works
  INFO: Using symlink to link files into build directory (use --link-method to change)
  INFO: Writing amalgamation to botan_all.h and botan_all.cpp
  ERROR: No header guard found in ./src/lib/pubkey/mce/gf2m_rootfind_dcmp.h

Configure.py fails by default when using MSVC

It seems that with a recent change Botan now includes the simd_sse2 module by default. Since configure.py currently thinks that MSVC cannot use SSE2 the build fails be default.

I see two possible solutions.
Solution 1: Remove the SSE2 deault dependency
Solution 2: MSVC supports intrinsics in the same manor as GCC (intrinsics map to their direct assembly equivalents). Enable configure.py to use the same modules as GCC/Clang.

MSVC supports the following intrinsics headers: intrin.h, ammintrin.h, emmintrin.h, immintrin.h, mmintrin.h, nmmintrin.h, pmmintrin.h, smmintrin.h, tmmintrin.h, wmmintrin.h, xmmintrin.h.

Note that this includes both AES-NI and AVX as well as everything prior to those ISA's.

In my opinion enabling instrinic support for MSVC would be the most valuable in the long-term.

If there are specific problems instrinics in MSVC please let me know, i'll ferret them out and propose solutions.