crocs-muni / cryptostreams Goto Github PK

Direct output of the stream to stdout would be a nice feature to have; we could then pipe the stream directly into other programs (batteries, rtt, ...). Also, we wouldn't have to care about default data file name as we could set our own.

Idea of new generator (processing).

It has given bit-length l. It generates all l binary rotation of the input. Implement following "python pseudocode"...

next():
    while True:
        _data = _source.next()
        for _ in range(l):
            circular_rotation_left(_data, 1)
            yield _data

The code of estream candidates is often modified from the reference implementation. Some parts are commented out, as was fixed in #87.

Migrated from crocs-muni/eacirc#49

Implement those functions:

• SHA-1, SHA-2, MD5
• MARS, RC6, Serpent
• Maybe A5/1, A5/2, Kasumi, IDEA, Streebog

After implementing them, document them in the wiki.

@rozsa117

Hello, I would like to ask you to review my changes to crypto functions done in testsuite PR. The changes I done was mostly done so that I found reference implementation and compare it using diff with our implementation and look for differences.

The tests are passing with test vectors I found, so what I would like to get is just some feedback that I haven't done something stupid for example completely ruined our round reduction or something like that.

Whole PR: #11

Functions changed:

• SIMD - There was problem with unsigned long having 8 Bytes, but function relied on the fact that type used have 4 Bytes (commit: d29420d)
• Lesamnta - Problem was that variable i was fixed in our implementation while in reference implementation I found i was changed in loop, so I needed to change it to loop (38ebfc7)
• JH - Probably I found different implementation, but I was not able to find test vectors for that one we had (commit: f28c033)
• Hamsi - Again maybe different implementation: (82b0bfa)
• Decim - We had two implementations of Decim marked decim-v2 and decim-128. I found that both implementations are identical. So I needed to update decim-v2 to actually use decim-v2 implementation not decim 128. This can ruin results we obtained for decim until now. (ef7e00e)

Can rnd-plt-ctx-stream and xor-stream be combined together at the moment somehow?
I.e., get the stream of: x_i XOR f(x_i)

Thanks!

For testing, it is difficult to find relevant information, when you are overwhelmed by the [info] messages.

I added a test for generating ciphertext in byte-by-byte manner and found bugs in 6 functions.

[ FAILED ] 5 tests, listed below:
[ FAILED ] chacha.test_vectors
[ FAILED ] hermes.test_vectors
[ FAILED ] lex.test_vectors
[ FAILED ] salsa20.test_vectors
[ FAILED ] trivium.test_vectors
Fubuki fails even with segmentation fault.

I suggest to inspire from implementation in crypto-libraries. The easiest possible fix is to generate keystream into a buffer. However, it is still a task to inspect, if some other variable's state is not discarded by end of function. (Please, C++, give us yield keyword and generators).

• Generalize eStream as stream ciphers.
• Move Chacha to it
• Move RC4 to it

Implement Skipjack - old NSA block cipher.

This is "good first issue" - for new contributors preferably, as the implementation should be exemplary for CryptoStreams workflow.

Steps:

1. Find no licence/MIT/BSD licensed implementation, preferably reference version (this is more difficult here, as Skipjack was classified crypto). Do not use heavilly optimized implementations, it will be hard to use them correctly.
2. Implement it to CryptoStreams, in appropriate folder, with appropriate naming - inspire yourself by AES or others.
3. Write test vectors.
4. Round reduce the function.
5. Document it in https://github.com/crocs-muni/CryptoStreams/wiki/Block-ciphers-list

https://github.com/crocs-muni/eacirc-core/blob/8ee0117993c6df7789aa0e96b39421d1a4691806/cmake/build_stream.cmake#L2

Hi!
My tests failed for Decim due to "Invalid IV length" error. IV is 16B by default, can it be changed somehow?

Thanks.

{
  "notes": "Configuration generated by poly-verif-egen",
  "seed": "1fe40505e131963c",
  "function_name": "DECIM",
  "stream_type": 1,
  "tv-size": 16,
  "tv-count": 65536,
  "stdout": true,
  "stream": {
    "type": "estream",
    "type_code": 1,
    "generator": "pcg32",
    "algorithm": "DECIM",
    "round": 6,
    "block-size": 16,
    "gen_inp": {
      "type": "false-stream",
      "scode": "0"
    },
    "gen_key": {
      "type": "hw_counter",
      "hw": 4,
      "scode": "hw4",
      "increase_hw": false,
      "randomize_start": false
    },
    "scode_inp": "0",
    "scode_key": "hw4",
    "scode_init": "e",
    "scode": "tpestream-aDECIM-r6-tv16-in0-khw4-rie",
    "init-frequency": "every-vector",
    "key-size": 16,
    "plaintext-type": {
      "type": "false-stream",
      "scode": "0"
    },
    "key-type": {
      "type": "hw_counter",
      "hw": 4,
      "scode": "hw4",
      "increase_hw": false,
      "randomize_start": false
    },
    "iv-type": {
      "type": "false-stream",
      "scode": "0"
    }
  }
}

Hi,

I was wondering what is the maintainers' opinion about the following possible stream types (may be already on your roadmap, discussed previously or already doable by another streams & configuration):

Low Hamming weight

Generate stream with low hamming weight. E.g., 1bit vectors, 2bit, etc. Could useful for I/O analysis of the ciphers. May reveal some new weaknesses when analyzing long output sequences.
The extended version could start with random first vector, then XOR the low hamming weight stream.

Gradual SAC

The idea is to get a long output sequence. It could work similarly to sac-2d-all-positions but after depleting the all potential positions the first vector could be slightly mutated. E.g., XOR with low hamming weight vector - using previous method. Then apply sac-2d-all-positions again. In this way we can get input block very similar to each other and slightly evolving. This can be also useful for long output sequence analysis.

Rho

Simple, maybe useful. Function-dependent. Output of the function is taken as an input to the next invocation. Not sure if this is possible at the moment.

LUT

@sysox 's random number generator with parametrizable randomness. Could be also nice to study the behaviour of the analyzed function in long streams.

EDIT1:

Distribution sampling

Another variant of non-random input. Inputs could be sampled from some specific distributions which could be then looked for in the function outputs.

Thanks for feedback!

Move around following files:

• main.c, generator{.h, .cc} to some level up directory (src?)
• similarly for stream.h, streams{.h, .cc}, but split streams to at least classes of streams (classification)
• version.h.in is needed?

(This issue is nice to have, definitely not necessary).

Migrated from crocs-muni/eacirc#217

There are few interesting PRNGs (over 40) implemented in the TestU01 test suite. It can be also found on GitHub.

They can be used for testing or benchmarking of the baseline or directly for experiments. Moreover parameters can be tweaked so be obtain stronger or weaker PRNGs.

For example there are:

• LSFR generators
• Linear congruential generators, also with instantiations used in the wild (e.g., C stdlib, MS Excel, Maple, VisualBasic, VAX, Fortran, ...),
• Multiplicative congruential generators
• Add-with-carry generators
• Generalized feedback shift register (GFSR)
• Twisted GFSR
• Tempered TGFSR (Mersenne Twister family)
• Inversive generators
• Cubic congruential generators
• Xorshift
• And many others proposed in an academic literature (e.g., by Knuth, Marsaglia, L’Ecuyer, Brent, ...)

Some of they are little bit tricky to seed - one has to pick some parameters while the parameters domains differ and some are better than others (e.g., primes vs. smooth numbers). I personally used GFSR, TGFSR, TTGFSR generators which are easy to seed and it worked well.

We may also add TinyMersenneTwister with only 127 bit state - recommended for small embedded devices.

Following ciphers are missing test vectors:

'sparx', 'rectangle', 'robin_star', 'road_runner'

Once this is fixed, please remove the ciphers from testsuite/scan_tests.py

After each commit, version.h is updated and then git status shows whole submodule as modified, forcing me to check what is the modification in case of searching for bug... Few possible solutions, sorted from the easiest:

• Add version.h to .gitignore
• Removing version.h completely (who is currently using it?)
• Moving version.h to CryptoStreams

What do you prefer?

• Write documentation for pipes.
• Cleanup deleted streams description in https://github.com/crocs-muni/CryptoStreams/wiki/Streams
• Write exemplary configs using pipes.

There is double division by 8 using all-in-one hash method.

Write several typical config files (we can start with configs for https://github.com/crocs-muni/CryptoStreams/wiki/Recommended-scenarios) and let them run whole CryptoStreams execution for them.

Check, whether parsing the config passes and we can also check hash of the output file.

It is hard to find test vectors for round-reduced function, but what we can test even without test vectors is encrypt -> decrypt test.

@rozsa117 Take a look at my repo, branch ph4:
https://github.com/ph4r05/eacirc-streams/tree/ph4

Path: /streams/hash

I implemented these round reduced functions + test vectors as I needed them for our paper submitting in a few days. Some of them you may have already implemented while others may be missing:

• Gost
• MD5
• Ripemd160
• SHA1
• SHA256
• Tiger
• Whirlpool

Add implemented ciphers here:
https://github.com/crocs-muni/CryptoStreams/wiki/Block-ciphers-list

Question: should we add it to the same table (split by empty line), or to extra table? I prefer the first one.

Write (probably Python) script, that accepts config file and visualize it. It can be either generation graphviz, or using some library (preferred due to follow-up issue).

Future feature - allow editing the config file in the graphical interface.

It can look like this

Migrated from crocs-muni/eacirc#207

Get CAESAR project working.

Update the wiki accordingly: https://github.com/crocs-muni/eacirc-streams/wiki/CAESAR

In case that the generator receives malformed json config (e.g. wrong function name), the execution is stopped, error is issued but empty file with correct name is still generated.

• Install CMake
• Generate Visual Studio 2015 project files: cmake -G "Visual Studio 14"
• VS Open existing project crypto-streams.sln
• Start fixing errors :)

Migrated from crocs-muni/eacirc#229

Resolve in unified way the clang-format. We want 4 space indentation again. Run the new clang-format on all project files (not on implementation of crypto functions, it would be messy in VCS).

Relates to this commit: crocs-muni/eacirc@6fadf26

Using code from https://github.com/crocs-muni/CryptoStreams/tree/dynamic_analysis_CI would be huge addition in secure coding. However, there are multiple issues that blocks accepting the changes into master.

• Fix bugs found by sanitizers.
• Anotate code that causes issues, but it cannot be fixed (external code).
• Modify cmake to simplify the Travis configuration + allow developers to use the sanitizers on their own machines.

Future plans:
Cooperation with Red Hat on testing environment, where the undefined behavior can lead to incorrect execution.
Use fuzzers from Google?

Clang vs GCC runs differently.

Problem should be probably with PCG32 with seeding.