crytic / medusa

Parallelized, coverage-guided, mutational Solidity smart contract fuzzing, powered by go-ethereum

medusa

medusa is a cross-platform go-ethereum-based smart contract fuzzer inspired by Echidna. It provides parallelized fuzz testing of smart contracts through CLI, or its Go API that allows custom user-extended testing methodology.

Disclaimer: Please note that medusa is an experimental smart contract fuzzer. Currently, it should not be adopted into production systems. We intend for medusa to reach the same capabilities and maturity that Echidna has. Until then, be careful using medusa as your primary smart contract fuzz testing solution. Additionally, please be aware that the Go-level testing API is still under development and is subject to breaking changes.

Features

medusa provides support for:

• ✔️Parallel fuzzing and testing methodologies across multiple workers (threads)
• ✔️Assertion and property testing: built-in support for writing basic Solidity property tests and assertion tests
• ✔️Mutational value generation: fed by compilation and runtime values.
• ✔️Coverage collecting: Coverage increasing call sequences are stored in the corpus
• ✔️Coverage guided fuzzing: Coverage increasing call sequences from the corpus are mutated to further guide the fuzzing campaign
• ✔️Extensible low-level testing API through events and hooks provided throughout the fuzzer, workers, and test chains.
• ❌ Extensible high-level testing API allowing for the addition of per-contract or global post call/event property tests with minimal effort.

Documentation

To learn more about how to install and use medusa, please refer to our documentation.

For a better viewing experience, we recommend you install mdbook and then running the following steps from medusa's source directory:

cd docs
mdbook serve

Contributing

For information about how to contribute to this project, check out the CONTRIBUTING guidelines.

License

medusa is licensed and distributed under the AGPLv3.

medusa's Issues

Support block.timestamp and block.number fuzzing

Currently our block timestamp is a dummy value that is increased with every added block by a fixed amount of time. Meanwhile, block numbers are only incremented for each new block sent to test some input.

We'll want to:

• Add the ability for the step size (advancing) of block.number and block.timestamp between a certain min/max set in our config.
• Ensure that a new block's timestamp and number do not precede the previous block (the chain must still make sense).

Notes:

• TestChain.CreateNewBlock() should be updated to use the current timestamp (we won't use this in the fuzzer, but it should be done).
• The FuzzerWorker's use of TestChain.CreateNewBlockWithParams() should consider a minimum/maximum jump value to randomly select the next call sequence element (block's) properties.

Allow fuzzer to continue on failed bytecode matching

Currently, the fuzzer will throw an error if bytecode matching fails. After #19 is merged, we should emit a warning when bytecode matching failed, but allow the fuzzer to continue.

Integrate slither's echidna printer

As @0xalpharush and @ggrieco-tob mentioned, we'll want to integrate slither's echidna printer to mine more interesting values and more importantly, contextualize them, for our ValueSet.

We currently mine some values from AST ourselves, but it's not perfect. We handle hex, decimal numbers, but possibly not numbers like 1 ETH or 1e8: https://github.com/trailofbits/medusa/blob/8036697794481b7bf9fa78c922ec7fa6a8a3005c/fuzzing/valuegeneration/value_set_from_ast.go#L13-L41

Reference for the echidna printer:
https://github.com/crytic/slither/blob/b96beeaa5742d5d7a862e964bc72b3cab67623f4/slither/printers/guidance/echidna.py#L350

Include function name, abi specification in corpus

Echidna's corpus format includes this info and allows for tools like Maat and Slither to enrich the corpus with "interesting" paths etc, but Medusa's format makes it difficult to tell what function was called because it doesn't include the name and ABI spec of the targeted function

Add tracking to TestChain for SELFDESTRUCT operations

Currently TestChain tracks new contract deployments and reverts. We should:

• Track SELFDESTRUCT operations (destroyed contracts)
• Add those variables to CallMessageResults, similar to how we track deployed contracts.
• Emit the relevant events for contract deployment being added/removed when SELFDESTRUCT occurs, or when the chain reverts past blocks that contained SELFDESTRUCT operations.
• The existing event handlers for those events should handle the rest for the fuzzer inherently.

This can be done in tandem with #21 (assertion testing), as we can test for SELFDESTRUCTs with that test case provider.

Support dynamic deployment matching

Currently medusa does not support byte-code matching for dynamically deployed contracts (via contracts that deploy other contracts).

Once matched, newly discovered deployments should be recorded and added to the list of contracts to test. This way the project can support testing invariants of dynamically deployed contracts.

This can also be used to match deployments which we extract from a dummy transaction sequence (captured by tools like etheno), so we can replay deployments and match them dynamically.

Add CODEOWNERS

We should add a CODEOWNERS file to enforce PR review by specific parties.

Me and @anishnaik should be one of those CODEOWNERS for now, as e.g. I have a lot of notions about how code should evolve and probably some undocumented considerations for why some code should look specifically as it does (so it can support future features we may not have now), that we don't want to step on. This will evolve with more people as we onboard everyone :)

Provide Docker environment for application + tests

We should create a Docker container that has all optional dependencies installed (solc, crytic-compile, hardhat, truffle, etc). We should also provide an entry point for unit tests to be run in a container.

Abstraction of testing layer

The project aims to have first party support for some testing strategies (property tests, assertion-based testing, etc). However, on an API level we should provide hooks such as pre-tx-execution and post-tx-execution that allow people to use the Fuzzer with their own hooks to report their own custom test results. Our own testing strategies should be implemented using the generic test case provider interface that others can also use, to remain as agnostic as we can and ensure the API supports all the operations one might expect.

This PR is dependent on #21 , which will introduce a generic test result interface to be used in this PR.

Replace go-ethereum's ABI API

go-ethereum's ABI API we rely on is meant to parse a full ABI and generate interfaces exactly matching those types, with native Go types. It is not meant to serialize individual ABI values in a human readable way (especially without a method definition) and there is no sensible intermediate representation for argument value representation. As such, the preparation for sequence mutation requires we resolve the abi.Method referenced when we first load a CallSequence from a corpus, so we have the correct provider to pack/unpack. In a future iteration, we should have ABI value types we can just pack directly without a method definition. It also requires type conversions through reflection.

Fixing this will clean up mutations/value generation a lot and strip away use of reflection to convert argument value types/structures.

However, for now we can proceed and can replace this later, so it's not worth detouring from our main objectives/timeline.

Add support for embedded library deployments

Add support for the deployment of embedded libraries.

Internal libraries are already supported because all the internal functions are added to the switch table and can be JUMPed to.

Key considerations for an embedded library:

• How do you tell that something is a library during compilation (maybe you can only find out during deployment)
• How do you safely update any contract bytecodes with library addresses before deployment?
• How do you handle the dynamic deployment of a library?

Add system-level specs and architecture diagrams

Need to add spec and arch diagrams.

Should highlight the following:

• Talk about how fuzzing is actually done
• Diagrams for visualization and show inheritance / relationships between objects
• Low-level discussion of each component / process flow

Abstract away coverage maps into generic coverage strategy providers

Currently, coverage maps are a singular definition that we should abstract away into a generic "coverage strategy" interface. When we build out this project's API at a later date, we'll want to be able to swap out coverage strategies that drive the fuzzing semantics.

In the end, coverage interfaces should:

• Implement the same methods as vm.EVMLogger so tracer calls can be forwarded to it for data collection.
  • Capture any relevant information the coverage implementation wants.
• Provide a method to aggregate two coverage maps together and indicate if new coverage was reached in the addition.
  • This will drive the fuzzer's notion of whether we should save an item to the corpus for use in mutation strategies.
• To be determined: Potentially be serializable

Do not start the fuzzing campaign if no properties are detected

Medusa will run without any error or warning if the property mode is used but even if no properties are defined (and assertins are detected). This will waste a lot of time, if the user defined the wrong prefix.

Add support for `optimization` mode

Optimization mode can be treated as another test case provider, similar to property or assertion mode. This is a great issue for someone who is starting to work with medusa because it touches on medusa's most critical packages, how medusa uses the subscribe/publish model to handle fuzz testing, and how easy it is to create a new testing strategy in medusa.

I would recommend understanding how Echidna's optimization mode works first. At a high level, the optimization mode in Echidna tries to maximize some value x. It probably does this by storing the latest maximum, max, and upon the running of a new call sequence will check whether the new max is greater than max. If so, it overwrites max with the new maximum. Realistically, it doesn't matter what x is, what you care about is the return value of the echidna_opt (medusa_opt) function.

This can be done in medusa as well, please refer to test_case.go, test_case_assertion.go, test_case_assertion_provider.go, test_case_property.go, and test_case_property_provider.go. Similarly, you will probably create a test_case_optimization.go and a test_case_optimization_provider.go.

Here are some key considerations:

1. Support for optimization mode in the config and CLI. Should be disabled by default.
2. Use the property and assertion test case providers as examples on how to build out optimization mode.
3. Create a unit test or two that runs medusa in optimization mode.

High level testing API

We currently have a good way to extend testing methodology using a lower level API. We'll want to add methods like RegisterTestCase and ReportTestCaseFinished into a higher level testing suite object that also wraps CallSequenceTestFunc and makes writing complex post call/event properties (per contract, method, or globally) just a few lines of code. It should contextualize all important info about the current state of execution in a test context object.

For more info, see here:
https://github.com/trailofbits/medusa/wiki/API-Overview-(WIP)#extending-testing-methodology

We envision this should look like for generic callback that just broadly take arguments for any call:
fuzzer.Testing.AddPostCallTest(contractName, functionName, callBackThatProvidesArgsAndTestContext)
fizzer.Testing.AddPostEventEmitTest(eventID, callback)

While we can also make method specific arguments that need to match the signature (but in Go) and will direct outputs into that callback's input, so we can (e.g. match our property test callback with event argument types, and the test suite will populate them for us through reflection maybe)?

Allow auto-complete filenames in sh/bash

Medusa requires a little bit more command line parameters than Echidna. This is fine, but there is one small detail that can be improved: auto-complete support (which echidna has already, in some degree). For instance, after typing medusa fuzz --target trying to autocomplete filenames will fail (will not give any suggestion at all). It will be nice to allow auto-complete here to speed up the typing.

Remove solc-version config from TestDeploymentsSelfDestruct and TestDeploymentsWithArgs

The test case TestDeploymentsWithArgs in fuzzer_test.go configures solc version to be 0.7.0 with a comment that it required this version. I tried testing it with solc version 0.8.17 and fixed compilation error of explicitly converting an address to address payable and it worked fine. I am not sure if there is any other reason for using solc version before 0.8.0 for this test. If there is no other reason then we should consider removing this config because it sets solc version to 0.7.0 for all the tests executed after this one. I had to configure solc version to 0.8.17 for my test case that depends on ABIEncoderV2 because the TestDeploymentsSelfDestruct sets solc version to 0.7.0 for all tests. Therefore we need to remove solc version configuration for these test cases.

Notes RE: Echidna

Capturing thoughts from @dev1644 and @snd on a discussion about the limits of Echidna, apropos a comment in townhall. Raising these here for prosperity, and so that perhaps Medusa can be designed early-on to avoid such issues.

Dev:
First, we had an issue when integrating Echidna into their codebase, as a lot of external libraries are being used. There were some errors with the linking of the external library. We used a workaround to use E2E as testing for it, it's helping with some basic property tests, but client's requirement was to focus more on testing a few complex economic exploits.

Max:

1. the client uses a lot of libraries with external functions. Those libraries need to be deployed and then linked. Echidna doesn't support this. We had to make all those functions internal so they get inlined.
2. The client wanted us to fuzz scenarios that involve a Uniswap v2 factory, router and pool. Uniswap v2 uses Solidity 0.5 and 0.6, the client's code 0.8. There's no way to mix those within an Echidna test. The only feasible workaround was to deploy Uniswap within a script, record the transactions with Etheno, and seed Echidna with them
3. Most importantly, the scenarios the client wanted us to fuzz require a setup of almost their entire system involving 15 contracts which all need to be wired up and configured correctly. It took a lot of debugging to figure out why deployment and fuzzing transactions revert. This process was frustrating and took a really long time: The ways to debug Echidna are very limited. The development loop is slow due to Echidna's startup time.

Max also mentioned he intends to create a presentation about these limitations.

Add a halt-on-failure configuration parameter, allow fuzzing to continue to discover multiple results

Currently the project will halt when the first failed test is discovered. Instead, we should have this as a configurable option where by default, we continue fuzzing to find other violations, but a user can specify to halt-on-failure if they choose.

This will require the following changes:

• Adding a configuration parameter that enables halt-on-failure.
• Refactoring of fuzzerWorker.testTxSequence to continue testing the sequence for other failures rather than stopping on the first failure.
• Refactoring fuzzerWorker.run to not halt when a unique test is reported for the first time.
• Refactoring the test results structures a bit
• Removing property tests to validate as we find ways to violate them (to avoid re-testing them).
• Adding logic to halt when there's nothing left to test.

This should be done with or after #21 , as that will change the results structure.

don't require deployment order for single contract

If there is only one contract being deployed, infer that it is the first contract to deploy rather than requiring it be explicit

Support duplicate contract names

Right now contract names are expected to be unique by the fuzzing package (but is supported by compilation). Instead we should come up with some contract ID or something similar to differentiate contracts. It has to factor in source file path. It needs to be human readable and intuitive (or we need an alternate approach), as the config file will have users specifying contract names for deployment order (per #11)

This is very low priority for now and not being considered yet.

Configs should be validated to be of proper form prior to fuzzing campaign

The project, fuzzer, account, and platform configs should all validate their fields to make sure they are properly handled and do not result in unexpected errors.

Note: A user may choose to omit some keys and we should continue using default values in such a case, rather than erroring out.

This is very low priority and should be left for later.

ValueGeneratorMutation prone to division by zero

The methods to generate strings and bytes in the mutational fuzzer is prone to division-by-zero panics if the input given to mutate is of length-zero. These methods should be broken apart so certain methods are only used in certain conditions that do not trigger these issues. We should also consider restricting the adding of strings or byte arrays of length zero to the BaseValueSet.

Add configurable parameters for value generators to balance strategy

This should be done after #28 .

Currently our value generators do not ingest configs that dictate the likelihood of a value in the BaseValueSet being mutated vs randomly generated.

For mutated value generation:
https://github.com/trailofbits/medusa/blob/191b5baf810281a37b543d044cd2eb8576656965/fuzzing/value_generation/value_gen_mutation.go#L8-L18

For random value generation:
https://github.com/trailofbits/medusa/blob/191b5baf810281a37b543d044cd2eb8576656965/fuzzing/value_generation/value_gen_random.go#L41-L43
https://github.com/trailofbits/medusa/blob/191b5baf810281a37b543d044cd2eb8576656965/fuzzing/value_generation/value_gen_random.go#L53-L59

We should also add a config option that allows us to select a given mutator (while also supporting passing of a custom value generator at an API-level) and remove the // TODO comment associated:
https://github.com/trailofbits/medusa/blob/191b5baf810281a37b543d044cd2eb8576656965/fuzzing/fuzzer.go#L132

This should likely be held off until we have #28 so that we can address the config layout for all of this at once and make it sensibly all encompassing.

Add support for multiple transactions per block to FuzzerWorker/CallSequence

Currently, the PR for the Corpus (#11, which addresses #27) supports saving a CorpusCallSequence where a block within it can have more than one transaction.

However, the CallSequence generated by the FuzzerWorker will currently only generate a sequence with one transaction per block.

Changes to be made:

• CallSequence needs to be modified to track per-call-per-block, not just per-call.
• TestChain supports multiple transactions per block but can fail (if you create too many transactions in a block and hit the gas limit) and all progress on that block construction should be lost. Instead of calling CreateNewBlock with all the transactions to include, we should change the semantics so CreateNewBlock is called, then a new method called AddTx can be called to try and add a transaction to the block (and error if we encounter an issue, so we know we can't put in any more transactions). Then finally a CommitBlock method should be called to commit the pending block.
• FuzzerWorker needs to build the call sequence iteratively by adding a transaction to the block, up to a configured "maximum transactions per block" value or until an error occurs adding another one, before committing the block and finalizing that call sequence element.

This should be considered to be done at the same time as #28 , as they touch related code.

config JSON keys should be consistent in casing

The crytic-compile platform config uses JSON keys in camelCase, as does the go-ethereum block header object. The rest of the application uses snake_case keys. We should convert all snake_case keys we can into camelCase for consistency.

Better constant mining and value generation

Not sure where is the issue in the constant mining or value generation, but Medusa should immediately be able to solve this example (tested with 612469e):

contract C {
   uint8 v; 
   function f(uint8 _v) public {
      v = _v; 
   }
   function echidna_test() public returns (bool) { 
      return v != 250;
   }
} 

Echidna output after shrinking:

Analyzing contract: /home/g/Code/echidna/8bits.sol:C
echidna_test: failed!💥  
  Call sequence:
    f(250)

Expose configurable options for chain config

We should add configurable support for different forks of Ethereum by making the TestChain take in a user provided params.ChainConfig. It currently uses params.TestChainConfig, which enables all updates at block number 0:
https://github.com/trailofbits/medusa/blob/8036697794481b7bf9fa78c922ec7fa6a8a3005c/chain/test_chain.go#L78

The chain config should be exposed so it can be queried by other parts of the program (e.g. if SELFDESTRUCT is changed to not destroy the contract, the testChainDeploymentsTracer should make use of this flag:
https://github.com/trailofbits/medusa/blob/8036697794481b7bf9fa78c922ec7fa6a8a3005c/chain/test_chain_deployments_tracer.go#L41-L46

It should also be considered when cloning the TestChain, such that all places we clone retain the exact same config.

max code size exceeded

For certain very large contracts, medusa fails to deploy:

contract deployment tx returned a failed status: max code size exceeded

While this is technically impossible to have in the blockchain, we still want to be able to deploy and run these.

Coverage reports

Similar to echidna, it will be very valuable to have a "coverage report" that outlines what portions of the code were covered.

These are the key features required:

1. The coverage report should be an HTML file
2. The HTML file should have links for each deployed contract
3. Similar to echidna, we will take the r, *, o methodology. Or come up with a new one if that makes sense
4. Create a --coverage-report CLI option that can be called separate from general fuzzing execution

Over time we can add more interesting coverage results or different types of reports. Ideally, we can also have:

1. Show how "often" a line of code was executed
2. TBD

Add hevm/foundry cheatcodes

prank, vm, etc...

Supporting hevm/foundry cheatcodes is going to be important for medusa's success. This might require us to keep a fork of geth.

Support msg.sender fuzzing

Currently msg.sender is set to one of the addresses of our accounts maintained by the Fuzzer. We'll later want to support fuzzing sending messages from hardcoded/magic addresses collected in our BaseValueSet.

Whether you will get a valid fuzzer account or try to derive a hardcoded one from the BaseValueSet should be derived from a configurable probability, as some values that will be interpreted as magic addresses may not actually be relevant and waste cycles for some projects.

Add support for constructor arguments in deployment config

Currently contract deployment configuration is limited to order of contracts to be deployed and does not support constructor arguments. For complex project setups we need the ability to deploy contracts with dynamic values provided as constructor arguments. We will also consider following things while working on this feature -

• Identify how to safely append constructor arguments in-order to the init bytecode before deployment based on argument type
• Identify any opportunities to create generic / general functions that can be re-used for similar use cases

Try to imitate echidna default configs

While the tool is internal, using echidna_ as the function prefix for detecting properties will be useful to speed up compatibility testing.

Refactor tx sequence printing

Currently, tx sequences are printing by relying on go-ethereum methods to serialize function input arguments as JSON, and inserting that into a string template like functionName(<here>). This makes for a slightly odd call sequence, as function parameters will be encapsulated in a array ([]), and byte-related objects will encode as a base64 string rather than showing individual bytes.

This should be refactored such that these caveats do not exist and tx sequence printing results in visually sensible output.

Refactor logging to make use of structured logging providers

Currently the project still has primitive Printf logging as a remnant from the project's inception. This should be replaced with a sensible structured logger, which can output console text, or output structure JSON logs to a file. The API should be consistent/clean throughout the application so an external application can parse structured logs and discern events from each other and act upon them sensibly.

It should also be flexible enough that we can choose not to output to stdout/stderr, or output to any arbitrary channel(s). For instance, logging to only JSON files, or logging to CLI and JSON files at the same time.

Other TODOs (thanks @ahpaleus):

• Quiet and verbose options
• Make UI more colorful/transparent
• medusa's version is output on startup

Add corpus-based tx sequence mutation strategies

Currently this project mutates immediate function input values based on values in the ValueSet, or generates new random inputs. We also store interesting tx sequences that resulted in new coverage into our corpus. We now want to take those interesting tx sequences and mutate them to reach potentially new interesting states/coverage.

We should:

• Have weights for corpus items assigned at runtime (with higher weights for sequences discovered and added to the corpus later in the fuzzing campaign, but the same value for all items loaded from the corpus on start). These values don't need to be stored in the corpus files, just set at runtime. The intent is to prioritize sequences found later in testing, to try and build on that more immediately. The weight can be based off the transaction/call number it's discovered at (obtained from FuzzerMetrics).
• Take a random entry from our weighted list of call sequences in the corpus
• Offer a variety of mutation methods to modify the corpus entry (tx sequence)
  • Configurable weights should be used to allow the balancing of mutation strategies.
• Executing the mutated corpus entry on our test chain with as much accuracy as we can get.

Additional goals:

• Corpus call sequences should have a custom routine for serializing CallMessages that splits the data field by its underlying ABI values, so it is human readable/editable, better for mutations on an API level.
• ValueSet should be fed values from coverage-increasing sequences.

Periodically generate calls to view methods in the CallSequence

We'll want to periodically call view methods to increase coverage. Additionally, assertion testing supports config-driven options to enable assert testing of view methods, but the FuzzerWorker only produces CallSequences with calls targeting state changing methods right now, so it's never hit.

Requirements:

• Record view functions separately, but alongside the existing stateChangingMethods (see the updateStateChangingMethods method)
• Add a really low probability of sourcing a target method from it (instead of stateChangingMethods) at the top of the generateFuzzedCall method.
• Ensure the corpus.UpdateCorpusAndCoverageMaps method checks if the last call was to a view method. If it was, do not record the call sequence in the corpus at that step, as it's not a coverage-increasing sequence we'd be interested in recording, as it was not state changing.

Add corpus support

In order to drive transaction sequence mutation, we must introduce a corpus.

Preliminary thoughts are that the corpus should:

• Be able to house block sequences (with block header info and underlying transactions) so that the sequence can be replayed.
• Support reading/writing to disk
  • Be I/O efficient so it does not slow down the fuzzer.
• Be abstracted behind an interface so different styles corpuses can be implemented/supported.

In using it, we should:

• Not allow semantically duplicate entries

Storing new entries should occur when a transaction sequence currently being tested at a given tx index reaches new coverage (not at the end of the tx sequence testing):
https://github.com/trailofbits/medusa/blob/191b5baf810281a37b543d044cd2eb8576656965/fuzzing/fuzzer_worker.go#L457-L465

Support assertion-based testing

Currently the project focuses on violated property tests. However, we also want to support a testing strategy where any failed assert(...) call in Solidity can be used to flag a failure. This will allow people to write tests as a series of assertion statements in their typical codebase.

ValueGenerator improvements

We'll want the ValueGenerator interface to optionally take in an input value as the base for a mutation, rather than creating a new integer from mutation. We'll also want to add all the 2**n - 1 numbers and maybe filter per integer size when obtaining numbers from the ValueSet to make the ValueGenerator more flexible.

There are probably some other improvements that could be made to improve this process, weighted values, etc.

crytic-compile fails to compile if JSON build artifacts contain one which does not have a compilation unit

crytic-compile sometimes can generate a .json file in the crytic-export directory when compiling, which do not contain compilation units. However, this should not stop parsing of other .json files which do contain them.

Currently if any file is found not to have a compilation unit, the compilation will fail:
https://github.com/trailofbits/medusa/blob/191b5baf810281a37b543d044cd2eb8576656965/compilation/platforms/crytic_compile.go#L134-L138
The error returned here should be replaced with a continue statement.

Consider removing tx gas limit for initial contract deployments

It's unintuitive that there is both a block and and tx gas limit? This deviates from the semantics of the runtime environment since tx's on mainnet can consume the entire block gas limit i.e. the tx gas limit is not less than the block gas limit, they're always the same

When `stopOnFailedTest` is false, stop when if all tests have failed.

Currently, medusa will continue running despite all tests having failed which I'm not sure there's a reason to do?

Better value generation with arrays

Not sure where is the issue in the value generation, but Medusa should immediately be able to solve this example (tested with 612469e):

contract A {

    mapping (address => mapping (address => bool)) allowance;

    function bulkChangeAllowance(address[][2] memory x) public {
        for(uint i = 0; i < x.length; i++) {
            address add1 = x[i][0];
            address add2 = x[i][1];
            allowance[add1][add2] = true;
        }
    }

    function echidna_test() public returns(bool) {
        return !allowance[msg.sender][msg.sender];
    }

}

Echidna output after shrinking:

Analyzing contract: /home/g/Code/echidna/nested.sol:A
echidna_test: failed!💥  
  Call sequence, shrinking (3861/5000):
    bulkChangeAllowance([[0x0, 0x0], [0x10000, 0x10000]])

Add basic CLI capabilities

We need to add some ability to modify configuration parameters through the command line. We do not have to be exhaustive at the moment but should be able to change at least the following:

1. Number of workers
2. Sequence Length
3. Test Limit
4. Coverage
5. Corpus Directory
6. Deployment Order
7. Assertion mode
8. Property mode

Add support for `multi-abi`

The multi-abi CLI flag or the multiABI config option would override the deploymentOrder configuration parameter and deploy every contract in the target directory. This would be a simple addition and the main changes would include:

1. Add CLI option (default is false)
2. Add config file option (default is false)
3. When honoring deployment order, if multi-abi is true, then deploy everything.

Step (3) can be done in a few different ways

Experiment with libfuzzer coverage-based feedback loop

Echidna coverage based feedback loop is not great, for a number of reason. The most important: it is too simple and it is unclear it is good enough to react quickly when there is a new coverage in the corpus. I suggest to test, even using some naive approach, the libfuzzer port to golang to see if it works better.

Refactor error handling/panics

The codebase should be refactored to remove most of the current panic statements. These are left over code snippets from the inception of the project.

Requirements:

• panics are only used if there is no sensible way to gracefully return a fatal error to the top level (e.g., in an execution tracer method that cannot return an error).
  • Otherwise, errors should be handled gracefully.
• Timeout/keyboard interrupt/test limit should all produce sensible messages.
• Errors should return context about whether they occurred (wrap errors where sensible with appropriate messages).

Examples to refactor:
https://github.com/trailofbits/medusa/blob/ec411887ed61cf254833cd30508558a5c1962057/main.go#L9-L12 (any error is translated to a panic right now)
https://github.com/trailofbits/medusa/blob/43d538e7b9aa534a367961201ce8bf99ee17f9f0/fuzzing/fuzzer_worker.go#L142
https://github.com/trailofbits/medusa/blob/43d538e7b9aa534a367961201ce8bf99ee17f9f0/fuzzing/fuzzer_worker.go#L153