brain-research / tensorfuzz Goto Github PK

I am using TF 1.15.0
I am trying to run the examples and get the following error during fuzzing,
TypeError: Cannot interpret feed_dict key as Tensor: The name 'save/Const:0' refers to a Tensor which does not exist. The operation, 'save/Const', does not exist in the graph.

Hi,

I am interested in the accuracy loss due to quantization and was running the quantized_fuzzer.py example. In the script I see that we first get a "result" when the objective function is not met, namely argmax for logits and quantized_logits differ. And then, we check whether the disagreement is correct or spurious. Is this to capture non-determinism in floating point operation? I see that the loop runs 10 times for the same input. Is that intentional?

Thanks!

First of all thank you very much for open-sourcing this! I just wanted to ask if there are any plans to accept PRs to adapt this code base to tensorflow2?

I'd be willing to discuss / submit some changes but want to know if this is on the roadmap or if I should consider a fork?

Hi tensorfuzz developers,

Thank you for making this tool public. I have a quick question about the quantization example. It seems that tensorfuzz works on a normalized image where each entry in the matrix is a fp value between [-1, 1]. So it appears to me that a mutated normalized image, despite having different prediction, could not map to a different image in original MNIST format where entries are integers.

I noticed that there's a piece of code that double check the validity of the mutated image. Is it related to this question?

I may miss something. Please let me know if it makes sense.

Could you guys push some example code(like detect NaNs, find disagreements on 32 bit model and 16bit model) on the github so i can understand the lib better~

I would appreciate it if you could show me some example ！

Hi, I'm currently running this fuzzer with the code provided in the /examples directory but it doesn't work out. For the dcgan example, I always get 'Fuzzing failed to satisfy objective function.' Just wondering what could be the reason for producing a none result? And for the nan and quantized example, there is an exception: ValueError: Cannot add function '__inference_Dataset_flat_map_read_one_file_44' because a different function with the same name already exists. I'm not sure if it is because there are some defects in TensorFlow 1.

I'm running it using python 3.6.9 and TensorFlow 1.15 by the way.

I could‘t figure out why the nan_fuzzer.py used the all_logit_coverage_function while the quantized_fuzzer and the dcgan_fuzzer used the raw_logit_coverage_fuction.Is there anyone could explain the reason about the qusetion?Thanks a lot.

This issue was automatically created by Allstar.

Security Policy Violation
Project is out of compliance with Binary Artifacts policy: binaries present in source code

Rule Description
Binary Artifacts are an increased security risk in your repository. Binary artifacts cannot be reviewed, allowing the introduction of possibly obsolete or maliciously subverted executables. For more information see the Security Scorecards Documentation for Binary Artifacts.

Remediation Steps
To remediate, remove the generated executable artifacts from the repository.

First 10 Artifacts Found

• examples/dcgan/pycache/ops.cpython-36.pyc
• examples/nans/pycache/dataset.cpython-36.pyc
• lib/pycache/init.cpython-36.pyc
• lib/pycache/corpus.cpython-36.pyc
• lib/pycache/coverage_functions.cpython-36.pyc
• lib/pycache/dataset.cpython-36.pyc
• lib/pycache/fuzz_utils.cpython-36.pyc
• lib/pycache/fuzzer.cpython-36.pyc
• lib/pycache/mutation_functions.cpython-36.pyc
• lib/pycache/sample_functions.cpython-36.pyc
• Run a Scorecards scan to see full list.

Additional Information
This policy is drawn from Security Scorecards, which is a tool that scores a project's adherence to security best practices. You may wish to run a Scorecards scan directly on this repository for more details.

Allstar has been installed on all Google managed GitHub orgs. Policies are gradually being rolled out and enforced by the GOSST and OSPO teams. Learn more at http://go/allstar

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.