wavmark / wavmark Goto Github PK

View Code? Open in Web Editor NEW

185.0 8.0 26.0 586 KB

AI-based Audio Watermarking Tool

License: MIT License

Python 100.00%

audio-watermarking digital-watermarking signal-processing watermarking

wavmark's People

Contributors

Stargazers

Watchers

wavmark's Issues

Request for Training Code to Train `my_model.Model` with Custom Audio Data

Dear WavMark Author,

I hope this message finds you well. I am currently working on an audio watermarking project using your WavMark repository, and I have a few questions regarding the use of custom data.

I would like to embed and extract watermarks using my own audio data. Does this require training a new model from scratch? If so, do you have any training scripts or code available that you could share to help with this process? Any guidance or resources you could provide would be greatly appreciated.

Thank you for your time and assistance.

Best regards,
Zhisheng Yao, Ph.D.
College of Telecommunications & Information Engineering
Nanjing University of Posts and Telecommunications
No. 66 Xinmofan Road, Nanjing 210003, China
email: [email protected]

How to deal with time stretching attack in training

Thanks for your wonderful work!

I have a question about time stretching attack in training. After applying the time stretching operation (Increasing the speech by 1.1x), the length of signal segment is changed. Then, the length of the model_out does not match the length of the target. How to deal with it? Thank you very much for taking the time to answer my question.

License?

Thanks for this great work - is there any plan to release it under a more permissive license? As it stands, we cannot incorporate this project into our MIT-licensed codebase.

bug

in the following code of init.py

if path == "default":
resume_path = hf_hub_download(repo_id="M4869/WavMark",
filename="step59000_snr39.99_pesq4.35_BERP_none

need to add an else branch, incase of loading the model file from local files.
else:
resume_path = path

FYI: it is very difficult to load id from huggingface for my country.

Could you please offer some codes for model training?

The brilliant work WavMark is really impressing, especially its innovated training strategies like the curriculum learning approach, shift module, and attack simulator, which are the parts I'm most interested in. While this repository seems to be a helpful tool for watermarking that only preserves the well-trained models, without those charming training procedures.

Would you consider adding modules for customed model training, like offering modules of curriculum learning approach, and attack simulator?

WavMark does not survive an Eleven Labs Cloning

I was attempting to use your project but it does not seem that it works as intended in the real world. The watermark can not survive against cloning by companies like Eleven Labs.

"WavMark has been designed with the intention of offering high stability and imperceptibility, making it suitable for protecting audio content against common forms of tampering and compression. However, the cloning of audio might present a unique challenge. Cloning can refer to a range of processes, from simple duplication to more sophisticated synthesis or transformation techniques that attempt to replicate the audio's characteristics without carrying over the embedded watermark information"

pip install watermark downgrades pytorch

while i applaud the project, when installed from pypi it downgrades torch to 1.13cu117 which messes up environments. Whatever you guys do should be as transparent as the watermark that you put.

I have not tried installing installing from setup, just noticed this while trying TTS projects requiring your work.

Thanks.

Question about random sampling of decoder in paper

In the paper, the decoder has an operation of random sampling but in the code, I find the random sampling is replaced with the following method:

wavmark/src/wavmark/models/my_model.py

Lines 40 to 43 in 682f0af

 def decode(self, signal): 

 signal_fft = self.stft(signal) 

 watermark_fft = signal_fft 

 _, message_restored_fft = self.enc_dec(signal_fft, watermark_fft, rev=True)

Is this implementation the same as the random sampling operation?

Replicate information about the model weights on the Hub!

Hi wavmark team,

I'm VB; I lead the advocacy efforts for Audio at Hugging Face. Thanks for your contribution and for releasing your weights on the Hugging Face Hub (https://huggingface.co/M4869/WavMark).

It'd be great if you could replicate the GitHub's README over to WavMark's model card as well. This will help in it's discovery and also would lead to greater visibility of the model as well.

Please do feel free to let me know if you need any support with this!

Cheers,
VB

请问这个类库支持MP3文件的加密吗？

I encountered an error during execution, can I see what I need to do to solve this problem?

Python Version: 3.12
OS: Mac

Error msg:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 61] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connection.py", line 213, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x1227047d0>: Failed to establish a new connection: [Errno 61] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.12/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /M4869/WavMark/resolve/main/step59000_snr39.99_pesq4.35_BERP_none0.30_mean1.81_std1.81.model.pkl (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1227047d0>: Failed to establish a new connection: [Errno 61] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1722, in _get_metadata_or_catch_error
    metadata = get_hf_file_metadata(url=url, proxies=proxies, timeout=etag_timeout, headers=headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1645, in get_hf_file_metadata
    r = _request_wrapper(
        ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 372, in _request_wrapper
    response = _request_wrapper(
               ^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 395, in _request_wrapper
    response = get_session().request(method=method, url=url, **params)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/utils/_http.py", line 66, in send
    return super().send(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /M4869/WavMark/resolve/main/step59000_snr39.99_pesq4.35_BERP_none0.30_mean1.81_std1.81.model.pkl (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1227047d0>: Failed to establish a new connection: [Errno 61] Connection refused'))"), '(Request ID: 5e910976-7672-4255-a7c7-ea060fb00c84)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/yinyue/workspace/audio-watermark/encode_utils.py", line 35, in <module>
    main()
  File "/Users/yinyue/workspace/audio-watermark/encode_utils.py", line 9, in main
    model = wavmark.load_model().to(device)
            ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/wavmark/__init__.py", line 10, in load_model
    resume_path = hf_hub_download(repo_id="M4869/WavMark",
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1221, in hf_hub_download
    return _hf_hub_download_to_cache_dir(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1325, in _hf_hub_download_to_cache_dir
    _raise_on_head_call_error(head_call_error, force_download, local_files_only)
  File "/opt/homebrew/lib/python3.12/site-packages/huggingface_hub/file_download.py", line 1826, in _raise_on_head_call_error
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

Process finished with exit code 1

Prediction of decoded message as mean of multiple segments?

In the paper, it is claimed that the model encodes watermark at a rate of 32 bit per second. This implies that we can encode a different message every second. However, the code in this repository tries to encode the same message every second (see here).

Moreover from this line, we see that the prediction is derived from an average of multiple segments. If different messages were encoded in different 1-sec segments, then averaging the prediction would be invalid.

My question is, is this averaged prediction used to calculate the Bit Error Rate in the paper?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble

	def decode(self, signal):
	signal_fft = self.stft(signal)
	watermark_fft = signal_fft
	_, message_restored_fft = self.enc_dec(signal_fft, watermark_fft, rev=True)