nlp-greyfoss / nlp-in-action-public

Natural language processing projects in action.

nlp-in-action-public

Natural language processing projects in action.

nlp-in-action-public's Issues

tokenized_dataset = load_from_disk(f"{train_args.dataset_name}") #doupo-dataset2

tokenized_dataset = load_from_disk(f"{train_args.dataset_name}") #doupo-dataset2 报错了，不能下载这个数据集呀老大

运行： python data_process.py 报错：

Traceback (most recent call last):
File "d:\Anaconda3\envs\py3.10\lib\site-packages\huggingface_hub\utils_errors.py", line 261, in hf_raise_for_status
response.raise_for_status()
File "d:\Anaconda3\envs\py3.10\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/simple-gpt2-doupo/resolve/main/tokenizer_config.json

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "d:\Anaconda3\envs\py3.10\lib\site-packages\transformers\utils\hub.py", line 417, in cached_file
resolved_file = hf_hub_download(
File "d:\Anaconda3\envs\py3.10\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "d:\Anaconda3\envs\py3.10\lib\site-packages\huggingface_hub\file_download.py", line 1195, in hf_hub_download
metadata = get_hf_file_metadata(
File "d:\Anaconda3\envs\py3.10\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "d:\Anaconda3\envs\py3.10\lib\site-packages\huggingface_hub\file_download.py", line 1541, in get_hf_file_metadata
hf_raise_for_status(r)
File "d:\Anaconda3\envs\py3.10\lib\site-packages\huggingface_hub\utils_errors.py", line 293, in hf_raise_for_status
raise RepositoryNotFoundError(message, response) from e
huggingface_hub.utils._errors.RepositoryNotFoundError: 401 Client Error. (Request ID: Root=1-6641c3e9-391b067911e954345955c2d5;1b4614e3-a8ce-4dc4-b3d2-e331c8541b82)

Repository Not Found for url: https://huggingface.co/simple-gpt2-doupo/resolve/main/tokenizer_config.json.
Please make sure you specified the correct repo_id and repo_type.
If you are trying to access a private or gated repo, make sure you are authenticated.
Invalid username or password.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "E:\wll\phpsys\WWW\start\u_server\u3_test\test\resnet\pytorch\transformer\nlp-in-action-public-master\transformers\mygpt2\data_process.py", line 59, in
tokenizer = AutoTokenizer.from_pretrained(train_args.model_name)
File "d:\Anaconda3\envs\py3.10\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 643, in from_pretrained
tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
File "d:\Anaconda3\envs\py3.10\lib\site-packages\transformers\models\auto\tokenization_auto.py", line 487, in get_tokenizer_config
resolved_config_file = cached_file(
File "d:\Anaconda3\envs\py3.10\lib\site-packages\transformers\utils\hub.py", line 433, in cached_file
raise EnvironmentError(
OSError: simple-gpt2-doupo is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with use_auth_token or log in with huggingface-cli login and pass use_auth_token=True.

数据集合说是：
如果这是一个专用存储库，请确保使用“use_auth_token”传递对此存储库具有权限的令牌，或者使用“huggingface cli login”登录并传递“use_aauth_token=True”。