lottopotato / note

skill, reference, method.. note

.

note's Issues

토치 모델 파라미터, BPE 토크나이저(바이트레벨) 디코드, transformers verbose 옵션

토치 모델 파라미터 개수

def count_parameters(model):
    n_parameters = sum(p.numel() for p in model.parameters() if p.requires_grad)
    print(f'{n_parameters:,}')

BPE 토크나이저(바이트레벨) 디코드

from tokenizers.decoders import ByteLevel
decoder = ByteLevel()
decoder.decode([ 'ĠÑģлово' ])

transformers verbose(huggingface/transformers#14285 (comment))

transformers.logging.set_verbosity_error()

jupyter notebook, BufferError: Existing exports of data: object cannot be re-sized

dask/distributed#1704 (comment)
https://stackoverflow.com/a/49305034
https://discourse.jupyter.org/t/jupyter-notebook-zmq-message-arrived-on-closed-channel-error/17869/7

jupyter notebook --NotebookApp.iopub_data_rate_limit=1.0e10
jupyter notebook --NotebookApp.iopub_msg_rate_limit=1.0e10

tornado6.1, jupyter-client==7.3.2

참고.

Failed to initialize NVML: Driver/library version mismatch

https://stackoverflow.com/a/45319156

lsmod | grep nvidia

sudo rmmod nvidia_drm
sudo rmmod nvidia_modeset
sudo rmmod nvidia_uvm

sudo rmmod nvidia

Troubleshooting
sudo lsof /dev/nvidia*

pytorch gpu 확인 관련

torch.cuda.is_available()
torch.cuda.device_count()
torch.cuda.current_device()

RAG 관련

Post processing

• Moderation - RAI: checking as harmful or unfavorable, ...
• Regas: grounding check for output answer by question, contexts, groundtruth, ...

pytorch transfomer 마스크 까먹을때

https://stackoverflow.com/a/68396781

Difference between src_mask and src_key_padding_mask

src_mask [Tx, Tx] = [S, S] ...
tgt_mask [Ty, Ty] = [T, T] ...
memory_mask [Ty, Tx] = [T, S] ...
src_key_padding_mask [B, Tx] = [N, S] ...
tgt_key_padding_mask [B, Ty] = [N, t] ...
memory_key_padding_mask [B, Tx] = [N, S] ...

대충 정리하면 _mask는 attention layer관련 cheating 방지 right shift or left shift 마스크가 맞고
_padding_mask는 loss, optimize 관련 패딩 -> ignore_pad 와 비슷한 개념

... 더있음

pandas 분위값

.quantile()

konlpy 태그셋

https://github.com/konlpy/konlpy/tree/master/konlpy/data/tagset

tqdm notebook 안보일때

jupyter nbextension enable --py widgetsnbextension