clovaai / lffont Goto Github PK
View Code? Open in Web Editor NEWOfficial PyTorch implementation of LF-Font (Few-shot Font Generation with Localized Style Representations and Factorization) AAAI 2021
License: Other
Official PyTorch implementation of LF-Font (Few-shot Font Generation with Localized Style Representations and Factorization) AAAI 2021
License: Other
Would you please tell me whether the chn_decomposed.json is created by yourself? And how can I know the number of the each component of the character? Thanks
CombTestDataset class
def __getitem__(self, index):
font_name, trg_uni = self.fus[index]
font_idx = self.fonts.index(font_name)
style_unis, trg_comp_ids = self.sample_char(font_name, trg_uni)
style_imgs = torch.stack([self.env_get(self.env, font_name, uni, self.transform) for uni in style_unis])
font_idx = torch.LongTensor([font_idx])
trg_dec_uni = torch.LongTensor([self.to_int(trg_uni)])
content_img = self.env_get(self.env, self.content_font, trg_uni, self.transform)
ret = (
font_idx.repeat(len(trg_comp_ids)),
torch.LongTensor(trg_comp_ids),
style_imgs,
font_idx,
trg_comp_ids,
trg_dec_uni,
content_img
)
I think return value doesn't contain the information of style_comp_ids, only trg_comp_ids
Can I get an explanation about this?
Thank you.
Line 53 in 2e45e91
Hi, when I trained phase1 , I am so happy to look this performance. and then I will train phase2, it have same bug during evaluate....
the bug is :
Lines 56 to 73 in 2e45e91
(in_style_ids, in_comp_ids, in_imgs, trg_style_ids, trg_comp_ids, trg_unis, content_imgs, trg_imgs)
But, when phase2 evaluate, the FactTestDataset() function only return 7 elements ,like :
ret = (torch.LongTensor(in_style_ids),
[*map(torch.LongTensor, in_comp_ids)],
torch.cat(in_imgs),
torch.LongTensor(trg_style_id),
torch.LongTensor(trg_comp_ids),
torch.LongTensor(trg_dec_uni),
content_img
)
the ret_targets is False it was be set here , so it only return 7 elements and the program is crashed!!! at infer_loader() function
Hi, I test the model, it is good。 And I have some question about the framework:
Hi, thanks for this great work! I have implemented this code into a single font style transferring task. That means I only use one content font (6000+ chars) and a style font (500+ chars), and try to generate the rest chars of that style. When I trained the model for phase1, the output is really strange and the generator accuracy is really low (no more than 40%). Could you give some hints? I am looking forward to your reply and further discussions.
Again, it is a fantastic work! Thank you very much!
Frank
Hi, I'm so appreciate of your work. And I have three questions about the training details.
question one: when train the phase1 model, how choose training fonts to benefit to the convergence of the model ?
question two: first train the phase1 model, then initialize phase1, train the phase2 model, the technological process is right?
question three: almost training fonts don't include 371 components,whether need to update components json?
thank you .
python build_dataset.py
--lmdb_path path/to/dump/lmdb
--meta_path path/to/meta/file
--json_path path/to/save/dict
can you give me an example of how to "Build meta file to dump lmdb environment" and how to "python build_dataset.py "?
because I just cant understand according to your description about "lmdb", "meta file",,,
Hi, if increase the input size 128128 to 256256, can the project generate more qualitative font img the 128128?
Thank you.
gpu: -1
INFO::07/07 13:42:05 | Unique name: NAME_phase1
INFO::07/07 13:42:05 | Get dataset ...
INFO::07/07 13:42:05 | Build model ...
INFO::07/07 13:42:14 | Start training ...
During training, the model stops at "Start training..."
Hi, I am very interested in your work, so I would really like to train my own dataset using your network. But I'm having some problems with your network, I made the dataset according to the build_dataset.py file, but the format of the dataset you use in training is not quite the same as this, and I'd like to ask about the exact way to make the training data. Thank you for reading and I look forward to your reply!
Hi, Thank you for your fantastic work! But when training with my dataset, I met a problem.
When training for phase 1, generator doesn't have a emb_block, but in phase 2, generator has a emb_block, so how to load the weight of the phase 1 in the phase 2?
when i ran train.py(phase 1),i got problem is "env_get = lambda env, x, y, transform: transform(read_data_from_lmdb(env, f'{x}_{y}')['img']) TypeError: 'NoneType' object is not subscriptable" .then i found that the return vaule “data” is None in function "read_data_from"
Looking forward to your help!
(base) D:\lffont>python train.py stkb2jingji1 ./cfgs/custom.yaml ./cfgs/combined.yaml
failed to import apex
INFO::08/25 18:03:41 | Run Argv:
> train.py stkb2jingji1 ./cfgs/custom.yaml ./cfgs/combined.yaml
INFO::08/25 18:03:41 | Args:
name = stkb2jingji1
config_paths = ['./cfgs/custom.yaml', './cfgs/combined.yaml']
resume = None
use_unique_name = False
INFO::08/25 18:03:41 | Configs:
use_half: False
half_type: O2
use_ddp: False
port: 7890
phase: comb
overwrite: False
work_dir: result
batch_size: 8
comb_iter: 800000
fact_iter: 50000
seed: 2
g_lr: 0.0002
d_lr: 0.0008
ac_lr: 0.0002
n_workers: 0
adam_betas:
- 0.0
- 0.9
init: kaiming
n_comps: 371
n_in_chars: 3
n_each_chars: 3
n_targets: 9
n_shots: 8
content_font: font1
language: chn
data_path: ./data/stkb2jingji
data_meta: ./data/stkb2jingji_train.json
dec_dict: meta/chn_decompose.json
dset_args:
n_sample_min: 3
n_sample_max: 5
dset_aug:
normalize: True
random_affine: False
cv_n_unis: 30
cv_n_fonts: 10
C: 32
g_args:
comp_enc:
norm: in
activ: relu
pad_type: zero
skip_scale_var: False
sigmoid: True
emb_block:
emb_dim: None
dec:
norm: IN
activ: relu
pad_type: zero
out: sigmoid
content_enc:
C_out: 256
content_sigmoid: False
norm: in
d_args:
w_norm: spectral
activ: relu
gap_activ: relu
pad_type: zero
res_scale_var: False
ac_args:
norm: IN
activ: relu
pad_type: zero
conv_dropout: 0.3
clf_dropout: 0.2
pixel_loss_type: l1
pixel_w: 0.1
gan_w: 1.0
fm_layers: all
fm_w: 1.0
ac_w: 0.1
ac_gen_w: 0.1
dec_const_w: 0.0
save: all-last
print_freq: 1000
val_freq: 10000
save_freq: 10000
tb_freq: 100
unique_name: stkb2jingji1
name: stkb2jingji1
gpu: -1
INFO::08/25 18:03:41 | Unique name: stkb2jingji1
INFO::08/25 18:03:41 | Get dataset ...
INFO::08/25 18:03:41 | Build model ...
INFO::08/25 18:03:42 | Start training ...
Traceback (most recent call last):
File "D:\lffont\train.py", line 253, in <module>
main()
File "D:\lffont\train.py", line 249, in main
train(args, cfg)
File "D:\lffont\train.py", line 235, in train
trainer.train(trn_loader, st_step, cfg[f"{cfg.phase}_iter"])
File "D:\lffont\trainer\combined_trainer.py", line 54, in train
for (in_style_ids, in_comp_ids, in_imgs,
File "D:\lffont\datasets\datautils.py", line 12, in cyclize
for x in loader:
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 652, in __next__
data = self._next_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 692, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\lffont\datasets\p1dataset.py", line 93, in __getitem__
trg_unis, trg_comp_ids = self.get_available_combinations(avail_unis, style_comp_ids)
File "D:\lffont\datasets\p1dataset.py", line 66, in get_available_combinations
comps = self.decompose_dict[uni]
KeyError: '71FC'
Hi, I do some test and using my dataset training with some unicodes, and I find if there are some characters that can not be divided into radicals, the evaluate will be failed.
Hello, I have the following questions:
I need you to explain to see fonts, unseen fonts, see characters, unseen characters
I build the train.json as ReadMe said.
train.json has no "test" in keywords , only have ("train", "valid", "avail")
this error occur in uniform_sample function
Maybe this is "valid" keyword ??
I wonder whether you used another pre-trained model to produce Korean fonts for Figure 11 or you just used the uploaded pre-trained for the Korean fonts as well.
If you have a Korean-specific trained model, may I ask you to share it?
Hi, in phase2, there is a function named check_and_sample, the trg_fonts is random choiced, and it's stype don't match to the in_fonts. Isn't it a bug?Thank you.
for example:
/media/chengls/Elements/lffont/images/in_img_3.jpg
/media/chengls/Elements/lffont/images/content_img_7.jpg
/media/chengls/Elements/lffont/images/trg_img_7.jpg
they are in_img、content_img and trg_img, but the style of trg_img is different from in_img.
Hi, when evaluator phase1, it occurs a error "TypeError: init() missing 1 required positional argument: 'emb_block"。what wrong with it? Thank you.
您好,我大致看了一下您的论文,对于数据集部分有些疑问,需要您对seen fonts,unseen fonts,seen characters,unseen characters进行一下解释。尤其是unseen fonts,训练中没有出现的字体种类,测试时可以生成该种类字体吗
In the paper, there are no details about korean generator; the number of sample fonts, and the number of seen/unseen characters.
I'm just curious about training details of korean.
Hi, I see there is pertrained weight in weight/, and I run the code like the ReadMe
But has error:
lmdb.Error: /path/to/lmdb : No such file or directory
because it find lmdb by defaults.yaml , and I did not modify it.
If I want to test pretrained weight, :
?? Am I right?
As my opinion, if I want to test the generator it must be need content_font, so the it must be in test_lmdb file when Step2 (make it).
您好!请教您一些问题:
1.lmdb_path文件下保存的.mdb文件是包括训练集和测试集的所有数据吗?
2.readme里面叙述的train meta下的"avail"数据应该是什么?我没有理解"avail",您可以解释一下吗。
3.您是如何拆分部件的,依据的标准是什么?
期待您的回复,谢谢!
Hi,
Thank you for your fantastic work! It's very helpful.
But when I execute the 'evaluator.py', there are some problems.
python evaluator.py
cfgs/factorize.yaml
--weight weight/generator.pth
--img_dir path/to/save/images
--test_meta meta/test.json
error output: lmdb.Error: /path/to/lmdb: No such file or directory
Best regards
GMT
when I train phase 1 , this error occur in here
I debug it and the local variables self.availds.keys() is ["seen_fonts", "unseen_fonts", "seen_unis", "unseen_unis"]
maybe this is a bug ?
INFO::08/25 20:17:59 | Validation at Epoch = 0.323
Traceback (most recent call last):
File "D:\lffont\train.py", line 253, in <module>
main()
File "D:\lffont\train.py", line 249, in main
train(args, cfg)
File "D:\lffont\train.py", line 235, in train
trainer.train(trn_loader, st_step, cfg[f"{cfg.phase}_iter"])
File "D:\lffont\trainer\combined_trainer.py", line 173, in train
self.evaluator.cp_validation(self.gen_ema, self.cv_loaders, self.step, phase="comb",
File "D:\lffont\evaluator.py", line 44, in cp_validation
self.comparable_val_saveimg(gen, loader, step, tag=f"comparable_{tag}_{ext_tag}",
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "D:\lffont\evaluator.py", line 19, in decorated
ret = val_fn(self, gen, *args, **kwargs)
File "D:\lffont\evaluator.py", line 50, in comparable_val_saveimg
compare_batches = self.infer_loader(gen, loader, phase=phase, reduction=reduction)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "D:\lffont\evaluator.py", line 19, in decorated
ret = val_fn(self, gen, *args, **kwargs)
File "D:\lffont\evaluator.py", line 60, in infer_loader
for i, (in_style_ids, in_comp_ids, in_imgs, trg_style_ids, trg_comp_ids,
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 652, in __next__
data = self._next_data()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\dataloader.py", line 692, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\ProgramData\Anaconda3\lib\site-packages\torch\utils\data\_utils\fetch.py", line 49, in <listcomp>
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\lffont\datasets\p1dataset.py", line 197, in __getitem__
style_unis, trg_comp_ids = self.sample_char(font_name, trg_uni)
File "D:\lffont\datasets\p1dataset.py", line 186, in sample_char
style_uni = random.choice(avail_style_uni)
File "C:\ProgramData\Anaconda3\lib\random.py", line 346, in choice
return seq[self._randbelow(len(seq))]
IndexError: list index out of range
Hi, thanks for the nice work! I have few questions about the evaluation classifiers while reading the paper. In your paper, you trained “style-aware (S) and content-aware (C) classifiers”. I'm curious about how they have been trained.
Are they trained on both the seen and unseen fonts? If so, what are the training samples? Are all samples from training and testing datasets used to train the classifiers? And what architecture you were using for these classifiers?
Again, LFFont is a great work. Thanks a lot for your time.
James
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.