jarveelee / synthtext_chinese_version

Modify from https://github.com/ankush-me/SynthText.git to generate chinese character

SynthText from Ankush

I can no longer maintain this OCR and SynthText project because I change my research domain according to the requirement of my new affiliation.....Anyone want to hold this can tell me , I can give the master to you as long as you can really understand the code in details.

Modify from https://github.com/ankush-me/SynthText.git to generate chinese character

My OS is Ubuntu opencv2.4 But I am not sure whether it can run on other OS

I changed some func,just run gen.py will be OK,in gen.py I change the depth prediction map with gray map for generating char on cartoon image , for natural img you need to change back to depth map ,other gen**.py contains similar code with different path I do for myself...

0,Before running this code make sure your OS support unicode for chinese.. which as well cost me hours....Added chinese may not make sense because in English words are saperated by blank meanwhile in chinese words are saperated by meaning.

1,In synthGen I added a function called is_chinese(char ) to or with is_english to cal num of valid chars.

2,Updated the .tff char style files and the path.txt,then

3,some utf-8 decoded and encoded for chinese char ....Ah I forgot the details....

4,So you can add more pic into the dataset and check with issue under the anthor to fix mistakes......

5,If you want to add more img , firstly you need to compute the segmentation and depth prediction by the 2 matlab code and 1 python code provided by author, and then use the add_more_data.py to generate a new big dset.h5 , containing all of imgs and their seg and depth, then rerun gen.py to see its performance.

These are some samples I do.

** Synthetic Scene-Text Image Samples**

Code for generating synthetic text images as described in "Synthetic Data for Text Localisation in Natural Images", Ankush Gupta, Andrea Vedaldi, Andrew Zisserman, CVPR 2016.

** Synthetic Scene-Text Image Samples**

The library is written in Python. The main dependencies are:

pygame, opencv (cv2), PIL (Image), numpy, matplotlib, h5py, scipy

Generating samples

python gen.py --viz

This will download a data file (~56M) to the data directory. This data file includes:

• dset.h5: This is a sample h5 file which contains a set of 5 images along with their depth and segmentation information. Note, this is just given as an example; you are encouraged to add more images (along with their depth and segmentation information) to this database for your own use.
• data/fonts: three sample fonts (add more fonts to this folder and then update fonts/fontlist.txt with their paths).
• data/newsgroup: Text-source (from the News Group dataset). This can be subsituted with any text file. Look inside text_utils.py to see how the text inside this file is used by the renderer.
• data/models/colors_new.cp: Color-model (foreground/background text color model), learnt from the IIIT-5K word dataset.
• data/models: Other cPickle files (char_freq.cp: frequency of each character in the text dataset; font_px2pt.cp: conversion from pt to px for various fonts: If you add a new font, make sure that the corresponding model is present in this file, if not you can add it by adapting invert_font_size.py).

This script will generate random scene-text image samples and store them in an h5 file in results/SynthText.h5. If the --viz option is specified, the generated output will be visualized as the script is being run; omit the --viz option to turn-off the visualizations. If you want to visualize the results stored in results/SynthText.h5 later, run:

python visualize_results.py

Pre-generated Dataset

A dataset with approximately 800000 synthetic scene-text images generated with this code can be found here.

[update] Adding New Images

Segmentation and depth-maps are required to use new images as background. Sample scripts for obtaining these are available here.

• predict_depth.m MATLAB script to regress a depth mask for a given RGB image; uses the network of Liu etal. However, more recent works (e.g., this) might give better results.
• run_ucm.m and floodFill.py for getting segmentation masks using gPb-UCM.

For an explanation of the fields in dset.h5 (e.g.: seg,area,label), please check this comment.

Further Information

Please refer to the paper for more information, or contact me (email address in the paper).

synthtext_chinese_version's Issues

Font with shadow

Hi，I want to create the text without shadow， how can I do？Thanks！

opencv error

Traceback (most recent call last):
File "gen.py", line 224, in
main(args.viz)
File "gen.py", line 131, in main
RV3 = RendererV3(DATA_PATH,max_time=SECS_PER_IMG)
File "/Users/zhangshihao/SynthText_Chinese_version/synthgen.py", line 370, in init
self.colorizer = Colorize(data_dir)
File "/Users/zhangshihao/SynthText_Chinese_version/colorize3_poisson.py", line 145, in init
self.font_color = FontColor(col_file=osp.join(model_dir,'models/colors_new.cp'))
File "/Users/zhangshihao/SynthText_Chinese_version/colorize3_poisson.py", line 54, in init
self.colorsLAB = np.squeeze(cv.cvtColor(self.colorsLAB[None,:,:],cv.cv.CV_RGB2Lab))
AttributeError: 'module' object has no attribute 'cv'

What should i do if i don't want to install opencv 2.0? Or i have to install opencv2.0?

how can i solve this problem

File "gen.py", line 142, in main
img_resize=img.resize(db['depth'][imname].shape)
File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 1745, in resize
return self._new(self.im.resize(size, resample, box))
TypeError: argument 1 must be sequence of length 2, not 3

wordBB problem

aaaaa

valueError

Hi, I got a problem when I try to run add_more_data.py.

/home/deeplearn/anaconda2/lib/python2.7/site-packages/h5py/_hl/files.pyc in make_fid(name, mode, userblock_size, fapl, fcpl, swmr)
117 fid = h5f.create(name, h5f.ACC_EXCL, fapl=fapl, fcpl=fcpl)
118 else:
--> 119 raise ValueError("Invalid mode; must be one of r, r+, w, w-, x, a")
120
121 try:

ValueError: Invalid mode; must be one of r, r+, w, w-, x, a

Can you give me some advice, much appreciated.

several questions

Hi, thanks a lot for your great job!
I can use your code now, but when I run your code, an error still happened.
File "gen.py", line 143, in main
img_resize=img.resize(db['depth'][imname].shape)
File "/usr/local/lib/python2.7/dist-packages/PIL/Image.py", line 1745, in resize
return self._new(self.im.resize(size, resample, box))
TypeError: must be sequence of length 2, not 3
So,I change a little, img_resize=img.resize(db['depth'][imname].shape) to
img_resize=img.resize((db['depth'][imname].shape[1], db['depth'][imname].shape[2]))

More, after a long time, I only got few images. And "Timed out" occurred frequently. Why?
Could you give me some suggestions?

The number of characters in the sample is not uniform

I generated 300000 samples. But I found two problems. (1)The number of occurrences of characters have a big difference between samples. The biggest number of occurrences of characters is 25 times the smallest number of occurrences of characters. (2)So many characters have a small number of occurrences. It accounted for half of the overall.
So, The number of characters in the sample is not uniform. Can you give me some suggestion?

These scripts are coded on Python2.7, anyone run on python3 and windows?

These scripts are coded on Python2.7, have anyone run the code on python3(like python3.5), and on windows?
I debuged for a long time and still have Two promblems.

请问作者如何做中文样本

我下载一万张图片，随机截取作为背景，随机字体，随机大小，随机颜色，做到这些够了吗？请问作者该怎么做样本

UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position 0

When I run gen.py on Windows, this error occurs：UnicodeDecodeError: 'utf8' codec can't decode byte 0xe8 in position 0。

Do you know how to solve it？

The text is mirrored.

Hi, Lee

I used this code for Chinese text generation and followed the readme to update some files. All things seem ok, but I find some text is mirrored. What's more, they often appear in the same position on certain bg-img. Do you have some ideas about this?

Which gen.py should I run?

Hi, there. You've done really great work! Still there are several gen.py-like python files in the root dic. May I ask which one can generate Chinese text in my own picture? It's gen.py or generateDataNew.py or gen_cartoon.py and what's the function of them separately? Would you help me find this out？Any reply is genuinely appreciated.

i want use generateDataNew.py to generate some chinese txt picture,but there has some data has not? how to solve this problem?

How to generate char_freq.cp and font_px2pt.cp

Is there any scripts to generate these file?
thanks for any help!

I found a problem and fix it about don't support chinese space, so don't need time_limit

text_utils.py: lines[i] = lines[i][:len(lines[i])-lines[i][::-1].find(' ')].strip() , don't support chinese space, add one line : lines[i] = lines[i][:len(lines[i])-lines[i][::-1].find(u'\u3000')].strip().

Hi i have a problem

when i run gen.py, i have problem ,my opencv version is 2.4.8,i don't know how to solve

wit@wit:~/SynthText_Chinese_version-master$ python gen.py
getting data..
-> done
Storing the output in: results/SynthText_cartoon_viz.h5
data/newsgroup/安徒生童话故事.txt
data/newsgroup/男生贾里-秦文君.txt
data/newsgroup/节日集会朗诵诗选.txt
data/newsgroup/小时候就在想的事.txt
data/newsgroup/小灵通漫游未来.txt
data/newsgroup/一千零一夜.txt
data/newsgroup/**童话百篇.txt
data/newsgroup/浪漫鼠德佩罗.TXT
data/newsgroup/猫咪分局.txt
20551
Traceback (most recent call last):
File "gen.py", line 223, in
main(args.viz)
File "gen.py", line 130, in main
RV3 = RendererV3(DATA_PATH,max_time=SECS_PER_IMG)
File "/home/wit/SynthText_Chinese_version-master/synthgen.py", line 370, in init
self.colorizer = Colorize(data_dir)
File "/home/wit/SynthText_Chinese_version-master/colorize3_poisson.py", line 147, in init
self.font_color = FontColor(col_file=osp.join(model_dir,'models/colors_new.cp'))
File "/home/wit/SynthText_Chinese_version-master/colorize3_poisson.py", line 56, in init
self.colorsLAB = np.squeeze(cv.cvtColor(self.colorsLAB[None,:,:],cv.cv.CV_RGB2Lab))
AttributeError: 'module' object has no attribute 'cv'

IOError: Unable to open file (Truncated file: eof = 15030288384, sblock->base_addr = 0, stored_eoa = 15807826032)

您好，
我下载了bg_img,seg.h5,depth.h5 然后运行add_more_data.py,就报错了
Traceback (most recent call last):
File "add_more_data.py", line 73, in
add_more_data_into_dset(DB_FNAME,more_img_file_path,more_depth_path,more_seg_path)
File "add_more_data.py", line 39, in add_more_data_into_dset
depth_db=get_data(more_depth_path)
File "add_more_data.py", line 31, in get_data
return h5py.File(DB_FNAME,'r')
File "/home/zhengjingqing/anaconda2/lib/python2.7/site-packages/h5py/_hl/files.py", line 260, in init
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/home/zhengjingqing/anaconda2/lib/python2.7/site-packages/h5py/_hl/files.py", line 89, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2579)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/home/ilan/minonda/conda-bld/work/h5py/_objects.c:2538)
File "h5py/h5f.pyx", line 76, in h5py.h5f.open (/home/ilan/minonda/conda-bld/work/h5py/h5f.c:1816)
IOError: Unable to open file (Truncated file: eof = 15030288384, sblock->base_addr = 0, stored_eoa = 15807826032)

请问支持python3吗？

你fork的那个版本，不支持python3.

How to speed up the generation of new images？

When generating texts in images, it takes severals second for each image and there are several warinings of "Times Out".
so is there any advice?

** instance # : 0
sample_output sample_para_output None
[]
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output sample_para_output None
[]
*** Timed out!

Bangla SynthText issue

Hi I downloaded 5 Bangla Fonts from http://www.freebanglafont.com/download.php?id=752 and appended them in fontlist including Ubuntu fonts you have provdied

Added lines like following in newsgroup.txt:-

বিয়ে বাড়ীতে হই চই সেগুন মেলার ফার্ণিচার কই
কালেকশন পাঞ্জাবী স্কুল ড্রেস বেডসীট ২৫৪১-৪২৮৪
অঞ্জনা ### ফ্যান্সী শাড়ী চুড়িদার পিস
এখানে সমস্ত রকমের মূদি ও
ছাত্রসাথী উপহার ও লিখন সামগ্রী ### ### ###
বিশ্বকবি রবীন্দ্রনাথ
রণে এই তোরণটি নির্মিত দমদম পৌরসভা
এম. বি. রোড পূব পাড়া মোড়
অনামিকা টেলার্স জেন্টস্ স্পেশালিষ্ট
২০ এম বি. রোড পূবপাড়া মোড়
ইন্দু ডেকরেটার্স
সাম্প্রদায়িক সম্প্রীতি রক্ষায় সুন্দর হোক
রকমের সরঞ্জাম ভাড়া দেওয়া হয়। বেলঘরিয়া কোল- ৭০০০৮৩
বিয়ে বাড়ীূূতে হই চই ### ### সেগুন মেলার ফার্ণিচার কই মা
দেওয়া ২ বিভিন্ন অনুষ্ঠানে বাড়ী ভাড়া মঙ্গলদীপ হয়। এম. রোড কোল- বি. ৪৯
সেগুন শ্রী মা মনসা'র
বিউটি প্লাজা
সেগুন মেলা
শারদোৎসবে বিউটি প্লাজার নিবেদন লেডিস স্পা জংশন
উপলক্ষে বিশাল অফার সব সময়ের জন্য ধনতেরাস দি সুভাষ জুয়েলার্স ৮৪/১ ফিডার ###

Created char_freq.cp using following code(reding utf-8, should I also dump cpickle file in utf-8? although as per this https://stackoverflow.com/questions/32957708/python-pickle-error-unicodedecodeerror/32957930 I just saved in raw binary and unpickling in raw binary "rb" as against default "r" in text_utils.py):-

from collections import Counter
import cPickle as cp
import pickle;
import numpy as np
import codecs

def normalize(d, target=1.0):
   raw = sum(d.values())
   factor = target/raw
   return {key:value*factor for key,value in d.items()}# iteritems for  python 2.7

filename ='/home/rohit/SceneTextDataset/SynthText/code/SynthText/data/newsgroup/newsgroup.txt'
with codecs.open(filename, encoding='utf-8') as f:
	c = Counter()
	for x in f:
		c += Counter(x.strip())
print(c)
d = dict(c)
print(d,sum(d.values()))
d = normalize(d)
print(d,sum(d.values()))

with open("char_freq.cp",'wb') as f:
	cp.dump(d,f)

And wrote font_px2pt.cp using invert_font_size.py

I also modified line 516 in text_utils.py as "with codecs.open(fn, encoding='utf-8') as f:" instead of " with open(fn, 'r') as f:"

Now when I run $ python gen.py --viz, I get following warnings and get images with font not rendered properly:-

$ python gen.py --viz
getting data..
	-> done
Storing the output in: results/SynthText.h5
0 of 4
 ** instance # : 0
/home/rohit/SceneTextDataset/SynthText/code/SynthText/text_utils.py:447: RuntimeWarning: divide by zero encountered in true_divide
  r = np.abs(sizes[:,1]/sizes[:,0]) # width/height
/home/rohit/SceneTextDataset/SynthText/code/SynthText/text_utils.py:447: RuntimeWarning: invalid value encountered in true_divide
  r = np.abs(sizes[:,1]/sizes[:,0]) # width/height
/home/rohit/SceneTextDataset/SynthText/code/SynthText/synthgen.py:352: MatplotlibDeprecationWarning: pyplot.hold is deprecated.
    Future behavior will be consistent with the long-time default:
    plot commands add elements without first clearing the
    Axes and/or Figure.
  plt.hold(True)
/usr/local/lib/python2.7/dist-packages/matplotlib/__init__.py:917: UserWarning: axes.hold is deprecated. Please remove it from your matplotlibrc and/or style files.
  warnings.warn(self.msg_depr_set % key)
/usr/local/lib/python2.7/dist-packages/matplotlib/rcsetup.py:152: UserWarning: axes.hold is deprecated, will be removed in 3.0
  warnings.warn("axes.hold is deprecated, will be removed in 3.0")
Traceback (most recent call last):
  File "gen.py", line 122, in main
    add_res_to_db(imname,res,out_db)
  File "gen.py", line 73, in add_res_to_db
    db['data'][dname].attrs['txt'] = res[i]['txt']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 171, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1611, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1633, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1694, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U19')
>>>> CONTINUING....
1 of 4
 ** instance # : 0
Traceback (most recent call last):
  File "gen.py", line 122, in main
    add_res_to_db(imname,res,out_db)
  File "gen.py", line 73, in add_res_to_db
    db['data'][dname].attrs['txt'] = res[i]['txt']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 171, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1611, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1633, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1694, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U22')
>>>> CONTINUING....
2 of 4
 ** instance # : 0
Traceback (most recent call last):
  File "gen.py", line 122, in main
    add_res_to_db(imname,res,out_db)
  File "gen.py", line 73, in add_res_to_db
    db['data'][dname].attrs['txt'] = res[i]['txt']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 171, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1611, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1633, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1694, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U17')
>>>> CONTINUING....
3 of 4
 ** instance # : 0
Traceback (most recent call last):
  File "gen.py", line 122, in main
    add_res_to_db(imname,res,out_db)
  File "gen.py", line 73, in add_res_to_db
    db['data'][dname].attrs['txt'] = res[i]['txt']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 171, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1611, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1633, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1694, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U23')
>>>> CONTINUING....
4 of 4
 ** instance # : 0
Traceback (most recent call last):
  File "gen.py", line 122, in main
    add_res_to_db(imname,res,out_db)
  File "gen.py", line 73, in add_res_to_db
    db['data'][dname].attrs['txt'] = res[i]['txt']
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 95, in __setitem__
    self.create(name, data=value, dtype=base.guess_dtype(value))
  File "/home/rohit/.local/lib/python2.7/site-packages/h5py/_hl/attrs.py", line 171, in create
    htype = h5t.py_create(original_dtype, logical=True)
  File "h5py/h5t.pyx", line 1611, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1633, in h5py.h5t.py_create
  File "h5py/h5t.pyx", line 1694, in h5py.h5t.py_create
TypeError: No conversion path for dtype: dtype('<U17')
>>>> CONTINUING....

Here are the images:-

Please help resolving it.

用自己的深度图合成

你好，我想使用已有的深度图（有相机坐标的三维信息和平面图的信息和H矩阵），请问怎么把字合成上去呀

when I run python gen.py --viz

It returns a error:
Traceback (most recent call last):
File "gen.py", line 142, in main
img_resize=img.resize(db['depth'][imname].shape)
File "/home/ray/anaconda2/lib/python2.7/site-packages/PIL/Image.py", line 1529, in resize
return self._new(self.im.resize(size, resample))
TypeError: argument 1 must be sequence of length 2, not 3

Would you help me to solve the problem

Error using dir Too many input arguments.

thank s for sharing your code ,can you tell me ,how to solve theses problems?
1、
Error using dir
Too many input arguments.

Error in predict_depth (line 43)
imnames = dir(fullfile(opts.imdir),'*');
2、
Error using imread (line 349)
File "/home/user/demo/img/._tmp.png" does not exist.

Error in read_img_rgb (line 17)
[img_data, map] = imread(tmp_img_file);

Error in predict_depth (line 50)
img = read_img_rgb(fullfile(opts.imdir,imname));

合成样本时有些汉字生成不出来

您好，我在合成中文样本时，出现这样一个问题，就是有的汉字是合成不出来的，例如：“玥”
假如要合成的文本行是“宇文玥宇文玥”，生成的图像里会发现“玥”这个汉字是缺少的了，有的是直接跳过生成这个汉字了，变成“宇文宇文”，有的是会出现空格，变成“宇文 宇文 ”，请问您知道这是什么问题吗？
非常感谢！

未定义函数或变量 'read_img_rgb'。

can you provide the function for read_img_rgb, I got this error when I tried to run predict_depth.m
thanks a lot

Nothing in pictures

hi ，i run the gen.py --viz.
but i got picture which only got some box not chars and no chinese，so，how to solve this problem

ValueError: conversion from RGB to HSV not supported

...
[ 2.0680037 5.5457587]
[ 2.0680037 5.5474877]
[ 2.0680037 5.577759 ]]

[[ 1.8009567 3.3199155]
[ 1.8009567 3.3199134]
[ 1.8009567 3.3199053]
...
[ 2.0680037 5.545406 ]
[ 2.0680037 5.546324 ]
[ 2.0680037 5.577694 ]]]
depth max min 81.71362 1.3432088
Traceback (most recent call last):
File "gen.py", line 153, in main
hsv_img=np.array(rgb2hsv(img_resize))
File "gen.py", line 102, in rgb2hsv
return image.convert('HSV')
File "/home/yangliu/anaconda2/envs/python27/lib/python2.7/site-packages/PIL/Image.py", line 702, in convert
im = im.convert(mode, dither)
ValueError: conversion from RGB to HSV not supported

when i run python gen.py --viz
I have met this problem. How should I deal with it?Thanks a lot~
(ps:ubuntu16.04 + opencv2.4.9 +python2.7 + PIL 1.1.7)

add new images to depth

Hello,
When I run the program, I encounter the following problems

Who encountered a similar problem, how to solve?
@JarveeLee
Can predict_depth.m be used directly?

The chinese character become frame in the picture

My operate system is Ubuntu 14.0, python 2.7, opencv 2.4.9

I fix some bug, it succefully run, but the result like the picture below

Could anyone tell me what's wrong and how to fix it? Or maybe this problem someone solve it before,
Could this person tell me the reason ?
Thank you !

Undefined function or variable 'read_img_rgb'.

when run predict_depth.m script got the error:
Warning: Name is nonexistent or not a directory: /root/fayao-dcnf-fcsp/libs/matconvnet_20141015/matlab/mex

In path (line 109)
In addpath (line 88)
In vl_setupnn (line 13)
In run (line 96)
In predict_depth (line 17)
loading trained model...

** No GPU found. Using CPU...
1 of 7
Undefined function or variable 'read_img_rgb'.

Error in predict_depth (line 50)
img = read_img_rgb(fullfile(opts.imdir,imname));
how to @@Solve this problem @JarveeLee
thank you!

Hi i have a problem

when i run gen.py, i have problem ,my opencv version is 2.4.8,i don't know how to solve

wit@wit:~/SynthText_Chinese_version-master$ python gen.py
getting data..
-> done
Storing the output in: results/SynthText_cartoon_viz.h5
data/newsgroup/安徒生童话故事.txt
data/newsgroup/男生贾里-秦文君.txt
data/newsgroup/节日集会朗诵诗选.txt
data/newsgroup/小时候就在想的事.txt
data/newsgroup/小灵通漫游未来.txt
data/newsgroup/一千零一夜.txt
data/newsgroup/**童话百篇.txt
data/newsgroup/浪漫鼠德佩罗.TXT
data/newsgroup/猫咪分局.txt
20551
Traceback (most recent call last):
File "gen.py", line 223, in
main(args.viz)
File "gen.py", line 130, in main
RV3 = RendererV3(DATA_PATH,max_time=SECS_PER_IMG)
File "/home/wit/SynthText_Chinese_version-master/synthgen.py", line 370, in init
self.colorizer = Colorize(data_dir)
File "/home/wit/SynthText_Chinese_version-master/colorize3_poisson.py", line 147, in init
self.font_color = FontColor(col_file=osp.join(model_dir,'models/colors_new.cp'))
File "/home/wit/SynthText_Chinese_version-master/colorize3_poisson.py", line 56, in init
self.colorsLAB = np.squeeze(cv.cvtColor(self.colorsLAB[None,:,:],cv.cv.CV_RGB2Lab))
AttributeError: 'module' object has no attribute 'cv'

cannot find the font

When I run the gen.py, the following error occurred:

Traceback (most recent call last):
File "/Users/humeng/Downloads/work/SynthText_Chinese_version/synthgen.py", line 665, in render_text
regions['homography_inv'][ireg])
File "/Users/humeng/Downloads/work/SynthText_Chinese_version/synthgen.py", line 502, in place_text
render_res = self.text_renderer.render_sample(font,collision_mask)
File "/Users/humeng/Downloads/work/SynthText_Chinese_version/text_utils.py", line 357, in render_sample
f_h = self.font_state.get_font_size(font, f_h_px)
File "/Users/humeng/Downloads/work/SynthText_Chinese_version/text_utils.py", line 477, in get_font_size
m = self.font_model[font.name]
KeyError: 'FZXiHei I-Z08S'

And the same error occurred for the other Chinese fonts.

For help how to encode and decode another languages which are different from Chinese and English such as Arabic！

Hello ,friends .Thank you for your good job .Can you share me more experience for encode and decode Chinese or another languages like Arabic, @JarveeLee

怎样生成固定字符数量的样本

你好，我想生成固定字符数的样本，比如“我很开心啊”这样固定5个字符的样本图片，应该修改哪个参数呢

Can't put chinese text in the picture

After running the program, I fail to see any chinese texts except bounding_box in the picture. But when the program produced English words or paragraphs, everything is normal. I have read the source code and traced back to the text_utils.py , but have not figured out the problems. Maybe the problem about fonts. @JarveeLee

generate more pictures

how do you do，there are only 4 pictures in this code ，so i wonder how could i generate more pictures here。

about dataset

Now I am anxious to use some image datasets generated in Chinese, but I have not reproduced your papers. Can I have a similar dataset available for me to use online?

For help how to run the code.

Hi, JarveeLee, when I run "python gen.py --viz". The following tips were prompt, I don't know why,please help me, thank you!

Traceback (most recent call last):
File "gen.py", line 22, in
import wget, tarfile
ImportError: No module named wget

someting error when run the python gen.py --viz

Thanks for your great job!
As described in the title, when i ran the demo with error below:

"../SynthText_Chinese_version/synthgen.py", line 623, in render_text
    regions = self.filter_for_placement(xyz,seg,regions)
……
TypeError: 'numpy.float64' object cannot be interpreted as an index

I changed the code from "xyz = su.DepthCamera.depth2xyz(depth)" to "xyz = su.DepthCamera.depth2xyz(depth).astype(int)" with error:

 File "../SynthText_Chinese_version/synthgen.py", line 617, in render_text
    regions = TextRegions.get_regions(xyz,seg,area,label)
……
    assert np.abs(np.linalg.norm(coeffs[:3])-1) < 1e-3

@JarveeLee do you have some suggestion?
@Oliver-Q did you encounter the problem when debug the demo? (I saw your good show in the blog! Great job! )

请问代码在GPU环境下能加速么？还是仅仅需要CPU运行？

未定义函数或变量 'gen_supperpixel_info'。

loading trained model...

1 of 13

super-pix
未定义函数或变量 'gen_supperpixel_info'。

出错 predict_depth>get_depth (line 82)
supix = gen_supperpixel_info(im_rgb, opts.sp_size);

出错 predict_depth (line 59)
depth = get_depth(img,model.(imtype),opts);

plz....~

how to input my own picture and output the result of detecting

I want to give the code one picture of my own, and let is show me the detecting result, but I don't know how, looking forward to your reply!
Thanks very much!

NameError: global name 'get_data' is not defined

It seems that 'get_data' is a function but not imported in <add_more_data.py>?And the same situation as the object 'Image'?
@JarveeLee

Chinese character is shown as a blank grid

conversion from RGB to HSV not supported

Hello, I encounter the following problems when I run the program
Traceback (most recent call last):
File "/home/soulie/project/SynthText_Chinese_version/gen.py", line 151, in main
hsv_img=np.array(rgb2hsv(img_resize))
File "/home/soulie/project/SynthText_Chinese_version/gen.py", line 100, in rgb2hsv
return image.convert('HSV')
File "/usr/lib/python2.7/dist-packages/PIL/Image.py", line 740, in convert
im = im.convert(mode, dither)
ValueError: conversion from RGB to HSV not supported
could you help me,thanks!

请问你创造样本的思路时什么

请问你创造样本的思路时什么

'NoneType' object is not subscriptable

I am running code with python3.5 and getting error
File "gen.py", line 90, in main
imnames = sorted(db['image'].keys())
TypeError: 'NoneType' object is not subscriptable

Issue when calling python gen.py --viz

Here's some example output

1 of 4
** instance # : 0
sample_output sample_para_output None
[]
sample_para_output [u'\uff33\u732a\u2462']
before the if judge Ｓ猪③
pass the text filter
pass in mask array size
text in synthgen.py/place_text to return Ｓ猪③
text in synthgen.py/render_text append into itext Ｓ猪③
sample_output sample_para_output None
[]
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output sample_para_output None
[]
sample_para_output [u'\u5feb\u5feb\u53d7\u6b7b\u5427\uff01\u201d', u'\u7eed\u4e86\u597d\u591a\u5929\u3002']
before the if judge 快快受死吧！”
续了好多天。
pass the text filter
pass in mask array size
sample_output sample_para_output [u'\u6240\u4ee5\u5b83\u5c31\u5f97\u5230\u4e86\u516c\u4e3b\u3002', u'\u201c\u518d\u4f1a\uff0c\u201d\u516c\u4e3b\u56de\u7b54\u8bf4\u3002']
所以它就得到了公主。
“再会，”公主回答说。
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output sample_para_output None
[]
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output sample_para_output [u'\u886c\u886b\u9886\u5b50', u'\u8e2a\u4e86\u3002\u201d']
衬衫领子
踪了。”
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output sample_para_output [u'&\u4e0b&\u4e66_']
&下&书_
sample_para_output [u'\u4e86\u3002']
before the if judge 了。
pass the text filter
pass in mask array size
text in synthgen.py/place_text to return 了。
text in synthgen.py/render_text append into itext 了。
sample_output *** Timed out!
sample_output *** Timed out!
sample_output sample_para_output None
[]
sample_para_output [u'\u6728\u9e21\u517b\u9053']
before the if judge 木鸡养道
pass the text filter
pass in mask array size
text in synthgen.py/place_text to return 木鸡养道
text in synthgen.py/render_text append into itext 木鸡养道
sample_output sample_para_output None
[]
*** Timed out!
sample_output sample_para_output [u'\u5b89\u7136\u7761\u5230\u5929\u660e\u3002']
安然睡到天明。
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output sample_para_output None
[]
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output sample_para_output None
[]
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output sample_para_output [u'\u52a8\u5730\u8bf4\uff1a']
动地说：
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output sample_para_output [u'\u4f60\u4eec\u3002\u201d']
你们。”
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output sample_para_output None
[]
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output *** Timed out!
sample_output *** Timed out!
sample_output sample_para_output [u'\u70e6\u607c\u3002']
烦恼。
sample_para_output None
before the if judge []
didn't pass because of len(text)==0
sample_output sample_para_output [u'\u3000\u3000\u5bf9\u4e86\u3002']
对了。
sample_para_output [u'\u4e0b{ \u4e66']
before the if judge 下{ 书
pass the text filter
pass in mask array size
text in synthgen.py/place_text to return 下{ 书
text in synthgen.py/render_text append into itext 下{ 书
sample_output *** Timed out!
[u'\uff33\u732a\u2462', u'\u4e86\u3002', u'\u6728\u9e21\u517b\u9053', u'\u4e0b{ \u4e66']
Traceback (most recent call last):
File "gen.py", line 192, in main
ninstance=INSTANCE_PER_IMAGE,viz=viz)
File "/home/pothole/text_recognition/SynthText_Chinese_version/synthgen.py", line 693, in render_text
viz_textbb(1,img, [idict['wordBB']], alpha=1.0)
File "/home/pothole/text_recognition/SynthText_Chinese_version/synthgen.py", line 351, in viz_textbb
plt.figure(fignum)
File "/usr/local/lib/python2.7/dist-packages/matplotlib/pyplot.py", line 535, in figure
**kwargs)
File "/usr/local/lib/python2.7/dist-packages/matplotlib/backends/backend_tkagg.py", line 81, in new_figure_manager
return new_figure_manager_given_figure(num, figure)
File "/usr/local/lib/python2.7/dist-packages/matplotlib/backends/backend_tkagg.py", line 89, in new_figure_manager_given_figure
window = Tk.Tk()
File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1767, in init
self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
TclError: no display name and no $DISPLAY environment variable

CONTINUING....
depth shape,img shape (446, 600, 2) (743, 1000, 3)
depth info [[[ 2.28360033 81.91452789]
[ 2.28360033 81.91452789]
[ 2.28360033 81.91451263]
...,
[ 2.09834814 81.91157532]
[ 2.09834814 81.91158295]
[ 2.09834814 81.91158295]]

[[ 2.28360033 81.91453552]
[ 2.28360033 81.91452789]
[ 2.28360033 81.91452026]
...,
[ 2.09834814 81.91158295]
[ 2.09834814 81.91158295]
[ 2.09834814 81.91159058]]

[[ 2.28360033 81.91454315]
[ 2.28360033 81.91454315]
[ 2.28360033 81.91453552]
...,
[ 2.09834814 81.91159821]
[ 2.09834814 81.91159058]
[ 2.09834814 81.91159058]]

...,
[[ 1.34355438 6.31256342]
[ 1.34355438 6.31259346]
[ 1.34355438 6.31277084]
...,
[ 1.44243908 5.58638668]
[ 1.44243908 5.58641577]
[ 1.44243908 5.5864315 ]]

[[ 1.34355438 6.3125391 ]
[ 1.34355438 6.31220341]
[ 1.34355438 6.31213331]
...,
[ 1.44243908 5.58516884]
[ 1.44243908 5.58522034]
[ 1.44243908 5.58521461]]

[[ 1.34355438 6.31251287]
[ 1.34355438 6.31235409]
[ 1.34355438 6.3122406 ]
...,
[ 1.44243908 5.58518744]
[ 1.44243908 5.58520794]
[ 1.44243908 5.58521891]]]
depth max min 81.9185 0.7133
hsv_img_shape (446, 600, 3)
H_channel (600, 446) [[ 255. 255. 255. ..., 230. 236. 234.]
[ 255. 255. 255. ..., 233. 234. 235.]
[ 255. 255. 255. ..., 232. 235. 234.]
...,
[ 255. 255. 255. ..., 251. 253. 251.]
[ 255. 255. 255. ..., 253. 253. 251.]
[ 255. 255. 255. ..., 252. 252. 250.]]
H_max min 255.0 11.0
scale 0.321249
depth shape (446, 600, 2)
depth max min 81.9185 0.7133
seg info (479, 647) (198,) (198,)

JarveeLee

@JarveeLee Thank you for your perfect job。When I run the program, the picture shows only a box, no words, what is the reason for this? like this picture below:

thank you

Time out, problem. Only get frame on picture not text

Traceback (most recent call last):
File "/home/zju/lkj/SynthText_Chinese_version/synthgen.py", line 668, in render_text
regions['homography_inv'][ireg])
File "/home/zju/lkj/SynthText_Chinese_version/synthgen.py", line 507, in place_text
render_res = self.text_renderer.render_sample(font, collision_mask)
File "/home/zju/lkj/SynthText_Chinese_version/text_utils.py", line 371, in render_sample
text = self.text_source.sample(nline,nchar,text_type)
File "/home/zju/lkj/SynthText_Chinese_version/text_utils.py", line 661, in sample
print('sample_output',self.fdictkind)
File "/home/zju/lkj/SynthText_Chinese_version/text_utils.py", line 710, in sample_para
lines = self.get_lines(nline, nword, nchar_max, f=0.35)
File "/home/zju/lkj/SynthText_Chinese_version/text_utils.py", line 649, in get_lines
if not np.any([ch.isspace() for ch in lines[i]]):
File "/home/zju/lkj/SynthText_Chinese_version/text_utils.py", line 649, in
if not np.any([ch.isspace() for ch in lines[i]]):
File "/home/zju/lkj/SynthText_Chinese_version/common.py", line 38, in signal_handler
raise(TimeoutException, colorize(Color.RED, " *** Timed out!", highlight=True))
TypeError: exceptions must derive from BaseException