GithubHelp home page GithubHelp logo

sunpinyin / sunpinyin Goto Github PK

View Code? Open in Web Editor NEW
495.0 45.0 125.0 3.53 MB

A statistical language model based Chinese input method

Home Page: http://sunpinyin.org

License: Other

Python 6.98% C++ 51.49% C 31.20% Objective-C 5.53% Objective-C++ 2.77% Makefile 0.18% CMake 0.97% Shell 0.06% Rich Text Format 0.14% Cython 0.68%

sunpinyin's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sunpinyin's Issues

Pressing a key more than 511 times, an error occurs.

Pressing a key more than 511 times, an error occurs.
You can reproduce in fcitx.

hodong@debian:~$ dpkg --list | grep sunpinyin
ii  fcitx-sunpinyin                       0.4.1-1                              amd64        fcitx wrapper for Sunpinyin IM engine
ii  libsunpinyin-dev:amd64                2.0.3+git20140127-1                  amd64        Simplified Chinese Input Method from SUN (development)
ii  libsunpinyin3:amd64                   2.0.3+git20140127-1                  amd64        Simplified Chinese Input Method from SUN (runtime)
ii  sunpinyin-data                        0.1.22+20131212-1                    amd64        Statistical language model data from open-gram

test code:

void
AimWinHandler::updatePreedit(const IPreeditString* ppd)
{
  if (ppd)
    g_message ("ppd->size() : %d", ppd->size());
}

result:

aim-Message: ppd->size() : 1
aim-Message: ppd->size() : 3
aim-Message: ppd->size() : 5
aim-Message: ppd->size() : 7
...
aim-Message: ppd->size() : 1013
aim-Message: ppd->size() : 1015
aim-Message: ppd->size() : 1017
aim-Message: ppd->size() : 1019
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

key nv is wrong in shuangpin ms2003

In ms2003 shuangpin scheme, y should be v, like ny should be .
But now, nv is .

I use Ubuntu 18.04, plasmashell 5.12.6, fcitx Version: 1:4.2.9.6-1, fcitx-sunpinyin Version: 0.4.2-1.

failed to save preferences

env:
ubuntu 18.04,
ibus-sunpinyin version: 2.0.3+git20181120-1
ibus version: 1.5.17-3ubuntu4

open preference, enable "[/]" on pagedown, then Appply, following error happened

Traceback (most recent call last):
  File "/usr/share//ibus-sunpinyin/setup/main.py", line 603, in on_main_ok_clicked
    self.__write_config()
  File "/usr/share//ibus-sunpinyin/setup/main.py", line 559, in __write_config
    opt.write_config()
  File "/usr/share//ibus-sunpinyin/setup/main.py", line 118, in write_config
    self.write(v)
  File "/usr/share//ibus-sunpinyin/setup/main.py", line 83, in write
    return self.config.set_value(section, key, type(self.default)(v))
  File "/usr/lib/python3/dist-packages/gi/overrides/IBus.py", line 87, in set_value
    super(Config, self).set_value(section, name, value)
TypeError: argument value: Expected GLib.Variant, but got int

Can't open preference in mint mate 18

Previous version of mint & ubuntu also has this issue, but could be fixed via sudo apt-get install python-ibus.

But in mint mate 18, which is based on ubuntu 16.04.1, there is no python-ibus package, here is the error tip: E: Unable to locate package python-ibus.

So, how to fix that?

当运行 ./sunpinyin-dictgen 时, 提示没有权限.

无法下载词库.

wget 'http://open-gram.googlecode.com/git/'/lm_sc.3gm.arpa-20140820.tar.bz2
--2014-09-06 23:12:52--  http://open-gram.googlecode.com/git//lm_sc.3gm.arpa-20140820.tar.bz2
Connecting to 127.0.0.1:8087... connected.
Proxy request sent, awaiting response... 403 
2014-09-06 23:12:54 ERROR 403: (no description).

make: *** [lm_sc.3gm.arpa-20140820.tar.bz2] Error 8

sun-pinyin MacOS-2.0.4.1104 cannot input anything on OS X 10.10 beta

What steps will reproduce the problem?

  1. Install sun-pinyin on OS X 10.10 public beta
  2. Select sun-pinyin
  3. Input

What is the expected output? What do you see instead?
Input text

What version of the product are you using? On what operating system?
Nothing inputed on OS X 10.10 public beta

有关 `帧' 的发音的问题.

我使用 Fcitx 当中的 sunpinyin 模块.

默认的 Fcitx 输入法, 帧 发因为 zhen', 而 sunpinyin 发音为zheng'.

我想一般绝大多数人都是发音做 zhen 的吧.

FTBFS with gcc-4.7

This is due to not include unistd.h praporly.

This is the patch

Index: sunpinyin/SConstruct

--- sunpinyin.orig/SConstruct 2012-04-04 16:57:33.000000000 +0800
+++ sunpinyin/SConstruct 2012-04-04 17:00:28.016700927 +0800
@@ -313,7 +313,8 @@
conf.CheckCHeader('sys/param.h')
conf.CheckCHeader('sys/stat.h')
conf.CheckCHeader('sys/types.h')

  • conf.CheckCHeader('unistd.h')

  • if conf.CheckCHeader('unistd.h'):

  •    conf.Define('DHAVE_UNISTD_H', 1)
    

    conf.CheckCHeader('wchar.h')

    add essential package requirements

    Index: sunpinyin/src/portability.h

    --- sunpinyin.orig/src/portability.h 2012-04-04 16:57:33.000000000 +0800
    +++ sunpinyin/src/portability.h 2012-04-04 16:59:39.476460231 +0800
    @@ -44,14 +44,14 @@

    include

    include

-#if defined(sun)
+#if defined(HAVE_UNISTD_H)
#include <unistd.h>
#ifdef __cplusplus
#include
#else
#include <sys/ddi.h>
#endif //__cpluscplus
-#endif //defined(sun)
+#endif //defined(HAVE_UNISTD_H)

#ifndef HOST_OS_GNUC_2

transplant into iOS

How to transplant this project to use in the iOS as custom keyboard?
Any hints or guidelines?
Thanks

github 版 的 sunpinyin 的拼音输入的一个问题.

新版本的 sunpinyin, 在输入 省略 的时候, 我(一般人都会吧) , 输入为 shenglue, 但是会输出
生路恶, 只有输入 shenglve 才有效.

请问, 这是有意为之吗? 这给我造成了很大的困扰!! 其他输入法(包括早期版本都没有这个问题).

ibus-sunpinyin can't work fine with ibus 1.4.99

The config tool can't be opened .

linux@linux:~> /usr/lib/ibus-sunpinyin/ibus-setup-sunpinyin

** (process:4196): WARNING **: Trying to register gtype 'GMountMountFlags' as enum when in fact it is of type 'GFlags'

** (process:4196): WARNING **: Trying to register gtype 'GDriveStartFlags' as enum when in fact it is of type 'GFlags'

** (process:4196): WARNING **: Trying to register gtype 'GSocketMsgFlags' as enum when in fact it is of type 'GFlags'
Traceback (most recent call last):
File "/usr/share//ibus-sunpinyin/setup/main.py", line 42, in
import ibus
ImportError: No module named ibus

升级词库后, 临时文件的问题. 谢谢.

当我使用 sunpinyin-dictgen 生成词库后, 在 bin 目录下生成了很多文件.

_dict.utf8
dict.utf8-20130220.tar.bz2
dict.utf8
pydict_sc.bin
lm_sc.t3g.log
lm_sc.t3g.arpa-20121025.tar.bz2
lm_sc.t3g
lm_sc.t3g.orig
/tmp/.tmp-fcitx/bin/lm_sc.t3g.orig

请问这些文件, 那个是需要的, 那个是可删除的临时文件呢?

scons error when do a parallel install

This is the detailed error info for example: http://pastebin.com/ezZQUALk

When do a parallel install, scons will come up with this error.
Seems that scons will chdir to another directory when install files using relative path, so cannot find those files anymore. Thus error occurs.

I am not quite familiar with scons. Can you prevent scons doing a parallel install in sunpinyin?

regular release

How is development progressing and when do you think we can expect a regular release again?

[Regression Bisected] fails to convert ue and iong syllables and crashes

I updated on Debian from libsunpinyin3 2.0.3+git20120607-1 to 2.0.3+git20130507-1. (Also see Debian bug #730821)

xsunpinyin will not display any candidate choices for yue, jiong, qiong, and xiong.

xsunpinyin will provide candidates for jue, que, and xue with actual candidates of jiong, qiong, and xiong.

After inputting yue, jiong, qiong, or xiong, the cursor will move to the front. Moving left or right at the moment will crash xsunpinyin with segfault:

Program terminated with signal 11, Segmentation fault.
#0  CIMIClassicView::_moveLeftSyllable (this=this@entry=0xc6e610,
mask=@0x7ffff27e728c: 5,
    searchAgain=searchAgain@entry=true) at src/ime-core/imi_view_classic.cpp:583

Bisection shows regression introduced in 4f9745c.

@CasperVector

Mac OS binary version

I do not know whether this project is under active maintenance. I do not have Xcode installed on my laptop, I wounder if it is possible to publish an official build of sunpinyin in Mac OS. The old builds in Google code cannot run correctly in new Mac OSX system.

Regards,

Sunpinyin floods console

Sunpinyin generates numerous messages in the console, simply saying "here."
I believe the "puts("here");" in src/ime_core/ic_history.cpp on line 280 should most likely be commented out.

Sunpinyin 3.0.0 still mentions 2.0 in various places

Header files in:
usr/include/sunpinyin-2.0/

pkgconfig file is:
/usr/lib/pkgconfig/sunpinyin-2.0.pc

SConstruct file still says
version = "2.0.4"
conf.Define('PACKAGE_STRING', '"sunpinyin 2.0"')
conf.Define('PACKAGE_TARNAME', '"sunpinyin"')
conf.Define('PACKAGE_VERSION', '"2.0"')
conf.Define('VRESION', '"2.0"')
etc.

build fail, gentoo


➜  ~ git:(master) ✗ eum fcitx-sunpinyin

 * IMPORTANT: 4 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Calculating dependencies... done!

>>> Verifying ebuild manifests

>>> Starting parallel fetch

>>> Emerging (1 of 3) app-i18n/sunpinyin-core-9999 from gentoo-zh
 * QA Notice: ECLASS 'user' inherited illegally in app-i18n/sunpinyin-core-9999 setup
>>> Unpacking source...
 * git.eclass is deprecated.
 * Please update your ebuilds to use git-2 instead. For details, see
 * http://archives.gentoo.org/gentoo-dev/msg_b7ba363cae580845819ae3501fb157e9.xml
 * GIT NEW clone -->
 *    repository:       git://github.com/sunpinyin/sunpinyin.git
Cloning into bare repository /usr/portage/distfiles/git-src/sunpinyin...
remote: Counting objects: 5095, done.
remote: Compressing objects: 100% (1292/1292), done.
remote: Total 5095 (delta 3775), reused 4982 (delta 3702)
Receiving objects: 100% (5095/5095), 3.03 MiB | 1 KiB/s, done.
Resolving deltas: 100% (3775/3775), done.
 *    at the commit:        e1fe9bce13fabc964efde8183e11a8d2c14c88fd
 *    branch:           master
 *    storage directory:    "/usr/portage/distfiles/git-src/sunpinyin"
Cloning into /var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999...
done.
>>> Unpacked to /var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999
>>> Source unpacked in /var/tmp/portage/app-i18n/sunpinyin-core-9999/work
>>> Preparing source in /var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999 ...
scons: Reading SConscript files ...
Warning: you've set CFLAGS in the environmental variable!
Warning: you've set CXXFLAGS in the environmental variable!
Warning: you've set LDFLAGS in the environmental variable!
Checking for pkg-config... yes
Checking for sqlite3... yes
Checking for Python library...yes
Checking for C header file assert.h... yes
Checking for C function bind_textdomain_codeset()... yes
Checking for C function dcgettext()... yes
Checking for C header file dlfcn.h... yes
Checking for C function exp2()... yes
Checking for C header file fcntl.h... yes
Checking for C header file getopt.h... yes
Checking for C function getopt_long()... yes
Checking for C function getpagesize()... yes
Checking for C function get_opt()... no
Checking for C header file iconv.h... yes
Checking for C header file inttypes.h... yes
Checking for C header file locale.h... yes
Checking for C header file libintl.h... yes
Checking for C header file limits.h... yes
Checking for C header file locale.h... yes
Checking for C function log2()... yes
Checking for C header file memory.h... yes
Checking for C function memset()... yes
Checking for C function mmap()... yes
Checking for C function munmap()... yes
Checking for C function setlocale()... yes
Checking for C function strndup()... yes
Checking for C header file sys/mman.h... yes
Checking for C header file sys/param.h... yes
Checking for C header file sys/stat.h... yes
Checking for C header file sys/types.h... yes
Checking for C header file unistd.h... yes
Checking for C header file wchar.h... yes
scons: done reading SConscript files.
scons: Building targets ...
scons: Configure: creating config.h
g++ -o src/portability.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/portability.cpp
g++ -o src/slm/slm.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/slm.cpp
g++ -o src/slm/tslmendian/writer.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslmendian/writer.cpp
g++ -o src/lexicon/trie_writer.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/lexicon/trie_writer.cpp
g++ -o src/lexicon/genPYT.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/lexicon/genPYT.cpp
g++ -o src/lexicon/pytrie.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/lexicon/pytrie.cpp
g++ -o src/lexicon/pytrie_gen.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/lexicon/pytrie_gen.cpp
g++ -o src/pinyin/pinyin_data.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/pinyin/pinyin_data.cpp
g++ -o build/genpyt -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/slm.o src/slm/tslmendian/writer.o src/lexicon/trie_writer.o src/lexicon/genPYT.o src/lexicon/pytrie.o src/lexicon/pytrie_gen.o src/pinyin/pinyin_data.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/getWordFreq/getWordFreq.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/getWordFreq/getWordFreq.cpp
g++ -o build/getWordFreq -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/slm.o src/slm/getWordFreq/getWordFreq.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/ids2ngram/idngram_merge.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/ids2ngram/idngram_merge.cpp
src/slm/ids2ngram/idngram_merge.h: In function ‘void DoIdngramMerge(FILE*, CMultiWayFileMerger<CSIM_IdngramFreq<N> >&) [with int N = 1, FILE = _IO_FILE]’:
src/slm/ids2ngram/idngram_merge.h:60:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:61:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:68:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:69:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h: In function ‘void DoIdngramMerge(FILE*, CMultiWayFileMerger<CSIM_IdngramFreq<N> >&) [with int N = 2, FILE = _IO_FILE]’:
src/slm/ids2ngram/idngram_merge.h:60:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:61:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:68:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:69:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h: In function ‘void DoIdngramMerge(FILE*, CMultiWayFileMerger<CSIM_IdngramFreq<N> >&) [with int N = 3, FILE = _IO_FILE]’:
src/slm/ids2ngram/idngram_merge.h:60:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:61:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:68:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:69:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/idngram_merge -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/ids2ngram/idngram_merge.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/ids2ngram/ids2ngram.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/ids2ngram/ids2ngram.cpp
src/slm/ids2ngram/ids2ngram.cpp: In function ‘void ProcessingRead(FILE*, FILE*, std::vector<long int>&, size_t) [with int N = 1, FILE = _IO_FILE, size_t = long unsigned int]’:
src/slm/ids2ngram/ids2ngram.cpp:88:5: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp: In function ‘void WriteOut(FILE*, std::map<CSIM_Idngram<N>, unsigned int>&) [with int N = 1, FILE = _IO_FILE]’:
src/slm/ids2ngram/ids2ngram.cpp:68:9: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp:69:9: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp: In function ‘void ProcessingRead(FILE*, FILE*, std::vector<long int>&, size_t) [with int N = 2, FILE = _IO_FILE, size_t = long unsigned int]’:
src/slm/ids2ngram/ids2ngram.cpp:88:5: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp: In function ‘void WriteOut(FILE*, std::map<CSIM_Idngram<N>, unsigned int>&) [with int N = 2, FILE = _IO_FILE]’:
src/slm/ids2ngram/ids2ngram.cpp:68:9: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp:69:9: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp: In function ‘void ProcessingRead(FILE*, FILE*, std::vector<long int>&, size_t) [with int N = 3, FILE = _IO_FILE, size_t = long unsigned int]’:
src/slm/ids2ngram/ids2ngram.cpp:88:5: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp: In function ‘void WriteOut(FILE*, std::map<CSIM_Idngram<N>, unsigned int>&) [with int N = 3, FILE = _IO_FILE]’:
src/slm/ids2ngram/ids2ngram.cpp:68:9: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/ids2ngram.cpp:69:9: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h: In function ‘void DoIdngramMerge(FILE*, CMultiWayFileMerger<CSIM_IdngramFreq<N> >&) [with int N = 1, FILE = _IO_FILE]’:
src/slm/ids2ngram/idngram_merge.h:60:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:61:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:68:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:69:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h: In function ‘void DoIdngramMerge(FILE*, CMultiWayFileMerger<CSIM_IdngramFreq<N> >&) [with int N = 2, FILE = _IO_FILE]’:
src/slm/ids2ngram/idngram_merge.h:60:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:61:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:68:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:69:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h: In function ‘void DoIdngramMerge(FILE*, CMultiWayFileMerger<CSIM_IdngramFreq<N> >&) [with int N = 3, FILE = _IO_FILE]’:
src/slm/ids2ngram/idngram_merge.h:60:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:61:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:68:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/ids2ngram/idngram_merge.h:69:17: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/ids2ngram -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/ids2ngram/ids2ngram.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/sim_dict.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/sim_dict.cpp
g++ -o src/slm/sim_sen.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/sim_sen.cpp
g++ -o src/slm/mmseg/mmseg.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/mmseg/mmseg.cpp
src/slm/mmseg/mmseg.cpp: In function ‘void output(int, const TWCHAR*, TSIMWordId, TSIMWordId, int&)’:
src/slm/mmseg/mmseg.cpp:199:62: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/mmseg/mmseg.cpp: In function ‘void output_stok(int&)’:
src/slm/mmseg/mmseg.cpp:163:58: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/mmseg -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/sim_dict.o src/slm/sim_sen.o src/slm/mmseg/mmseg.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/sim_slmbuilder.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/sim_slmbuilder.cpp
src/slm/sim_slmbuilder.cpp: In member function ‘void CSlmBuilder::Write(FILE*)’:
src/slm/sim_slmbuilder.cpp:581:44: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/sim_slmbuilder.cpp:582:50: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/sim_slmbuilder.cpp:589:40: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/sim_slmbuilder.cpp:594:50: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/sim_slmbuilder.cpp:598:46: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o src/slm/slmbuild/slmbuild.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/slmbuild/slmbuild.cpp
g++ -o build/slmbuild -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/sim_slmbuilder.o src/slm/slmbuild/slmbuild.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/slminfo/slminfo.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/slminfo/slminfo.cpp
src/slm/slminfo/slminfo.cpp: In function ‘void PrintSimple(FILE*)’:
src/slm/slminfo/slminfo.cpp:264:32: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:265:50: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:269:44: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp: In function ‘void PrintARPA(FILE*, const char*, bool)’:
src/slm/slminfo/slminfo.cpp:251:32: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp: In function ‘void PrintARPALevel(int, FILE*, TReverseLexicon*, bool)’:
src/slm/slminfo/slminfo.cpp:125:34: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:126:50: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:132:38: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:148:65: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:149:65: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:154:65: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:169:69: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:211:67: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slminfo/slminfo.cpp:215:67: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/slminfo -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/slminfo/slminfo.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/sim_slm.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/sim_slm.cpp
src/slm/sim_slm.cpp: In member function ‘bool CSIMSlm::Load(const char*)’:
src/slm/sim_slm.cpp:52:42: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/sim_slm.cpp:56:62: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/sim_slm.cpp:59:62: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o src/slm/slmprune/slmprune.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/slmprune/slmprune.cpp
src/slm/slmprune/slmprune.cpp: In member function ‘void CSlmPruner::Write(const char*)’:
src/slm/slmprune/slmprune.cpp:123:34: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slmprune/slmprune.cpp:124:50: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slmprune/slmprune.cpp:125:40: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slmprune/slmprune.cpp:127:52: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slmprune/slmprune.cpp:129:48: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/slmprune -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/sim_slm.o src/slm/slmprune/slmprune.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/slmseg/slmseg.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/slmseg/slmseg.cpp
src/slm/slmseg/slmseg.cpp: In function ‘void output(int, const TWCHAR*, TSIMWordId, TSIMWordId, int&)’:
src/slm/slmseg/slmseg.cpp:194:62: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/slmseg/slmseg.cpp: In function ‘void output_stok(int&)’:
src/slm/slmseg/slmseg.cpp:159:58: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/slmseg -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/sim_dict.o src/slm/sim_sen.o src/slm/slm.o src/slm/slmseg/slmseg.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/thread/ValueCompress.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/thread/ValueCompress.cpp
g++ -o src/slm/thread/slmthread.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/thread/slmthread.cpp
src/slm/thread/slmthread.cpp: In function ‘int main(int, char**)’:
src/slm/thread/slmthread.cpp:427:35: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:429:42: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:433:41: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:435:61: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:437:45: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:439:63: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:443:45: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:447:29: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/thread/slmthread.cpp:448:74: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o build/slmthread -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/sim_slm.o src/slm/thread/ValueCompress.o src/slm/thread/slmthread.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/thread/test_vc.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/thread/test_vc.cpp
g++ -o build/testvc -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/slm/thread/ValueCompress.o src/slm/thread/test_vc.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/tslmendian/slm_file.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslmendian/slm_file.cpp
g++ -o src/slm/tslmendian/slm_endian.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslmendian/slm_endian.cpp
g++ -o build/tslmendian -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/slm/tslmendian/slm_file.o src/slm/tslmendian/slm_endian.o src/slm/tslmendian/writer.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/tslminfo/tslminfo.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslminfo/tslminfo.cpp
g++ -o build/tslminfo -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/slm.o src/slm/tslminfo/tslminfo.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
g++ -o src/slm/tslmpack/slmpack.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslmpack/slmpack.cpp
src/slm/tslmpack/slmpack.cpp: In function ‘void write_out(const char*, const CArpaSlm&, CompressedTable&, CompressedTable&, const TNodeLevels&, const CThreadSlm::TLeaf*)’:
src/slm/tslmpack/slmpack.cpp:270:35: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:272:49: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:276:46: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:282:55: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:285:49: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:292:56: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:295:49: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:301:33: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
src/slm/tslmpack/slmpack.cpp:304:78: warning: ignoring return value of ‘size_t fwrite(const void*, size_t, size_t, FILE*)’, declared with attribute warn_unused_result
g++ -o src/slm/tslmpack/arpa_conv.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslmpack/arpa_conv.cpp
g++ -o src/slm/tslmpack/arpa_slm.o -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/tslmpack/arpa_slm.cpp
g++ -o build/tslmpack -Wl,-O1 -Wl,--as-needed -Wl,-soname=libsunpinyin.so.3 src/portability.o src/slm/slm.o src/slm/thread/ValueCompress.o src/slm/tslmpack/slmpack.o src/slm/tslmpack/arpa_conv.o src/slm/tslmpack/arpa_slm.o -lpthread -ldl -lutil -lm -lpython3.2 -lsqlite3
Copy("data/genpyt", "build/genpyt")
Copy("data/tslmendian", "build/tslmendian")
Copy("data/tslmpack", "build/tslmpack")
g++ -o src/portability.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/portability.cpp
g++ -o src/slm/slm.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/slm/slm.cpp
g++ -o src/lexicon/pytrie.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/lexicon/pytrie.cpp
g++ -o src/pinyin/pinyin_data.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/pinyin/pinyin_data.cpp
g++ -o src/pinyin/pinyin_seg.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/pinyin/pinyin_seg.cpp
g++ -o src/pinyin/shuangpin_data.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/pinyin/shuangpin_data.cpp
g++ -o src/pinyin/shuangpin_seg.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/pinyin/shuangpin_seg.cpp
g++ -o src/pinyin/hunpin_seg.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/pinyin/hunpin_seg.cpp
g++ -o src/ime-core/imi_context.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_context.cpp
g++ -o src/ime-core/imi_data.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_data.cpp
g++ -o src/ime-core/lattice_states.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/lattice_states.cpp
g++ -o src/ime-core/imi_view.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_view.cpp
g++ -o src/ime-core/imi_uiobjects.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_uiobjects.cpp
g++ -o src/ime-core/imi_view_classic.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_view_classic.cpp
g++ -o src/ime-core/imi_winHandler.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_winHandler.cpp
g++ -o src/ime-core/ic_history.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/ic_history.cpp
src/ime-core/ic_history.cpp: In member function ‘bool CBigramHistory::loadFromFile(const char*)’:
src/ime-core/ic_history.cpp:190:36: warning: ignoring return value of ‘ssize_t read(int, void*, size_t)’, declared with attribute warn_unused_result
g++ -o src/ime-core/imi_funcobjs.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_funcobjs.cpp
g++ -o src/ime-core/imi_options.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_options.cpp
g++ -o src/ime-core/imi_option_event.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_option_event.cpp
g++ -o src/ime-core/userdict.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/userdict.cpp
g++ -o src/ime-core/imi_plugin.os -c -O2 -march=core2 -pipe -pipe -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/lib/sunpinyin/data"' -fPIC -I. -Isrc -Isrc/lexicon -Isrc/ime-core -Isrc/slm -Isrc/slm/tools -Isrc/slm/tslminfo -Isrc/slm/ids2ngram -Isrc/slm/tslmpack -Isrc/slm/slmbuild -Isrc/slm/mmseg -Isrc/slm/tslmendian -Isrc/slm/slminfo -Isrc/slm/thread -Isrc/slm/slmprune -Isrc/slm/getWordFreq -Isrc/slm/slmseg -Isrc/pinyin -I/usr/include/python3.2 src/ime-core/imi_plugin.cpp
src/ime-core/imi_plugin.cpp: In constructor ‘CIMIPythonPlugin::CIMIPythonPlugin(std::string)’:
src/ime-core/imi_plugin.cpp:102:44: error: ‘PyString_Check’ was not declared in this scope
src/ime-core/imi_plugin.cpp:103:40: error: ‘PyString_AsString’ was not declared in this scope
src/ime-core/imi_plugin.cpp:105:48: error: ‘PyString_Check’ was not declared in this scope
src/ime-core/imi_plugin.cpp:106:44: error: ‘PyString_AsString’ was not declared in this scope
src/ime-core/imi_plugin.cpp:108:58: error: ‘PyString_Check’ was not declared in this scope
src/ime-core/imi_plugin.cpp:109:54: error: ‘PyString_AsString’ was not declared in this scope
src/ime-core/imi_plugin.cpp: In function ‘wstring PyUnicode_AsWString(PyObject*)’:
src/ime-core/imi_plugin.cpp:145:60: error: cannot convert ‘PyUnicodeObject*’ to ‘PyObject*’ for argument ‘1’ to ‘Py_ssize_t PyUnicodeUCS4_AsWideChar(PyObject*, wchar_t*, Py_ssize_t)’
src/ime-core/imi_plugin.cpp: In function ‘void ExtractSequence(TPluginCandidates&, PyObject*)’:
src/ime-core/imi_plugin.cpp:164:54: error: ‘PyInt_Check’ was not declared in this scope
src/ime-core/imi_plugin.cpp:169:74: error: ‘PyInt_AsLong’ was not declared in this scope
src/ime-core/imi_plugin.cpp: In member function ‘virtual TPluginCandidates CIMIPythonPlugin::provide_candidates(const TPluginPreedit&, int*)’:
src/ime-core/imi_plugin.cpp:192:35: error: ‘PyInt_Check’ was not declared in this scope
src/ime-core/imi_plugin.cpp:193:47: error: ‘PyInt_AsLong’ was not declared in this scope
src/ime-core/imi_plugin.cpp:198:52: error: ‘PyInt_AsLong’ was not declared in this scope
src/ime-core/imi_plugin.cpp: In member function ‘virtual TPluginCandidate CIMIPythonPlugin::translate_candidate(const TPluginCandidate&, int*)’:
src/ime-core/imi_plugin.cpp:227:35: error: ‘PyInt_Check’ was not declared in this scope
src/ime-core/imi_plugin.cpp:228:47: error: ‘PyInt_AsLong’ was not declared in this scope
src/ime-core/imi_plugin.cpp: In member function ‘void CIMIPluginManager::initializePlugins()’:
src/ime-core/imi_plugin.cpp:283:48: warning: ignoring return value of ‘char* fgets(char*, int, FILE*)’, declared with attribute warn_unused_result
scons: *** [src/ime-core/imi_plugin.os] Error 1
scons: building terminated because of errors.
 * ERROR: app-i18n/sunpinyin-core-9999 failed (compile phase):
 *   compile failed
 * 
 * Call stack:
 *     ebuild.sh, line   85:  Called src_compile
 *   environment, line 4692:  Called _scons_do 'compile'
 *   environment, line  824:  Called die
 * The specific snippet of code:
 *       scons --prefix=/usr "${dest}" "${operation}" || die "${1} failed"
 * 
 * If you need support, post the output of 'emerge --info =app-i18n/sunpinyin-core-9999',
 * the complete build log and the output of 'emerge -pqv =app-i18n/sunpinyin-core-9999'.
 * This ebuild is from an overlay named 'gentoo-zh': '/var/lib/layman/gentoo-zh/'
 * The complete build log is located at '/var/tmp/portage/app-i18n/sunpinyin-core-9999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/app-i18n/sunpinyin-core-9999/temp/environment'.
 * S: '/var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999'

>>> Failed to emerge app-i18n/sunpinyin-core-9999, Log file:

>>>  '/var/tmp/portage/app-i18n/sunpinyin-core-9999/temp/build.log'

 * Messages for package app-i18n/sunpinyin-core-9999:

 * GIT NEW clone -->
 *    repository:       git://github.com/sunpinyin/sunpinyin.git
 *    at the commit:        e1fe9bce13fabc964efde8183e11a8d2c14c88fd
 *    branch:           master
 *    storage directory:    "/usr/portage/distfiles/git-src/sunpinyin"
 * ERROR: app-i18n/sunpinyin-core-9999 failed (compile phase):
 *   compile failed
 * 
 * Call stack:
 *     ebuild.sh, line   85:  Called src_compile
 *   environment, line 4692:  Called _scons_do 'compile'
 *   environment, line  824:  Called die
 * The specific snippet of code:
 *       scons --prefix=/usr "${dest}" "${operation}" || die "${1} failed"
 * 
 * If you need support, post the output of 'emerge --info =app-i18n/sunpinyin-core-9999',
 * the complete build log and the output of 'emerge -pqv =app-i18n/sunpinyin-core-9999'.
 * This ebuild is from an overlay named 'gentoo-zh': '/var/lib/layman/gentoo-zh/'
 * The complete build log is located at '/var/tmp/portage/app-i18n/sunpinyin-core-9999/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/app-i18n/sunpinyin-core-9999/temp/environment'.
 * S: '/var/tmp/portage/app-i18n/sunpinyin-core-9999/work/sunpinyin-core-9999'

 * IMPORTANT: 4 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Crash with sqlite 3.8.0

I'm getting the following crash after upgrading sqlite to 3.8.0.

Reproduce:

Switch to fcitx-sunpinyin, input "aaaser"

Output:

FCITX 4.2.8.1 -- Get Signal No.: 6
Date: try "date -d @1377568425" if you are using GNU date ***
ProcessID: 23271
fcitx[0x401568]
/usr/lib/libc.so.6(+0x35450)[0x7f4b84d8f450]
/usr/lib/libc.so.6(gsignal+0x39)[0x7f4b84d8f3d9]
/usr/lib/libc.so.6(abort+0x148)[0x7f4b84d907d8]
/usr/lib/libc.so.6(+0x72ed4)[0x7f4b84dcced4]
/usr/lib/libc.so.6(+0x7869e)[0x7f4b84dd269e]
/usr/lib/libc.so.6(+0x79377)[0x7f4b84dd3377]
/usr/lib/libsqlite3.so.0(sqlite3_free+0x6e)[0x7f4b741d60fe]
/usr/lib/libsqlite3.so.0(+0x51f8e)[0x7f4b7420ff8e]
/usr/lib/libsqlite3.so.0(+0x640c2)[0x7f4b742220c2]
/usr/lib/libsqlite3.so.0(+0x685d7)[0x7f4b742265d7]
/usr/lib/libsqlite3.so.0(+0x68abe)[0x7f4b74226abe]
/usr/lib/libsqlite3.so.0(+0x68f15)[0x7f4b74226f15]
/usr/lib/libsqlite3.so.0(sqlite3_prepare+0x15)[0x7f4b74226fa5]
/usr/lib/libsunpinyin.so.3(_ZN9CUserDict8getWordsERSt6vectorI9TSyllableSaIS1_EERS0_IN11CPinyinTrie11TWordIdInfoESaIS6_EE+0x29c)[0x7f4b744c03bc]
/usr/lib/libsunpinyin.so.3(_ZN11CIMIContext22_forwardSingleSyllableEjj9TSyllableRKN12IPySegmentor8TSegmentEb+0x1ec)[0x7f4b744a286c]
/usr/lib/libsunpinyin.so.3(_ZN11CIMIContext17_forwardSyllablesEjjRKN12IPySegmentor8TSegmentE+0x4a)[0x7f4b744a3c7a]
/usr/lib/libsunpinyin.so.3(_ZN11CIMIContext13_buildLatticeERSt6vectorIN12IPySegmentor8TSegmentESaIS2_EEjb+0xa4)[0x7f4b744a5ac4]
/usr/lib/libsunpinyin.so.3(_ZN15CIMIClassicView7_insertEjRj+0x5b)[0x7f4b744b42eb]
/usr/lib/libsunpinyin.so.3(_ZN15CIMIClassicView10onKeyEventERK9CKeyEvent+0x5b6)[0x7f4b744b6e26]
/usr/lib/fcitx/fcitx-sunpinyin.so(+0xaa76)[0x7f4b746dfa76]
/usr/lib/libfcitx-core.so.0(FcitxInstanceProcessKey+0x63b)[0x7f4b85763aeb]
/usr/lib/fcitx/fcitx-ipc.so(+0x46ba)[0x7f4b739f36ba]
/usr/lib/libdbus-1.so.3(+0x1d546)[0x7f4b83cf9546]
/usr/lib/libdbus-1.so.3(dbus_connection_dispatch+0x381)[0x7f4b83cec0f1]
/usr/lib/fcitx/fcitx-dbus.so(+0x2178)[0x7f4b83f25178]
/usr/lib/libfcitx-core.so.0(+0x9d66)[0x7f4b85757d66]
/usr/lib/libfcitx-core.so.0(FcitxInstanceRun+0x1e0)[0x7f4b85758710]
fcitx[0x400f68]
/usr/lib/libc.so.6(__libc_start_main+0xf5)[0x7f4b84d7bbc5]
fcitx[0x400fdd]

Notes:

I already tried to backup and remove ~/.sunpinyin/userdict, and rebuild sunpinyin against sqlite 3.8.0, but neither makes a difference.

Downgrading sqlite to 3.7.17 fixes this.

Request to join the sunpinyin development team

Hello, I am an open source developer (see my projects on http://code.google.com/p/caspervector/) and a user of sunpinyin. I have been researching sunpinyin for some days, and have recently read the code tours of sunpinyin (though still not understanding many details).

After joining, I will try to to do the following things (and perhaps more than that):

  • Restructure the sunpinyin codebase in a trainer/ime architecture just like libpinyin (I would prefer to explain it in detail in the mail list 1).
  • Unify the build tools (currently I see a collage of Makefile, CMake and Scons).
  • Apply the Chinese word segmentor algorithm in mmseg to the quanpin segmentor (I would elaborate about it in the mail list).
  • Implement (in the correct way) the advices in Issue #7.
  • Fix the issues I created and/or am interested on Google Code.
  • Improve the sunpinyin documentation, including the Wiki on Google Code and the comments in code.

Footnotes

  1. In addition, I have requested to join the [email protected] mail list, but got no reply for my request during a long time.

Default preferences.plist assume "Hei" font available

(lldb) p [NSFont fontWithName:@"Hei" size:16]
(id) $6 = nil

I don't have "Hei" available on my Yosemite system.

$ /Library/Input\ Methods/SunPinyin.app/Contents/MacOS/SunPinyin
2014-11-08 15:39:39.971 SunPinyin[3068:931302] An uncaught exception was raised
2014-11-08 15:39:39.971 SunPinyin[3068:931302] *** setObjectForKey: object cannot be nil (key: NSFont)
2014-11-08 15:39:39.971 SunPinyin[3068:931302] (
    0   CoreFoundation                      0x00007fff8c67e64c __exceptionPreprocess + 172
    1   libobjc.A.dylib                     0x00007fff910786de objc_exception_throw + 43
    2   CoreFoundation                      0x00007fff8c5667e6 -[__NSDictionaryM setObject:forKey:] + 1174
    3   SunPinyin                           0x0000000100019557 -[CandidateWindow setFont:] + 167
    4   SunPinyin                           0x00000001000043fa -[SunPinyinApplicationDelegate updateUISettings] + 1818
    5   SunPinyin                           0x00000001000074cb -[SunPinyinApplicationDelegate loadPreferences] + 43
    6   SunPinyin                           0x000000010000751e -[SunPinyinApplicationDelegate awakeFromNib] + 46
    7   CoreFoundation                      0x00007fff8c58c83f -[NSSet makeObjectsPerformSelector:] + 223
    8   AppKit                              0x00007fff8f34a09d -[NSIBObjectData nibInstantiateWithOwner:options:topLevelObjects:] + 1216
    9   AppKit                              0x00007fff8ee5c055 loadNib + 384
    10  AppKit                              0x00007fff8f3ca020 +[NSBundle(NSNibLoading) _loadNibFile:nameTable:options:withZone:ownerBundle:] + 313
    11  AppKit                              0x00007fff8ee5b725 -[NSBundle(NSNibLoading) loadNibNamed:owner:topLevelObjects:] + 201
    12  AppKit                              0x00007fff8ee5b4f1 +[NSBundle(NSNibLoading) loadNibNamed:owner:] + 344
    13  SunPinyin                           0x0000000100001825 main + 293
    14  SunPinyin                           0x00000001000016f4 start + 52
)
2014-11-08 15:39:39.972 SunPinyin[3068:931302] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** setObjectForKey: object cannot be nil (key: NSFont)'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff8c67e64c __exceptionPreprocess + 172
    1   libobjc.A.dylib                     0x00007fff910786de objc_exception_throw + 43
    2   CoreFoundation                      0x00007fff8c5667e6 -[__NSDictionaryM setObject:forKey:] + 1174
    3   SunPinyin                           0x0000000100019557 -[CandidateWindow setFont:] + 167
    4   SunPinyin                           0x00000001000043fa -[SunPinyinApplicationDelegate updateUISettings] + 1818
    5   SunPinyin                           0x00000001000074cb -[SunPinyinApplicationDelegate loadPreferences] + 43
    6   SunPinyin                           0x000000010000751e -[SunPinyinApplicationDelegate awakeFromNib] + 46
    7   CoreFoundation                      0x00007fff8c58c83f -[NSSet makeObjectsPerformSelector:] + 223
    8   AppKit                              0x00007fff8f34a09d -[NSIBObjectData nibInstantiateWithOwner:options:topLevelObjects:] + 1216
    9   AppKit                              0x00007fff8ee5c055 loadNib + 384
    10  AppKit                              0x00007fff8f3ca020 +[NSBundle(NSNibLoading) _loadNibFile:nameTable:options:withZone:ownerBundle:] + 313
    11  AppKit                              0x00007fff8ee5b725 -[NSBundle(NSNibLoading) loadNibNamed:owner:topLevelObjects:] + 201
    12  AppKit                              0x00007fff8ee5b4f1 +[NSBundle(NSNibLoading) loadNibNamed:owner:] + 344
    13  SunPinyin                           0x0000000100001825 main + 293
    14  SunPinyin                           0x00000001000016f4 start + 52
)
libc++abi.dylib: terminating with uncaught exception of type NSException
[1]    3068 abort      /Library/Input\ Methods/SunPinyin.app/Contents/MacOS/SunPinyin

Some scripts and advices to enhance sunpinyin

P.S.

  1. In fact I don't quite know why these issues did not get any attention on Google Code...
  2. Show I move the Unihan convert script to open-gram?

How to build this package

I want to compile this package and use it to build the fcitx-sunpinyin package which I have create an issue on fcitx/fcitx-sunpinyin#10.
But when I clone the source and didn't find a method to compile it. There is neither configure file nor Makefile to auto build.
Then I conclude maybe I should compile in an original way. But there aren't any file to indicate the way to build or I didn't have find it.
I am here to ask a way to build it in my linux computer which I didn't have an package manager like yum or apt-get to auto install this lib.

sunpinyin-dictgen 运行出错.

安装完之后, 当我手动生成词库时, 提示下面的信息:

./sunpinyin-dictgen 
tar xmf 
tar: Old option 'f' requires an argument.
Try 'tar --help' or 'tar --usage' for more information.
sunpinyin-dictgen:60: recipe for target 'lm_sc.3gm.arpa' failed
make: *** [lm_sc.3gm.arpa] Error 2

genpyt: bus error

Checkout commit 22afa31 (commit chosen to prove this bug already existed before my commits), and issue the following commands in the git repository:

$ scons
$ cd raw/
$ ../data/genpyt -i dict.utf8 -s ../data/lm_sc.t3g -l pydict_sc.log -o pydict_sc.bin

And a bus error will occur.
I might bisect and investigate which commit introduced this bug.

在 Slackware 14.1 编译安装 sunpinyin 时, scons 异常退出.

详细输出 log 如下:

scons: done reading SConscript files.
scons: Building targets ...
g++ -o src/portability.os -c -g -Wall -pipe -fPIC -DHAVE_CONFIG_H -DSUNPINYIN_DATA_DIR='"/usr/local/fcitx/share/sunpinyin"' -I. -Isrc -Isrc/ime-core -Isrc/slm -Isrc/slm/slmprune -Isrc/slm/mmseg -Isrc/slm/ids2ngram -Isrc/slm/slmseg -Isrc/slm/thread -Isrc/slm/tools -Isrc/slm/slmbuild -Isrc/slm/getwordfreq -Isrc/slm/tslmendian -Isrc/slm/tslminfo -Isrc/slm/slminfo -Isrc/slm/slmpack -Isrc/pinyin -Isrc/lexicon src/portability.cpp
src/portability.cpp: In function ‘size_t MBSTOWCS(TWCHAR*, const char*, size_t)’:
src/portability.cpp:197:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     while (dst - pwcs < n) {
                         ^
src/portability.cpp: In function ‘size_t WCSTOMBS(char*, const TWCHAR*, size_t)’:
src/portability.cpp:228:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
     while (dst - s < n) {
                      ^
src/portability.cpp:237:23: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
         if (dst - s > n) return -1;
                       ^
src/portability.cpp: In function ‘char* strndup(const char*, size_t)’:
src/portability.cpp:270:32: error: declaration of ‘char* strndup(const char*, size_t)’ has a different exception specifier
 strndup(const char *s, size_t n)
                                ^
In file included from src/portability.cpp:46:0:
src/portability.h:330:18: error: from previous declaration ‘char* strndup(const char*, size_t) throw ()’
 extern "C" char *strndup(const char *s, size_t n);
                  ^
scons: *** [src/portability.os] Error 1
scons: building terminated because of errors.

Could not build data files

I'm trying to:

scons --prefix=/usr

mkdir -p $srcdir/$_gitname-build/raw
cd $srcdir/$_gitname-build/raw
tar xjvf ${srcdir}/lm_sc.t3g.arpa.tar.bz2
tar xjvf ${srcdir}/dict.utf8.tar.bz2

PATH=$PATH:$srcdir/$_gitname-build/src
make -f ../doc/SLM.mk slm_bin

but I got the following error when trying the genpyt part:

scons: done building targets.
lm_sc.t3g.arpa
dict.utf8
slmbuild -n 3 -w 200000 -c 0,2,2 -d ABS,0.0005 -d ABS -d ABS -b 10 -e 9 \\
        -o lm_sc.3gram
Parameter input_file error

Usage:
  slmbuild options idngram

Description:
  This program generate language model from idngram file.

Options:
  -n --ngram     N            # 1 for unigram, 2 for bigram, 3 for trigram...
  -o --out       output       # output file name
  -l --log                    # using -log(pr), default use pr directly
  -w --wordcount N            # Lexicon size, number of different word
  -b --brk       id[,id...]   # set the ids which should be treat as breaker
  -e --exclude   id[,id...]   # set the ids which should not be put into LM
  -c --cut       c1[,c2...]   # k-gram whose freq <= c[k] are droped
  -d --discount  method,param # the k-th -d parm specify the discount method
      for k-gram. Possible values for method/param:
          GT,R,dis  : GT discount for r <= R, r is the freq of a ngram.
                      Linear discount for those r > R, i.e. r'=r*dis
                      0 << dis < 1.0, for example 0.999
          ABS,[dis] : Absolute discount r'=r-dis. And dis is optional
                      0 < dis < cut[k]+1.0, normally dis < 1.0.
          LIN,[dis] : Linear discount r'=r*dis. And dis is optional
                      0 < dis < 1.0

Notes:
      -n must be given before -c -b. And -c must give right number of cut-off,
  also -d must appear exactly N times specify discount for 1-gram, 2-gram...,
  respectively.
      BREAKER-IDs could be SentenceTokens or ParagraphTokens. Concepturally,
  these ids has no meaning when they appeared in the middle of n-gram.
      EXCLUDE-IDs could be ambiguious-ids. Concepturally, n-grams which
  contain those ids are meaningless.
      We can not erase ngrams according to BREAKER-IDS and EXCLUDE-IDs directly
  from IDNGRAM file, because some low-level information still useful in it.

Example:
      Following example read 'all.id3gram' and write trigram model 'all.slm'.
  At 1-gram level, use Good-Turing discount with cut-off 0, R=8, dis=0.9995. At
  2-gram level, use Absolute discount with cut-off 3, dis auto-calc. At 3-gram
  level, use Absolute discount wgenpyt -i dict.utf8 -s  -l pydict3_sc.log -o pyd
ict3_sc.bin
ith cut-off 2, dis auto-calc. Word id 10,11,12
  are breakers (sentence/para/paper breaker, etc). Exclude-ID is 9. Lexicon
  contains 200000 words. The result languagme model use -log(pr).

        slmbuild -l -n 3 -o all.slm -w 200000 -c 0,3,2 -d GT,8,0.9995
                 -d ABS -d ABS -b 10,11,12 -e 9 all.id3gram

make: *** [lm_sc.3gram] Error 100
make: *** Waiting for unfinished jobs....
Opening language model...open -l: No such file or directory
error!
make: *** [pydict3_sc.bin] Error 255

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.