emscripten-core / emscripten Goto Github PK
View Code? Open in Web Editor NEWEmscripten: An LLVM-to-WebAssembly Compiler
License: Other
Emscripten: An LLVM-to-WebAssembly Compiler
License: Other
On my Windows 7 system, running:
runner clang_0_0.test_conststructs
gives this output:
======================================================================
ERROR: test_conststructs (__main__.clang_0_0)
----------------------------------------------------------------------
Traceback (most recent call last):
File "Y:\emscripten\tests\runner.py", line 1117, in test_conststructs
self.do_test(src, '*70,97,15,3,3029,90*')
File "Y:\emscripten\tests\runner.py", line 322, in do_test
self.assertContained(expected_output, js_output)
File "Y:\emscripten\tests\runner.py", line 276, in assertContained
raise Exception("Expected to find '%s' in '%s'" % (limit_size(value), limit_size(string)))
Exception: Expected to find '*70,97,15,3,3029,90*' in '*70,97,9900,0.12,3029,90*
'
----------------------------------------------------------------------
Ran 1 test in 0.391s
FAILED (errors=1)
Here are the generated src.cpp.o.ll and src.cpp.o.js files:
http://pastebin.com/krJWXweS
http://pastebin.com/bkwFWGjy
The problem seems to be that the double member variable in the IUB struct is not correctly aligned/padded. Emscripten seems to think that the size of the IUB struct is 16 bytes, wheras clang will align and pad the double in a way that makes the struct 24 bytes.
The problem can be observed in the generated src.cpp.o.js file where Emscripten allocates 64 "bytes" (16 "bytes" for each entry in the iub array):
var $iub=__stackBase__+16;
var $after=__stackBase__+80;
but then goes ahead and copies 96 "bytes" when initializing the array:
_llvm_memcpy_p0i8_p0i8_i32($4, __ZZ4mainE3iub, 96, 8, 0);
It should be simple (and very useful) to have node.js running emscripten, as a npm module
I get this output in the Javascript file:
// Processing item for llvm line 33124
intertyper.js:617: TypeError: Cannot read property 'tokens' of undefined
item.params = parseParamTokens(item.tokens[3].item.tokens);
^
TypeError: Cannot read property 'tokens' of undefined
at Object.processItem (intertyper.js:617:57)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
at intertyper (intertyper.js:809:20)
at JSify (jsifier.js:33:30)
at /home/name/Dev/emscripten/src/compiler.js:79:1
This was the output at line 33124 in the .ll file:
invoke void bitcast (void (%"struct.std::__1::string", i32) @Z8toStringj to void (%"struct.std::basic_string<char,std::char_traits,std::allocator >", i32)_)(%"struct.std::basic_string<char,std::char_traits,std::allocator >"* noalias sret %3, i32 %99)
to label %invcont33 unwind label %lpad106
Here is the .ll file that was generated (plus the source file I think it may be caused by):
http://ge.tt/8wHRkn4
Affected version:
cca5278
To reproduce:
in ~/.emscripten
V8_ENGINE = ['/usr/bin/d8']
COMPILER_ENGINE=V8_ENGINE
Then, run emscripten.py anyfile.ll, leading to the following error (by the way, it would be better if errors were sent to stderr):
~/emscripten/src/compiler.js:30:Error loading file
var settings=JSON.parse(read(settings_file));^
To fix it:
tools/shared.py, line 47 in function run_js:
return timeout_run(Popen(engine + [filename] + (['--'] if 'v8' in engine[0] else []) + args,
stdout=stdout, stderr=stderr, cwd=cwd), 15*60 if check_timeout else None, 'Execution')
'v8' might not be in the pathname of the compiler. Changing it to 'd8' works in my case but it should be handled in a more general way.
Apparently Lua string functions is not fully supported in the compiled Lua interpreter. I successfully tried
string.byte("ABCDE", 2)
string.char(65,66,67,68,69)
string.format("%.7f", math.pi)
string.gsub("Hello banana", "banana", "Lua user")
string.gsub("banana", "(an)", "%1-")
string.rep("Lua ",5)
string.reverse("lua")
string.sub("Hello Lua user", -8, 9)
string.upper("Hello, Lua user!")
but the following functions produces an (error object is not a string)
which is not very helpful anyway:
string.find("Hello Lua user", "Lua")
string.gsub("Hello Lua user", "(%w+)", print)
for word in string.gmatch("Hello Lua user", "%a+") do print(word) end
string.match("I have 2 questions for you.", "%d+ %a+")
Since searching in strings is a very common task, the Lua interpreter is of little use with this limitations. I think the bug is in the compilation of lstrlib.c but I could not track down the error.
This might be a bit going against the strategy outlined in the README. It is stated that different sized pointer casting code will not work properly in emscripten.
However I wanted to compile a specific file format parsing library (st-guilde .HYP hepertext files http://code.google.com/p/libhyp/) into javascript along with its LH5 decompression algorithm. That required the pointer casting to work properly.
After adding some aliases and methods to src/library.js I succeeded to build a working application available at http://opichals.github.com/hypview/ (.hyp file viewer, one can download some .hyp file from http://libhyp.atari.org for tests).
The pointer casting changes are available at https://github.com/opichals/emscripten/tree/correct-pointers branch (on top of the nodejs branch I posted about before elsewhere). The core of the idea is that the resulting generated code emulates bigendian HEAP access for different size datatypes like e.g. "HEAP[v+1]<<8 | HEAP[v]" for reading 16bit int. No performance or other optimizations were accounted for - just the result correct behaviour.
Just wanted to make this available. I tend to think that this is the closest we can get to real LLVM bytecode memory management. Not sure whether this is something to be merged to master later on (perhaps as an option?).
Currently emscripten stubs or fakes most filesystem access calls. This causes plenty of software to either not work at all or fail subtly due to incomplete functionality. I propose implementing a lightweight file system that would allow users of emscripted programs to set up a access to their real files with the paths that the emscripted program expects.
Basically, I plan to follow the same pre-configuration approach currently used for STDIO.prepare()
. I imagine a main Module.FS
object which would be used to set up all the paths before running the module. To create a file you could pass actual data or a relative URL that would be loaded lazily when/if requested. In non-browser contexts, the relative URL can be used as a real filesystem path.
A similar method could be used to create virtual folders or explicitly or mark leaf folders for lazy-loading, so any reads of file from inside the folder would be forwarded to a given URL.
In the interest of keeping it lightweight, I see no reason to support any of the following:
Modifying anything (writing to files, deleting/creating folders, modifying permissions) by the emscripted program could either be handled by a user-specified callback, or happen in memory with the user being able to flush all the changes after the program is done running. The latter would be simpler, but the former significantly more flexible.
I believe this would allow emscripten to implement all of the Unix filesystem API at a minimum level.
In terms of actual implementation, I see the file system as a simple nested dictionary/Object
. A straightforward but memory-wasteful approach would be to have something like this for a simple file system:
root = {
read: true,
write: false,
isFolder: true,
timestamp: <Date>,
inodeNumber: 1,
contents: {
subfolder1: {
read: true,
write: true,
isFolder: true,
timestamp: <Date>,
inodeNumber: 2,
contents: {
myfile: {
read: true,
write: true,
isFolder: false,
timestamp: <Date>,
inodeNumber: 3,
url: '/path/to/myfile/on/server'
},
myotherfile: {
read: true,
write: true,
isFolder: false,
timestamp: <Date>,
inodeNumber: 4,
contents: <index-into-HEAP>
},
mylink: {
read: true,
write: true,
isFolder: false,
timestamp: <Date>,
inodeNumber: 5,
link: '/subfolder1/myfile'
},
mydevice: {
read: true,
write: true,
isFolder: false,
timestamp: <Date>,
inodeNumber: 6,
read: readFromDevice,
write: writeToDevice
},
subfolder3: {
read: true,
write: false,
isFolder: true,
timestamp: <Date>,
inodeNumber: 7,
url: '/getfileFromSubfolder?name=%s'
}
}
},
subfolder2: {
read: false,
write: false,
isFolder: true
timestamp: <Date>,
inodeNumber: 8,
}
}
};
In the interest of memory conservation, the permissions read
and write
could be merged into a single flags field, and instead of an Object
for each file, we can use a more flattened 5-tuple: permissions
, inode
, timestamp
, type
and content
, where type
implies isFolder
. The above structure would look as follows using this approach:
root = [1, 1, {
subfolder1: [3, 2, <Date>, 'contents', {
myfile: [3, 3, <Date>, 'url', '/path/to/myfile/on/server'],
myotherfile: [3, 4, <Date>, 'data', <index-into-HEAP>],
mylink: [3, 5, <Date>, 'link', '/subfolder1/myfile'],
mydevice: [3, 6, <Date>, 'device' [readFromDevice, writeToDevice]],
subfolder3: [1, 7, <Date>, 'url_pattern', '/getfileFromSubfolder?name=%s']
},
subfolder2: [0, 8, <Date>, 'contents', {}]
}];
emscripten latest git version
/dev/shm/tmp$ uname -a
Linux ubuntu32 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:50 UTC 2011 i686 i686 i386 GNU/Linux
/dev/shm/tmp$ clang -v
clang version 2.9 (tags/RELEASE_29/final)
Target: i386-pc-linux-gnu
Thread model: posix
/dev/shm/tmp$ llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: /home/duncan/llvm-2.9/32/llvmgcc42-2.9-release.src/configure --prefix=/home/duncan/llvm-2.9/32/Phase2/Release/llvmgcc42-2.9-release.install --program-prefix=llvm- --enable-llvm=/home/duncan/llvm-2.9/32/Phase2/Release/llvmCore-2.9-release.obj --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2.9)
:/opt/src/emscripten/tests$ python runner.py clang_0_0.test_zlib
Running Emscripten tests...
test_zlib (__main__.clang_0_0) ... ./configure
Checking for shared library support...
Tested /opt/src/emscripten/tools/emmaken.py -w -c -m32 ztest689.c
emmaken.py: /opt/src/emscripten/tools/emmaken.py -w -c -m32 ztest689.c
Running: /opt/src/llvm-2.9/bin/clang -w -c -m32 ztest689.c -m32 -U__i386__ -U__x86_64__ -U__SSE__ -UX87_DOUBLE_ROUNDING -UHAVE_GCC_ASM_FOR_X87 -m32 -emit-llvm -c
Tested cc -shared -m32 -o ztest689.so ztest689.o
ztest689.o: file not recognized: File format not recognized
collect2: ld returned 1 exit status
No shared library support; try without defining CC and CFLAGS
Building static library libz.a version 1.2.5 with /opt/src/emscripten/tools/emmaken.py.
Checking for off64_t... No.
Checking for fseeko... No.
Checking for unistd.h... No.
Checking whether to use vs[n]printf() or s[n]printf()... using s[n]printf().
Checking for snprintf() in stdio.h... No.
WARNING: snprintf() not found, falling back to sprintf(). zlib
can build but will be open to possible buffer-overflow security
vulnerabilities.
Checking for return value of sprintf()... No.
WARNING: apparently sprintf() does not return a value. zlib
can build but will be open to possible string-format security
vulnerabilities.
make libz.a
FAIL
======================================================================
FAIL: test_zlib (__main__.clang_0_0)
----------------------------------------------------------------------
Traceback (most recent call last):
File "runner.py", line 1963, in test_zlib
libraries=[self.get_library('zlib', os.path.join('libz.a'), make_args=['libz.a'])],
File "runner.py", line 1918, in get_library
self.do_link(map(lambda lib: os.path.join(project_dir, lib), generated_libs), bc_file)
File "runner.py", line 155, in do_link
assert output is None or 'Could not open input file' not in output, "Linking error: %s %s" % (output, cmd)
AssertionError: Linking error: /opt/src/llvm-2.9/bin/llvm-link: /dev/shm/tmp/building/zlib/libz.a: Could not open input file: No such file or directory
/opt/src/llvm-2.9/bin/llvm-link: error loading file '/dev/shm/tmp/building/zlib/libz.a'
/opt/src/llvm-2.9/bin/llvm-link /dev/shm/tmp/building/zlib/libz.a -o /dev/shm/tmp/building/zlib/bc.bc
----------------------------------------------------------------------
Ran 1 test in 2.700s
FAILED (failures=1)
If you use DataView, you are no longer required to divide the pointers (or bitshifting) with some constant but rather use [dataview].getInt32, getFloat64 and setInt32 etc. An other bonus is that you can support int8, int16, uints, float32 etc. and can provide better compability.
The question is of course of it is as fast as an array index at this moment, but likely will be in the future.
To improve the performance of compiled code, asynchronous callbacks should be used wherever possible. The nimble library provides the functionality. It can be used without node.js.
Help with making the code prettier would be great. It is really easy to improve this, just changing function names would be a good start. Bigger refactorings are also possible of course.
Emscripten has tons of automatic tests, so we can make these changes with no risk of regressions.
The reason I don't do more of this myself, is I am too used to the code to know what stands out as unreadable and in need of cleanup.
http://blog.llvm.org/2010/01/address-of-label-and-indirect-branches.html
Emscripten does not support the LLVM blockaddress
node nor indirectbr
instruction. These are rare instructions which are produced through using labels as values:
$ cat test.c
#include <stdio.h>
int
main(void) {
const void *addr = &&FOO;
goto *addr;
printf("bad\n");
return 1;
FOO:
printf("good\n");
return 0;
}
$ ~/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -Wall -W -emit-llvm -c -o test.o test.c
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-link -o=test.bc test.o
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-dis -show-annotations test.bc
$ ~/reposs/emscripten/emscripten.py test.ll
intertyper.js:303: Invalid token, cannot triage: // {"tokens":[{"text":"indirectbr"},{"text":"i8*"},{"text":"%2"},{"text":","},{"tex
// t":"[label %FOO]","item":{"tokens":[{"text":"label"},{"text":"%FOO"}],"indent":0
// },"type":"["}],"indent":2,"lineNum":19,"__uid__":35}
throw 'Invalid token, cannot triage: ' + dump(item);
And another example for blockaddress
:
$ cat test.c
#include <stdio.h>
int
main(void) {
const void *addr[] = {&&FOO, &&BAR};
goto *addr[0];
printf("bad\n");
return 1;
FOO:
printf("foo\n");
return 0;
BAR:
printf("bar\n");
return 0;
}
$ ~/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -Wall -W -emit-llvm -c -o test.o test.c
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-link -o=test.bc test.o
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-dis -show-annotations test.bc
$ ~/reposs/emscripten/emscripten.py test.ll
intertyper.js:347: Invalid segment: // [{"text":"i8*"},{"text":"blockaddress"},{"text":"(@main, %FOO)","item":{"tokens"
// :[{"text":"@main"},{"text":","},{"text":"%FOO"}],"indent":0},"type":"("}]
throw 'Invalid segment: ' + dump(segment);
^
pick_llvm_opts in tests/runner.py currently selects a subset of the LLVM optimizations, that appears to be safe and not generate nonportable code.
We might be missing some LLVM optimizations though. Also, in some cases LLVM optimizations help, in others they hurt. We should find out why.
Also, with typed arrays == 2, our memory model is much closer to the native one, and we should be able to use even nonportable optimizations.
Hey!
I've tried emscripten for converting our c++ projects to javascript. These programs read and write files simple text files. I have the content of the input files in javascript, and I want to process them with the converted c++ program, and receive the content of the output files in javascript.
I saw you have a STDIO object for reading and writing file streams, but I found that fscanf doesnt exist at all. There is an scanf and a function called fread.
Could you please implement a simple fscanf function?
How can I set the initial content of the input files? Can it be done with STDIO.prepare(...)?
I saw that there is an scanf implementation which does not supprt %f format yet. Do you think you can make that too?
What would happen if I open an input file for writing, change the content, and open it again for reading in the same running session? Can it store the file changes during the running of the program? I could add persistance on javascript side to save them to the HTML5 local storage, but at the moment I'm not sure if this idea would work.
Thanks,
Henko
You can get a disassembled llvm from following link: http://desti.ruree.net/core_simple.o.ll
Original source code is here: http://dosbox.svn.sourceforge.net/viewvc/dosbox/dosbox/tags/RELEASE_0_74/src/cpu/core_simple.cpp?revision=3609&view=markup
I don't know why, but it failed after long execution, and a error message is following:
Assertion failed: What variable is this? |getelementptr|
Stack: Error
at assertTrue (utility.js:60:23)
at getVarImpl (jsifier.js:435:5)
at jsifier.js:636:16
at Object.processItem (jsifier.js:495:19)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
at JSify (jsifier.js:792:34)
at JSify (jsifier.js:33:15)
utility.js:61: Assertion failed: What variable is this? |getelementptr|
throw msg;
^
I'm trying to compile this LLVM:
store i64 ptrtoint (i8* blockaddress(@vm_exec_core, %2882) to i64), i64* getelementptr inbounds ([1 x i64]* @finish_insn_seq, i32 0, i64 0), align 8
The new 'time' test fails on my machine, in all settings (clang, llvm_gcc, optimize, etc.). The output is
stime: -1
tzname[0] set: 1
tzname[1] set: 1
sec: 43
min: 22
hour: 3
day: 25
mon: 11
year: 102
wday: 3
yday: 358
dst: 0
off: 0
zone: GMT
timegm <-> gmtime: 1
old year: 102
new year: 70
old year again: 102
localtime timezone: 1
localtime daylight: 1
localtime tzname: 1
localtime <-> mktime: 1
old year: 102
new year: 69
old year again: 102
time: 1
difftime+: 268848637.000000
difftime-: -268848637.000000
1854 days: 365
2000 days: 366
2001 days: 365
2004 days: 366
asctime: Tue Dec 24 19:22:43 2002
old asctime: Tue Dec 24 19:22:43 2002
new asctime_r: Sat Jul 2 12:33:20 2011
old asctime again: Tue Dec 24 19:22:43 2002
clock: 0
ctime: 0
Only things I can think of offhand that could cause it to fail for me but succeed elsewhere, are headers? Or possibly different time values in JavaScript due to different local settings?
Here's python.opt.ll. Here's the stack trace:
Assertion failed: undefined
Stack: Error
at assertTrue (utility.js:60:23)
at toNiceIdent (parseTools.js:74:3)
at getLabelId (jsifier.js:565:13)
at makeBranch (jsifier.js:593:37)
at jsifier.js:602:23
at Object.processItem (jsifier.js:524:19)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
utility.js:61: Assertion failed: undefined
throw msg;
^
Running on an unoptimized version works fine.
To improve our Lua build, we should build it from source in the test runner. This will allow building with clang, as well as debugging the existing issues we have with Lua.
See test_the_bullet (the bullet test) in tests/runner.py for how we do this for Bullet. The same approach should be done for Lua.
This isn't exactly an issue - more of a request.
I noticed the previous ticket in google code and I realise that there is probably no CommonJS way to do this, however I would love if it could be done.
I help out with http://ncss.edu.au (we teach python to high-school students) and we're looking for a way to get Python running in the browser so that students who don't have admin on their machine (ie at school computers etc) can write code and keep up with our challenge.
I'm not looking for someone to do all the work - I just need someone to point me in the direction of a solution. I see a few different possible ones:
What do you think?
The javascript generated by emscripten from the following C program prints "0 0", instead of "1.00000 1.00000". It appears the offset of fixed size arrays in structs is sometimes wrong (zero).
#include <stdio.h>
struct {
int x;
double y[10];
double z[10];
} commonblock;
int main(int argc, char **argv)
{
int i;
for (i = 0; i < 10; ++i) {
commonblock.y[i] = 1;
commonblock.z[i] = 1;
}
printf("%f %f\n", commonblock.y[0], commonblock.z[0]);
return 0;
}
Code like this was obtained by converting a Fortran program to C. I worked around this issue by putting every fixed size array in its own common block.
The way that CMake uses the 'ar' tool doesn't seem to be compatible with the way that emmaken.py emulates it.
For example, using helloworld.c:
#include <stdio.h>
int helloworld(void) {
printf("helloworld \n");
return 0;
}
and CMakeLists.txt:
cmake_minimum_required(VERSION 2.8.3)
# these must be set *before* project()
if (EMSCRIPTEN)
set(EMSCRIPTEN_TOOLS ${EMSCRIPTEN_HOME}/tools)
SET(CMAKE_C_COMPILER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_CXX_COMPILER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_LINKER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_CXX_LINKER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_C_LINK_EXECUTABLE "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_CXX_LINK_EXECUTABLE "${EMSCRIPTEN_TOOLS}/emmaken.py")
endif(EMSCRIPTEN)
project(helloworld)
# these must be set *after* project()
if (EMSCRIPTEN)
SET(CMAKE_AR "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_RANLIB "${EMSCRIPTEN_TOOLS}/emmaken.py")
endif(EMSCRIPTEN)
# list of sources
set(SRCS helloworld.c)
# make a lib
add_library(helloworld ${SRCS})
will produce this output:
$ cmake .. -DEMSCRIPTEN=1 -DEMSCRIPTEN_HOME=/home/vail/Development/emscripten; make VERBOSE=1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vail/Development/helloworld/.build-js
/usr/bin/cmake -H/home/vail/Development/helloworld -B/home/vail/Development/helloworld/.build-js --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/vail/Development/helloworld/.build-js/CMakeFiles /home/vail/Development/helloworld/.build-js/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory `/home/vail/Development/helloworld/.build-js'
make -f CMakeFiles/helloworld.dir/build.make CMakeFiles/helloworld.dir/depend
make[2]: Entering directory `/home/vail/Development/helloworld/.build-js'
cd /home/vail/Development/helloworld/.build-js && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/vail/Development/helloworld /home/vail/Development/helloworld /home/vail/Development/helloworld/.build-js /home/vail/Development/helloworld/.build-js /home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/DependInfo.cmake --color=
Dependee "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/DependInfo.cmake" is newer than depender "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/depend.internal".
Scanning dependencies of target helloworld
make[2]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make -f CMakeFiles/helloworld.dir/build.make CMakeFiles/helloworld.dir/build
make[2]: Entering directory `/home/vail/Development/helloworld/.build-js'
/usr/bin/cmake -E cmake_progress_report /home/vail/Development/helloworld/.build-js/CMakeFiles 1
[100%] Building C object CMakeFiles/helloworld.dir/helloworld.c.o
/home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
emmaken.py: /home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
emmaken.py: /home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
Running: /home/vail/Development/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c -m32 -U__i386__ -U__x86_64__ -U__SSE__ -UX87_DOUBLE_ROUNDING -UHAVE_GCC_ASM_FOR_X87 -emit-llvm -c
Linking C static library libhelloworld.a
/usr/bin/cmake -P CMakeFiles/helloworld.dir/cmake_clean_target.cmake
/usr/bin/cmake -E cmake_link_script CMakeFiles/helloworld.dir/link.txt --verbose=1
/home/vail/Development/emscripten/tools/emmaken.py cr libhelloworld.a CMakeFiles/helloworld.dir/helloworld.c.o
emmaken.py: /home/vail/Development/emscripten/tools/emmaken.py cr libhelloworld.a CMakeFiles/helloworld.dir/helloworld.c.o
emmaken.py: /home/vail/Development/emscripten/tools/emmaken.py cr libhelloworld.a CMakeFiles/helloworld.dir/helloworld.c.o
Running: /home/vail/Development/llvm-2.9/cbuild/bin/llvm-link cr libhelloworld.a CMakeFiles/helloworld.dir/helloworld.c.o
/home/vail/Development/llvm-2.9/cbuild/bin/llvm-link: cr: Could not open input file: No such file or directory
/home/vail/Development/llvm-2.9/cbuild/bin/llvm-link: error loading file 'cr'
make[2]: *** [libhelloworld.a] Error 1
make[2]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make[1]: *** [CMakeFiles/helloworld.dir/all] Error 2
make[1]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make: *** [all] Error 2
With CMake its possible to compile .c files as C++. This is an unfortunate yet common practice in large, legacy codebases. emmaken.py uses the extension to figure out which compiler it needs to use, but this fails with .c files that are really c++.
For example, using helloworld.c:
#include <stdio.h>
// C++ headers
#include <cmath>
#include <limits>
int helloworld(void) {
printf("helloworld \n");
return 0;
}
and CMakeLists.txt:
cmake_minimum_required(VERSION 2.8.3)
# these must be set *before* project()
if (EMSCRIPTEN)
set(EMSCRIPTEN_TOOLS ${EMSCRIPTEN_HOME}/tools)
SET(CMAKE_C_COMPILER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_CXX_COMPILER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_LINKER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_CXX_LINKER "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_C_LINK_EXECUTABLE "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_CXX_LINK_EXECUTABLE "${EMSCRIPTEN_TOOLS}/emmaken.py")
endif(EMSCRIPTEN)
project(helloworld)
# these must be set *after* project()
if (EMSCRIPTEN)
SET(CMAKE_AR "${EMSCRIPTEN_TOOLS}/emmaken.py")
SET(CMAKE_RANLIB "${EMSCRIPTEN_TOOLS}/emmaken.py")
endif(EMSCRIPTEN)
# list of sources
set(SRCS helloworld.c)
# compile all files as C++, no matter their extension
set_source_files_properties( ${SRCS} PROPERTIES LANGUAGE CXX )
# make a lib
add_library(helloworld ${SRCS})
has this output:
$ cmake .. -DEMSCRIPTEN=1 -DEMSCRIPTEN_HOME=/home/vail/Development/emscripten; make VERBOSE=1
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- Check for working C compiler: /home/vail/Development/emscripten/tools/emmaken.py
-- Check for working C compiler: /home/vail/Development/emscripten/tools/emmaken.py -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /home/vail/Development/emscripten/tools/emmaken.py
-- Check for working CXX compiler: /home/vail/Development/emscripten/tools/emmaken.py -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vail/Development/helloworld/.build-js
/usr/bin/cmake -H/home/vail/Development/helloworld -B/home/vail/Development/helloworld/.build-js --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/vail/Development/helloworld/.build-js/CMakeFiles /home/vail/Development/helloworld/.build-js/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory `/home/vail/Development/helloworld/.build-js'
make -f CMakeFiles/helloworld.dir/build.make CMakeFiles/helloworld.dir/depend
make[2]: Entering directory `/home/vail/Development/helloworld/.build-js'
cd /home/vail/Development/helloworld/.build-js && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/vail/Development/helloworld /home/vail/Development/helloworld /home/vail/Development/helloworld/.build-js /home/vail/Development/helloworld/.build-js /home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/DependInfo.cmake --color=
Dependee "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/DependInfo.cmake" is newer than depender "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/depend.internal".
Dependee "/home/vail/Development/helloworld/.build-js/CMakeFiles/CMakeDirectoryInformation.cmake" is newer than depender "/home/vail/Development/helloworld/.build-js/CMakeFiles/helloworld.dir/depend.internal".
Scanning dependencies of target helloworld
make[2]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make -f CMakeFiles/helloworld.dir/build.make CMakeFiles/helloworld.dir/build
make[2]: Entering directory `/home/vail/Development/helloworld/.build-js'
/usr/bin/cmake -E cmake_progress_report /home/vail/Development/helloworld/.build-js/CMakeFiles 1
[100%] Building CXX object CMakeFiles/helloworld.dir/helloworld.c.o
/home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
emmaken.py: /home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
emmaken.py: /home/vail/Development/emscripten/tools/emmaken.py -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c
Running: /home/vail/Development/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -o CMakeFiles/helloworld.dir/helloworld.c.o -c /home/vail/Development/helloworld/helloworld.c -m32 -U__i386__ -U__x86_64__ -U__SSE__ -UX87_DOUBLE_ROUNDING -UHAVE_GCC_ASM_FOR_X87 -emit-llvm -c
/home/vail/Development/helloworld/helloworld.c:1:17: error: cmath: No such file or directory
/home/vail/Development/helloworld/helloworld.c:2:18: error: limits: No such file or directory
make[2]: *** [CMakeFiles/helloworld.dir/helloworld.c.o] Error 1
make[2]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make[1]: *** [CMakeFiles/helloworld.dir/all] Error 2
make[1]: Leaving directory `/home/vail/Development/helloworld/.build-js'
make: *** [all] Error 2
Since QUANTUM_SIZE
can only be set to 1 or 4 at this time, it makes sense to have it set from a boolean flag in settings.js
instead of the current numerical value, to show that other values (e.g. 2, 8) are not supported. This can be done by either replacing QUANTUM_SIZE
completely or creating a new flag and setting QUANTUM_SIZE
to the right integer on start up.
This is Python's _functools
module compiled to LLVM bytecode, and this is it after being emscripted as a shared lib. On line 1721, inside run()
, you can see:
HEAP[_partial_type+160]=_PyObject_GC_Del;
Note that _PyObject_GC_Del
is an external function that exists only in the parent module. It appears that it is not indexized as it should be in the GlobalVariablePostSet
pass.
I was compiling the cubescript example directly via the command line and encountered this:
Assertion failed: Failed to find the # of uses of var: $0
Stack: Error
at assertTrue (utility.js:60:23)
at analyzer.js:271:13
at Array.forEach (native)
at analyzer.js:260:20
at Array.forEach (native)
at Object.processItem (analyzer.js:238:22)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
utility.js:61: Assertion failed: Failed to find the # of uses of var: $0
throw msg;
^
The addresses of the corresponding cstdlib maths functions are not added to FUNCTION_TABLE
. This causes any dereferenced function pointer to a cstdlib maths function to die.
$ cat test.c
#include <math.h>
#include <stdio.h>
int
main(void) {
float (*fn)(float) = &sqrtf;
printf("fn(10) = %f\n", fn(10));
return 0;
}
$ ~/downloads/llvm-gcc-4.2-2.9.source/cbuild/install/bin/llvm-gcc -emit-llvm -c -o test.o test.c
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-link -o test test.o
$ ~/downloads/llvm-2.9/cbuild/Release/bin/llvm-dis -show-annotations test
$ ~/reposs/emscripten/emscripten.py test.ll > test.js
$ ~/downloads/v8/d8 test.js
test.js:727: TypeError: Property 'function sqrt() { [native code] }' of object 0,0 is not a function
var $2=FUNCTION_TABLE[$1](10);
^
TypeError: Property 'function sqrt() { [native code] }' of object 0,0 is not a function
at _main (test.js:727:33)
at Object.callMain (test.js:762:10)
at run (test.js:779:18)
at test.js:790:3
I'm going to guess they're not in FUNCTION_TABLE
since they're normally dynamically linked functions. In this case however, since emscripten is providing equivalent functions for these cstdlib function calls, they should be added to FUNCTION_TABLE
.
Apparently we fail lots of tests (but not all) on OS X. I suspect header differences.
Errors have been reported in dlmalloc, libcxx, debug and freetype.
(I don't have an OS X machine to test on myself - need help with this.)
We need a way to automate including library function that can be compiled from C. This would allow us to reduce the amount of hand-written JS code by quite a large margin. Two approaches have been proposed:
llvm-link
, then pass to the emscripten compiler. This is what's currently used for dlmalloc
.Input file is: http://desti.ruree.net/midi.o.ll
Output is following:
intertyper.js:303: Invalid token, cannot triage: // {"tokens":[{"text":"module"},{"text":"asm"},{"text":"\".symversnd_pcm_hw_params
// _get_access,snd_pcm_hw_params_get_access@ALSA_0.9\""}],"indent":0,"lineNum":5,"_
// _uid__":359}
throw 'Invalid token, cannot triage: ' + dump(item);
Input:
#include <cstdio>
#include <list>
#include <string>
using namespace std;
int main(void) {
list<string> string_list;
string_list.push_back("hello");
printf("%d\n", string_list.size());
return 0;
}
Output:
intertyper.js:137: TypeError: Cannot read property 'text' of undefined
while (tokens[openBrace].text != '{') openBrace --;
I've compiled the Python 2.7.1 source to a .ll
file successfully and passed it through emscripten successfully to get out a JavaScript file. The problem I now have is that the Closure Compiler dies when I try to run it over the emscripten-produced JavaScript file. In your Python demo, the JS file looks like it has been Closure Compiled. I was wondering if there was anything you need to do in order to get this to work?
$ ./emscripten.py ~/downloads/Python-2.7.1/bin/pylibs/python.ll > input.js
$ java -Xmx2g -jar ../compiler/build/compiler.jar --js_output_file output.js --js input.js
java.lang.RuntimeException: java.lang.RuntimeException: INTERNAL COMPILER ERROR.
Please report this problem.
Analysis did not terminate after 200000 iterations
...
$
emscripten.py fails on Python 2.6 and below due to argparse being 2.7+.
If there is no easy fix, then we should have a clear error message for this at least. Also, the test runner's error is even less clear, but that's a separate issue to fix.
Whether we need to rewrite anything depends on how popular 2.7 is, I guess. I wonder how many people still use 2.6. (I noticed this on a machine running Ubuntu 10.04, which is 2.6. I guess that's pretty old, but it is an LTS release...)
On the Python demo at http://syntensity.com/static/python.html type
print 'a', 'b'
Expected output:
a b
Actual output:
a
b
Basic idea is, add a PROFILING option to src/settings.js. When it is true, then entering and exiting a function will print to the console how much time was spent there. Then a commandline tool can parse that and generate profiling data, call stacks, etc.
While running the following I get an error:
$ ./emscripten.py demo.ll /home/chris/v8-trunk/d8
intertyper.js:393: TypeError: Cannot read property 'tokens' of undefined
params: parseParamTokens(item.tokens[2].item.tokens),
^
The full demo.ll file is here: https://gist.github.com/914021
Steps to reproduce (if desired). demo.hs:
main = putStrLn "Hello, World!"
Build:
/opt/ghc7/bin/ghc -fllvm -keep-llvm-files demo.hs -fforce-recomp # outputs demo.ll
I suspect this is probably not a bug in emscripten, as GHC may use some novel extensions to support its calling conventions and such. I don't really know anything about LLVM or how GHC uses it. I suppose it's worth posting; if other Haskellers try this they will get the same problem, undoubtedly.
When returning a structure from a function, only the first element is returned.
#include <stdio.h>
typedef struct {
int x; int y;
} data;
data f(int x, int y) {
data tmp;
tmp.x = x;
tmp.y = y;
return tmp;
}
int main() {
data z = f(5,6);
printf("%d %d\n", z.x, z.y);
}
This program prints "5 0" instead of "5 6". If adding a third field to the structure, d8 dies with the following error:
ReferenceError: $ is not defined
HEAP[$14]=$;; HEAP[$14+4]=1;; HEAP[$14+8]=5;
(llvm v2.9, v8 3.2.10.2, emscripten cca5278)
-- edit:
This only affects 64-bit compilation. Please compile with -m32 until 64-bit is properly supported.
Looks like the svn version of llvm produces things like this now:
%8 = call %2 @llvm.umul.with.overflow.i32(i32 %7, i32 4) ; [#uses=2 type={ i32, i1 }]
we crash on that. Also need to make sure we still parse the #uses info properly.
In a clean checkout, test_scriptaclass
fails on my mahcine due to being unable to find bindingtest.c
. Did you forget to check it in?
The InChI library is a somewhat complex C library used in chemistry.
The generated javascript is available at http:.//www.redbrick.dcu.ie/~noel/tmp/tmp.js.gz
"v8/d8 tmp.js" gives a Javascript error as follows:
$ ../v8-repo/d8 tmp.js
tmp.js:417618: SyntaxError: Unexpected token {
IHEAP[cnMe+6] = {$2{i16_2__i8_16__i8_0__i8_0__i8_undef}__%2_{i16_4__i8
It put a carat "^" underneath the first "{".
I'd really appreciate some help with this. The InChI library is available from http://www.iupac.org/inchi/download/version1.03/INCHI-1-API.zip. The makefile in INCHI-1-API/INCHI_API/gcc_so_makefile should be run as follows:
LINKER=pathto/emmaken.py SHARED_LINK=pathto/emmaken.py C_COMPILER=pathto/emmaken.py make
after commenting out the section "=== InChI Main Link rule ================" (whose compilation causes an llvm error).
Running LLVM opts sometimes produces calls to @llvm.uadd.with.overflow.i##
intrinsics, where ##
is 16/32/64. Emscripten does not have these implemented yet. Should be fairly simple to add.
The code: import os; print os.fork.__doc__
leads to:
JS crash: |ReferenceError: _dlopen is not defined|. Please let us know about this problem!
In theory it should be straightforward to support OpenGL ES, which is very close to WebGL. Some initial work is in src/library_gl.js There are some differences though, WebGL doesn't have clientside arrays for example.
It might also be possible to do full OpenGL, by emulating the function calls. This might end up being very slow. Might be interesting to look at ShaderGen, an open source tool to generate shader code for fixed-function code.
http://mew.cx/glsl/shadergen/
http://www.gp32x.com/board/index.php?/topic/55746-shadergen-for-linux/
When I try runnig test_indirectbr using clang on the master branch, I get the following in src.cpp.o.js
:
parseTools.js:499: TypeError: Cannot read property 'substr' of undefined
if ((!type || type == 'double' || type == 'float') && (value.substr && value
^
TypeError: Cannot read property 'substr' of undefined
at parseNumerical (parseTools.js:499:63)
at jsifier.js:166:61
at Array.map (native)
at parseConst (jsifier.js:166:36)
at Object.processItem (jsifier.js:192:22)
at Object.process (framework.js:155:26)
at framework.js:104:25
at Array.forEach (native)
at Object.solve (framework.js:92:27)
at JSify (jsifier.js:834:34)
Works fine on llvm-gcc though.
Currently the type info, in particular pre-calculated struct layouts, is not available to library code at either compile- or run-time. Since this info is already calculated from the LL, it would be nice to have it available directly, so one could call Runtime.generateStructInfo('foo')
(or something similar) and get an object defining the size of the foo
struct and the offsets of its members.
Right now malloc()
simply extends the stack address and free()
is a no-op. This means severe memory leaks if any serious emscripted program is used for any length of time. A real memory manager would be very useful indeed.
Moonlight is the open source, client-side .NET runtime.
http://www.mono-project.com/Moonlight
Moonlight is currently being compiled by Xamarin @ http://xamarin.com (Miguel de lcaza) to iPhone, iPad, & Android devices.
However, the most important platform - is the browser. Our goal is to have a .NET runtime for Javascript. This would open the flood gates for rich-client business applications of all kinds to be deployed to the browser, to run on any operating system or device.
Ryan
PS. Mono compiler has full, built-in LLVM output support. http://www.mono-project.com/Mono_LLVM
Running Emscripten with llvm 2.9 on my Windows system, test_raytrace fails with this output in src.cpp.o.js:
analyzer.js:285: TypeError: Cannot read property 'derivedTypes' of undefined
if (!func.variables[item.value.ident].derivedTypes) {
^
The problem seems to be related to Emscripten not understanding this llvm function signature:
define internal %struct.node_t* @_Z6createP6node_tii3v_tS1_d(%struct.node_t* %n, i32 %lvl, i32 %dist, %struct.v_t* byval align 4, %struct.v_t* byval align 4, double %r)
The complete llvm output can be found here (please note that I had to change iostream usage with printfs to make it compile):
http://pastebin.com/7s7vV5xJ
If I pass the v_t structs by const reference instead of by value in the create() function the test passes. In this case the llvm function signature looks like this:
define internal %struct.node_t* @_Z6createP6node_tiiRK3v_tS3_d(%struct.node_t* %n, i32 %lvl, i32 %dist, %struct.v_t* %c, %struct.v_t* %d, double %r)
If I insert these lines before line 285 in analyzer.js:
print(JSON.stringify(item.value.ident));
print(JSON.stringify(func.variables));
I get that item.value.ident is "$0", and then there is this oddly looking entry in func.variables:
"4":{"ident":"4","type":"%struct.v_t*byvalalign","origin":"funcparam","lineNum":216,"uses":null}
I'm guessing the "4" here comes from the parameter declaration:
%struct.v_t* byval align 4
Running emscripten on python.ll
with OPTIMIZE = 1
, RELOOP=1
and ASSERTIONS=1
produces many invalid blocks, of the general form:
if ($n) {
...
}
else {
...
}
else { throw "Bad multiple branching: " + __label__ + " : " + (new Error().stack); }
The extra else is added at jsifier.js:402. Removing these lines from the result produces a working file (which is about 10 times faster than the unoptimized version).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.