GithubHelp home page GithubHelp logo

swiftandroid / swift Goto Github PK

View Code? Open in Web Editor NEW

This project forked from swiftlang/swift

721.0 55.0 35.0 83.02 MB

Port of Apple's reference Swift toolchain to Android; doesn't quite work yet

License: Apache License 2.0

Emacs Lisp 0.10% CMake 0.75% C 0.14% C++ 57.87% Python 1.17% Swift 38.31% Objective-C++ 0.47% DTrace 0.01% Objective-C 0.56% Shell 0.40% LLVM 0.15% Ruby 0.01% Makefile 0.01% Perl 0.01% Vim Script 0.04% Assembly 0.01% D 0.01%

swift's Introduction

Swift logo

# Swift Programming Language
Status
OS X Build Status
Ubuntu 14.04 Build Status
Ubuntu 15.10 Build Status

Welcome to Swift!

Swift is a high-performance system programming language. It has a clean and modern syntax, offers seamless access to existing C and Objective-C code and frameworks, and is memory safe by default.

Although inspired by Objective-C and many other languages, Swift is not itself a C-derived language. As a complete and independent language, Swift packages core features like flow control, data structures, and functions, with high-level constructs like objects, protocols, closures, and generics. Swift embraces modules, eliminating the need for headers and the code duplication they entail.

Documentation

To read the documentation, start by installing the Sphinx documentation generator tool by running the command:

easy_install -U Sphinx==1.3.4

More recent versions are currently not supported.

Once complete, you can build the Swift documentation by changing directory into docs and typing make. This compiles the .rst files in the docs directory into HTML in the docs/_build/html directory.

Many of the docs are out of date, but you can see some historical design documents in the docs directory.

Another source of documentation is the standard library itself, located in stdlib. Much of the language is actually implemented in the library (including Int), and the standard library gives some examples of what can be expressed today.

Getting Started

These instructions give the most direct path to a working Swift development environment. Options for doing things differently are discussed below.

System Requirements

OS X, Ubuntu Linux LTS, and the latest Ubuntu Linux release are the current supported host development operating systems.

For OS X, you need the latest Xcode.

For Ubuntu, you'll need the following development dependencies:

sudo apt-get install git cmake ninja-build clang python uuid-dev libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config

Note: LLDB currently requires at least swig-1.3.40 but will successfully build with version 2 shipped with Ubuntu.

If you are building on Ubuntu 14.04 LTS, you'll need to upgrade your clang compiler for C++14 support and create a symlink:

sudo apt-get install clang-3.6
sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-3.6 100
sudo update-alternatives --install /usr/bin/clang++ clang++ /usr/bin/clang++-3.6 100

Getting Sources for Swift and Related Projects

Via HTTPS For those checking out sources as read-only, HTTPS works best:

git clone https://github.com/apple/swift.git
cd swift
./utils/update-checkout --clone

Via SSH For those who plan on regularly making direct commits, cloning over SSH may provide a better experience (which requires uploading SSH keys to GitHub):

git clone [email protected]:apple/swift.git
cd swift
./utils/update-checkout --clone-with-ssh

CMake

CMake is the core infrastructure used to configure builds of Swift and its companion projects; at least version 2.8.12.2 is required. Your favorite Linux distribution likely already has a CMake package you can install. On OS X, you can download the CMake Binary Distribution, bundled as an application, copy it to /Applications, and add the embedded command line tools to your PATH:

export PATH=/Applications/CMake.app/Contents/bin:$PATH

Ninja

Ninja is the current recommended build system for building Swift and is the default configuration generated by CMake. If you're on OS X or don't install it as part of your Linux distribution, clone it next to the other projects and it will be bootstrapped automatically:

Build from source

Via HTTPS

git clone https://github.com/ninja-build/ninja.git && cd ninja
git checkout release
cat README

Via SSH

git clone [email protected]:ninja-build/ninja.git && cd ninja
git checkout release
cat README

Install via third-party packaging tool (OS X only)

Homebrew

brew install cmake ninja

MacPorts

sudo port install cmake ninja

Building Swift

The build-script is a high-level build automation script that supports basic options such as building a Swift-compatible LLDB, building the Swift Package Manager, building for iOS, running tests after builds, and more. It also supports presets which you can define for common combinations of build options.

To find out more:

utils/build-script -h

Note: Arguments after "--" above are forwarded to build-script-impl, which is the ultimate shell script that invokes the actual build and test commands.

A basic command to build Swift with optimizations and run basic tests with Ninja:

utils/build-script -r -t

Developing Swift in Xcode

build-script can also generate Xcode projects:

utils/build-script -x

The Xcode IDE can be used to edit the Swift source code, but it is not currently fully supported as a build environment for SDKs other than OS X. If you need to work with other SDKs, you'll need to create a second build using Ninja.

Testing Swift

See docs/Testing.rst.

Contributing to Swift

Contributions to Swift are welcomed and encouraged! Please see the Contributing to Swift guide.

To be a truly great community, Swift.org needs to welcome developers from all walks of life, with different backgrounds, and with a wide range of experience. A diverse and friendly community will have more great ideas, more unique perspectives, and produce more great code. We will work diligently to make the Swift community welcoming to everyone.

To give clarity of what is expected of our members, Swift has adopted the code of conduct defined by the Contributor Covenant. This document is used across many open source communities, and we think it articulates our values well. For more, see the Code of Conduct.

swift's People

Contributors

adrian-prantl avatar akyrtzi avatar annazaks avatar aschwaighofer avatar atrick avatar benlangmuir avatar bitjammer avatar cwakamo avatar cwillmor avatar davezarzycki avatar devincoughlin avatar douggregor avatar eeckstein avatar gottesmm avatar gribozavr avatar jckarter avatar jopamer avatar jrose-apple avatar lattner avatar milseman avatar mxswd avatar nadavrot avatar nkcsgexi avatar practicalswift avatar rjmccall avatar rudkx avatar slavapestov avatar swiftix avatar tkremenek avatar trentxintong avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

swift's Issues

libicu doesn't link at runtime on android systems with their own icu

I have been playing around with the JNI from within Android Studio and got my first direct Java->Swift->Java call working in a normal APK.

The problem I ran into is that libicu doesn't seem to load in the same way al the other libraries do. I repeatedly get a libswiftCore couldn't load because it couldn't find a certain libicu symbol (which I established is in libicui18n.so). Putting libicui18n.so in the system /lib directory gets the app running, but this requires the device to be rooted as far as I understand (I'd never heard of Android Studio let alone the JNI a month ago, bear with me).

I was wondering whether we can statically (instead of dynamically) link libicu for libswiftCore, and whether this may solve the problem of needing to link it at runtime (which apparently doesn't work). Who knows, maybe I'm just overlooking something, but I found the experience of manually setting all the shared libraries to load kind of painful overall. I also wonder if we can save on overall binary size with static linking? Excuse the noob questions, I'm new to this whole world but learning fast...

Find the correct way to package libicu data files

Swift needs at least the collation data; a full libicudata.so is like 25MB; need to find better solution for packaging it...

Swift needs something in the icudata, anyways, since for MakeRootCollator() without data it dies after

newfstatat(AT_FDCWD, "./share/icu/55.1/icudt55l/coll/ucadata.icu", 0xbed26088, 0) = -1 ENOENT (No such file or directory)
brk(0x41268000)                         = 0x41268000
futex(0x40f15054, FUTEX_WAKE_PRIVATE, 2147483647) = 0
newfstatat(AT_FDCWD, "./share/icu/55.1/icudt55l.dat", 0xbed26058, 0) = -1 ENOENT (No such file or directory)

JNI Support

I have a pretty solid base of a Swift JNI Wrapper ready. It doesn't do anything complex, but allows calling Java methods with varying levels of complexity, wraps some of the create jobject (e.g. Array) functions etc. At the moment it's coupled into my program code, but I'd like to make it a separate module and publish it as a part of SwiftAndroid.

For some reason I could only get JNI importing if I include $(NDK_PATH_TO_INCLUDE)/jni.h in the Bionic wrapper at the swift codebase level. Instead of that we'd ideally make JNI a separate Module so we can import JNI instead of import Glibc. I tried adding another target and ended up with no change in the output (my experience so far is that the build system seems to just ignore everything I do) โ€“ so I evidently don't understand the build system enough to do that.

I also couldn't seem to make a standard module out of it using the Package Manager, but that'd probably be the most maintainable solution in the medium-term. As a short aside, different users will want to target different Android-JDK versions, so maybe it'd be best to have users build the module themselves with a short script.

There's also the question of whether we include the Swift wrapper or just export the raw C functions. The C functions are a bit of a nightmare to work with, especially without an IDE, so my vote would be to include the Swift wrapper.

So this is me gauging interest / requesting help in packaging the module correctly and working on the wrapper for various use-cases.

1_stdlib/VarArgs.swift

******************** TEST 'Swift :: 1_stdlib/VarArgs.swift' FAILED ********************
Script:
--
rm -rf /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/test-android-armv7/1_stdlib/Output/VarArgs.swift.tmp && mkdir -p /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/test-android-armv7/1_stdlib/Output/VarArgs.swift.tmp && /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc -target armv7-none-linux-androideabi -Xlinker --allow-shlib-undefined -Xlinker -pie  -module-cache-path '/tmp/swift-testsuite-clang-module-cache0D5zbw'   -module-cache-path '/tmp/swift-testsuite-clang-module-cache0D5zbw' /home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift -o /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/test-android-armv7/1_stdlib/Output/VarArgs.swift.tmp/a.out -module-name main -Xfrontend -disable-access-control && /home/modocache/GitHub/apple/swift/test/../../swift-android-extras/rundroid /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/test-android-armv7/1_stdlib/Output/VarArgs.swift.tmp/a.out -parse-stdlib /home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift | FileCheck /home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift
--
Exit Code: 1

Command Output (stderr):
--
/home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift:15:25: error: cannot convert value of type '(_) -> Int32' to expected argument type 'CVaListPointer -> _'
  withVaList(arguments) {
                        ^
/home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift:39:20: error: cannot convert value of type '(_) -> Int32' to expected argument type 'CVaListPointer -> _'
  withVaList(args) {
                   ^
/home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift:63:20: error: cannot convert value of type '(_) -> Int32' to expected argument type 'CVaListPointer -> _'
  withVaList(args) {
                   ^

test/1_stdlib/VarArgs.swift fails to compile when targeting android-armv7. @hpux735 tells me it also fails for Linux armv6/armv7.

I found the error message hard to understand. cannot convert value of type '(_) -> Int32' to expected argument type 'CVaListPointer -> _' -- huh? So I made the following change:

diff --git a/test/1_stdlib/VarArgs.swift b/test/1_stdlib/VarArgs.swift
index 2791711..6e537f0 100644
--- a/test/1_stdlib/VarArgs.swift
+++ b/test/1_stdlib/VarArgs.swift
@@ -12,8 +12,8 @@ typealias CGFloat = Double
 #endif

 func my_printf(format: String, _ arguments: CVarArgType...) {
-  withVaList(arguments) {
-    vprintf(format, $0)
+  withVaList(arguments) { (vaList: CVaListPointer) in
+    vprintf(format, vaList)
   }
 }

This results in the following error message, which is much easier to understand:

/home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift:16:21: error: cannot convert value of type 'CVaListPointer' to expected argument type '__va_list'
    vprintf(format, vaList)
                    ^~~~~~

Aha! So vprintf expects a __va_list, but we're giving it a CVaListPointer.

As far as I understand, it's lib/ClangImporter's responsibility to convert certain raw C types like __va_list to CVaListPointer. Sure enough, there are a few commits related to va_list:

  1. swiftlang@2b6bee7 -- Imports C va_list as CVaListPointer. The author struggles with behavior on arm vs. x86_64--perhaps the same sort of problem we're seeing here?
  2. swiftlang@9d74d25 -- "va_list on i386 and arm64 is a pointer. On armv7, it's a struct that /contains/ a pointer. But on x86_64, it's an array of one struct element, which means it decays to a pointer as an argument but not as a variable or
    struct field." The patch cleans up the va_list import implementation.

It appears that on android-armv7 and linux-armv7, va_list is not being imported as CVaListPointer, and as a result our vprintf function takes a __va_list instead of a CVaListPointer.

Just for fun I tried adding the following mapping:

diff --git a/lib/ClangImporter/MappedTypes.def b/lib/ClangImporter/MappedTypes.def
index 474a78b..ccf4530 100644
--- a/lib/ClangImporter/MappedTypes.def
+++ b/lib/ClangImporter/MappedTypes.def
@@ -126,6 +126,7 @@ MAP_STDLIB_TYPE("u_int64_t", UnsignedInt, 64, "UInt64", false, DoNothing)
 // stdarg.h types.
 // FIXME: why does this not catch va_list on x86_64?
 MAP_STDLIB_TYPE("va_list", VaList, 0, "CVaListPointer", false, DoNothing)
+MAP_STDLIB_TYPE("__va_list", VaList, 0, "CVaListPointer", false, DoNothing)

 // libkern/OSTypes.h types.
 MAP_STDLIB_TYPE("UInt", UnsignedInt, 32, "CUnsignedInt", false, DoNothing)

This actually gets 1_stdlib/VarArgs.swift to compile, but it fails with the following error:

/home/modocache/GitHub/apple/swift/test/1_stdlib/VarArgs.swift:21:12: error: expected string not found in input
 // CHECK: The answer to life and everything is 42, 42, -42, 3.14
           ^
<stdin>:1:1: note: scanning from here
The answer to life and everything is 42, 42, -42, 0.000000
^

Anyway, something is wrong with va_list importing on android-armv7 and linux-armv7, but I'm not sure what yet.

Current status in NDK usage?

This isn't so much an issue as it is a usage question:

I have a small Swift framework that I'd rather not port into Java. It's only using NSObject for the setValue:forKey: magic, and that could be easily replaced. If it's written in pure Swift, how would I interface it from the Java side of things?

Get tests running

test/lit.cfg needs to have a way to push executables to Android and get stdout, stderr back correctly. Also, linker flags in lit.cfg needs to be tweaked to work on Android.

LLVM lit had a patch for running tests on an Android device but the patch was never merged. :(

The current plan is to make a small wrapper script that pushes to the device and use it in target_run

stdlib/FixedPoint.swift.gyb

******************** TEST 'Swift :: stdlib/FixedPoint.swift.gyb' FAILED ********************
Script:
--
rm -rf /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp && mkdir -p /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp && /home/modocache/GitHub/apple/swift/validation-test/stdlib/../../utils/gyb /home/modocache/GitHub/apple/swift/validation-test/stdlib/FixedPoint.swift.gyb -o /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp/FixedPoint.swift
/home/modocache/GitHub/apple/swift/validation-test/stdlib/../../utils/line-directive /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp/FixedPoint.swift -- /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc -target armv7-none-linux-androideabi -Xlinker --allow-shlib-undefined -Xlinker -pie  -module-cache-path '/tmp/swift-testsuite-clang-module-cacheeWSCZC'   /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp/FixedPoint.swift -o /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp/a.out
/home/modocache/GitHub/apple/swift/validation-test/stdlib/../../utils/line-directive /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp/FixedPoint.swift -- /home/modocache/GitHub/apple/swift/utils/android/adb_test_runner.py /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/validation-test-android-armv7/stdlib/Output/FixedPoint.swift.gyb.tmp/a.out
--
Exit Code: 1

Command Output (stdout):
--
/home/modocache/GitHub/apple/swift/validation-test/stdlib/FixedPoint.swift.gyb:284:7: warning: variable 'x' was never mutated; consider changing to 'let' constant
  var x = numericCast(0b11_1010_00) as T
  ~~~ ^
  let
/home/modocache/GitHub/apple/swift/validation-test/stdlib/FixedPoint.swift.gyb:285:7: warning: variable 'y' was never mutated; consider changing to 'let' constant
  var y = numericCast(0b10_1100_10) as T
  ~~~ ^
  let
/home/modocache/GitHub/apple/swift/validation-test/stdlib/FixedPoint.swift.gyb:292:7: warning: variable 'z' was never mutated; consider changing to 'let' constant
  var z = T.allZeros
  ~~~ ^
  let
Executable exited with a non-zero code on the Android device.
Device stdout:

To debug, run:
$ adb shell /data/local/tmp/2da39969-f/__executable_with_args

Re-running executable with command:
$ adb shell /data/local/tmp/2da39969-f/__executable_with_args

[ RUN      ] FixedPoint.UInt8(truncatingBitPattern:)
err>>> CRASHED: SIGSEGV
the test crashed unexpectedly
[     FAIL ] FixedPoint.UInt8(truncatingBitPattern:)
[ RUN      ] FixedPoint.Int8(truncatingBitPattern:)
err>>> CRASHED: SIGSEGV
the test crashed unexpectedly
[     FAIL ] FixedPoint.Int8(truncatingBitPattern:)
[ RUN      ] FixedPoint.UInt16(truncatingBitPattern:)
err>>> CRASHED: SIGSEGV
the test crashed unexpectedly
[     FAIL ] FixedPoint.UInt16(truncatingBitPattern:)
[ RUN      ] FixedPoint.Int16(truncatingBitPattern:)
err>>> CRASHED: SIGSEGV
the test crashed unexpectedly
[     FAIL ] FixedPoint.Int16(truncatingBitPattern:)
[ RUN      ] FixedPoint.UInt32(truncatingBitPattern:)
[       OK ] FixedPoint.UInt32(truncatingBitPattern:)
[ RUN      ] FixedPoint.Int32(truncatingBitPattern:)
[       OK ] FixedPoint.Int32(truncatingBitPattern:)
[ RUN      ] FixedPoint.UInt64(truncatingBitPattern:)
[       OK ] FixedPoint.UInt64(truncatingBitPattern:)
[ RUN      ] FixedPoint.Int64(truncatingBitPattern:)
[       OK ] FixedPoint.Int64(truncatingBitPattern:)
[ RUN      ] FixedPoint.UInt(truncatingBitPattern:)
[       OK ] FixedPoint.UInt(truncatingBitPattern:)
[ RUN      ] FixedPoint.Int(truncatingBitPattern:)
[       OK ] FixedPoint.Int(truncatingBitPattern:)
[ RUN      ] FixedPoint.UInt8(bitPattern: Int8)
[       OK ] FixedPoint.UInt8(bitPattern: Int8)
[ RUN      ] FixedPoint.Int8(bitPattern: UInt8)
[       OK ] FixedPoint.Int8(bitPattern: UInt8)
[ RUN      ] FixedPoint.UInt16(bitPattern: Int16)
[       OK ] FixedPoint.UInt16(bitPattern: Int16)
[ RUN      ] FixedPoint.Int16(bitPattern: UInt16)
[       OK ] FixedPoint.Int16(bitPattern: UInt16)
[ RUN      ] FixedPoint.UInt32(bitPattern: Int32)
[       OK ] FixedPoint.UInt32(bitPattern: Int32)
[ RUN      ] FixedPoint.Int32(bitPattern: UInt32)
[       OK ] FixedPoint.Int32(bitPattern: UInt32)
[ RUN      ] FixedPoint.UInt64(bitPattern: Int64)
[       OK ] FixedPoint.UInt64(bitPattern: Int64)
[ RUN      ] FixedPoint.Int64(bitPattern: UInt64)
[       OK ] FixedPoint.Int64(bitPattern: UInt64)
[ RUN      ] FixedPoint.UInt(bitPattern: Int)
[       OK ] FixedPoint.UInt(bitPattern: Int)
[ RUN      ] FixedPoint.Int(bitPattern: UInt)
[       OK ] FixedPoint.Int(bitPattern: UInt)
[ RUN      ] FixedPoint.UInt8.hashValue
[       OK ] FixedPoint.UInt8.hashValue
[ RUN      ] FixedPoint.Int8.hashValue
[       OK ] FixedPoint.Int8.hashValue
[ RUN      ] FixedPoint.UInt16.hashValue
[       OK ] FixedPoint.UInt16.hashValue
[ RUN      ] FixedPoint.Int16.hashValue
[       OK ] FixedPoint.Int16.hashValue
[ RUN      ] FixedPoint.UInt32.hashValue
[       OK ] FixedPoint.UInt32.hashValue
[ RUN      ] FixedPoint.Int32.hashValue
[       OK ] FixedPoint.Int32.hashValue
[ RUN      ] FixedPoint.UInt64.hashValue
[       OK ] FixedPoint.UInt64.hashValue
[ RUN      ] FixedPoint.Int64.hashValue
[       OK ] FixedPoint.Int64.hashValue
[ RUN      ] FixedPoint.UInt.hashValue
[       OK ] FixedPoint.UInt.hashValue
[ RUN      ] FixedPoint.Int.hashValue
[       OK ] FixedPoint.Int.hashValue
[ RUN      ] FixedPoint.BitwiseOperations/UInt8
[       OK ] FixedPoint.BitwiseOperations/UInt8
[ RUN      ] FixedPoint.BitwiseOperations/UInt16
[       OK ] FixedPoint.BitwiseOperations/UInt16
[ RUN      ] FixedPoint.BitwiseOperations/UInt32
[       OK ] FixedPoint.BitwiseOperations/UInt32
[ RUN      ] FixedPoint.BitwiseOperations/UInt64
[       OK ] FixedPoint.BitwiseOperations/UInt64
[ RUN      ] FixedPoint.OverflowCheck
[       OK ] FixedPoint.OverflowCheck
[ RUN      ] FixedPoint.String.init
[       OK ] FixedPoint.String.init
[ RUN      ] FixedPoint.byteSwapped
[       OK ] FixedPoint.byteSwapped
FixedPoint: Some tests failed, aborting
UXPASS: []
FAIL: ["UInt8(truncatingBitPattern:)", "Int8(truncatingBitPattern:)", "UInt16(truncatingBitPattern:)", "Int16(truncatingBitPattern:)"]
SKIP: []
To debug, run:
$ /data/local/tmp/2da39969-f/__executable --stdlib-unittest-in-process --stdlib-unittest-filter "UInt8(truncatingBitPattern:)"
Aborted 

FixedPoint.swift.gyb generates a massive amount of test cases: one for each Int type (such as UInt8 and Int64) as a source, to one of each Int type as a destination, for each of 27 bit patterns. By my math that's 8 * 8 * 27 == 1728 tests. Tests for four Int types fail on android-armv7:

  • UInt8
  • Int8
  • UInt16
  • Int16

There are three patterns that cause these tests to fail on android-armv7--deleting the following gets the tests passing:

diff --git a/validation-test/stdlib/FixedPoint.swift.gyb b/validation-test/stdlib/FixedPoint.swift.gyb
index ff78ea3..0cc6361 100644
--- a/validation-test/stdlib/FixedPoint.swift.gyb
+++ b/validation-test/stdlib/FixedPoint.swift.gyb
@@ -46,9 +46,6 @@ test_bit_patterns = [
   0x00007f123456789a,
   0x0000007f12345678,
   0x000000007f123456,
-  0x00000000007f1234,
-  0x0000000000007f12,
-  0x000000000000007f,
 ]

 def prepare_bit_pattern(bit_pattern, dst_bits, dst_signed):

The weird part is that the Int16 and UInt16 tests stop failing if any of the three patterns are removed. They pass if any two of those patterns is tested, but not all three!

Removing Int64 from the source Int types also gets Int16 and UInt16 tests passing (removing it from the destination types does not):

diff --git a/validation-test/stdlib/FixedPoint.swift.gyb b/validation-test/stdlib/FixedPoint.swift.gyb
index ff78ea3..8ae0c2a 100644
--- a/validation-test/stdlib/FixedPoint.swift.gyb
+++ b/validation-test/stdlib/FixedPoint.swift.gyb
@@ -85,7 +85,7 @@ truncating_bit_pattern_test_template = gyb.parseTemplate("truncating_bit_pattern

 FixedPoint.test("${Dst}(truncatingBitPattern:)") {

-%   for src_ty in all_integer_types(word_bits):
+%   for src_ty in list(all_integer_types(word_bits))[:-1]:
 %     Src = src_ty.stdlib_name
 %     if should_define_truncating_bit_pattern_init(src_ty=src_ty, dst_ty=dst_ty):
 %

Another interesting note: the tests appear to be crashing with a SIGSEGV. This happens on the line where expectEqual is called. Normally a failed expectation doesn't cause a crash. The semantics of expectEqual could be affecting the test--for example, the following causes Int16 and UInt16 tests to stop failing:

diff --git a/validation-test/stdlib/FixedPoint.swift.gyb b/validation-test/stdlib/FixedPoint.swift.gyb
index ff78ea3..54a0cb4 100644
--- a/validation-test/stdlib/FixedPoint.swift.gyb
+++ b/validation-test/stdlib/FixedPoint.swift.gyb
@@ -97,7 +97,8 @@ FixedPoint.test("${Dst}(truncatingBitPattern:)") {
 %       input = prepare_bit_pattern(input, src_ty.bits, False)
     let output = get${Dst}(${Dst}(truncatingBitPattern: input))
     let expected = get${Dst}(${prepare_bit_pattern(input, dst_ty.bits, dst_ty.is_signed)})
-    expectEqual(expected, output)
+    let e = output == expected
+    expectEqual(e, true)
   }

 %       end

We can diff what gyb generates before and after removing the three failing bit patterns to see the Swift for the failing tests, but there are just too many tests--we need a smarter way of pinpointing what's causing these tests to fail.


It was just impossible to debug these tests without getting a more granular idea of what was failing. So I tried splitting the test cases up:

diff --git a/validation-test/stdlib/FixedPoint.swift.gyb b/validation-test/stdlib/FixedPoint.swift.gyb
index ff78ea3..a0e8885 100644
--- a/validation-test/stdlib/FixedPoint.swift.gyb
+++ b/validation-test/stdlib/FixedPoint.swift.gyb
@@ -82,15 +82,13 @@ truncating_bit_pattern_test_template = gyb.parseTemplate("truncating_bit_pattern

 % for dst_ty in all_integer_types(word_bits):
 %   Dst = dst_ty.stdlib_name
-
-FixedPoint.test("${Dst}(truncatingBitPattern:)") {
-
 %   for src_ty in all_integer_types(word_bits):
 %     Src = src_ty.stdlib_name
 %     if should_define_truncating_bit_pattern_init(src_ty=src_ty, dst_ty=dst_ty):
 %
 %       for bit_pattern in test_bit_patterns:

+FixedPoint.test("${Dst}(truncatingBitPattern:) from ${Src}(${bit_pattern})") {
   do {
 %       input = prepare_bit_pattern(bit_pattern, src_ty.bits, src_ty.is_signed)
     let input = get${Src}(${input})
@@ -99,6 +97,7 @@ FixedPoint.test("${Dst}(truncatingBitPattern:)") {
     let expected = get${Dst}(${prepare_bit_pattern(input, dst_ty.bits, dst_ty.is_signed)})
     expectEqual(expected, output)
   }
+}

 %       end

@@ -112,7 +111,8 @@ FixedPoint.test("${Dst}(truncatingBitPattern:)") {

 %   end

-}
+// This comment prevents gyb from miscompiling this file.
+// <rdar://problem/17548877> gyb miscompiles a certain for loop

 % end
 """)

Lo and behold, this actually got the tests passing. Is it possible that if a test case is too long, it crashes? I tried making 10,000 assertions in each test:

diff --git a/validation-test/stdlib/FixedPoint.swift.gyb b/validation-test/stdlib/FixedPoint.swift.gyb
index ff78ea3..f066e29 100644
--- a/validation-test/stdlib/FixedPoint.swift.gyb
+++ b/validation-test/stdlib/FixedPoint.swift.gyb
@@ -82,23 +82,24 @@ truncating_bit_pattern_test_template = gyb.parseTemplate("truncating_bit_pattern

 % for dst_ty in all_integer_types(word_bits):
 %   Dst = dst_ty.stdlib_name
-
-FixedPoint.test("${Dst}(truncatingBitPattern:)") {
-
 %   for src_ty in all_integer_types(word_bits):
 %     Src = src_ty.stdlib_name
 %     if should_define_truncating_bit_pattern_init(src_ty=src_ty, dst_ty=dst_ty):
 %
 %       for bit_pattern in test_bit_patterns:

-  do {
+FixedPoint.test("${Dst}(truncatingBitPattern:) from ${Src}(${bit_pattern})") {
+% for i in range(10000):
 %       input = prepare_bit_pattern(bit_pattern, src_ty.bits, src_ty.is_signed)
+  do {
     let input = get${Src}(${input})
 %       input = prepare_bit_pattern(input, src_ty.bits, False)
     let output = get${Dst}(${Dst}(truncatingBitPattern: input))
     let expected = get${Dst}(${prepare_bit_pattern(input, dst_ty.bits, dst_ty.is_signed)})
     expectEqual(expected, output)
   }
+% end
+}

 %       end

@@ -112,7 +113,8 @@ FixedPoint.test("${Dst}(truncatingBitPattern:)") {

 %   end

-}
+// This comment prevents gyb from miscompiling this file.
+// <rdar://problem/17548877> gyb miscompiles a certain for loop

 % end
 """)

But this caused an out-of-memory error in gyb:

Traceback (most recent call last):
  File "/home/modocache/GitHub/apple/swift/validation-test/stdlib/../../utils/gyb", line 3, in <module>
    gyb.main()
  File "/home/modocache/GitHub/apple/swift/utils/gyb.py", line 1072, in main
    args.target.write(executeTemplate(ast, args.line_directive, **bindings))
  File "/home/modocache/GitHub/apple/swift/utils/gyb.py", line 977, in executeTemplate
    ast.execute(executionContext)
  File "/home/modocache/GitHub/apple/swift/utils/gyb.py", line 594, in execute
    x.execute(context)
  File "/home/modocache/GitHub/apple/swift/utils/gyb.py", line 670, in execute
    result = eval(self.code, context.localBindings)
  File "/home/modocache/GitHub/apple/swift/validation-test/stdlib/FixedPoint.swift.gyb", line 5, in <module>

  File "/home/modocache/GitHub/apple/swift/utils/gyb.py", line 978, in executeTemplate
    return ''.join(executionContext.resultText)
MemoryError

Repeating the same assertion 100 times caused the tests to pass as well. Baffling!

I have a few theories as to what might be happening:

  1. There's something wrong with gyb.
  2. There are limitations to the size of tests when run on an Android host, for some definition of "size".
  3. Some sort of spell or curse that has been cast upon me or my CPU.

I submitted swiftlang#1324, which miraculously gets this test passing.

Can share info about how to start porting to Windows..

Hi,
maybe some effort porting to Android can be shared for porting to Windows platform..
if yes can you share some plan on how to start a Swift port to Windows?
Would be even better if you can get interest in doing an early swift Windows port

Enums and Protocol conformance causes segfault on android

I finally managed to get my own compiled version of the swift toolchain working with the help of your swiftc-android script. I am now testing some code examples to see what works.

Compiling and linking multiple swift files seems to be fine.
Glibc seems to work fine (avoiding arc4_random of course ;) - pow, sqrt etc. at least..

The current issue is that even a simple enum like this one:

enum MusicalNote {
    case C, Csharp, D, Dsharp, E, F, Fsharp, G, Gsharp, A, Asharp, B
}

causes a segfault at runtime on android, even if no enum members are instantiated (it is enough to just have the above in any file you compile). I wouldn't have a clue how to start tracking this down. Maybe I should rebuild a debug version of the toolchain to try to get some more information about the crash?

I also tested a struct conforming to CustomDebugPrintable and Equatable and also got a segfault. I haven't narrowed that case down yet but it may well be related.

Cross-compile for Android from OSX

I have just pushed a branch called osx-crosscompile that is at the stage of correctly compiling the objects (as verified using nm from the different toolchains). It fails at the linker stage. I don't understand my way around the OSX toolchain just yet, or how to change it to use the Android toolchain.

Basically it's failing because the OSX linker doesn't understand --sysroot, up until that point, we need the --sysroot linker option because otherwise the Android components attempt to link against the OSX /usr/include, which throws all sorts of errors because of wrong architecture etc.

Another thought was that maybe we can use the iOS linker, since it is also aware of armeabi-v7a. Not sure how to do this, or whether it's a good idea.

I'm going to try figure out how to use the linker from a different toolchain tomorrow but given my (non-existent) experience in this field I'm not sure how far I'll get. Any hints or help are most welcome.

libicu data files need to be pushed to device

@modocache I don't think the build script pushes libicu yet; should that be added to the build script? I was getting "Illegal instruction" errors which turned out to be an abort() called by the runtime since libicu was missing.

Lib Bionic doesn't compile

I'm using your pre-built toolchain and I can't get swiftc-android to compile anything. It first tries to build Bionic, which doesn't work because the module map contains absolute paths:

// C standard library
  module C {
    //module complex {
    //  header "/home/zhuowei/ndk/platforms/android-16/arch-arm/usr/include/complex.h"
    //  export *
    //}
    module ctype {
      header "/home/zhuowei/ndk/platforms/android-16/arch-arm/usr/include/ctype.h"
      export *
    }

etc

After replacing these absolute paths with my own, Bionic still fails to build. It seems to be missing a lot of internal dependencies. I've tried setting C_INCLUDE_PATH=$ANDROID_NDK_HOME/platforms/android-16/arch-arm/usr/include, which gets me slightly further, as does reorganising the module map to import the C standard library first etc., but I think I'm missing a step here that will make it "just work". Any ideas?

Problem building on Ubuntu 15.10

I am trying to build the swift stdlib for Android from your fork and am getting stuck right at the start of the build process. I'm using Ubuntu 15.10 and have updated the paths to /home/zhuowei with equivalent paths of my own (that contain the expected files of course : )

The process hangs in two different locations, depending whether I'm starting with a clean build folder or not. With a clean folder it looks like this:

~# swift/swift/utils/build-script -R
Building the standard library for: swift-stdlib-linux-x86_64 swift-stdlib-android-armv7
Running Swift tests for: check-swift-linux-x86_64 check-swift-android-armv7

cmark: using gold linker
+ cd /root/swift/build/Ninja-ReleaseAssert/cmark-linux-x86_64
+ /usr/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=clang -DCMAKE_CXX_COMPILER:PATH=clang++ -DCMAKE_EXE_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_SHARED_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_BUILD_TYPE:STRING=Release /root/swift/cmark
-- The C compiler identification is Clang 3.6.2
-- The CXX compiler identification is Clang 3.6.2
-- Check for working C compiler using: Ninja

The process hangs here and appears not to pass the C compiling test (it doesn't crash though, I have to Ctrl-C to exit). I went into the build/Ninja-Release/cmark.../CMake-Temp folder and manually ran clang c-compiler-test.c, which compiles immediately without issue (the resulting a.out file didn't exist beforehand though).

With or without the manual C compiling step, if I break the original build-script process and try again, it looks like this:

# build-script -R
Building the standard library for: swift-stdlib-linux-x86_64 swift-stdlib-android-armv7
Running Swift tests for: check-swift-linux-x86_64 check-swift-android-armv7

cmark: using gold linker
+ cd /root/swift/build/Ninja-ReleaseAssert/cmark-linux-x86_64
+ /usr/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=clang -DCMAKE_CXX_COMPILER:PATH=clang++ -DCMAKE_EXE_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_SHARED_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_BUILD_TYPE:STRING=Release /root/swift/cmark
-- Performing Test HAVE_FLAG_ADDRESS_SANITIZER

Again, this is the end of the line. I get no error, the process doesn't crash or continue from here. Did you see this in your development process yourself? Any ideas what could be causing it?

I'm excited by your work on this. I fear I may be too inexperienced with Cxx to really help out here but if I see an opportunity to help out I will jump on it. Thanks

Setting up environment

I'm not an Android developer but would love to help in this one. I have a question how you have your environment set up. As I'm a mac user I would try to address this issue:

  • The current stdlib build system also assumes that the host and the target are of the same OS family, so currently only Linux hosts can target Android. Is there anything I can do to allow Mac hosts to also target Android?

It's a cool project!

IRGen/objc_simd.sil

******************** TEST 'Swift :: IRGen/objc_simd.sil' FAILED ********************
Script:
--
/home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swift -frontend -target armv7-none-linux-androideabi -sdk /home/modocache/android-ndk-r10e/platforms/android-16/arch-arm   -enable-source-import -sdk '/home/modocache/GitHub/apple/swift/test'/Inputs/clang-importer-sdk -I '/home/modocache/GitHub/apple/swift/test'/Inputs/clang-importer-sdk/swift-modules   -emit-ir /home/modocache/GitHub/apple/swift/test/IRGen/objc_simd.sil | FileCheck /home/modocache/GitHub/apple/swift/test/IRGen/objc_simd.sil --check-prefix=armv7
--
Exit Code: 1

Command Output (stderr):
--
/home/modocache/GitHub/apple/swift/test/IRGen/objc_simd.sil:34:17: error: expected string not found in input
// armv7-LABEL: define <3 x float> @simd_c_args_float3(<4 x i32>)
                ^
<stdin>:12:46: note: scanning from here
define <4 x float> @simd_c_args(<4 x float>) #0 {
                                             ^
<stdin>:54:1: note: possible intended match here
define <3 x float> @simd_c_args_float3(<3 x float>) #0 {
^

The problem is that android-armv7 has @simd_c_args_float3 take an argument of <3 x float>, like the x86_64 check expects. But the check for armv7 expects <4 x i32> for some reason.

Running the tests for iphoneos-armv7 works (once you remove this pesky warning from the lit config):

diff --git a/test/lit.cfg b/test/lit.cfg
index cbc3277..4c84e2e 100644
--- a/test/lit.cfg
+++ b/test/lit.cfg
@@ -503,8 +503,7 @@ if run_vendor == 'apple':
         (config.variant_triple, stdlib_resource_dir_opt, mcp_opt))
     target_options_for_mock_sdk_after = sdk_overlay_dir_opt

-    if 'arm' in run_cpu and swift_test_mode != 'non_executable':
-        raise RuntimeError('Device tests are not currently supported.')

     if 'arm' in run_cpu:
        # iOS/tvOS/watchOS device
swift $ ../llvm/utils/lit/lit.py -v ../build/Ninja-ReleaseAssert/swift-macosx-x86_64/test-iphoneos-armv7/IRGen/objc_simd.sil
lit.py: lit.cfg:249: note: using swift: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift
lit.py: lit.cfg:249: note: using swiftc: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swiftc
lit.py: lit.cfg:249: note: using sil-opt: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/sil-opt
lit.py: lit.cfg:249: note: using sil-extract: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/sil-extract
lit.py: lit.cfg:249: note: using lldb-moduleimport-test: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/lldb-moduleimport-test
lit.py: lit.cfg:249: note: using swift-ide-test: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift-ide-test
lit.py: lit.cfg:249: note: using swift-reflection-test: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift-reflection-test
lit.py: lit.cfg:249: note: using clang: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/bin/clang
lit.py: lit.cfg:249: note: using llvm-link: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/llvm-macosx-x86_64/bin/llvm-link
lit.py: lit.cfg:249: note: using swift-llvm-opt: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swift-llvm-opt
lit.py: lit.cfg:290: note: Using resource dir: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/lib/swift
lit.py: lit.cfg:316: note: Using Clang module cache: /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/swift-testsuite-clang-module-cachevjTIdD
lit.py: lit.cfg:320: note: Using code completion cache: /var/folders/ry/2ryfdsb56b30092626qprw6d3rb3ss/T/swift-testsuite-completion-cacheuAPXjd
lit.py: lit.cfg:511: note: Testing iOS armv7-apple-ios7.0
lit.py: lit.cfg:622: note: Running tests on iPhone OS version 9.2 (13C75)
lit.py: lit.cfg:801: note: Using platform module dir: /Users/bgesiak/GitHub/apple/build/Ninja-ReleaseAssert/swift-macosx-x86_64/lib/swift/%target-sdk-name/armv7
-- Testing: 1 tests, 1 threads --
PASS: Swift :: IRGen/objc_simd.sil (1 of 1)
Testing Time: 0.05s
  Expected Passes    : 1

In other words, it looks like the armv7 check works for most platforms. @hpux735 never mentions it in his lists of failing tests, so I'm assuming this works for linux-armv7 as well.

So why does simd behave differently on android-armv7?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.