google / android-riscv64

Issues and discussions around RISC-V support in AOSP.

Android/riscv64

What is this?

This github project is solely for issue tracking/discussion/documentation purposes.

All Android/riscv64 work is being done directly in AOSP. Patches should be sent to AOSP using the usual AOSP contribution process and not as pull requests here.

You might also want to subscribe to the sig-android mailing list. (You need to send mail to [email protected] and then click "Join" on the mailing list web site.)

Status

We're currently (2023Q1) still working on cuttlefish virtual devices and ART, so for now all you'll have is a shell and command-line tools, and all the libraries they rely on. Until we have cuttlefish working you'll have to manually run them in qemu or on your own hardware.

You can see the current status of the riscv64 build in the aosp_riscv64 column of ci.android.com.

Can I try it?

Download the source using the usual AOSP download process and then:

$ cd aosp
$ source build/envsetup.sh
$ lunch aosp_cf_riscv64_phone-userdebug

============================================
PLATFORM_VERSION_CODENAME=UpsideDownCake
PLATFORM_VERSION=UpsideDownCake
TARGET_PRODUCT=aosp_cf_riscv64_phone
TARGET_BUILD_VARIANT=userdebug
TARGET_ARCH=riscv64
TARGET_ARCH_VARIANT=riscv64
TARGET_CPU_VARIANT=generic
HOST_OS=linux
HOST_OS_EXTRA=Linux-5.19.11-1rodete1-amd64-x86_64-Debian-GNU/Linux-rodete
HOST_CROSS_OS=windows
BUILD_ID=AOSP.MASTER
OUT_DIR=out
============================================
$ make -j

If you want to check whether a particular directory builds, cd into that directory and use mm -j.

Cuttlefish (emulator) setup

To launch cuttlefish, follow the general AOSP cuttlefish setup instructions.

(Note that in addition to the general setup mentioned above, if your host Linux distro doesn't already have it, you will have to apt install qemu-system-riscv64. If your host Linux distro's qemu version is too old -- we recommend 7.2.0 -- you may need to build your own. In that case, you'll need the -qemu_binary_dir= option when calling launch_cvd to point it at the correct copy of qemu.)

Getting to a shell (faster, but no graphics)

After building, run this following command from the same shell:

$ launch_cvd -cpus=4 --memory_mb=8192

After about 10s you should be able to use adb shell to connect to your riscv64 cuttlefish!

Getting to the home screen (slower, but with graphics)

After building, run this following command from the same shell:

$ launch_cvd -cpus=8 --memory_mb=8192 --gpu_mode=drm_virgl

You can then use vncviewer localhost:6444 to connect to your riscv64 cuttlefish!

(Note that even on a fast Xeon workstation it takes several minutes to get to the boot animation and tens of minutes to get to the home screen!)

How do I contribute?

Consult the regular AOSP Contributing documentation for more information on how to send us your patches.

Note that changes to projects under external/ (other than to Android.bp files) typically need to go to the upstream open source project in question first, and will then be merged into AOSP from there. In many cases, the METADATA file will point you to the upstream source, but feel free to ask here if not!

Review our Community Guidelines

This project follows Google's Open Source Community Guidelines.

android-riscv64's Issues

bionic/: assembler versions of the __mem*_chk functions?

would we actually benefit from them, or are the C versions good enough? (this bug is specifically about the _FORTIFY_SOURCE wrappers like https://cs.android.com/android/platform/superproject/+/master:bionic/libc/arch-riscv64/string/memset_chk.c, not the "real" functions like memset --- that's #7.)

external/boringssl: optimization

no risc-v assembler (openssl doesn't seem to have any either)

some of the https://github.com/riscv/riscv-crypto/releases/tag/v1.0.1-scalar instructions might be useful?

it seems like most people -- hardware and software -- would prefer to go straight to the vector crypto instructions instead, and that's what's been uploaded as patches to openssl already.

ship libFuzzer.a

both arm64 and x86_64 ship this in prebuilts/clang/host/linux-x86/clang-r475365/runtimes_ndk_cxx/

external/skia: optimization

lots of neon/sse intrinsics; we'll want similar for risc-v.

external/scudo/: optimize CRC32 when we have useful instructions

it looks like Zbr didn't make it? is Zbc's clmulr useful?

what's the ifunc story? hwcap.h

how are we going to do ifuncs if hwcap is limited/lagging?

there's still not even V in https://github.com/torvalds/linux/blob/master/arch/riscv/include/uapi/asm/hwcap.h and what does B look like in a world where it's been split into Zb[a-z]? those specific cases won't matter, because they'll be in the base Android riscv64 ABI, but if major mainstream extensions like that aren't getting the software support to enable ifuncs, what will the long tail look like?

clang: check that the global clang driver's riscv64 default flags make sense for Android

we've typically had bits of Android-specific configuration for the llvm targets, and we'll obviously want that to specify our base ABI, but we should run clang -v or whatever and manually check that all the defaults make sense.

• check defaults for C programs at -O0
• check defaults for C++ programs at -O0
• check defaults for C programs at -O2
• check defaults for C++ programs at -O2
• Create separate tasks for changes to defaults as needed.

llvm: missing libunwind support

it looks like no-one's tried to upstream riscv-android-src/toolchain-llvm-project@839f6ee yet (and worse, it looks like upstream has rewritten that code quite a bit, so there will be merge conflicts to deal with: https://github.com/llvm/llvm-project/blob/main/libunwind/src/UnwindCursor.hpp).

external/renderscript-intrinsics-replacement-toolkit/: optimization

depending on how much use of of this we have from code that's moved off renderscript (but not to C++ or vulkan), we might want to have rv64 implementations of these.

hardware: better atomics

See https://lists.riscv.org/g/tech-unprivileged/message/382 for a proposal.

llvm: make LTO work

(internal bug http://b/254713216)

currently worked around by disabling LTO, but we should fix that.

bazel: microdroid will need bazel support for riscv64

really just filing this for completeness. having talked to the bazel folks, this is not expected to be much work, but it does need to be done at some point (probably for microdroid first?).

since riscv64 is already in CI, no actual action should be needed from us?

external/libopus/: do we need optimization? (probably not?)

external/libopus/ has optimization for arm/x86/x86-64, but not for arm64, so i'm assuming this isn't interesting any more?

(check with essick, since he's in OWNERS.)

external/libopus/

Studio: gradle support

something along the lines of https://android-review.googlesource.com/c/platform/tools/base/+/2280530

external/flac/: need V optimization

external/flac/ has optimized code (using intrinsics rather than in .S files) for all the other architectures. we'll likely want something for risc-v's V extension.

bionic: switch over last builtins in libm once new clang lands

blocked on internal bug http://b/264965700 for the clang update that includes https://reviews.llvm.org/D136508. this should let us add || defined(__riscv) to the remaining #if defined(__aarch64__) blocks in https://cs.android.com/android/platform/superproject/+/master:bionic/libm/builtins.cpp.

simpleperf

(googlers: see internal bug http://b/267670561)

Currently simpleperf uses external/llvm, which is too old to build on riscv.
So currently simpleperf is disabled on riscv, in system/extras/simpleperf/Android.bp L69.
To fix it, we need to export llvm libraries needed by simpleperf (listed in Android.bp L34) in clang prebuilts.
And switch simpleperf to the prebuilt libraries.

external/lzma: optimization (probably not?)

there's arch-specific "custom filter chain" stuff in lzma, but it doesn't look like it was extended to arm64 (and does support sparc!), so it's presumably of historical interest only?

kernel: HAVE_EFFICIENT_UNALIGNED_ACCESS

bpf will fail when this feature is not enabled, and obviously Android won't support architectures where this isn't true, but the upstream situation seems unclear?

earlier lkml thread: https://lore.kernel.org/all/mhng-3469a0a8-a337-4b56-8e4f-b5a8bc47c4e9@palmerdabbelt-glaptop/

external/zlib optimization

external/zlib (upstream: https://chromium.googlesource.com/chromium/src/third_party/zlib/) has SSSE3 and NEON intrinsics for adler32 among other things. Should RISC-V have similar changes?

system/core/init/: more ASLR bits when we have Kconfig support for more

init can ask for more aslr bits when we have newer kernels: https://android-review.git.corp.google.com/c/platform/system/core/+/2251618/1..2/init/security.cpp#b120

TODO: find out what kernel version improves this; ToT still only has 24 in arch/riscv/Kconfig --- https://github.com/torvalds/linux/blob/master/arch/riscv/Kconfig#L150 ... it looks like maybe VA_BITS has been increased already (https://github.com/torvalds/linux/blob/master/arch/riscv/include/asm/pgtable.h#L60) but no-one came back to fix the Kconfig, so there are still upstream fixes needed before we can use this in init?

llvm: fix emutls for risc-v (probably not?)

Android's build system's global default is emutls because all the other architectures shipped before we had elf tls. there's no reason for risc-v to use emutls (but see #3 for exactly what we'll use instead), but it's a bit odd that emutls is completely broken, and might cause porting issues for others (apps/middleware) who're also explicitly specifying emutls in their build systems?

external/libyuv: optimization

there's lots of inline assembler for arm/intel; we'll want similar for risc-v.

external/aac: inline assembler (clz, saturating arithmetic)

external/aac/ needs inline assembler risc-v clz, saturating arithmetic etc (see include/arm for arm and arm64.)

bionic/: assembler versions of the mem* and str* functions using the V extension

we don't have any assembler for mem* or str* functions yet, but now the V extension is available, we should fix that.

external/XNNPACK: optimization

this whole project is mostly optimized assembler :-)

Zvediv looks like it would be very useful for them.

lldb: does it work sufficiently well for Studio?

at some point we should make sure that lldb works well enough for all the functionality exposed by Studio to work correctly. (if nothing else, "does it pass the manual QA tests?".)

llvm: make sure https://reviews.llvm.org/D131230 "[RISCV] Allow mismatched SmallDataLimit and use Min for conflicting values" gets merged

already too late for our january llvm update, but it would be nice to have https://reviews.llvm.org/D131230 "[RISCV] Allow mismatched SmallDataLimit and use Min for conflicting values" merged upstream in time for our next llvm update...

external/libhevc/: optimization

(this is an ittiam library, so this work will presumably be done by them.)

there's some optimized assembler that needs writing using V.

Studio: do we need goldfish too?

obviously we need cuttlefish for CI (#25), but will we need goldfish for Studio use?

alibaba has their goldfish patch up (which we're ignoring for now as we work on cuttlefish): https://android-review.git.corp.google.com/c/platform/external/qemu/+/2142497

at some point we'll have to talk to Studio about this.

external/libvpx: missing optimized assembler

external/libvpx (https://android-review.googlesource.com/c/platform/external/libvpx/+/2168948) is missing optimized assembler (present for arm/arm64/x86/x86-64), using C fallback for now.

llvm: should sqrt/sqrtf and the lrint/lrintf/llrint/llrintf family be builtins?

both riscv64 (https://cs.android.com/android/platform/superproject/+/master:bionic/libm/riscv64/) and arm64 (https://cs.android.com/android/platform/superproject/+/master:bionic/libm/arm64/) just have two .S files in libm, and they're all single-instruction functions for both architectures. this seems like something that llvm should just know about, so it can turn calls to sqrt etc into single instructions?

godbolt on a trivial function calling sqrt() shows that this isn't happening currently for arm64 or riscv64 (tested at -O2).

Investigate the current state of Auto-vectorization for RISC-V targets

• Compiling TSVC benchmark would be a good way to find out if commonly found loop structures are getting vectorized.
• Instruction scheduling of vectorized loops. If there is no instruction scheduling for RISC-V vectors, then we might have to create a separate task for this.
• Eliminate redundant vsetvl instructions. If we are not doing it already, a simple reaching definition analysis should accomplish this.

bionic/: should we have vdso support for cache flushing and <sys/cacheflush.h>?

glibc has a <sys/cacheflush.h> for risc-v, and linux has a vdso entry that might motivate it ... but if you look at that (https://github.com/torvalds/linux/blob/master/arch/riscv/kernel/vdso/flush_icache.S) it's just calling the system call anyway (since Android is always SMP), so i think the traditional "just use __builtin___clear_cache()" advice (https://man7.org/linux/man-pages/man2/cacheflush.2.html) makes sense here too, and we should not do this?

external/libpng: optimization

there's arm/intel-specific .c source that uses neon or sse intrinsics; we'll want similar for risc-v.

security: enable software CFI

until we have #15 (which hopefully we'll get in the Android/riscv64 ABI), we should at least enable software-only CFI.

(we had to disable this in the build system's global settings, suggesting at least some llvm work is needed here.)

llvm: function alignment

do we care? is the llvm default good enough? best to leave it to llvm to decide on a per-SoC basis rather than having anything in the build system?

security: hardware cfi ("landing pads") support

if https://github.com/riscv/riscv-cfi lands early enough, we'll want that as part of the ABI. even if it doesn't, we'll want to support it and enable it by default for the kernel and OS.

external/libjpeg-turbo: optimization

missing optimized code using V extension, using portable C fallback for now.

bionic/tests/sys_ptrace_test.cpp: add an instruction that writes more than 64 bits

we need to grab a V instruction to finish the missing piece of this test:

static void watchpoint_imprecise_child(Uint128_t& data) {
  raise(SIGSTOP);  // Synchronize with the tracer, let it set the watchpoint.

#if defined(__i386__) || defined(__x86_64__)
  asm volatile("movdqa %%xmm0, %0" : : "m"(data));
#elif defined(__arm__)
  asm volatile("stm %0, { r0, r1, r2, r3 }" : : "r"(&data));
#elif defined(__aarch64__)
  asm volatile("stp x0, x1, %0" : : "m"(data));
#elif defined(__riscv)
  UNUSED(data);
  GTEST_LOG_(INFO) << "missing riscv64 instruction to store > 64 bits in one instruction";
#endif
}

external/freetype: optimization (?)

src/smooth/ftgrays.c has one bit of arm64-specific configuration. check whether we need similar for rv64. (there's a lot more stuff for intel, but if arm64 hasn't needed it, does it matter?)

cuttlefish: get riscv64 cuttlefish up and running

adelva's working on it...

security: hardware shadow call stack support

we currently have https://reviews.llvm.org/D84414 available, but if/when https://github.com/riscv/riscv-cfi lands we should switch to an implementation based on that.

bionic/: implement TLSDESC (not yet in psabi)

risc-v is the first Android ABI to ship after we added ELF TLS support. emutls doesn't even seem to work (with LLVM at least) for risc-v, but we can't go straight to using TLSDESC relocations because they haven't been standardized for risc-v yet. blocked on riscv-non-isa/riscv-elf-psabi-doc#94 getting some traction.

llvm: prebuilts for hwasan support

when https://github.com/riscv/riscv-j-extension lands, we'll want to make sure we have working hwasan support.

external/libmpeg2: optimization

(this is an ittiam library, so this work will presumably be done by them.)

there's some optimized assembler that needs writing using V.

lld: investigate state of linker relaxation

alibaba had questions (what, exactly?) about the state of linker relaxation.

@MaskRay in https://maskray.me/blog/2022-07-10-riscv-linker-relaxation-in-lld had at least one suggestion for further work, but aiui he's saying we should not have gp relaxation (which isn't useful for realistic modern systems) and free up a register: riscv-non-isa/riscv-elf-psabi-doc#298

external/libavc/: optimization

(this is an ittiam library, so this work will presumably be done by them.)

looks like this library needs saturating arithmetic, for which there's no extension yet.

but right now, we need some optimized assembler written using V.

renderscript: go from deprecation to removal

(http://b/206676167 is the internal bug for the overall "remove renderscript", http://b/255624710 the more specific "remove *.rscript files from the tree".)

renderscript's been deprecated for a while, but there was no real motivation (and some downside) to actually remove it until now. with risc-v, we either have to (a) backport risc-v support to a decade-old fork of llvm, (b) forward port renderscript to current llvm, or (c) finally remove renderscript. despite the unfortunate disruption to legacy code, option (c) is the least difficult of these choices.

i've removed all the renderscript from frameworks/av/ already (after proving it was unused), and obviously all the renderscript tools and the CTS tests of renderscript itself are easy to delete, but we'll need to manage the Studio side of things (they ship some of the tools).

the main issues are these camera tests, all of which will need to be rewritten:

https://cs.android.com/search?q=f:%5C.rscript$%20-f:cts%2Ftests%2Ftests%2Frenderscript%20-f:cts%2Ftests%2Ftests%2Frscpp%20-f:rsblas%20-f:frameworks%2Fcompile%20-f:frameworks%2Frs%20-f:development%2Fsamples&sq=package:%5Eandroid-internal$