cmusatyalab / vmnetx-android Goto Github PK

When the viewport is completely zoomed out, and the user shows/hides the soft keyboard, rotates the screen, or enters/exits fullscreen mode, we abruptly rescale the display (several times on entering fullscreen). Smoothly animate the rescale. android.view.animation may help.

While a connection is open, display a notification to remind the user that they have an active VM and to give them an easy way to return to it if they switch away from the app. This might also help remind them to terminate their VM when they're done with it.

Consider using a different API.

RelativeMouseHandler gets sensitivity and acceleration settings from RemoteCanvasActivity, where they are hard-coded. Make them user-configurable or hardcode them directly in RelativeMouseHandler.

GestureDetector imposes a delay before a single-tap is processed, since it's waiting to see if more taps will follow. This makes the viewer UI less responsive than it could be. On the other hand, we probably do want to detect double-taps ourselves, rather than passing two independent single-taps to the guest, since it reduces the chance that the guest will misdetect the taps as two separate single-taps due to network jitter.

See android.widget.Scroller.

Right now we permit everything. Instead, use a sane StrictMode policy and fix the problems it finds.

Eclipse is deprecated.

Note that the GStreamer SDK writes source files into project directories that are supported by Eclipse but not used by Android Studio. This will require manual hacking, or an updated GStreamer SDK that supports Android Studio.

Try to run all of the separate manual build steps (build-deps.sh, ndk-build) from Gradle tasks, so there's one unified top-level build command.

Celt 0.5.1 is deprecated; the replacement is Opus. Opus is not supported by the SPICE servers in RHEL 6 or 7, so we will need to support both codecs. However, Opus has an internal copy of Celt, so if one or both are built as static libraries, there will be symbol conflicts. Build Celt as a shared library, then enable Opus in the GStreamer bundle SO.

Cerbero now has a bundle-source command. If the GStreamer 1.6 SDK binaries are shipped alongside a source tarball, ship those and stop building the SDK ourselves. If not, switch to using bundle-source rather than tarring up the Cerbero build tree.

GestureHandler implements delays between mouse button state changes by calling sleep() on the UI thread. Implement via callbacks instead, but beware of introducing races.

Allow launching multiple VMs and switching between them.

When closing the control connection, we should be sure to transmit all pending data first (unless an error or timeout occurs).

We detect the disappearance of the network via control protocol pings, and the reappearance of the network via repeated connection attempts. This is power-inefficient. To the extent possible, optimize by subscribing to platform notifications of network status.

Currently, the user has to touch with one finger, then touch and drag vertically with a second finger. This is unintuitive. Permit scrolling by touching and dragging with two fingers at once; this will need to be distinguished from a pinch gesture.

If the application is running in the background and is terminated by Android to free memory, the control connection will be closed and the server will terminate the VM, causing the user to lose data. Improve the survivability of the control connection by moving it to a service, so its lifetime can be independent of the viewer UI.

When handling an ACTION_MULTIPLE KeyEvent with a keycode other than KEYCODE_UNKNOWN, we should respect the reported repeat count.

It's divided between RemoteKeyboard and SpiceCommunicator.

The Android IME API wants a lot of context about the text being edited (for autocorrect, word completion, etc.) that we're not able to provide. The problem is that we want an on-screen keyboard, but the IME system provides a generic input API and expressly not a keyboard. The default Android IME, unlike some other IMEs, does provide a fallback mode that returns key-at-a-time output, which usually works well enough for our purposes. However, this mode is deprecated and doesn't always work properly. (For example, Swype-style input doesn't return a word until after the next word is input.)

The recommended fix is to avoid the IME API and implement our own keyboard UI. This could perhaps be based on the Hacker's Keyboard. It might also permit elimination of the existing on-screen keys.

We currently ignore check_display requests from the server.

Android N adds an API to set the mouse pointer. Use it to hide the system pointer, which:

• in absolute mode, uselessly duplicates the SPICE pointer, and
• in relative mode, is completely unrelated to the position of the remote pointer.

The full complement of gestures is not necessarily obvious, so we should provide some sort of documentation. Ideally this would be presented in bite-sized chunks in some natural way, rather than as a wall-of-text help screen.

Absolute mode

• To click, tap the screen
• To drag, tap and move your finger
• To right-click, tap with two fingers
• To scroll, press with one finger, then press and drag vertically with a second finger
• To zoom, pinch
• To pan when zoomed, press and hold until vibration and message
• To show/hide soft keyboard, press keyboard button in toolbar
• To show/hide extra keys (arrow keys, modifiers, Tab, Escape), press joystick button in toolbar
• To exit full-screen mode, swipe from top or bottom of screen

Relative mode

• To move the mouse, tap and move your finger
• To click, tap the screen
• To drag, press and hold until vibration, then move your finger
• To right-click, tap with two fingers
• To scroll, press with one finger, then press and drag vertically with a second finger
• To zoom, pinch
• To show/hide soft keyboard, press keyboard button in toolbar
• To show/hide extra keys (arrow keys, modifiers, Tab, Escape), press joystick button in toolbar
• To exit full-screen mode, swipe from top or bottom of screen

Reliably triggers on Nexus 5/Android 6.0.1 and Pixel C/Android 7.0. I believe this is the crash that spice-marshaller-sigbus.patch is meant to fix.

08-24 16:54:41.949 15478 15542 F libc    : Fatal signal 7 (SIGBUS), code 1, fault addr 0x9e47109e in tid 15542 (Thread-2630)
08-24 16:54:42.051  1439  1439 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-24 16:54:42.052  1439  1439 F DEBUG   : Build fingerprint: 'google/hammerhead/hammerhead:6.0.1/MOB30Y/3067468:user/release-keys'
08-24 16:54:42.052  1439  1439 F DEBUG   : Revision: '11'
08-24 16:54:42.052  1439  1439 F DEBUG   : ABI: 'arm'
08-24 16:54:42.053  1439  1439 F DEBUG   : pid: 15478, tid: 15542, name: Thread-2630  >>> org.olivearchive.vmnetx.android <<<
08-24 16:54:42.053  1439  1439 F DEBUG   : signal 7 (SIGBUS), code 1 (BUS_ADRALN), fault addr 0x9e47109e
08-24 16:54:42.105  1439  1439 F DEBUG   :     r0 9e47109e  r1 00000012  r2 f85c74aa  r3 000006b4
08-24 16:54:42.105  1439  1439 F DEBUG   :     r4 b3b34e48  r5 b3b34e60  r6 9e456338  r7 9e456470
08-24 16:54:42.105  1439  1439 F DEBUG   :     r8 9e456338  r9 00000000  sl 00000000  fp 9d8ebcc4
08-24 16:54:42.106  1439  1439 F DEBUG   :     ip 00000012  sp 9d67a820  lr 9e471094  pc 9d923438  cpsr 20000010
08-24 16:54:42.106  1439  1439 F DEBUG   : 
08-24 16:54:42.106  1439  1439 F DEBUG   : backtrace:
08-24 16:54:42.106  1439  1439 F DEBUG   :     #00 pc 0009a438  /data/app/org.olivearchive.vmnetx.android-2/lib/arm/libspice.so (spice_marshaller_add_uint64+24)
08-24 16:54:42.444  1439  1439 F DEBUG   : 
08-24 16:54:42.444  1439  1439 F DEBUG   : Tombstone written to: /data/tombstones/tombstone_04
08-24 16:54:42.444  1439  1439 E DEBUG   : AM write failed: Broken pipe
08-24 16:54:42.445  1919 15549 W ActivityManager:   Force finishing activity org.olivearchive.vmnetx.android/.RemoteCanvasActivity

• Convert buttons to nine-patch images
• Ensure button sizes are device-independent

Display authors, copyright information, and license blurb.

Split the codebase into two repos:

1. A library providing a generic SPICE client in the form of a Fragment, plus a simple application to exercise it.
2. The VMNetX UI, control connection, and viewer connection setup code, wrapped around the generic code.

The input subpackage should probably be moved into the base package to reduce the exposed API surface of the library.

The Message mechanism isn't very clean API, so consider handling event callbacks via a callbacks object, in which the application subclasses a base class with null implementations (so new callbacks can be added without breaking ABI).

• Android shows "App may not support split-screen" popup when launching in split-screen mode
• When running in the bottom pane, absolute pointer mode calculates touch coordinates incorrectly
• When running in the bottom pane with YouTube in the top pane, opening the soft keyboard causes YouTube to stop playing video after a few seconds

If the control connection drops, or if pinging stops for an extended period of time, the user will lose data.