[DEVEL] Bluetooth Support about gnome-shell-extension-gsconnect

[andyholmes]

A short status update.

• I seem to have a fair amount of this working (standard plugins, connecting, etc)
• File transfers aren't implemented yet, but should be possible soon
• sshfs seems to be out of the question, unless a custom FUSE fs is implemented
• It's unclear how pairing is going to work with bluetooth connections (even in KDE Connect), so for now a device has to be paired over LAN first
• Sometimes there are hangs when connecting if the mobile device's screen is off :/

Some things to keep in mind:

• This won't be released until the rewrite is finished. You can test the rewrite branch now by building from git, although it's still unstable.
• Bluetooth still isn't a mainline feature in KDE Connect, so you'll have to build to the app yourself or use the linked build above.

[andyholmes]

Current status is that a multiplex connection protocol for Bluetooth was merged into kdeconnect-android about two months ago, while there is a WIP bluetooth-multiplex branch in GSConnect.

To test this, you'll have to uncomment the bluetooth link provider in kdeconnect-android and build an Apk, then fetch/rebase the bluetooth-multiplex branch in GSConnect. Last I checked, it seemed like the GC had trouble keeping up with the amount of objects being created to chunk out network packets by the multiplexer.

In reality, no one has really done any testing except the KDE Connect developer who designed the multiplexer and me, so development is basically stalled.

[andyholmes]

@getzze

Since there's been some work on bluetooth upstream to get payloads and such working, I've started actually working on this based on your implementation and SDP record.

I've had some limited success, although I'm worried GJS isn't equipped to handle file descriptors over DBus and from your implementation it seems Vala can automatically cast those to GSockets. The branch in general is a bit unstable, but I'll try and clear things up a bit so it's at least runnable.

Let me know if you're still interested in helping out with that. I'm attaching a link to a bluetooth build of v1.8.2 of the Android app I was given by one of the developers:

https://drive.google.com/open?id=1MynMgweRB68vJjfaeEP9WgEFboLVL8HW

UPDATE

Nevermind that about file-descriptors; got that working. Just working on getting the rest going now :)

[rugk]

I think this issue is not important at all. As long as you cannot use it with the precompiled KDEConnect app and have to re-compile it (with some options) by yourself, few people will use that feature, actually.

[Tuxino88]

Hi guys, I hope that with the new developers it will be possible to implement bluetooth

[andyholmes]

@getzze I'm pretty close to releasing v13, and since bluetooth support wouldn't be here at all without your help I thought of adding you to the About Dialog credits. Since your name/email would be there I thought I should ask if you'd like that or not. Let me know!

[getzze]

@andyholmes Of course, no problem! Thanks

[andyholmes]

As far as I know bluetooth is still considered an unstable feature, since it isn't even enabled in the beta Android app. There was some discussion a few months ago on reddit with the author, but it didn't sound especially useful. As far as I can tell there hasn't been any progress in about 7 months.

Implementing it might be possible, I haven't investigated what introspected libraries are available for implementing this. The Lan handler code would need to be separated from the Device channel code so that a Bluetooth handler can be implemented without duplicating a bunch of code.

Any help with this would be appreciated.

[getzze]

I have implemented a basic but working bluetooth link in Vala:
https://github.com/getzze/gconnect/blob/master/lib/channels/bluetooth/bluetoothlinkprovider.vala
It is hard for me to reimplement it all in js but I can help.
The main difficulty was to find the correct SPD record to connect with the phone, it is here;
https://github.com/getzze/gconnect/blob/master/lib/channels/bluetooth/bluetoothdbus-iface.vala
The rest is how to handle bluez with dbus, there are some tutorials around.
If you have a branch with a template for a bluetooth handler I can work on it.

[andyholmes]

That would be great, I'll admit I know very little about bluetooth so I hadn't looked much into it yet. My understanding is the only part really different is the socket is obtained from DBus, then the normal TLS protocol?

DBus clients in GJS can be simple using Gio.DBus.makeProxyWrapper("<xml>") like shown here, although I usually prefer making heavier GObject proxies that can call "notify::" like here where I used this WIP class as a base.

If you think you can get a DBus proxy working you could make a "BluetoothListener" along the lines of TcpListener that yields incoming connections. For now, you could probably try subclassing LanChannel and overriding the open(), accept() and request() methods to handle those sockets. This is basically what I did for LanUploadChannel and LanDownloadChannel.

Unfortunately I decided on doing a big rewrite and broke a bunch of things in branch "v10" which is where I'm doing all my work right now. Any help would be appreciate though, so feel free to pester me with javascript questions..

[andyholmes]

Sorry I've been so busy lately. I've been putting some work into protocol.js to make it easier to extend the Channel class (previously LanChannel) and getting a DBus skeleton for bluez mostly setup to make it easy on you.

As soon as what I have for v10 is remotely stable I'll push it and I'd be happy to hear your thoughts since you seem have a good grasp of GLib.

[getzze]

Sorry, I am really busy and I haven't find the time to help on this, but I will try to test this!
Great job!

One question, how do you test changes of the extension? Restarting the extension does not always work, I guess because of caching, and restarting gnome-shell (via f2-r) neither. So I usually logout-login, which is quite annoying.

[andyholmes]

Sorry, I am really busy and I haven't find the time to help on this, but I will try to test this!

No problem, your original work is the only reason I could figure it out since Qt is so much different.

One question, how do you test changes of the extension?

Yeah, even in the rewrite there's no button to restart the daemon (I'll put one in tonight). The daemon watches its own file for moves/deletes, so if you're using the ninja install-zip command it should automatically restart. Restarting the shell extension is only really useful for starting the daemon if its stopped.

Note

I mean it when I say it's unstable ;) It shouldn't break anything, but its rough around the edges in a number of places and some things just don't work.

[bepolymathe]

Thank you for what you've embarked upon.

I would find this feature very useful, especially for people who only have a non-shareable wired connection (which is the case at my job).... and who want to keep the phone in silent mode ;-)

Good luck to you. Sorry I can't help in this particular area.

[getzze]

I finally managed to test the bluetooth function! I could see my phone from Gnome and also see Gnome from my phone, that's great!
Then I cannot pair because Gsconnect is receiving the pairing requests but the phone is not receiving any packet (except from the initial identity).
I have to look more closely, probably build the app in debug mode.

Also in device.js, in the function _setPaired(bool), I changed this:

        if (bool) {
            if (this._channel.type === 'bluetooth') {
                debug(`Skip storing certificate for bluetooth connection`);
            } else {
                this.settings.set_string(
                    'certificate-pem',
                    this._channel.certificate.certificate_pem
                );
            }

[andyholmes]

Yeah, pairing is a little weird right now. I talked briefly with Matthijs who I think is continuing the bluetooth for KDE Connect about it, but not sure he understood what I meant. Currently devices are identified by hostname and verified by TLS certificate, but since bluetooth has no TLS certificate to verify I think it would be really easy to spoof a paired device by changing your hostname. Maybe it's not a problem since bluetooth devices have to be paired, but I think a lot of people want to use bluetooth at work/school on shared computers where it might be an issue.

The part I got stuck on was file transfers. Incoming transfers get a service UUID instead of a port, but I couldn't get a device to respond to one; same for outgoing transfers where you send a service UUID.

Oh also, if you use TCP and bluetooth with the same device you have to manually change it back and forth in dconf-editor right now, since I don't have any UI setup for it yet.

[getzze]

For bluetooth, the pairing is actually managed idependently from gsconnect, by bluez, and encrypting also. So pairing in kdeconnect should be always accepted automatically.
Is the communication through the bluetooth channel encrypted by gsconnect? I think it should not be.

For file transfer, I would say to use bluez, there is a file transfer protocol (https://en.wikipedia.org/wiki/List_of_Bluetooth_protocols#Object_exchange_(OBEX)). Probably you need to add this capability in the spd. Are file transfer already working in kdeconnect?

[andyholmes]

No extra encryption is added over Bluetooth in gsconnect, it just means the only identification is hostname. But as long as paired Bluetooth devices are assumed trusted for kdeconnect also that's okay.

In kdeconnect transfers are the same as TCP. Regular device channel with identity exchange, then a packet with size, service UUID (instead of TCP port) and optional checksum is sent. They don't use obex at all for transfers.

[getzze]

I see, sorry I had not understood what you meant by identification by hostname only. I guess it's fine, as all the identification/trusting/encrytion chain is managed by the bluetooth protocol.

In Vala, I never managed to add a UUID capability to the bluetooth connection without reconnecting. If it's the same for gjs, then the current approach for file transfer is not doable. Did you ask them why they are not using Obex? It's the same goal and it's already implemented!
Also the bluetooth protocol has a telephony protocol that would be great to integrate in kdeconnect.

[andyholmes]

That's okay it's not obvious until you play with the pairing code and you see there's sort of a hole where spoofing could occur. I have a bunch of TODOs I didn't commit reminding me to revisit it when KDE Connect makes a decision how to handle it.

I'm still fuzzy on how service UUIDs work, since I only use the DBus API for bluez. KDE Connect uses the Qt bindings which are really convenient and I wish GLib had a proper GIR for them. I have a prototype of a Bluez socket wrapper I wrote somewhere, I'll post tomorrow to see if you have any thoughts.

I've pushed a few fixes wrt pairing. I guess I broke device notifications and pairing awhile ago and never noticed, but it should be working now.

[getzze]

With the fixes I can pair my phone, thanks!
I tried to ping gsconnect from my phone and it works, but I couldn't find the button to ping my phone ( in the dbus interface either). Locate my phone works. For the remote plugin, the mouse is slow, lagging behind and the keyboard is not working, receiving the appropriate packet in gsconnect but not showing anything. It is probably a pb in the plugin, not the bluetooth connection. I cannot test via TCP now to see if the lag of the mouse is due to bt.
To be able to see my phone in the first place, I needed to restart the daemon, a pb I had in Vala also. That was a problem in how discovery was done I think.
Also, I cannot unpair from gsconnect (my phone unpairs well), but it is probably not a bt problem (again I could not check via TCP to be sure).

For the bt pairing, I would trust the bt protocol to be secure enough, so that we don't have to worry about spoofing. KDEConnect is actually mimicking features from the bt protocol (pairing, secure communication, file sharing, maybe audio bridge for telephony) to TCP, so we don't need to duplicate them on the bt channel.

The UUIDs seem really hard to work with in bluez5, that's why it seemed to me it would be easier to use native bt features like OBEX. It's supported out of the box in qt: https://doc.qt.io/qt-5/qtbluetooth-overview.html and most probably in Android also. But yeah, that would be a decision for the KDEConnect people.

[andyholmes]

Yeah, I moved almost all the "actions" from dedicated interfaces to Gio.Actions. I found the GActions DBus interface is a lot faster, they can be individually disabled and used with the GMenu API to pass whole menus over DBus easily. It's harder to test in D-Feet, but I've been working on a special debug interface for that I'll get back to later.

I fixed the mousepad plugin, it was just an oversight. I reorganized the code path to a switch statement, with mouse, scroll, keyboard first, so I think it's as fast as it will get.

Yeah, the most awkward part of bluetooth in KDE Connect is it conflicts with all the features they duplicate. Pairing, transfers, contacts...I'm not too worried since it's still not a standard feature and tethering can be used if necessary.

In the Qt bindings you basically just open a "socket" with the UUID, which I assume is the same as connecting a profile and waiting for NewConnection() to be called, but it never seems to connect. KDE Connect registers a dedicated service UUID for each transfer, so it should be a matter of swapping out the KDE Connect UUID for the one received in the payTransferInfo object, but I haven't been able to get that to work yet. The app also doesn't seem to unregister old UUIDs properly so my phone has dozens of them at all time now.

[getzze]

I submitted a PR #134 that fixes some problems with pairing.

I asked on KDEconnect issue tracker (https://phabricator.kde.org/T7447), I was wrong, OBEX is not fully working in Qt, so they are not planning on using it.
As I understood, bt UUIDs are not channels but service descriptors. Using one for the kdeconnect communication should be enough, then changing the bt channel for the transfers (I think there are 8 channels available). But I'm not sure I understood correctly the bt protocol...

I tested the mousepad plugin with TCP on another computer and it's working fast. On this same computer on bt it was fast also, so maybe a pb with the first computer.

The clipboard plugin is not working at all, the clipboard content is not sent to the phone. I have enable both Copy and Paste in the UI. What is this.device.activate_action('clipboardCopy', null) actually doing? Do I have to press a button to trigger clipboardCopy?

[andyholmes]

I submitted a PR #134 that fixes some problems with pairing.

That's awesome! But...after seeing your PR I took a much longer look at all the (current) code, then I looked more closely at KDE Connect's handling. You definitely caught some things I missed, but I also missed some other things. The problem with using NA as the certificate-pem is that will choke later when connecting on LAN, so I added a 'paired' GSetting (similar to KDE Connect). There were a bunch of other changes I wanted to make too, so I did them all in one go. One thing I noticed is the Android app doesn't seem to be hooked up to the paired setting so it unpairs if you later connect to LAN, but I think that Mattijs mentioned that will be done later.

I tested the mousepad plugin with TCP on another computer and it's working fast. On this same computer on bt it was fast also, so maybe a pb with the first computer.

It could also be the debug messages. I removed a lot of them in mousepad.js because even empty calls seemed to slow it down so disabling debug when testing might help

The clipboard plugin is not working at all, the clipboard content is not sent to the phone. I have enable both Copy and Paste in the UI. What is this.device.activate_action('clipboardCopy', null) actually doing? Do I have to press a button to trigger clipboardCopy?

Yeah, I sort of broke that and have some unstaged changes to get back to. A couple people have requested "sync on demand" and the only way to do that is to buffer them and offer a GAction that can be bound to a keybinding, I'll try and get back to that asap.

I'm pushing now, but there's some unstaged stuff that might still be broken, bear with me :)

[andyholmes]

I haven't done much work on this recently, but I built an Apk from git with bluetooth support, so I thought I'd share. This is 1.8.4+ whatever's in git: kdeconnect-bt-1.8.4.apk.

Reminder for those subscribed here: bluetooth is experimental at best, not supported and only in wip-rewrite, not master.

[mu4um]

something cool . i am using master in pc and f-droid version on android (i mean not bluetooth version). when they are connected by bluetooth gsconnect not working but when i enable bluetooth tethering on my phone gsconnect starts working with bluetooth . i tried some of features and they worked perfect . did you know this ?

[andyholmes]

Yep, tethering bluetooth will work with both KDE Connect and GSConnect since it's TCP over a bluetooth connection.

[eX00r]

Is Bluetooth now working in GSConnect with KDE Connect from Playstore? Or do I need to do something else?

[andyholmes]

Bluetooth is still experimental in GSConnect, KDE Connect Android and KDE Connect desktop. The original author of Bluetooth support in KDE Connect hasn't been around in a very long time, and no one else has picked up the project

To test it with GSConnect you'll have to build the Android app from git, and uncomment all the lines containing "bluetooth" that have been commented out.

[andyholmes]

Personal reminder for later: https://phabricator.kde.org/D17789

[ferdnyc]

Personal reminder for later: https://phabricator.kde.org/D17789

FYI, scheduled reminders are one of the handier installable features of Probot.

[arjan-s]

Hi, what is currently the status on this?

[kartheek7895]

Hi, what's the progress on this? Also if it's still in the experimental stage, what are steps one needs to do to test it?

[andyholmes]

There is no progress; development is still stalled without contributors or testers. You should expect this to require some effort to get it working at all.

To start contributing to GSConnect you'll have to pull the bluetooth-mutliplexing branch and probably rebase it on master.

To start working on kdeconnect-android you'll have to build an Apk with Android Studio, uncomment any lines that have been commented out (like here, here and here). You'll probably want to reference the Mutliplex Protocol and contact the upstream developers.

[devmrfitz]

Is there a way to implement bluetooth calls like Windows Your Phone app does?

[Tuxino88]

Is there a way to implement bluetooth calls like Windows Your Phone app does?

I think it's beyond the capabilities of the developer.

[andyholmes]

I'm going to go ahead and lock this now; no comment made since December 2018 is relevant to resolving the issue.

Please feel free to contact a maintainer to re-open the issue if you are interested in working on this feature.