connectbox / connectbox-pi

ConnectBox is a media sharing device based on small form factor computers like the Raspberry Pi

TheWell version of ConnectBox

TheWell is a variant of ConnectBox that adds Moodle Learning Management System (v. 3.9.3), PHP (v. 7.4) and MySQL (MariaDB) (vv 10.3) to bring training system and learning content to the ConnectBox platform.

Summary Of Changes:

• ConnectBox Ansible roles are updated to build ConnectBox with Moodle, PHP and MySQL
• TheWell is for Debian OS (Raspbian) on Raspberry Pi (with modifications) or other Linux host
• Refer to Relay Trust Moodle Repo for Documentation Of Changes
• Default Moodle MySQL database is located in this repo under ansible/roles/moodle/templates/
• Legacy Connectbox File Serving is now at {{{hostname}}} such that Connectbox is http://thewell, Moodle is http://learn.thewell and Admin is http://thewell/admin
• (There will be more as this gets built out)

ConnectBox

ConnectBox is a media sharing device based on small form factor computers including the Raspberry Pi 3, Raspberry Pi Zero W, NanoPi NEO, Orange Pi Zero and Pine64.

Making a ConnectBox

See docs/deployment.md

Making a Connectbox on AWS

See [docs/awsinstall.md](docs /docs/awsinstall.md)

Connectbox setup and administration

See docs/administration.md

Developing the ConnectBox Software

See docs/development.md

MicroSD Card Images/Releases

TBD

connectbox-pi's Issues

Allow ssid, channel and hostname setting without admin UI

Allow specification of the ssid, channel and hostname via txt files in the root of the USB stick. These files will be read and applied only at boot-time and the changes will be done via the same script that is used to perform actions on behalf of the admin interface. These config files should not be deleted, so they reapply on next reboot (unlike the reset file in #53)

Match icons to font awesome directly

Currently we map key directory names to specific icons in the client interface. We will be providing instructions to connectbox administrators about how to layout files and directories on their USB stick place in order to take advantage of the iconography, so instead of constraining them to a small predefined mapping, let’s direct them to the FontAwesome Icon List and tell them that they can name their directories based on the FA icon names. e.g. a directory called paint-brush maps to http://fontawesome.io/icon/paint-brush (and the logic puts the element in the fa-paint-brush class).

@furnox / @kldavis4 , is one of you able to help with this?

Thanks for the suggestion, @GeoDirk

Package database is sometimes locked after initial playbook reboot

fatal: [192.168.88.33]: FAILED! => {"changed": false, "cmd": "apt-get install python-apt -y -q", "failed": true, "msg": "E: Could not get lock /var/cache/apt/archives/lock - open (11: Resource temporarily unavailable)\nE: Unable to lock directory /var/cache/apt/archives/", "rc": 100, "stderr": "E: Could not get lock /var/cache/apt/archives/lock - open (11: Resource temporarily unavailable)\nE: Unable to lock directory /var/cache/apt/archives/\n", "stderr_lines": ["E: Could not get lock /var/cache/apt/archives/lock - open (11: Resource temporarily unavailable)", "E: Unable to lock directory /var/cache/apt/archives/"], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n  python-apt-dbg python-apt-doc\nThe following NEW packages will be installed:\n  python-apt\n0 upgraded, 1 newly installed, 0 to remove and 52 not upgraded.\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", "  python-apt-dbg python-apt-doc", "The following NEW packages will be installed:", "  python-apt", "0 upgraded, 1 newly installed, 0 to remove and 52 not upgraded."]}

I’ve seen this when the unattended upgrades are running. We haven’t discussed whether we’d expect these devices to automatically upgrade if/when they have internet connection - if we don’t, then we can simply remove the unattended-upgrades package and this shouldn’t happen again.

Connectbox client UI should resize user supplied icons to be consistent with FA icons

Admin interface allows SSIDs over the maximum length

SSIDs are limited to 32 octets (http://standards.ieee.org/getieee802/download/802.11-2007.pdf). With the enabling of UTF-8 SSIDs and multi-byte characters, this means that it's quite easy to exceed the maximum length. We should reject updates via the API for SSIDs that exceed this length and constrain the interface (or give feedback) when the maximum length is reached.

There are two skipped tests, test33CharacterPlainSSIDSet and test33CharacterUnicodeSSIDSet that can be un-skipped once checking is being performed in the API.

Disable bluetooth for improved wifi on Raspbian

@matheweis and @GeoDirk found that disabling bluetooth at the boot prompt on RPi devices and in systemd (to remove the red "failed" message during boot) improves wifi performance. We should at least offer this as a build time choice, if not make it the default.

Allow out-of-band reset to defaults

If someone is locked out of the admin UI, there’s no way to get back in.

We need to allow triggering of a reset-to-defaults action if a particular file is present on the USB stick. The file should be automatically removed after the reset-to-defaults action. For ease of implementation, and to avoid having a periodic task, this action should only happen at boot time. i.e. one workflow would be:

1. yank power
2. unplug USB stick and create magic file (say, _reset.txt in the root of the USB stick),
3. re-insert USB stick
4. re-attach power

Add popular content list to the front page

@GeoDirk requested that popular content is highlighted on the front page. This might be something like a top 10 popular files. This need not be accurate to the minute i.e. we can batch process the webserver logs and create a file that describes the top content.

Build failures related to AWS instantiation

Build 254 failed due to problems on the AWS side. It timed out waiting for ssh access with the following error:

Error applying plan:
1 error(s) occurred:
* Resource 'aws_subnet.default' does not have attribute 'id' for variable 'aws_subnet.default.id’

The terraform destroy then failed with:

Error applying plan:
1 error(s) occurred:
* aws_vpc.default: DependencyViolation: The vpc 'vpc-37803851' has dependencies and cannot be deleted.
	status code: 400, request id: 65192abf-2a43-42f0-95ac-39d50361033c

The immediate followup terraform apply then failed with:

Error applying plan:
1 error(s) occurred:
* aws_subnet.default: Error creating subnet: InvalidSubnet.Conflict: The CIDR '10.0.1.0/24' conflicts with another subnet
	status code: 400, request id: a8b911bc-3300-4f1d-99c7-2074165a1fef

Then the final terraform destroy failed with the same sort of dependency violation:

Error applying plan:
1 error(s) occurred:
* aws_vpc.default: DependencyViolation: The vpc 'vpc-37803851' has dependencies and cannot be deleted.
	status code: 400, request id: adc58245-20e4-48b1-a1c2-211b6ba1c90c

The first problem is new since I fixed terraform apply. The failure also happened in build #278 but I re-ran it before capturing the output.

Write first-run admin docs

Dnsmasq doesn’t always answer DHCP requests after wifi unplug/replug

This is a dependency issue - dnsmasq needs to be restarted after the wlan interface is brought up.

@matheweis suggested:

I was going to disable the auto start of dnsmasq and put post-up and pre-down callback in the interfaces configuration that started/stopped it, or something with similar effect if there is a better way.

Also, confirm whether this has been seen at boot.

php5-fpm package not always found on Armbian

Sometimes the php5-fpm package can't be found on Armbian

(Specific image for this case was Armbian_5.25_Orangepizero_Ubuntu_xenial_default_3.4.113.img)

Allow serving of static site

Currently the nginx config is tightly coupled to the icon-only interface. We want to allow groups to use the device to display a regular website instead of the icon-only interface.

There is a separate task ( #127 ) to exposing a toggle in the admin interface to switch between icon-only mode and website mode. Until that is done we'd need to provide an icon-only image, and a website image.

Fix Android and Windows captive portal behaviour

We don’t have any captive portal logic for Android or Windows (phone/desktop) so I’m expecting that we’re still trapped inside the captive portal browsers (I don’t have devices to test, though)

The following resources should be sufficient to show what needs to be implemented:

• https://github.com/httpwg/wiki/wiki/Captive-Portals
• https://serverfault.com/questions/679393/captive-portal-popups-the-definitive-guide#
• https://msdn.microsoft.com/en-us/library/windows/hardware/dn408681.aspx

Decision: Development mode or non-dev mode for pilot

Dev mode means that passwords are easy to break and ssh is enabled.

Create CI pipeline for Ubuntu

If we’re supporting it, we need to run our test suite against it.

Mainline kernel gives different eth0 MAC addresses on each boot

While this is certainly a pain while developing and testing, this might not be a problem for deployment if we don't expect people to access the device via ethernet interface.

Enable internal USB ports on NEO

When we get to using the daughterboard to access to the internal USB ports on the Neo (for mainline), we may need to play with the device trees/overlays to get them to work.

There's a guy on the armbian forums that seems to have got it sorted, so for reference: https://forum.armbian.com/index.php?/topic/4470-nanopi-neo-internal-usb-seems-disabled/

Decide on OS/Kernel combo for GA

This involves side-by-side testing for debian vs ubuntu and mainline vs legacy for unit performance and battery life.

dns-dhcp task fails if wlan0 is not up

TASK [dns-dhcp : Start and enable dnsmasq] *************************************
fatal: [192.168.1.136]: FAILED! => {"changed": false, "failed": true, "msg": "Job for dnsmasq.service failed because the control process exited with error code. See "systemctl status dnsmasq.service" and "journalctl -xe" for details.\n"}

Admin interface should allow uploading of content to USB stick

Decision: Which OS + Kernel combination for pilot

Mainline and Legacy kernel
Debian and Ubuntu OS

Allow display of non-USB content alongside USB content

#28 removed the ability to ship content on the Pi, and have it appear in the client interface alongside content that is stored on the USB device. Being able to ship content seems like a good feature, so we should add this back in.

Add chat interface

@GeoDirk mentioned that a group chat interface is desirable (simple broadcast, not one-to-one chats) and that it should be visible from the front page. Chat history should not be maintained, but a 1h rolling window of histroy - don't keep messages longer than this duration, even if they aren't visible in the chat window.

Improve structure of client html

Create build to combine and minify JS, CSS, etc
Have build pull in current version of font awesome css assets automatically (or a tag)

Keyboard Support

By default, the Raspberry Pi's are coming with keyboards configured for the UK. It would enhance our configuration if the keyboard layout was defaulted to the 'US' layout.

All that needs to be done is modify the one line in /etc/default/keyboard to look like this:

XKBLAYOUT="us"

Filter standard operating system folders on USB from icon interface

Originally mentioned by @algebur . A windows-formatted USB key has a folder called System Volume Information which shows in the interface. We should filter this out, as it’ll never have any end-user managed files.

Similarly for .DS_Store directories that might be present on USB keys used on a Mac.

Migrate admin api from php to python

Improve initial connect experience

When someone connects to the wifi there’s no indication what they need to do next. While entering an http URL will bounce them to the connectbox interface, one with an explicit https protocol won’t and a site that uses HSTS also won’t.

Look at using a standard captive portal workflow on connect.

Make deployment of sample content optional

testSetSSID does not return SSID to original value

This means that running the tests results on the SSID ultimately being of the form ssid-1[0-9]{9}. It should do what it's parents said, and leave things just the way it found it.

Internal NEO USB ports stop working when kernel is updated

a /boot/dtb directory is created on each kernel update, however we create an overlay in that directory on our initial run in order to activate the internal USB ports. It’s hard to create that overlay on each kernel update, so we have a problem.

Thoughts:

• blacklist the kernel from being updated
• get the overlay added upstream. Even though the sun8i setup is shared with lots of boards, it may be accepted because the overlays aren’t activated without a line in armbianEnv.txt.

RT5372 devices aren’t using all chipset features

There’s an ht_capab line in hostapd that sets chipset specific parameters. We should do this so we’re using the features of the device. The group_vars/rt5372 has incorrect parameters (my bad) and this issue is to capture them.

There’s a separate issue to make sure we enable them on our shipping image.

Allow persistent setting of wifi tx power

The admin interface currently exposes a method to set the tx power but it’s not currently hooked up to do anything. It’s likely that we can do something with iw dev <devname> set txpower <auto|fixed|limit> [<tx power in mBm>], but I don’t know how to make that stick across a reboot, short of putting it in an ifup type script.

I think it’s worth considering the use-cases for adjusting tx power - under what circumstances will the default be inappropriate, and can we simplify the interface so that it has low power (short range) and max power?

Setting Wifi channel, Hostname or SSID through admin interface while on wifi reports an error

Right after updating the channel, the admin interface says: "Error updating channel” - Unexpected error setting property: error”. Same applies for updating the hostname or SSID. The channel/ssid are successfully updated though.

This doesn’t happen when you’re connected via ethernet, but if you’re connected via wifi, the wifi will drop and if the device connects to another network the admin interface will never get the SUCCESS response back from the API

Disable new “consistent naming” of wlan interfaces

Ubuntu names wlan interfaces based on the MAC of the adapter. This breaks config like hostapd.conf where we need to provide the wlan interface name. We can revert to the old naming with a symlink under /etc ( @matheweis knows what) or using some boot params per: https://github.com/ConnectBox/wifi-test-framework/blob/master/ansible/roles/setup-system/tasks/main.yml#L25

I don’t mind which one we use.

Admin interface - error dialog button off screen

On an Android phone, whenever the error dialog pops up, the close button is located too far to the right to be able to hit it to close the dialog. Suggest that the button be moved to the lower left underneath the error message to ensure that it can be seen and closed.

Admin interface uses http

Admin credentials pass over cleartext, which is bad. We'll probably have to self-sign a certificate, which means that admin connections will get certificate warnings (perhaps there's another way?)

iptables not always installed on Armbian

Sometimes the playbook fails on Armbian due to iptables not being installed. mikegleasonjr.firewall should ensure iptables is installed first.

(Specific image for this case was Armbian_5.25_Orangepizero_Ubuntu_xenial_default_3.4.113.img)

Improve hostapd config

The hostapd config is the example config, with only minimal changes. It should be reviewed and updated with sensible settings, and config lines removed if they're the defaults.

We should at least consider:

• Whether hostapd is built with config_acs, which would allow use to use auto channel selection
• Understand what needs to be done to enable 802.11n support (the Pi 3 supports it, but we're only using 802.11g in the config)
• country_code (and how it would be set - perhaps in the admin interface as a part of language selection for the interface)
• How to set the tx power

Doco: hostapd

Split file-browser UI into separate repo

This will allow it to be used by an OpenWRT-based platform, albeit one that needs to implement the same interface provided by nginx with autoindex and autoindex_format json. It’s not clear that we’ll need to split out the admin interface at this stage, so let’s leave it where it is.

Top level content and unmatched directories are not visible

It’s reasonable to expect connectbox owners to place files in the root directory of their USB stick, and to name their directories as something that doesn’t currently match our mapping (or that doesn’t correspond to a font-awesome icon once #44 has been implemented).

• We should display these root directory files below the icons for the top-level directories.
• We should also display these unmatched folders (perhaps witha generic folder icon, or even the name directory given this is something that the user will have created and will be in an appropriate language - other suggestions?)

@furnox / @kldavis4 , are you interested in taking a shot at this? Better ideas on how to achieve this?

Thanks for the suggestion, @GeoDirk.

Disable selenium tests

Remove travis integration and reimplement the single test using requests.

Logout functionality in the admin UI

Currently using basic auth in nginx which makes logout implementation challenging. Consider implementing authentication in the admin UI so there is a session that can be cleared.

Implement admin functionality for client ui configuration

Top-level folder name display toggle
Icon metadata prefix

Provide statistics on downloads

This is for administrators to see what files are being downloaded.

It might be something that we display in the admin interface (though perhaps it needs to be locked down more aggressively if that's the case).

It might also be something that's uploaded to a central site when an internet connection is detected. If this were the case, we could use a unique identifier present on the hardware (say a serial number, or the MAC of the wireliess interface) to identify the connectbox. We may want to disable upload stats by default, and we may want to show the link to the stat summary in the connectbox admin interface. We may want to hash the identifier if it helps with privacy.

Expose toggle in admin interface for icon-only mode vs website mode

This allows us to provide a single image, and have groups enable website or icon-only by using the admin interface.

Builds on #126

Raspbian lite 2016-11-25 doesn’t work with instructions

Raspbian 2016-11-25 disabled ssh by default, so our instructions to download the image and run ansible need to be changed to include instructions on enabling ssh.

There might be other things that come up when running against this version too.

https://downloads.raspberrypi.org/raspbian/release_notes.txt

Consider dropping 802.11b support for potential improved wifi performance

802.11b is probably unused, even in older smartphones (I can't ever remember seeing a 802.11b only smartphone) (https://mentor.ieee.org/802.11/dcn/14/11-14-0099-00-000m-renewing-2-4ghz-band.pptx).

We may be able to get better performance by having a g+n only network.

This might be achieved by use of the beacon_rate and preamble hostapd options. See: http://w1.fi/cgit/hostap/tree/hostapd/hostapd.conf

And also HT greenfield mode if the adapter supports it.

Improve OS/Account security

Despite the Raspbian Nov 2016 Security Update disabling ssh, we still enable it in order to run the playbooks. So, the current state is that we would ship a pi with the default password on the pi account, and sudo enabled and ssh enabled, which is bad*

I’d like to harden the install a bit, and I want to make sure we’re not going to make it hard for device developers, mass provisioners or single-device provisioners. Please discuss :-) - @kldavis4 @matheweis @GeoDirk comments most welcome

*: e.g. A malicious party could place malware in the file share, change config to start tracking users, have an internet-connected connectbox participate in a botnet.