openwrt / asu

An image on demand server for OpenWrt based distributions

OpenWrt Project is a Linux operating system targeting embedded devices. Instead of trying to create a single, static firmware, OpenWrt provides a fully writable filesystem with package management. This frees you from the application selection and configuration provided by the vendor and allows you to customize the device through the use of packages to suit any application. For developers, OpenWrt is the framework to build an application without having to build a complete firmware around it; for users this means the ability for full customization, to use the device in ways never envisioned.

Sunshine!

Download

Built firmware images are available for many architectures and come with a package selection to be used as WiFi home router. To quickly find a factory image usable to migrate from a vendor stock firmware to OpenWrt, try the Firmware Selector.

• OpenWrt Firmware Selector

If your device is supported, please follow the Info link to see install instructions or consult the support resources listed below.

An advanced user may require additional or specific package. (Toolchain, SDK, ...) For everything else than simple firmware download, try the wiki download page:

• OpenWrt Wiki Download

Development

To build your own firmware you need a GNU/Linux, BSD or macOS system (case sensitive filesystem required). Cygwin is unsupported because of the lack of a case sensitive file system.

Requirements

You need the following tools to compile OpenWrt, the package names vary between distributions. A complete list with distribution specific packages is found in the Build System Setup documentation.

binutils bzip2 diff find flex gawk gcc-6+ getopt grep install libc-dev libz-dev
make4.1+ perl python3.7+ rsync subversion unzip which

Quickstart

1. Run ./scripts/feeds update -a to obtain all the latest package definitions defined in feeds.conf / feeds.conf.default

2. Run ./scripts/feeds install -a to install symlinks for all obtained packages into package/feeds/

3. Run make menuconfig to select your preferred configuration for the toolchain, target system & firmware packages.

4. Run make to build your firmware. This will download all sources, build the cross-compile toolchain and then cross-compile the GNU/Linux kernel & all chosen applications for your target system.

Related Repositories

The main repository uses multiple sub-repositories to manage packages of different categories. All packages are installed via the OpenWrt package manager called opkg. If you're looking to develop the web interface or port packages to OpenWrt, please find the fitting repository below.

• LuCI Web Interface: Modern and modular interface to control the device via a web browser.

• OpenWrt Packages: Community repository of ported packages.

• OpenWrt Routing: Packages specifically focused on (mesh) routing.

• OpenWrt Video: Packages specifically focused on display servers and clients (Xorg and Wayland).

Support Information

For a list of supported devices see the OpenWrt Hardware Database

Documentation

• Quick Start Guide
• User Guide
• Developer Documentation
• Technical Reference

Support Community

• Forum: For usage, projects, discussions and hardware advise.
• Support Chat: Channel #openwrt on oftc.net.

Developer Community

• Bug Reports: Report bugs in OpenWrt
• Dev Mailing List: Send patches
• Dev Chat: Channel #openwrt-devel on oftc.net.

License

OpenWrt is licensed under GPL-2.0

asu's Issues

deliver usign in image response

as mentioned in #1 comment

platforms with generic images

For x86 platforms there are generic image names rather than board-specific. Yet, /tmp/sysinfo/board_name is populated, but a generic image should be requested on those platforms.
Ie. lede-x86-generic-combined-squashfs.img for x86/generic and lede-x86-64-combined-squashfs.img for x86/64 are used for all boards respective boards.
Anyway, just be aware that some platforms do and some platforms don't have board-specific images.
Maybe use a different BIN_DIR rather than just EXTRA_IMAGE_NAME for each build and fuzzy-match the image name, falling back on generic names if the board-name wasn't found?
However, the fact that a specific target/subtarget offers generic images needs to be known upfront to improve caching efficiency ie. maximize cache hits.

hidden dependencies

when generating an image, the PACKAGES list is passed to the imagebuilder which parses it and then thoughs it at opkg install ....
opkg now installs the packages listed including their dependencies.
hence the list of packages in an image-request does not map straight to the list of packages installed in the image. there are many images-requests which should point to the same image.

There are two possible ways to fix this:

1. re-implement opkg in SQL, ie. import all the dependency, provides, conflicts, ... into the database and evaluate the exact list of packages

2. collect the manifest generated by the IB which comes with the image and store the information about installed packages and versions in the database. mapping (multiple) image-requests to the same image can then be accomplished once the IB has run once for each synonymous image-request. That creates an avoidable overhead, but has the side-effect of verifying the reproducibility of the images built (as the hash of the image should be identical if the manifest is identical).

I have a strong tendency for option 2

add alias to distro and release

lime = LibreMesh
lede = LEDE
17.01.1 = Reboot

make the packages-list editable

when requesting the image there should be a possibility to VIEW AND CHANGE the list of packages

make workdir configurable

make work directory of the image server configurable
currently, tempdir is used for the downloads, however, $PWD/imagebuilder is still being used to extract the downloaded file.

only offer updates for sysupgrade-capable platforms

generally support on LEDE:

[daniel@box linux]$ find -name platform.sh
./adm5120/base-files/lib/upgrade/platform.sh
./pistachio/base-files/lib/upgrade/platform.sh
./mvebu/base-files/lib/upgrade/platform.sh
./kirkwood/base-files/lib/upgrade/platform.sh
./ppc40x/base-files/lib/upgrade/platform.sh
./au1000/base-files/lib/upgrade/platform.sh
./mediatek/base-files/lib/upgrade/platform.sh
./lantiq/base-files/lib/upgrade/platform.sh
./orion/generic/base-files/lib/upgrade/platform.sh
./ixp4xx/base-files/lib/upgrade/platform.sh
./bcm53xx/base-files/lib/upgrade/platform.sh
./adm8668/base-files/lib/upgrade/platform.sh
./x86/base-files/lib/upgrade/platform.sh
./brcm2708/base-files/lib/upgrade/platform.sh
./oxnas/base-files/lib/upgrade/platform.sh
./imx6/base-files/lib/upgrade/platform.sh
./apm821xx/base-files/lib/upgrade/platform.sh
./rb532/base-files/lib/upgrade/platform.sh
./mpc85xx/base-files/lib/upgrade/platform.sh
./ramips/base-files/lib/upgrade/platform.sh
./ipq806x/base-files/lib/upgrade/platform.sh
./ath25/base-files/lib/upgrade/platform.sh
./brcm47xx/base-files/lib/upgrade/platform.sh
./cns3xxx/base-files/lib/upgrade/platform.sh
./brcm63xx/base-files/lib/upgrade/platform.sh
./ar71xx/base-files/lib/upgrade/platform.sh
./octeon/base-files/lib/upgrade/platform.sh
./sunxi/base-files/lib/upgrade/platform.sh

On LibreMesh: https://github.com/libremesh/lime-sdk/blob/master/targets.list

ar71xx/generic
ramips/rt305x
ramips/rt3883
ramips/mt7620
ramips/mt7621
ramips/mt7628
ramips/mt7688
lantiq/xrx200
lantiq/xway
x86/generic
x86/64
mvebu/generic
ipq806x/generic
brcm2708/bcm2708
brcm2708/bcm2709
brcm2708/bcm2710
octeon/generic
sunxi/generic
mediatek/generic
arm64/generic
armvirt/generic

problems with LEDE SNAPSHOTS on cpe510

the possibility for upgrading LEDE snapshots is missing at the moment.

error: unknown device, please check model and board params

Repace Python tarfile with system native tar

Seems like the python tarfilelib requires to much memory.

imagebuilder: check PGP signature

Example:

• populate keyring with LEDE release pubkeys
• Download https://downloads.lede-project.org/releases/17.01.2/targets/ramips/mt7620/sha256sums and https://downloads.lede-project.org/releases/17.01.2/targets/ramips/mt7620/sha256sums.gpg
• verify sha256sums PGP signature
• verify sha256 of lede-imagebuilder-* against sha256sums

split existing opkg package and add package for upgrade-server

The attended-sysupgrade package should be split into

• rpcd-mod-imageinfo
• luci-app-attendedsysupgrade

The upgrade-server should also be provided as an OpenWrt package. Ideally, the parameters of the update-server should be read from UCI and an init script should be created.

find user removed packages and keep them uninstalled

If the user removed default packages the sysupgrade image should auto remove them as well.
This can be done by parsing imagebuilders make info default packages and compare it to currently user installed packages

Add overview of faild requests

Show failed images to simplify debugging

checkbox to flash firmware WITHOUT keeping settings

especially with auto-config firmwares (gluon, libremesh,..) there is no need to keep settings (all files in /etc/config/ )
so the attended-sysupgrade should have a checkbox for "keep no settings / delete all settings"

on standard firmware-upgrade webui-page there is such a checkbox for years

deliver image size in image response

Routers must check how much space is needed and free memory if needed

make layout great again

the layout is getting even more worse with every change/wish.
some changes should be done for a consistent style of this project.

• there should be a small description about that package. (see other luci-subpages). of course with translations (at least for german language).

• button: "search for updates" sounds inconsistent, because the headline and name of the project is "... upgrade"! so the name "updates" isnt nice.. the result of the search is not "many updates", but one "upgrade". you might change the name to "online search" to clarify the online-character of this project

• the checkbox "Search for package updates": i really do not understand the purpose of that checkbox. also my problem with "updates", while the project does an "upgrade".
  a translation also would be nice. (i8n)
  if that checkbox does something during the "search" it should be ABOVE the search-button, because its an option for that search!? (see other luci-pages)

• The message "imagebuilder not ready, please wait" should be in the same style as other luci-pages. please have a look at the layout of "Änderungen werden angewandt".

• it also should be editabl to use a different updateserver

uhttpd-mod-ubus is needed

trying freifunk-meshkit with luci via imagebuilder makes me need to install uhttpd-mod-ubus manually to get the attended-sysupgrade running.

there was no error message (ob webui) about that (or about problems)

capture imagebuilder log output along with the created image

and allow user to review imagebuilder log before flashing the image.

bug: packagelist will not be sent atm

trying the todays trunk makes me wonder.
sysupgrade was working, but there was no routingprotocol or luci anymore

browserproblems with firefox

with firefox i cant click the "search" button.
with cromium all is set. my OS is ubuntu

use existing methods to upload/flash firmware

within luci1 and even luci2 there are possibilities to upload/flash the firmware.
so there might be no need to program a new way, and no need to need the cgi-io package

tmpdir or tempdir?

8f60c79 introduced a config variable 'tempdir' but later on sets 'tmpdir'. Also 7cb84bd didn't fix that.

in config.yml.default

tmpdir: None

in imagebuilder.py

with tempfile.TemporaryDirectory(dir=self.config.get("tempdir")) as tar_folder:

auto clean outdated snapshot images

currently snapshot builds are stored even if there's a newer ImageBuilder.
A cron job or timer in the server should fix the problem

problem: reboot LEDE before using attended-sysupgrade

the postinstall scripts seems to have problems:
when installing atrtended-sysupgrade on openwrt/lede (via opkg) this project cant be used until rebooting the device.

also, manually restart the rpcd- and uhttpd service doenst help. there might be problems on the postinstall-scripts.

create uci options

add a uci config for the following settings

• update server
  • full url of update server with port
• notify package updates - the current approach can not only update releases but updated packages as well. This is useful if opkg cannot be used to install updated packages because of space constraints
• auto check - check on opening the luci web interface

not working for Ubiquity devices

there is an error when trying it with UBNT devices: board not found

improve the settings in /etc/config

on https://github.com/openwrt/luci/blob/master/applications/luci-app-attendedsysupgrade/luasrc/view/attendedsysupgrade.htm#L90 there is a

settings[0].

please change it to a better openwrt standard

button doenst work atm: Search for update -> Access denied

today i was trying last lede-release on x86, and was installing official lede trunk packages for attended sysupgrade.

atm all ubus-http-request (without pressing any button) got "Access denied"

possibility of different updateserver

before clicking the "search update" button the adress of the update-server should appear.

it also should be editable! (in order to use a different updateserver)

support for NAND and block-based devices

currently only NOR-based devices with squashfs-sysupgrade images are supported.
Support also devices which are block-device or NAND-flash based by picking up the required files generated by the ImageBuilder.

useful for sysupgrade:

• most NOR based devices: *-${FSTYPE}-sysupgrade.bin
• Broadcom NOR devices: *-${FSTYPE}.trx, *-${FSTYPE}.chk, *-${FSTYPE}-sysupgrade.bin, *-${FSTYPE}.bin
• NAND based devices: *-${FSTYPE}-sysupgrade.tar
• MMC based devices: *-${FSTYPE}-sdcard.img.gz
• x86 MBR: *-combined-${FSTYPE}*
• x86 GPT: *-uefi-gpt-${FSTYPE}*

Hence, to pick the right image for sysupgrade, we need to know the rootfs-type (squashfs vs. jffs2/ubifs/ext4/f2fs) as well as the type of the rootdisk's partition table on x86-based devices (MBR vs. GPT/UEFI).

As there are images files generated for other purposes (factory, ...) it makes sense to serve them as well for use with jefe.tk. To achieve that, I suggest to simply match all filenames containing the EXTRA_IMAGE_NAME substring supplied to IB.

Enable multible workers

• improve the 'imagebuilder_requests' handling
• let workers "upload" the image
• store public worker key in database
• create separate database login for workers
  • limit database access of workers

simplify database.py with PREPARE statements

https://www.postgresql.org/docs/9.6/static/sql-prepare.html

create rpcd-mod-imageinfo

rpcd-mod-imageinfo will be a package reporting the currently installed release with packages to the updateserver. This will enable the server to see if an upgrade worked, mark images as stable or similar

store version via ids not as text

attendedsysupgrade=# \d+
public | packages_available_table     | table    | postgres | 37 MB      | 

Table is bigger than needed

Remove hash in image when using standard packages

This should result in the same build tree as for the official buildbot

deliver md5sum in image response

The routers should verify the checksum before installation

implement package list transformations

packages are sometimes removed (kmod-ipv6), renamed (libpolarssl -> libmbedtls) or split into several build variants (openvpn -> openvpn-openssl, openvpn-mbedtls, ...).
Hence when generating the list of packages to be included in an updated image, those changes need to be taken into account.
Implement this by having a set of rules specifying the replacement operations in a context-aware fashion (ie. use openvpn-mbedtls if libmbedtls is already going to be installed)

auto update libremesh network-profiles

cronjob or python based update of the network-profiles

target currently not supported x86/generic

Rumors came up that there were changes on the project to allow to de-activate opkg-packages while upgrading via attended-sysgrade. i couldnt find something like that. (this morning i was upgrading a vm several times.)

this afternoon im not able to upgrade that same vm anymore:

target currently not supported x86/generic

Add last_sync for imagebuilders

Store the last sync of packages for imagebuilders to answer faster on update-requests

safe image hash to database

right now a hash of all pre installed packages is created. These hashes need to be stored in a database to reverse instelld packages of an image

make the firmware-destination editable

when i see "new update avaiblae", i can see the prefered destination (f.i. "SNAPSHOT")

please make that textfield editable. i need the possibility to switch from LEDE-release to LEDE-trunk (and vice versa) or other firmware variation.

wrong mime-type for build log

*-sysupgrade.bin.log should be text/plain rather then application/octet-stream.
Probably a http server configuration issue.

install problems with freifunk-meshkit firmware

with 2 days old trunk i am on installing attended sysupgrade via opkg
there are some error messages

opkg update
Downloading http://gadow.freifunk.net:8004/meshkit/buildroots/ar71xx-lede-201710 19/packages//Packages.gz
Updated list of available packages in /var/opkg-lists/reboot_core
root@120-9:# opkg list | grep atten
luci-app-attendedsysupgrade - git-17.291.51172-a98548e-1 - LuCI support for atte nded sysupgrades
rpcd-mod-attendedsysupgrade - 1-1 - implements sysupgrade function with ubus add uci settings and needed acls
root@120-9:# opkg install luci-app-attendedsysupgrade
Installing luci-app-attendedsysupgrade (git-17.291.51172-a98548e-1) to root...
Downloading http://gadow.freifunk.net:8004/meshkit/buildroots/ar71xx-lede-201710 19/packages//luci-app-attendedsysupgrade_git-17.291.51172-a98548e-1_all.ipk
Installing uhttpd-mod-ubus (2017-08-19-3fd58e9b-1) to root...
Downloading http://gadow.freifunk.net:8004/meshkit/buildroots/ar71xx-lede-201710 19/packages//uhttpd-mod-ubus_2017-08-19-3fd58e9b-1_mips_24kc.ipk
Installing cgi-io (4) to root...
Downloading http://gadow.freifunk.net:8004/meshkit/buildroots/ar71xx-lede-201710 19/packages//cgi-io_4_mips_24kc.ipk
Installing rpcd-mod-packagelist (0.1-1) to root...
Downloading http://gadow.freifunk.net:8004/meshkit/buildroots/ar71xx-lede-201710 19/packages//rpcd-mod-packagelist_0.1-1_mips_24kc.ipk
Installing rpcd-mod-attendedsysupgrade (1-1) to root...
Downloading http://gadow.freifunk.net:8004/meshkit/buildroots/ar71xx-lede-201710 19/packages//rpcd-mod-attendedsysupgrade_1-1_mips_24kc.ipk
Configuring cgi-io.
Configuring rpcd-mod-packagelist.
Configuring rpcd-mod-attendedsysupgrade.
//usr/lib/opkg/info/rpcd-mod-attendedsysupgrade.postinst: .: line 130: attendeds ysupgrade: not found
Configuring uhttpd-mod-ubus.
//usr/lib/opkg/info/uhttpd-mod-ubus.postinst: .: line 130: 00_uhttpd_ubus: not f ound
Configuring luci-app-attendedsysupgrade.

store sysupgrade suffix in database

store strings like squashfs-sysupgrade.bin in the database to enable simple upgrades for users. Currently the sysupgrade file is renamend

ipq806x images inconsistent naming

trying to build a image for ipq806x/generic produces the following list of files, no sysupgrade is found, only a tar file.

DEBUG:image:['/tmp/tmpgt6us7pp/lede-ipq806x-EA8500-squashfs-factory.bin', 
'/tmp/tmpgt6us7pp/lede-ipq806x-EA8500-squashfs-sysupgrade.tar', 
'/tmp/tmpgt6us7pp/lede-ipq806x-squashfs-root.img', 
'/tmp/tmpgt6us7pp/lede-ipq806x-ubifs-root.img', 
'/tmp/tmpgt6us7pp/sha256sums', 
'/tmp/tmpgt6us7pp/lede-ipq806x-vmlinux.elf', 
'/tmp/tmpgt6us7pp/lede-ipq806x-device-ea8500.manifest']

@dangowrt should I implement auto-extracting if tar files are found? What else could fix this?
Also, why are there img and bin files?

build problems: Profile does not exist!

atm sysupgrade wont work for me:

[
"lede",
"snapshot",
"ar71xx",
"generic",
"tl-wr902ac-v1",
"f5b148faad84",
""
]

Profile "tl-wr902ac-v1" does not exist!
Use "make info" to get a list of available profile names.
Makefile:197: recipe for target 'image' failed
make: *** [image] Error 1

Depend on luci-base instead of luci

Current attended-sysupgrade package depends on "luci" which is a collection. Would it be possible to depend only on luci-base as libremesh is doing for webui package?

https://github.com/libremesh/lime-packages/blob/develop/packages/lime-webui/Makefile

This is IMO a requirement to better fit into 4MB routers.

remove inactive workers

If there was no heartbeat for a specific time, remove the worker

reload snapshot strucutre daily

happens that targets/subtargets are renamed in snapshots