olf0 / sfos-upgrade Goto Github PK
View Code? Open in Web Editor NEWUpgrading SailfishOS fail-safe and semi-automated at the command line
Home Page: https://openrepos.net/content/olf/sfos-upgrade
License: GNU Lesser General Public License v2.1
Upgrading SailfishOS fail-safe and semi-automated at the command line
Home Page: https://openrepos.net/content/olf/sfos-upgrade
License: GNU Lesser General Public License v2.1
Consider performing
rm -r /var/cache/ssu/
ssu ur
But when? Likely both, before and after updating (on SFOS < 4.4.0), because users may start on any affected release and end on one, but never use sfos-upgrade
before or after that, again.
Original report at OpenRepos by @dalas_revo:
"Unfortunately upgrading from 3.0.2.8 to 3.0.3.10 on the Aigo (Jolla Tablet) fails for me with sfos-upgrade, complaining about a missing /sys/class/power_supply/battery/uevent."
I might introduce an override for this after some more analysis:
mount | fgrep sysfs
ls -l /sys/class/power_supply
ls -l /sys/class/power_supply/battery
cat /sys/class/power_supply/battery/uevent
Checking the battery charge is important, thus I want to avoid implementing an override, if a proper solution can be introduced.
I haven't pondered it deeply but I suspect there is a typo in line 182 where
$recent_stop_releases
should be$recent_sfos_releases
(because that is defined in line 171).
Ack!
Thank you, I wanted to get it out quickly, and was just tested per bash -n
, which does not catch this kind of error.
That suspicion is deepened by the fact that renaming that variable makes everything work normally.
Exactly.
Within the current 2.7-1 release I found an error that prevented me from executing the script in the first run:
Within the for-loop starting from line 406 (current master branch) you forgot to end the (outer) if clause at around line 415 (between the emit_newline and the end-of-for-loop line).
Little hint: As I found out last week, bash also has an integrated syntax check with the option '-n', I used frequently since then, but maybe you know it already.
Nonetheless, great script and great work in general.
Best regards,
Frank
Hi,
trying to use the script from sfos-upgrade-3.5-5.noarch.rpm on a Gemini PDA under SFOS 3.1.0.11 fails with the script exiting silently (exit code 127).
bash -x tracing reveals that:
POWER_SUPPLY_STATUS=Not charging
the space causing an incorrect sourcing.
I worked around this by rewriting that section to:
if [ -e "${battery_uevents}/uevent" ]
then
sed 's/STATUS=\(.*\)/STATUS="\1"/' "${battery_uevents}/uevent" > /tmp/battuevent$$
source /tmp/battuevent$$
battery_info="sourced"
break
fi
which is a dirty, dirty hack but works for me.
Having gone through a bumpy upgrade process from a SFOS 1.x release to current on a Jolla Tablet, the idea arose that, as the upgrade process basically does
... and repeats that process every time a particular update step is performed, why not skip step 1. by having the option to supply a prepopulated cache directory.
Consider the case of a J1 or Tablet which has been reset to factory.
One would now go through the step releases to update to a certain target release. It's likely one has performed such an update previously, and it's likely the packages needed to do the upgrade have not changed since the last time.
So in case a (large enough) SD card is available, that could be used to persist the cache dir for every (stop) release, and supply it for repeated from-factory-reset upgrade tasks.
Hi,
Would you mind adding the pre-upgrade script for themepacksupport?
Its path is /usr/share/harbour-themepacksupport/ocr.sh
(It restores default themes/display options in one click. I have a systemd service if the upgrade is triggered via UI, but I'm pretty sure it doesn't work if the upgrade is triggered via cli.).
Br,
Fra
SailfishOS VERSION: Sailfish OS 4.4.0.72 (Vanha Rauma)
HARDWARE: Gemini PDA 0.0.5.1
sfos-upgrade VERSION: 3.9.11
The "not enough free space" warning gives twice the amount that df reports.
defaultuser@GeminiPDA:~ $ df -h /; df -k /; devel-su sfos-upgrade
Filesystem Size Used Available Use% Mounted on
rootfs 2.3G 1.9G 430.0M 82% /
Filesystem 1K-blocks Used Available Use% Mounted on
rootfs 2454256 1971912 440368 82% /
Password:
Notice: Less than 1 GiB (880736 KiB) free space on the root filesystem.
Please consider to clean up or to enlarge the root filesystem, see:
https://gitlab.com/Olf0/sailfishX#33-increasing-the-root-lvm-volume-size
Not at all sure where the 880736 KiB come from, because executing the command for $free_space
in sfos-upgrade gives:
(relevant line in script)
defaultuser@GeminiPDA:~ $ df -k / | sed -n '2p' | rev | grep '^/ ' | tr -s ' ' | cut -s -f 3 -d ' ' | rev
440368
defaultuser@GeminiPDA:~ $ echo $(( 440368 * 2 ))
880736
df -k / && devel-su sfos-upgrade
It's not a busybox vs bash thing:
defaultuser@GeminiPDA:~ $ echo $BASH_VERSION
5.0.18(1)-release
defaultuser@GeminiPDA:~ $ busybox bash -c "df -k / | sed -n '2p' | rev | grep '^/ ' | tr -s ' ' | cut -s -f 3 -d ' ' | rev"
440368
defaultuser@GeminiPDA:~ $ /bin/bash -c "df -k / | sed -n '2p' | rev | grep '^/ ' | tr -s ' ' | cut -s -f 3 -d ' ' | rev"
440368
Hey,
Is this warning dangerous for the upgrade operation? What could I do about it?
# sfos-upgrade 3.4.0.24
[D] unknown:0 - Got http_proxy from environment, will not talk to connman
Warning: Failed to extract the current "stop releases" from https://jolla.zendesk.com/hc/en-us/articles/201836347#4.1
Hence using an internal, potentially outdated list of stop releases instead.
Do you really want to continue? (Y/N)
Also, why those information is needed?
Any help would be appreciated.
See the first try to adapt to obs with subsequent discussion and a separate discussion, the first one spawned / experimental commit a31be78.
SRPM is now provided, see this comment for details.
But I still don't know / understand, if this (an SRPM) resolves the original issue (integration into build systems, e.g. obs).
P.S.: Building SRPMs works now, per commit 41d29a4.
An SRPM can be downloaded e.g. per curl -sSLO https://github.com/Olf0/%{name}/releases/download/%{version}-%{release}/%{name}-%{version}-%{release}.src.rpm
, while the original TAR.GZ source archive can be downloaded e.g. per curl -sSLo %{name}-%{version}-%{release}.tar.gz https://github.com/Olf0/%{name}/archive/%{version}-%{release}.tar.gz
(that is equivalent to curl -sSLo %{source} %{source1}
, when SOURCE1 was still set, i.e. before aforementioned commit 41d29a4).
On Jolla 1 version 1.0.8.19 the default (on line 364) in sfos-upgrade
btrfs_allocation="$(btrfs filesystem df / | grep '^Data, ' | cut -s -f 2 -d ':' | tr ',' '\n' | tr -d ' ' | rev | grep '^Bi*G[0-9][0-9]\.[0-9][0-9]*=' | sed 's/^Bi*G//g' | tr -d '.' | rev)"
gives incorrect results.
But if used (grep '^Data'
instead of grep '^Data, '
):
btrfs_allocation="$(btrfs filesystem df / | grep '^Data' | cut -s -f 2 -d ':' | tr ',' '\n' | tr -d ' ' | rev | grep '^Bi*G[0-9][0-9]\.[0-9][0-9]*=' | sed 's/^Bi*G//g' | tr -d '.' | rev)"
It gives correct results
[root@Jolla nemo]# btrfs filesystem df / | grep '^Data' | cut -s -f 2 -d ':' | tr ',' '\n' | tr -d ' ' | rev | grep '^Bi*G[0-9][0-9]\.[0-9][0-9]*=' | sed 's/^Bi*G//g' | tr -d '.' | rev
total=964
used=287
I don't have other devices to test the btrfs command with, so I dont know how these differ in different versions, and what is a universal fix.
Comparing SailfishOS version strings lexicographically may fail, if a version field is a single digit in one version string and double digit in the other strings.
Hence, e.g. 3.0.3.9 -> 3.0.3.10 is incorrectly determined to be a downgrade (as described in the original report).
SailfishOS VERSION (Settings → About product → Build): 4.4.0.64
HARDWARE (Settings → About product → Manufacturer & Product name): Xperia 10iii
sfos-upgrade VERSION ( rpm -qi sfos-upgrade
): 3.9.9-release3
Update to 4.4.0.72 not detected
run
sfos-upgrade 4.4.0.72
observe that nothing happens at all. No error msg, no updating, just nothing.
From doing things like sh -x
it tuens out the release is not yet listed in the docs.s.o wiki/site. So the parsing done by the script fails.
Snippet if the last few lines of sh -x sfos-update 4.4.0.72
:
1.0.0.3
1.0.0.1
0.99.6.8
0.99.6.3
0.99.5.11
0.99.5.8
0.99.5.6'
++ curl -sSkL https://github.com/sailfishos/docs.sailfishos.org/raw/master/Releases/README.md
++ cut -s -d '|' -f 2,5
++ tr -d ' '
+ sailfishdocs_sfos_list=
++ echo ''
++ cut -s -d '|' -f 1
++ grep '^[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$'
+ sailfishdocs_sfos_releases=
I realize that is not the fault of the script.
But it should output something in this case, not silently exit.
<Please consider which other pieces of information may be relevant: Denote if this is not always reproducible, if this is a regression (then name to which older version), attach relevant data such as log files or the systemd journal, provide text output via copy & paste or output-redirecton (no screenshots, please!) etc.>
SailfishOS VERSION (Settings → About product → Build): 4.3.0.15
HARDWARE (Settings → About product → Manufacturer & Product name): Sony Xperia XA2 (4113)
sfos-upgrade VERSION ( rpm -qi sfos-upgrade
): 3.9.4-release3
/usr/bin/sfos-upgrade: line 177: final_sfos_releases: unbound variable
/usr/bin/sfos-upgrade: line 196: final_sfos_releases: unbound variable
seems the newly added feature pulling releases from sailfishos does not work.
start from 4.3.0.12
run sfos-update 4.4.0.64 (can't remember, which version it was run on)
stop release 4.3.0.15 get's detected
install, reboot, run post_sfos-upgrade
this refreshes repositories
at some point the new version 3.9.4.-release3 of sfos-upgrade get's installed
the remaining step sfos-update 4.4.0.64 is not successful because of error message for line 177 and 196
at the time of reporting, /sailfishos/docs.sailfishos.org/master/Releases/README.md does not contain 4.4.0.64 yet, while coderus.openrepos.net/whitesoft/sailversion does
Unfortunately OBS doesn't use the Release:
tag part of the RPM Spec and overwrites the Version:
part with the latest tag information and expects that to be part of the tarball naming.
To fix this, i would suggest a change to the way you number releases, which can be a 3 digit versioning number.
eg. 3.5.8
instead of 3.5-8
r0kk3rz@cbfd004
https://build.sailfishos.org/package/show/home:r0kk3rz/sfos-upgrade
For ported devices stop releases aren't always necessary or possible depending on the state of the adaptation repositories for that device.
it would be nice to tell sfos-update to skip the stop release and go straight to the version its been told to, with appropriate warnings of course
SailfishOS VERSION (Settings → About product → Build): 3.4.0.24
HARDWARE (Settings → About product → Manufacturer & Product name): F(x)tec Pro¹ (qx1000, not the Pro¹X which is qx1050)
sfos-upgrade VERSION ( rpm -qi sfos-upgrade
): 3.9.13
After a fresh reflash of SFOS on a F(x)tec Pro¹ (version 3.4.0.24, which is the latest working image), I tried to upgrade to the latest SFOS version. I downloaded sfos-upgrade 3.9.13 rpm from openrepos (via a browser search).
I tried running sfos-upgrade without arguments, with any supported version as argument, with "--verify", but got this error message :
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
sfos-upgrade
[root@Pro1 defaultuser]# ssu re
Device release is currently: 3.4.0.24
[root@Pro1 defaultuser]# sfos-upgrade
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
[root@Pro1 defaultuser]# sfos-upgrade --verify
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
[root@Pro1 defaultuser]# sfos-upgrade 4.4.0.72
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
[root@Pro1 defaultuser]# sfos-upgrade 4.0.1.45
/usr/bin/sfos-upgrade: line 179: upgrade_version: unbound variable
/usr/bin/sfos-upgrade: line 179: installed_version: unbound variable
Comparing versions failed when checking, if downloading lists of SailfishOS releases is necessary.
[root@Pro1 defaultuser]# pkcon get-details sfos-upgrade
Resolving [ ] (0%) More than one package matches:
1. sfos-upgrade-3.9.13-release6.noarch [installed]
2. sfos-upgrade-3.9.13-1.2.1.jolla.noarch [sailfishos-chum]
Then I installed version 3.9.12 (also from openrepos website), and it worked :
[root@Pro1 defaultuser]# pkcon get-details sfos-upgrade
Resolving [ ] (0%) More than one package matches:
1. sfos-upgrade-3.9.12-release5.noarch [installed]
2. sfos-upgrade-3.9.13-1.2.1.jolla.noarch [sailfishos-chum]
#
# at this point, I had made some tests, including a `ssu re 4.4.0.72` and forgot to reset SSU release to 3.4.0.24, which explains why at first it tries to upgrade to 4.4.0.72
#
[root@Pro1 defaultuser]# sfos-upgrade
Notice: Upgrading from 3.4.0.24 to 4.4.0.72 would omit installing 4.0.1.48 as a stop release!
Warning: Doing so will likely break this SailfishOS installation, unfortunately often only subtly so the issues caused might be observed way later!
Do you really want to continue? (y/N)
Aborting upon user request.
[root@Pro1 defaultuser]# sfos-upgrade 4.0.1.48
Notice: The installed version 3.4.0.24 is smaller than the one currently set for SSU (4.4.0.72).
A possible reason for this is, that the 'sailfish-osupdateservice osupdate-check' invoked by osupdate-check.service (which is regularly triggered by osupdate-check.timer) does more than just checking (observed with SailfishOS 3.2.1 and earlier): E.g., setting ssu to the recent release version or next stop release.
Never mind, the version for SSU will be set correctly again, later on.
Notice: Mind that sfos-upgrade is best run on a freshly rebooted device.
Notice: Do you want to upgrade SailfishOS from 3.4.0.24 to 4.0.1.48? (y/N) y
Notice: For troubleshooting issues with the upgrade proper, please consult https://jolla.zendesk.com/hc/en-us/articles/360005795474
- Stopping osupdate-check.timer
- Setting SSU to SailfishOS release:
Changing release from 4.4.0.72 to 4.0.1.48
Your device is now in release mode!
- Fetching and installing the SailfishOS upgrade from 3.4.0.24 to 4.0.1.48 (this may take a while):
REFRESHING CACHE AND DOWNLOADING PACKAGES
Finished transaction (status=1, runtime=805987ms)
UPGRADING SYSTEM
Finished transaction (status=1, runtime=273105ms)
FINISHING
So it's probably a regression between version 3.9.12 and 3.9.13.
This bug is maybe related to #64
To properly support community adaptions of SailfishOS is probably a longer endeavor, as there currently are many unknowns (for me, at least):
sfos-upgrade
on one. Hence the current, "official" state of support for community adaptions is something between "unknown" and "unsupported".sfos-upgrade
does already basically work on devices with community adaptions of SailfishOS.As sfos-upgrade
up to now always uses version --dup
to perform the upgrade proper, which checks if rnd-dist-upgrade
is installed (usually in /usr/bin/
) and uses either that or pkcon upgrade
, respectively complains (since SFOS3.2.1?), I now wonder why the "zypper dance" / zypper upgrade
is the primary suggestion from SailfishOS porters for upgrading installations of community adaptions.
Hence I compiled a list of simple questions, trying to pose them in a way that they can be answered with "Yes" or "No".
For my basic understanding:
To understand the technical differences:
version
shell script preinstalled (usually in /usr/bin/
)?rnd-dist-upgrade
binary installed (usually in /usr/bin/
), when "developer mode" is enabled in the SailfishOS' Settings (which I assume every user of a "community device" has)?zypper upgrade
is preferred over pkcon upgrade
by "community porters", in contrast to Jolla's general preference of pkcon
over zypper
(also specifically for the upgrade case)?pkcon
and zypper
use libzypp
as backend (on SailfishOS) and hence should yield the same package transactions and result. Maybe zypper
's more elaborate output and configurability is the only reason, why porters prefer it.SailfishOS VERSION : 2.0.0.10
HARDWARE : Jolla Tablet
sfos-upgrade VERSION: 3.9.17
Internal list of stop releases: Is sfos-upgrade more zealous than the Jolla list?
Running sfos-upgrade on 2.0.0.10, wanting to update to 2.2.0.29 will output:
Notice: Upgrading from 2.0.0.10 to 2.2.0.29 would omit installing 2.0.5.6 as a stop release!
Now, the d.s.o page on Releases sais this step, 2.0.0.10 -> 2.2.0.29 should be possible.
I don't mind sfos-update being more careful than the list prescribes, but I was curious whether there was maybe some outdated internal list in the tool, or an omission on Jolla's list.
By the way, sfos-upgrade 3.9.17 is capable of performing its duty on SFOS 1.1.9.30 which was a bit surprising to me but kudos to you for making it so very portable.
And more by the way, actually sfos-upgrade does give an error when running it on such old releases, namely:
[root@Jolla nemo]# sfos-upgrade 2.2.0.29
/usr/bin/sfos-upgrade: line 205: sailfishdocs_sfos_releases: unbound variable
Warning: "2.2.0.29" does not seem to be a publicly released SailfishOS version!
Do you really want to continue? (y/N) y
I haven't investigated deeply but I assume it's simply because ancient ssl certificates on such old releases will make the curl
call(s) fail. Harmless in my book so I'm ignoring that.
SailfishOS VERSION (Settings → About product → Build): 4.5.0.21
HARDWARE (Settings → About product → Manufacturer & Product name): Pine64 PinePhone Pro
sfos-upgrade VERSION ( rpm -qi sfos-upgrade
): 3.11.1
When relying on btrfs
(as opposed to btrfs-balancer
), the way allocation is computed is mistaken.
this code segment gets as an inputs the Data chunks allocated, and how much file data is written into them. The ratio computed is the occupancy of the chunks themselves, this doesn't give any information about the unallocated space on the device.
Instead btrsf filesystem show
should be used (or btrfs filesystem usage
on more recent versions).
balance start -v -dusage=80 -musage=80 /
)sfos-upgrade 4.5.0.24
)Aborting: Less than 2 GiB unallocated data space (.6 GiB) on the root filesystem (BTRFS)!
Please balance the btrfs root filesystem before retrying.
Here are my numbers (after balancing):
### Just for reference, it's on a 8GiB partition:
$ df -h /
Filesystem Size Used Avail Use% Mounted on
/dev/mmcblk2p2 7.8G 3.3G 4.4G 44% /
### Allocated chunks on the device(s) on /:
### on device 1, we have allocated 4.20GiB in various chunks, the total space usable on that device is 7.75GiB
$ btrfs filesystem show /
Label: 'root' uuid: 7d24fbb1-73c3-42e2-bae1-c02321491dd5
Total devices 1 FS bytes used 3.26GiB
devid 1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2
Btrfs v3.16
### Content of the chunks:
### here's exactly how the allocated 4.20GiB chunks are split between Data, System and Metadata, and how much is written into them
$ btrfs filesystem df /
Data, single: total=3.91GiB, used=3.15GiB
System, single: total=32.00MiB, used=16.00KiB
Metadata, single: total=256.00MiB, used=114.08MiB
unknown, single: total=9.64MiB, used=0.00
Currently, sfos-upgrade computes using this line:
Data, single: total=3.91GiB, used=3.15GiB
and this gets the information that .8 GiB are unwritten on the allocated chunks.
(This is expected: after balancing the chunks will probably be very compact).
What it should do is use all per fdevice line(s):
devid 1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2
Total number of space on device available is 7.75, 4.20 has been allocated to chunks, 3.55 is unallocated and can still further be allocated to Data or Metadata chunks.
A possible piece of bash code that does this:
btrfs_allocation="$(btrfs filesystem show / )"
btrfs_total=0
btrfs_used=0
while read ld i ls t lu u lp p; do
# look for lines like:
# devid 1 size 7.75GiB used 4.20GiB path /dev/mmcblk2p2
if [[ "$ld" != 'devid' || "$ls" != 'size' || "$lp" != 'path' ]]; then
continue
fi
# parse numbers with regex and tally them
[[ "$t" =~ ([0-9]*)\.([0-9]*)GiB ]] && (( btrfs_total += "${BASH_REMATCH[1]}${BASH_REMATCH[2]}" ))
[[ "$u" =~ ([0-9]*)\.([0-9]*)GiB ]] && (( btrfs_used += "${BASH_REMATCH[1]}${BASH_REMATCH[2]}" ))
done <<< "$btrfs_allocation"
btrfs_unallocated="$((btrfs_total-btrfs_used))"
echo $btrfs_unallocated
I've tested it in GNU/bash (I don't know if it works in busybox bash. Does it support regex? Otherwise the tr+sed+grep+etc. route would be needed) .
Use case:
So if a user keeps both a backup of system.img and the patched one, the /opt partition doesn't have enough room left for an updated system.img and the upgrade fails.
Checking that the partition usage is <60% should do the trick
@pcfe, @direc85, @Okxa, @Self-Perfection,
I want to change the license from MIT to LGPL-2.1-only.
Because you all contributed to sfos-upgrade, I want to align this license change with you.
If you do not concur with this change, please notify me (preferably here in this issue) before 1 May 2021.
sfos-upgrade fails to work on Inoi R7.
[root@Sailfish nemo]# sfos-upgrade 3.1.0.11
Notice: The installed version 3.0.0.11+omp0.1.3.10 is smaller than the one curr
ently set for SSU (3.1.0.11).
A possible reason for this is, that the \'sailfish-osupdateservice osupdate-che
ck\' invoked by osupdate-check.service (which is regularly triggered by osupdat
e-check.timer) does more than just checking (observed with SailfishOS 3.0.2 and
earlier): E.g., setting ssu to the recent release version or next stop release
.
Never mind, the version for SSU will be set correctly again, later on.
/usr/bin/sfos-upgrade: line 74: [: 11+omp0: integer expression expected
/usr/bin/sfos-upgrade: line 78: [: 11+omp0: integer expression expected
Warning from function compare_versions: Version strings "3.0.0.8" and "3.0.0.11
+omp0.1.3.10" are not comparable!
[root@Sailfish nemo]#
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.