Docker image containing GOLANG that is compatible to the Raspberry Pi.
make build
make version
- First, use a
docker login
with username, password and email address - Second, push Docker Image to the official Docker Hub
make push
Command line script to flash SD card images of any kind
License: MIT License
Docker image containing GOLANG that is compatible to the Raspberry Pi.
make build
make version
docker login
with username, password and email addressmake push
This (see headline) holds e.g. when flashing Hypriot Cluster Lab. Then this is what happens:
$ ./flash hypriot_20160121-235123_clusterlab.img.zip
/usr/bin/unzip
Uncompressing /home/[...]/hypriot_20160121-235123_clusterlab.img.zip ...
Archive: /home/[...]/hypriot_20160121-235123_clusterlab.img.zip
inflating: /tmp/hypriot-rpi-20160121-235123.img
Use /tmp/ hypriot-rpi-20160121-235123.img
Filesystem 1K-blocks Used Available Use% Mounted on
udev 8046676 4 8046672 1% /dev
tmpfs 1612604 1816 1610788 1% /run
[...]
Is /dev/mmcblk0 correct? y
Synchronizing Filesystems
[sudo] password for ###:
Unmounting /dev/mmcblk0 ...
Flashing /tmp/ hypriot-rpi-20160121-235123.img to /dev/mmcblk0 ...
stat: cannot stat ‘hypriot-rpi-20160121-235123.img’: No such file or directory
cat: /tmp/: Is a directory
cat: hypriot-rpi-20160121-235123.img: No such file or directory
0B 0:00:00 [ 0B/s] [> ] 0%
0+0 records in
0+0 records out
0 bytes (0 B) copied, 6,177e-05 s, 0,0 kB/s
Flushing Buffers
Mounting Disk
Mounting /dev/mmcblk0p1 to customize
Unmounting /dev/mmcblk0 ...
Finished.
If archive name is identical with image name inside the archive, everything works fine.
This isn't a bug, but it is a suggestion that I think would be helpful.
One safety check that could be added to this script is to enumerate only devices that show as removable so that system disks and similar are not even suggested as options for flashing. This can be done on Linux by reading the file /sys/block/XXX/removable, where XXX is your device name (e.g. sda).
using flash did not add credentials correclty, manually editing the credentials files and running occi
did not work and fails with occidentalis not installed
and when manually trying to run the occidentalis install script that fails as well.
using fresh Elizabeth img.
First of all, thank you for an awesome tool!
Is there anyway to setup a static ip like the option hostname? Would save some time, when building a Raspberry Pi cluster.
Thank you in advance.
I have coreutils installed via bre which means dd and stat are differnet that the stock osx ones.
The follow output is what i get.
↪ flash --hostname pi-docker hypriot-rpi-20160306-192317.img.zip
Uncompressing hypriot-rpi-20160306-192317.img.zip ...
Archive: hypriot-rpi-20160306-192317.img.zip
inflating: /tmp/hypriot-rpi-20160306-192317.img
Use /tmp/hypriot-rpi-20160306-192317.img
No SD card found. Please insert SD card, I'll wait for it...
Filesystem Size Used Avail Use% Mounted on
/dev/disk1 465G 409G 57G 88% /
/dev/disk6s1 7.4G 4.2M 7.4G 1% /Volumes/PI
Is /dev/disk6 correct?
Please answer yes or no.
Is /dev/disk6 correct? yes
Unmounting disk6 ...
Unmount of all volumes on disk6 was successful
Unmount of all volumes on disk6 was successful
Flashing /tmp/hypriot-rpi-20160306-192317.img to disk6 ...
Password:
stat: cannot read file system information for '%B': No such file or directory
pv: -s: integer argument expected
dd: invalid number: ‘1m’
Unmounting and ejecting disk6 ...
Unmount of all volumes on disk6 was successful
Unmount failed for /dev/disk6s2
Disk /dev/disk6 ejected
🍺 Finished.
I've fixed it by changing the paths to point directly at the programs installed in /usr/bin
for stat and /bin
for dd and it works fine.
OSX: 10.11.3
I have this error after flashing the SD.
Using Ubuntu 16.04
Unmounting /dev/mmcblk0p1 ...
Flashing /tmp/hypriotos-rpi-v1.0.0.img to /dev/mmcblk0p1 ...
1000MiB 0:00:02 [ 375MiB/s] [==========================================================================>] 100%
116+6527 records in
116+6527 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 96.6965 s, 10.8 MB/s
Waiting for device...
/dev/mmcblk0p1:
re-reading partition table
BLKRRPART failed: Invalid argument
This is odd, because the variable is quoted in the umount
section:
ryan@washington [09:53:28 PM] [~]
-> % flash -d /dev/sdc backup.img
NAME SIZE TYPE MOUNTPOINT
sda 238.5G disk
├─sda1 200M part /boot/efi
├─sda2 152.6G part
├─sda3 619.9M part
├─sda4 3.7G part
│ └─cryptswap1 3.7G crypt [SWAP]
└─sda5 81.4G part /
sdc 7.5G disk
├─sdc1 1.7G part /media/ryan/Linux Mint 18 MATE 64-bit
└─sdc2 2.3M part
Is /dev/sdc correct? yes
Unmounting /dev/sdc ...
umount: 64-bit: mountpoint not found
Issue found on 64-bit Linux.
@aetherical has added a nice feature specifiing the card with a command line option. We should port this back to the Mac version.
--device
and -d
option from Linux to MacQuick note, where should I file bugs on the flash utility. If you have a mounted .dmg file, it doesn't work correctly (for instance if you have one mounted from an install).
Reported by @richtong
https://github.com/hypriot/flash/blob/master/Linux/flash#L150
On Linux, I need to change
if [[ echo "${disk}"|awk '{print NF}'
-gt 1 ]]; then
to
if [[ echo "${disk}"|awk '{print NF}'
> 1 ]]; then
Right now the default timezone that a flashed image comes up with is Europe/Berlin
from /etc/timezone
and it seems to be set by /root/firstboot.sh
Might be a good idea to take the default timezone from the Mac/Linux system that the cards are flashed from, as that will often match the systems/cluster one wants to setup.
Could set the timezone after the flashing before unmounting and ejecting the SD card.
I fail to connect to the wireless LAN with the --ssid option and the --password option. hypriot is version 1.7.1, flash is the latest version. The same is true when using device-init.yaml. Please help me.
Just ran the flash script and got an error on line 114: syntax error near unexpected token `)'
Hi,
When I try to flash and install Hypriot on SD card from openSUSE Tumbleweed, terminal answer :
flash hypriotos-rpi-v1.4.0.img.zip Using cached image /tmp/hypriotos-rpi-v1.4.0.img Unknown OS: linux
Any idea ?
I see there's a setting for connecting to a specified WLAN, but is there also a way to set a static IP adress for eth0?
If the script has already downloaded an image version it should not do it again if the corresponding file is already there. It should just use the already existing image and flash it.
`
The script needs pv in Linux. It does not work without.
Suggestion: Check if this package is available beforehand - and install if not. The installation would require sudo...
Any better ideas?
For instance for downloading from Amazon S3 the script needs a working version of awscli present.
When the script is started it should first check if all necessary dependencies are there and if not should abort with a notice about what needs to be done to make it work.
My use case is running docker and a web browser on each raspberry.
Is this possible ?
sudo -E flash -n athenas -s HACK-MEEE -p password https://downloads.hypriot.com/hypriotos-rpi-v1.0.0.img.zip
sudo -E flash -n esparta -s HACK-MEEE -p password https://downloads.hypriot.com/hypriotos-rpi-v1.0.0.img.zip
sudo -E flash -n zeus -s HACK-MEEE -p password https://downloads.hypriot.com/hypriotos-rpi-v1.0.0.img.zip
HypriotOS/armv7: pirate@athenas in ~
$ cat /var/lib/dbus/machine-id
9989a26f06984d6dbadc01770f018e3b
HypriotOS/armv7: pirate@esparta in ~
$ cat /var/lib/dbus/machine-id
9989a26f06984d6dbadc01770f018e3b
HypriotOS/armv6: pirate@zeus in ~
$ cat /var/lib/dbus/machine-id
9989a26f06984d6dbadc01770f018e3b
On debian machines there is a hash under cat /var/lib/dbus/machine-id
that uniquely identifies each machine. My program depends on it, but the flash tool seems to bring its own from https://downloads.hypriot.com/hypriotos-rpi-v1.0.0.img.zip. Is there another way I can uniquely identify a machine under Hypriot?
It works with Pine64?
I tried to flash the latest official Raspbian LITE image, but found out that the script doesn't expand the zip file as the URL does not end with .zip
. The script should check if the downloaded file is a ZIP archive and not just check the file extension.
flash https://downloads.raspberrypi.org/raspbian_lite_latest
Since brew implies bringing it's own particular complications and it's possible to have broken brew installations [I've been struggling with mine which resists all simple repair]
...and since your script appears to use curl, why not simplify the instructions for download of the flash script and remove the reference to brew altogether..?
curl -O https://raw.githubusercontent.com/hypriot/flash/master/$(uname -s)/flash chmod +x flash
If the extra parameters are at the end, like in unix style, are ignored:
flash hypriotos-rpi-v1.4.0.img.zip --device disk4 --config device-init.yaml --bootconf config.txt
device, config and bootconf are being ignored without raising any warning.
The current way is:
flash --device disk4 --config device-init.yaml --bootconf config.txt hypriotos-rpi-v1.4.0.img.zip
which is a bit weird.
Thanks
Considering broader deployment options like home-brew as in #55 , it might be a good idea to rename the script from flash
to hypriot-flash
to avoid current and future naming conflicts.
When I run this command:
wget https://raw.githubusercontent.com/hypriot/flash/master/$(uname -s)/flash
I receive this error:
zsh: parse error near `)'
I don't get it!?
Brew is freshly installed.
Thank in advance :)
In my opinion, a command should be available (ex: --version
or -v
) to display the version of the script.
On:
Linux Mint 17.1
As the image file actually contains data for 2 partitions, after writing to the SD card using dd, Linux does not know that the partition table may have changed.
This can be fixed by using hdparm to re-read the partition table on the SD card.
So, after the dd (or udevadm settle) and before the mount and file write processes, insert:
hdparm -z "${disk}"
Cheers,
After Matt's blog post http://matthewkwilliams.com/index.php/2015/09/07/troubleshooting-microsd-card-problems/ I got the same issue on my Mac today. Just plugged in the micro SD card adapter once too often I think, the slider went to write-protect.
The flash script just hangs with
Is /dev/disk2s1 correct? y
Unmounting disk2 ...
Unmount of all volumes on disk2 was successful
Unmount of all volumes on disk2 was successful
Flashing /tmp/hypriot-rpi-20150908-173232.img to disk2 ...
Password:
dd: /dev/rdisk2: Permission denied
64KiB 0:00:00 [9.09MiB/s] [> ] 0%
Investigating the problem shows:
$ df
Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on
/dev/disk1 974700800 724069208 250119592 75% 90572649 31264949 74% /
devfs 666 666 0 100% 1153 0 100% /dev
map -hosts 0 0 0 100% 0 0 100% /net
map auto_home 0 0 0 100% 0 0 100% /home
/dev/disk2s1 130780 58392 72388 45% 512 0 100% /Volumes/NO NAME
~
$ ls -l /dev/disk2
br--r----- 1 root operator 1, 5 Sep 8 19:50 /dev/disk2
~
$ ls -l /dev/rdisk2
cr--r----- 1 root operator 1, 5 Sep 8 19:50 /dev/rdisk2
After removing the write-protection and plugging in the SD card again it shows
$ ls -l /dev/disk2
brw-r----- 1 root operator 1, 5 Sep 8 19:52 /dev/disk2
~
$ ls -l /dev/rdisk2
crw-r----- 1 root operator 1, 5 Sep 8 19:52 /dev/rdisk2
~
$ ls -l /dev/disk2s1
brw-r----- 1 root operator 1, 6 Sep 8 19:52 /dev/disk2s1
Documentation should include how to use the AWS support to pull an image from S3.
I curled the latest image forgetting to put the -L
argument resulting in a .zip file containing HTML code indicating the redirection to S3:
$ file hypriotos-rpi-v1.1.3.img.zip
hypriotos-rpi-v1.1.3.img.zip: HTML document text, ASCII text, with very long lines, with no line terminators
However when attempting to flash I had little feedback:
Is /dev/disk2 correct? y
Unmounting disk2 ...
Unmount of all volumes on disk2 was successful
Unmount of all volumes on disk2 was successful
Flashing hypriotos-rpi-v1.1.3.img.zip to disk2 ...
Password:
605 B 0:00:00 [6.27MiB/s] [=====================================================>] 100%
dd: /dev/rdisk2: Invalid argument
0+1 records in
0+0 records out
0 bytes transferred in 0.000066 secs (0 bytes/sec)
Unmounting and ejecting disk2 ...
Unmount of all volumes on disk2 was successful
Unmount failed for /dev/disk2s2
Disk /dev/disk2 ejected
🍺 Finished.
The final message give the impression that everything have gone smoothly and the only indications are the number of record transferred and the dd: /dev/rdisk2: Invalid argument
message which gives the impression of a problem with the device rather than a problem with the image file.
For information, when downloaded correctly the final image file should look like:
file hypriotos-rpi-v1.1.3.img
hypriotos-rpi-v1.1.3.img: DOS/MBR boot sector; partition 1 : ID=0xc, start-CHS (0x0,32,33), end-CHS (0x8,73,1), startsector 2048, 131072 sectors; partition 2 : ID=0x83, start-CHS (0x8,73,2), end-CHS (0x7f,122,58), startsector 133120, 1914879 sectors
I don't know if checking DOS/MBR boot sector
will be too restrictive or if there is a smarter way to check this. At any rate it might be great to give at least a warning before proceeding to the dd part
Hi,
Just tried to use flash 1.0.0 over a version of 0.8 that wasn't installed using flash:
~ flash --hostname XXX --ssid XXX --password XXX https://downloads.hypriot.com/hypriotos-rpi-v1.0.0.img.zip
Downloading https://downloads.hypriot.com/hypriotos-rpi-v1.0.0.img.zip ...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 231M 100 231M 0 0 8231k 0 0:00:28 0:00:28 --:--:-- 10.1M
Uncompressing /tmp/image.img.zip ...
Archive: /tmp/image.img.zip
inflating: /tmp/hypriotos-rpi-v1.0.0.img
Use /tmp/hypriotos-rpi-v1.0.0.img
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk0s2 238Gi 75Gi 162Gi 32% 19805272 42499598 32% /
devfs 184Ki 184Ki 0Bi 100% 638 0 100% /dev
map -hosts 0Bi 0Bi 0Bi 100% 0 0 100% /net
map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /home
/dev/disk1s1 64Mi 30Mi 34Mi 47% 512 0 100% /Volumes/HypriotOS
s /dev/disk1 correct? yes
Unmounting disk1 ...
Unmount of disk1 failed: at least one volume could not be unmounted
Unmount of disk1 failed: at least one volume could not be unmounted
Flashing /tmp/hypriotos-rpi-v1.0.0.img to disk1 ...
Password:
dd: /dev/rdisk1: Resource busy
0 B 0:00:00 [ 0 B/s] [> ] 0%
Set hostname=XXX
Set wlan0/ssid=XXX
Set wlan0/password=XXX
Unmounting and ejecting disk1 ...
Unmount of disk1 failed: at least one volume could not be unmounted
Unmount of disk1 failed: at least one volume could not be unmounted
Volume failed to eject
👎 Someting went wrong.
At I guess is it because of how the card has been formatted?
After flashing an ODROID SD card image there is no FAT filesystem. Re-flashing this SD card does not work with flash
on the Mac.
flash
only uses df
to get the list of devices. The SD card does not show up there.
$ df
Filesystem 512-blocks Used Available Capacity iused ifree %iused Mounted on
/dev/disk1 974700800 683874848 290313952 71% 85548354 36289244 70% /
devfs 666 666 0 100% 1154 0 100% /dev
map -hosts 0 0 0 100% 0 0 100% /net
map auto_home 0 0 0 100% 0 0 100% /home
But using diskutil
can list the SD card as well:
$ diskutil list
/dev/disk0 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: GUID_partition_scheme *500.3 GB disk0
1: EFI EFI 209.7 MB disk0s1
2: Apple_CoreStorage Macintosh HD 499.4 GB disk0s2
3: Apple_Boot Recovery HD 650.0 MB disk0s3
/dev/disk1 (internal, virtual):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_HFS Macintosh HD +499.0 GB disk1
Logical Volume on disk0s2
/dev/disk2 (internal, physical):
#: TYPE NAME SIZE IDENTIFIER
0: FDisk_partition_scheme *15.8 GB disk2
1: Linux 837.3 MB disk2s1
/dev/disk3 (disk image):
#: TYPE NAME SIZE IDENTIFIER
0: Apple_partition_scheme +17.4 MB disk3
1: Apple_partition_map 32.3 KB disk3s1
2: Apple_HFS Flash Player 17.4 MB disk3s2
And with diskutil info
we can check further things like write protection:
$ diskutil info /dev/disk2
Device Identifier: disk2
Device Node: /dev/disk2
Whole: Yes
Part of Whole: disk2
Device / Media Name: SD Card Reader
Volume Name: Not applicable (no file system)
Mounted: Not applicable (no file system)
File System: None
Content (IOContent): FDisk_partition_scheme
OS Can Be Installed: No
Media Type: Generic
Protocol: USB
SMART Status: Not Supported
Total Size: 15.8 GB (15811477504 Bytes) (exactly 30881792 512-Byte-Units)
Volume Free Space: Not applicable (no file system)
Device Block Size: 512 Bytes
Read-Only Media: Yes
Read-Only Volume: Not applicable (no file system)
Device Location: Internal
Removable Media: Yes
Media Removal: Software-Activated
Virtual: No
OS 9 Drivers: No
Low Level Format: Not supported
The Read-Only Media: Yes
or No
shows the write protection switch of the SD card.
Actually, the generated wifi configuration doesn't provide hidden ssid support.
It would be nice to manage hidden wifi networks by adding an optional param scan-ssid to device-init.yaml such as:
hostname: black-pearl
wifi:
interfaces:
wlan0:
ssid: "MyNetwork"
scan-ssid: 1
password: "secret_password"
This way, generated wlan0 file would look like
allow-hotplug wlan0
auto wlan0
iface wlan0 inet dhcp
wpa-ssid MyNetwork
wpa-scan-ssid 1
wpa-psk generated_psk
Very useful tool to flash images to SD cards, great work!
Is it also possible to change the hostname of a raspbian image or is that only possible with a Hypriot image?
The flash script should set wifi name and password in new HypriotOS SD images.
When I run
./flash --hostname somenode hypriotos-rpi-v1.0.1.img
... the flashing itself is successful, but the customizing afterwards is not. I get the error
"Permission denied"
Does anyone else run into this issue?
The terminal output:
Flashing hypriotos-rpi-v1.0.1.img to /dev/sdb ...
1000MiB 0:00:00 [1,71GiB/s] [================================>] 100%
24+7681 records in
24+7681 records out
1048576000 bytes (1,0 GB, 1000 MiB) copied, 42,5457 s, 24,6 MB/s
Waiting for device...
/dev/sdb: Permission denied
Environment:
On:
Linux Mint 17.1 Xfce
The syntax of the pv and dd command may function better as per method 1 on:
https://askubuntu.com/questions/215505/how-do-you-monitor-the-progress-of-dd
So, the line in flash:
pv -s "${size}" < "${image}" | sudo dd bs=1M "of=${disk}"
could be:
dd if="${image}" status=none | pv -s "${size}" | dd bs=1M of="${disk}"
echo "Done."
As there is a pause before dd finishes completely, status update of 'Done' maybe useful also.
Cheers,
I tried running the flash script on my Ubuntu 14.04 (trusty) system that had hdparm installed and got an error. I enabled xtrace and found the problem:
+ hdparm -z /dev/sdd
/dev/sdd: Permission denied
Incoming pull request for this issue.
On the osx version when gnu-sed is installed, it breaks, it should be hard coded to /usr/bin/sed
Thanks,
I got this working by applying this patch to flash:
diff -Naur orig/flash new/flash
--- orig/flash 2017-09-20 11:36:11.953099540 -0400
+++ new/flash 2017-09-20 11:30:56.619773917 -0400
@@ -277,6 +277,8 @@
if beginswith /dev/mmcblk "${disk}" ;then
dev="${disk}p1"
+elif beginswith /dev/loop "${disk}" ;then
+ dev="${disk}p1"
else
dev="${disk}1"
fi
And using the following approach:
# wrapper.sh [...]
qemu-img create -f raw test.img 4G
disk=$(sudo losetup --show -Pf test.img)
sudo bash ./flash -d $disk $@
sudo losetup -d $disk
Later I could test it with qemu-system-arm
Perhaps this could be integrated in the script?
Any way that I can pass the mac address of the boot device to become the host name during first boot? MAC+fixed named.
The current flash tool is almost 100% applicable to Raspbian and sometimes Raspbian is a better option for some use cases (for example if you need some software that it is only in stretch). Would you be open to extend the current tool to support additional configuration options for Raspbian?
I have created a first version of the code here. I am more than happy to create a pull request.
Regards
I have used this tool before on previous releases of images to flash SD cards.
Since I upgraded MAC to Sierra I am getting a bad device.
Not sure if this is a problem or if I have screwed up something on mac. The SD cards do not seem to be getting written properly at all anymore.
Not sure why the change in device name to rdisk2 occurs or why the DD is failing.
pogo:flash-master sbeeker$ sudo flash --hostname pi1 https://blog.hypriot.com/hypriotos-rpi-v1.1.3.img.zip
Downloading https://blog.hypriot.com/hypriotos-rpi-v1.1.3.img.zip ...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11329 0 11329 0 0 58897 0 --:--:-- --:--:-- --:--:-- 59005
No SD card found. Please insert SD card, I'll wait for it...
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk1 931Gi 741Gi 189Gi 80% 2811341 4292155938 0% /
devfs 186Ki 186Ki 0Bi 100% 642 0 100% /dev
map -hosts 0Bi 0Bi 0Bi 100% 0 0 100% /net
map auto_home 0Bi 0Bi 0Bi 100% 0 0 100% /home
map -fstab 0Bi 0Bi 0Bi 100% 0 0 100% /Network/Servers
localhost:/fDQm93gxSUdjdppJeemvo8 931Gi 931Gi 0Bi 100% 0 0 100% /Volumes/MobileBackups
/dev/disk2s1 30Gi 1.9Mi 30Gi 1% 0 0 100% /Volumes/PI
Is /dev/disk2 correct? y
Unmounting disk2 ...
Unmount of all volumes on disk2 was successful
Unmount of all volumes on disk2 was successful
Flashing /tmp/image.img.zip to disk2 ...
11.1KiB 0:00:00 [62.1MiB/s] [================================>] 100%
dd: /dev/rdisk2: Invalid argument
0+1 records in
0+0 records out
0 bytes transferred in 0.000399 secs (0 bytes/sec)
Unmounting and ejecting disk2 ...
Unmount of all volumes on disk2 was successful
Unmount failed for /dev/disk2s2
Disk /dev/disk2 ejected
🍺 Finished.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.