GithubHelp home page GithubHelp logo

pikvm / pi-builder Goto Github PK

View Code? Open in Web Editor NEW
95.0 95.0 60.0 14.78 MB

Extensible tool to build Arch Linux ARM for Raspberry Pi on x86_64 host using Docker

Home Page: https://pikvm.org

License: GNU General Public License v3.0

Makefile 43.77% Shell 1.42% Python 53.63% C 1.18%
archlinuxarm arm binfmt-misc build-environment build-system build-tool builder docker makefile pi-builder pi-kvm pikvm raspberry-pi raspberrypi sd-raspberry-pi

pi-builder's Introduction

Open and inexpensive DIY IP-KVM based on Raspberry Pi

Discord Reddit

A very simple and fully functional Raspberry Pi-based KVM (Keyboard-Video-Mouse) over IP that you can make with your own hands. This device helps to manage servers or workstations remotely, regardless of the health of the operating system or whether one is installed. You can fix any problem, configure the BIOS, and even reinstall the OS using the virtual CD-ROM or Flash Drive.

The website: pikvm.org. Also check out the documentation and join to the Discord Community Chat for news, questions and support!

Web UI

Features

  • Can be made based on Raspberry Pi 2, 3, 4 and Zero2W;
    • Raspberry Pi 5 is not supported right now. It doesn't have GPU video encoders, therefore, there is no point in using it for PiKVM, it will not give any performance boost for this case. The Pi 5 is a great device, just not suitable for PiKVM.
  • FullHD video using advanced HDMI-to-CSI bridge or USB dongle;
  • Extra low video latency with MJPEG or H.264 / WebRTC (for CSI bridge);
  • Bootable Virtual CD-ROM and Flash Drive, ability to store images on NFS;
  • USB Keyboard and mouse (with leds and the wheel), PS/2 keyboard, Bluetooth HID;
  • Control the server power using ATX functions;
  • Access via Web UI or VNC;
  • Ability to use IPMI BMC, IPMI SoL, Redfish and Wake-on-LAN to control the server;
  • The ready-made OS with read-only filesystem;
  • Extensible authorization and HTTPS out of the box;
  • Health monitoring of the Pi;
  • Control GPIO ports and USB relays;
  • 100% Open Source!

Variants

PiKVM supports several different hardware configurations, referred to as platforms. All of them uses our common open source software stack.

  • PiKVM V4 and V3 are our own plug-and-play, ready-made devices (Raspberry Pi included).
    Supports H.264 video & audio transmission, host power management, mass-storage emulation and much more. They are really well-made, reliable things which you can use yourself or provide to your clients. V4 and V3 also have more features than DIY builds.

  • PiKVM V2 and V1 devices are DIY so you can make it with your own hands.
    If you don't know where to put your old Raspberry Pi, here's a great goal.

Look at what Tech Bloggers are saying about us :)

PiKVM V4 Plus & Mini PiKVM V3 DIY PiKVM
Novaspirit Tech
Level1Techs
Jeff Geerling
~~~ LINUS TECH TIPS ~~~
Novaspirit Tech
Level1Techs
Techno Tim
The Geek Freaks (DE)
Novaspirit Tech
Hackaday
Tom's HARDWARE
Elector MAG
Our boring presentation of DIY

~~~ BUY IT NOW ~~~



~~~ BUY IT NOW ~~~



DIY Getting Started



⭐ ⭐ ⭐ CORE SYSTEM ⭐ ⭐ ⭐
V4 Plus V4 Mini V3 DIY V2 DIY V1
Plug and Play out of box! DIY DIY
Base Raspberry unit CM4
Included!
CM4
Included!
RPi4
Included!
RPi4
Zero2W
RPi3
RPi2
Zero2W
⭐ ⭐ ⭐ VIDEO ⭐ ⭐ ⭐
V4 Plus V4 Mini V3 DIY V2 DIY V1
Video passthrough to local display
HDMI 1920x1200@60Hz support
for big displays
HDMI 1920x1080@60Hz support
for better BIOS/UEFI compatibility
HDMI audio capture
Super fast H.264 & MJPEG video:
720p - 60fps, 1080p - 30fps for H.264
⭐ ⭐ ⭐ PERIPHERY ⭐ ⭐ ⭐
V4 Plus V4 Mini V3 DIY V2 DIY V1
USB keyboard/mouse emulation
Virtual Flash Drive / CD-ROM emulation
Ability to simulate "eject/insert" for USB
Onboard ATX controller for power management of the host DIY DIY
Built-in OLED with IP and other info
Multiport KVM switches support
⭐ ⭐ ⭐ HARDWARE ABILITIES ⭐ ⭐ ⭐
V4 Plus V4 Mini V3 DIY V2 DIY V1
Internal secured USB 3.0 storage port
USB serial console port
CISCO-style RJ-45 console port
A real-time clock for accurate logging
Locator LED to find device in the rack
mPCIe slot with USB lines for LTE/5G cards
SIM card slot for modem
External antenna support
USB host support (external USB devices connectivity)
⭐ ⭐ ⭐ POWER AND COOLING ⭐ ⭐ ⭐
V4 Plus V4 Mini V3 DIY V2 DIY V1
Cooling system / fan type Quiet
Radial
Fanless Axial DIY DIY
PWM fan controller
Fan speed and health monitoring
Power consumption in idle mode 3.3W 2.65W 3.3W
⭐ ⭐ ⭐ MISCELLANEOUS ⭐ ⭐ ⭐
V4 Plus V4 Mini V3 DIY V2 DIY V1
USB relays, lamps, smartplugs control
Pi's GPIO ports control from the Web UI
More indepth details about the PiKVM features

Fully-featured and modern IP-KVM

  • Cheaper, but better than commercial solutions
    Costs between $30 and $100 depending on the features desired. Even the most expensive configuration will be cheaper than a $500 commercial IP-KVM.
  • Easy to build - For the V1 variant
    A ready-to-use OS that can be created just by running make build and installed to an SD-card using make install. The hardware can be made in half an hour and without soldering.
  • The widest hardware support
    There are many ways to build a PiKVM. Video capture devices can be attached using the CSI-2 or USB interfaces. Raspberry Pi models 2, 3, 4 or Zero2W may be used. Any combination of hardware is supported, and PiKVM implements the maximum possible set of features.
  • Very low latency
    ~100 milliseconds of video latency. This is the smallest delay of all existing solutions.
  • Extra lightweight and fancy Web UI
    No weird proprietary clients. No ugly Java applets. Just use your favorite browser to connect to the PiKVM. No flash plugins either!
  • Keyboard and mouse
    Mouse usage works directly in the browser. The keyboard emulator supports displaying the state of the keyboard LEDs.
  • Mass Storage Drive
    On the Raspberry Pi 4 and Zero2W, PiKVM can emulate a virtual CD-ROM or Flash Drive. A live image can be uploaded to boot the attached server.
  • ATX power management
    With a very simple circuit that can be assembled on a breadboard, the power button of the attached server can be controlled using the ATX button headers on the motherboard.
  • Security
    PiKVM can be opened to the Internet and no one will get access without the password. SSL is used to protect traffic from being intercepted by third parties.
  • Local monitoring
    PiKVM monitors the health of the Raspberry Pi board and will warn you if it's too hot or is not receiving enough power.
  • GPIO management You can control GPIO and USB relays via the web interface. The extensible interface allows you to make anything out of it.

Production-ready

  • Linux-based embedded OS
    The PiKVM OS is based on Arch Linux ARM and can be customized for any needs. Thousands of pre-built binary packages are at your service and can be updated with a single command!
  • Read-only filesystem
    By default the OS runs in read-only mode, as an embedded system should. This prevents damage to the memory card due to a sudden power outage.
  • VNC server
    The managed server and its BIOS can be accessed using a regular VNC client which supports JPEG compression.
  • IPMI BMC and Redfish
    Use ipmitool, ipmiutil or any existing network infrastructure that supports IPMI to monitor and manage the server's power. Redfish and redfishtool supported too.
  • Wake-on-LAN
    PiKVM can be configured to power up a managed server using WoL. This will be available in the Web menu.
  • Extensible authorization methods
    Multiple PiKVMs can be configured to use a common authorization service. PAM and its rich authorization mechanisms can also be used to integrate PiKVM into your existing authentication infrastructure.
  • Macro scripts
    If repetitive actions must be performed on the attached server (or on several different servers), a macro can be recorded with keyboard & mouse actions and can be played back multiple times.

Open Source

  • Community & support
    PiKVM has a large community. Join the discord chat if you have any questions.
  • Extensible
    PiKVM was designed as a set of microservices with a plug-in architecture. It's very easy to modify and maintain.
  • Comprehensive API
    Everything that can be done via the user interface can also be done via a powerful HTTP API.
  • Know-how
    We created our very own MJPG video server written in C with multi-threading support and GPU video encoding - the fastest streaming solution available to provide the best video quality for PiKVM. We also tested a lot of hardware configurations so that you can be sure devices you assemble will work reliably.

DIY Getting Started

PiKVM supports several different DIY platforms. Now available: V2 and V1.

  • Recommended: V2 is the most powerful implementation for Raspberry Pi 4 and Zero2W supporting all of the features of PiKVM including the Mass Storage Drive. It's also the easiest to make.
  • V1 was designed to work with Raspberry Pi 2 and 3 that do not have USB emulation port and requires a few more components for a basic implementation. It also does not support the Mass Storage Drive feature.

The full list of parts is at the very beginning according to the assembly instructions for the corresponding platform. You can review both options to decide which one is most suitable for you.

So, again, in short:

  • DIY PiKVM V2:

  • DIY PiKVM V1:

    • Here are the instructions and a complete list of components.
    • Requires Raspberry Pi 2 or 3.
    • Requires Raspberry Pi Pico for the keyboard/mouse emulation.
    • Requires extra wiring and little bit more parts.
    • Supports H.264 (for recommended CSI capture).
    • Doesn't support CD-ROM and Flash Drive (mass storage) emulation.

Donate

This project is developed by Open Source enthusiasts. If you find PiKVM useful or it has saved you a long trip to check on an unresponsive server, you can support us by donating a few dollars on Patreon or Paypal or buying our devices. With this money, we will be able to buy new hardware (Raspberry Pi boards and other components) to test and maintain various configurations of PiKVM, and generally devote significantly more time to the project. At the bottom of this page are the names of all the people who have helped this project develop with their donations. Our gratitude knows no bounds!

If you wish to use PiKVM in production, we accept orders to modify it for your needs or implement custom features you require. Contact us via live chat or email the lead developer at: [email protected]


Special thanks

These kind people donated money to the PiKVM project and supported work on it. We are very grateful for their help, and commemorating their names is the least we can do in return.

View all people!
  • A. Isenring
  • Aaron Graubert
  • Aaron Heise
  • Aaron Stein
  • Accalia
  • Adam Goodbar
  • Adam S
  • Adam Stuart
  • AdamBomb
  • adipisicing
  • Adrian Basham
  • Adrian Popescu
  • Ahmed Syed
  • Alberto Bassi
  • alejandro
  • Aleksei Brusianskii
  • Alessio Curri
  • Alex T
  • Alex Z
  • Alexander Karmanov
  • Alexander Lahuerta
  • Alexander Martin
  • Alexander Pankov
  • Alexandre Jablonski
  • Alexey Kamenskiy
  • alm0241
  • Alok Anand
  • Alucard
  • Ananthaneshan Elampoornan
  • Andreas Marufke
  • Andreas Schmid
  • Andrew Brant
  • Andrew Melton
  • Andrew Reusch
  • Andrew Ruan
  • Andrzej V
  • Andy
  • Andy Keys
  • Anish Patel
  • Anix
  • Anonymous
  • Anthony Junk
  • Anton Kovalenko
  • Armen
  • Aron Green
  • Aron Perelman
  • Artem Simonov
  • Arthur Mayer
  • Arthur Woimbée
  • Ashlesh Chaudhari
  • Asim Shakour
  • Augusto Becciu
  • AVS Computer
  • awkspace
  • Badal Patel
  • baddog
  • Bao Tin Hoang
  • Bean Co.
  • Bela Bargel
  • Belf Igor
  • Ben Gordon
  • Ben Scott
  • Benedikt Heine
  • Benedikt Meier
  • Benjamin Frewert
  • Benjamin Melancon
  • Benjamin Schwartz
  • Benjamin Stegmann
  • Benni Stauder
  • Bernhard Fitzke
  • Beu
  • bikmaek
  • bitjoe
  • Bits and Bytes Computers LLC
  • Bjoern Petsch
  • Blair Hasler
  • Blindside
  • Blue Frog LLC
  • Bootstrapper - Programmierung erklärt
  • Bosco
  • Bradford King
  • Brainspore Networks
  • Branden Shaulis
  • Brandon Daniels
  • Brian
  • Brian Moses
  • Brian T Mulcahy
  • Brian Vecchiarelli
  • Brian White
  • Bruno Gomes
  • Bryan Adams
  • Bryan Montgomery
  • Buzzer
  • C P ELSE
  • Calanish
  • Cameron Hatcher
  • Cameron Tacklind
  • Carl Mercier
  • Carl-Fredrik Johansson
  • Carlos Eduardo Porter Herrera
  • Carlos Garcia
  • Carlos Manuel Torres
  • cbad536
  • César Nascimento
  • CHINATERA LIMITED
  • Chris Blackmon
  • Chris Burton
  • Chris Campbell
  • Chris Jackson
  • Chris Lewis
  • Chris Rizio
  • Christi King
  • Christian Schlögl
  • Christian Svensson
  • Christoph Dette
  • Christof Maluck
  • Christoffer Lund
  • Christopher Bulla
  • Christopher Gelatt
  • Christopher Hearn
  • Christopher Mandlbaur
  • Christopher Mendoza
  • Christopher Simms
  • Chucktastic
  • Cihan VURAL
  • clauskj3r
  • Clifford Coleman
  • Clinton Lee Taylor
  • Cole Imhoff
  • Colin Goodman
  • Corey Layton
  • Corey Lista
  • Craig Keenan
  • Crossfactor
  • Cruzzer
  • ctag
  • Curt Sammer
  • CyB0rgg
  • DeMentor
  • Desmond Whitt
  • Daegara
  • DailyAneurism
  • Damon Meledones
  • Dan Berkowitz
  • Dan Brakeley
  • Daniel Bowder
  • Daniel Cabrera
  • Daniel Davila
  • Danilo Saft
  • Danne
  • Dariusz Techmański
  • David
  • David Brausewetter
  • David Davis
  • David Godibadze
  • David Howell
  • David Irvine
  • David Klinkman
  • David Niemann
  • David Shay
  • David Ye
  • David York
  • Denis
  • Denis Andreev
  • Denis Yatsenko
  • Dennis Becker
  • Dennis Joslin
  • Dennis Lomet
  • Derek Jarvis
  • Derek Yap
  • Didrik
  • digitalbaconbits
  • Dimitrij Jedich
  • dixon wong
  • dizztrukshin
  • Dmitry Shilov
  • DogeLabs
  • Dominic Phoon
  • Dominik Klonowski
  • Donald Hays
  • Edmon Abdul Nur
  • Edward Wang
  • Egan Ford
  • Elani Ferri
  • Elliot Woo
  • Entt
  • Eric Phenix
  • Ethan Shold
  • Eugene Sukhodolin
  • ewook
  • eye-catcher.com
  • Fabian Druschke
  • Fabiano Sidler
  • Far Pin Solutions, LLC
  • Felyx Gabryel
  • Fergus McKay
  • Finn Ebenritter
  • floppy
  • fo0bar
  • Foad Yousef
  • Foamy
  • Foli Ayivoh
  • Folkert Weistra
  • Francisco Pavon
  • Frank
  • Frank Sander
  • Frederick Czajka
  • Fredrik Idréus
  • Garrett Dangerfield
  • Ge Men
  • Geekworm
  • Genkinger Andreas
  • Geijer
  • Geoffrey Wright
  • George Becker
  • Georgy Brodsky
  • Gerald
  • Gerardus Vernimmen
  • Gernot Neuschröer
  • Giovanni Fulco
  • GK
  • Glen Dragon
  • Greg Winterstein
  • Gregory Smith
  • Gregory Treantos
  • grewil
  • Grey Cynic
  • Guenter Honisch
  • Guido Bernacchi
  • Gustin Johnson
  • György Tamás Vizi
  • Haiberg GmbH
  • Haven Zheng
  • Heibunny
  • Heikki Tiittanen
  • Helio Leonardo Pinheiro e Mota
  • Henrik Ählström
  • Henry Hood
  • HimKo
  • HouseFPV
  • Howard Simons
  • HyunohRyu
  • Icculus
  • iks
  • INFO TRX INC
  • Invader Monks
  • Ioannis Karageorgos
  • Isaac
  • IT Lifesaver
  • Ivan Ganev
  • Ivan Josiah Lapis
  • Ivan Shapovalov
  • iwbjhbweriuhf
  • J
  • J L
  • Jaanus
  • Jackson Wyatt
  • Jacob Karaffa
  • Jacob Morgan
  • James Cadd
  • James Cobb
  • James Edwards
  • James Kocher
  • James Mayhugh
  • James Noonan
  • James Ye
  • Jamie Murphy
  • Jamie Scott
  • Jan Niehusmann
  • Jannick Oursin
  • Jari Hiltunen
  • Jason Crossley
  • Jason Downey
  • Jason Toland
  • Jasper Backer
  • Jay Davis
  • Jay Isaacs
  • Jazereel Goh
  • Jean-Daniel Croteau
  • Jean-Philippe Guilbault
  • Jeff
  • Jeff Bowman
  • Jeff Urlwin
  • Jennifer Herting
  • Jennifer Rowlett
  • Jeremy Abel
  • Jeremy Combs
  • Jeremy Hines
  • Jerremy Holland
  • Jerry Nall
  • Jerry Y. Chen
  • Jim Bailey
  • Jim Harbin
  • Jimmy Burgett
  • Jimmy Stanley
  • Joachim Bruening
  • Joe Hanson
  • Joe Hinteregger
  • Joe Ventura
  • Joel Jacobs
  • Johannes Heigermose
  • John Andersen
  • John Copeland
  • John F Glenn
  • John Holmes
  • John Kelley
  • John McGovern
  • Johnny Henson
  • Jon Ferguy
  • Jon-Eric
  • Joni Ruuskanen
  • Jonas Fischer
  • Jonathan Slenders
  • Jonathan Vaughn
  • Joost Backer
  • Joseph Swift
  • Josh Nethery
  • Josh Ricker
  • Josh VanDeraa
  • Joshua Futterer
  • Jordan Blake
  • Jordi Pakey-Rodriguez
  • Joris van Embden
  • Jozef Riha
  • Jörgen Fredriksson
  • Julian Forero
  • Julian Schneider
  • Julien Angelier
  • Justin
  • Justin Waters
  • Kai Hadler
  • Kamil Chyba
  • Kari Matti Korpi
  • Karl Dunne
  • Karl Moos
  • Keith Muggleton
  • Ken Lee
  • Kenneth Younger III
  • Kenny Hui
  • KeonWoo PARK
  • Kevin Bajohr
  • Kevin Schwartz
  • Kevin Sherwood
  • Kiera Kujisawa
  • Kiran Schuler
  • Koloman
  • Konrad Neitzel
  • Krzysztof Żelaśkiewicz
  • Lance Ward
  • Larry Meaney
  • Lars
  • Lars Reinhardt
  • Lee Wilkinson
  • LeeNX
  • Leon Siegl
  • Leonard Feineis
  • Lewis Wild
  • Liran
  • Liviu Dimitriu
  • Lizardo Hernandez
  • LoCascio
  • Lordbob75
  • Lothar Schweikle-Droll
  • Louis Müller
  • LSDTripp
  • Ľubor Slušný
  • Luca Di Diomede
  • Lucio De Carli
  • Luiz Bizzio
  • Lukas Bischof
  • Lukas Kammerer
  • Lukas Söder
  • Maksim Terehin
  • Malcolm Cameron
  • Manfred Radeschnig
  • Marc Khouri
  • Marcin Wilk
  • Marcio Zimbres
  • Marco Rossi
  • Marcos Wolf
  • Marek Marczykowski-Górecki
  • Marius
  • Mar. Balske
  • Mark Farrell
  • Mark Gilbert
  • Mark Knam
  • Mark Robinson
  • Markrosoft
  • Markus Halm
  • Markus Schicker
  • Markus Sobczack
  • Marshall Bjerke
  • Marten Hermans
  • Martin Gasser
  • Martin Hofbauer
  • Martin Raine
  • Martin Suelmann
  • Martin Wilhelmi
  • Marvin Honderboom
  • Mateusz Grabowski
  • Mathias Uhl
  • Matt Kane
  • Matthew Cameron
  • Mauricio Allende
  • Max Evans
  • Mecky
  • Mehmet Aydoğdu
  • Michael Bartholomew
  • Michael Bell
  • Michael Bombe
  • Michael Collins
  • Michael Copeland
  • Michael Ho
  • Michael Kovacs
  • Michael Lynch
  • Michael Pennington
  • Michael Sage
  • Michael Stella
  • Michael Thalmann
  • Michael Wu
  • MichaelZ
  • Michel Bissonnette
  • Mikael Wikström
  • Mike Mason
  • Mikhael Mariano
  • Milan Múčka
  • Miles Davis
  • Minh Tang
  • Moez Tharani
  • Morgan Helton
  • Murad Khasawneh
  • Myron Weber
  • N Patel
  • Nathaniel Griswold
  • Nelson Lee
  • nezu
  • Nicholas Jeppson
  • Nicholas Kopas
  • Nicholas Walczak
  • Nick Leffler
  • Nick Roethemeier
  • Nico Baumgartner
  • Nicolai Kragh-Hansen
  • Nigel Smith
  • Nihal Fernando
  • Nils Orbat
  • Nis Wechselberg
  • Nithin Philips
  • Nod Swal
  • Nolan Haynes
  • nubbn
  • nybble
  • Oh Be
  • Oliver Schwarz
  • Oliver Zimmer
  • Omar El-Domeiri
  • Omar Siam
  • Oscar
  • Patrick
  • Patrick Fortin-Ducharme
  • Patrick McDowell
  • Patrick Wagstrom
  • Paul Bishop
  • Paul De La Rosa
  • Paul Pietkiewicz
  • Paul Tan
  • Pawel Trofimiuk
  • Peder Madsen
  • Peter
  • Peter Drayton
  • Peter Farrelly
  • Peter Okelmann
  • Petra Lohmann
  • Petri Heiskanen
  • Phil Wu
  • Philip Edwards
  • Philip Merricks
  • Pierre Brassart
  • Pierre Peine
  • posicat
  • pozitron03
  • Przemysław Szypowicz
  • P_Dmitrij
  • Qteal
  • Quattro Uno
  • Quentin Peten
  • Ralph Borchers
  • Ranc1d
  • Randall D Bilbrey
  • RandomJerk
  • Ref Chowdhury
  • Raphael Schitz
  • René Rathenau
  • ReysDad
  • Ricardo Marques
  • Richard
  • Richard Bernarts
  • Richard Fancher
  • Richard Freemantle
  • Richard Michael
  • Rico Cantrell
  • Rob
  • Rob Holden
  • Rob Tongue
  • Robert Klauco
  • Robert Weemhoff
  • Robin Gfatter
  • Rodion DENISYUK
  • Rohit Priyadarshi
  • Rolfs 3D UG
  • Ronald LeBaron
  • Ronald Wells
  • Ronny Haldorsen
  • rotx
  • Rufo Sanchez
  • Russell Scott
  • Ryan
  • Ryan Peacock
  • Samed Ozoglu
  • Sameul Davies
  • Samuel Cote
  • Samuel Vetsch
  • Samuel Walker
  • Sarah Foster
  • Sarten X
  • Satish Alwani
  • Scott
  • Scott Gagon
  • Scott Spicola
  • Scott Tusing
  • Scott Worthington
  • Scuba
  • Sean
  • Sean Akers
  • SEAT
  • Sebastian
  • Seonwoo Lee
  • Sergey Lukjanov
  • Seth Jennings
  • Shane Selling
  • Shawn Butts
  • Sheran Gunasekera
  • Shichun Chen
  • Shin Guey Wong
  • Simon Evans
  • Simon Sundgaard
  • Simplistic Realities
  • Sirmo
  • Snowy Maslov
  • Solve Technology
  • srepac
  • Stefan Bautz
  • Stefan Müller
  • Stefan Stemmer
  • Stefan Vaillant
  • Stephan Schmidt
  • Stephen
  • Stephen Hocking
  • Steve Jones
  • Steve Kerr
  • Steve Ovens
  • Steve Stringham
  • Steven Richter
  • Stratagem Solutions Ltd
  • Sven Breckler
  • sudo34
  • SuperHiTech
  • Tango_Echo_Alpha
  • Tarlak Desaydrone
  • TechBear
  • techlobo
  • Ted
  • Tejun Heo
  • TheSnowedOne
  • TheTechGiant
  • Thomas Charisoulis
  • Thomas Gitlin
  • Thomas Hagenmaier
  • Thomas Hedberg Jensen
  • Thomas Price
  • Thomas Søfteland
  • Tim Lenz
  • Tim Wilkinson
  • Timo Brinkmann
  • Timothee Besset
  • TitomusPrime
  • Tom Lawson
  • Tom York
  • Tomas Kuchta
  • Tomáš hrubý
  • Torsten Droste
  • Torsten Knoll
  • Tobias Schafferhans
  • Tracy Fitch
  • Tristan Schoening
  • Truman Kilen
  • turbochris
  • tutanak
  • Tyler
  • Udo Schroeter
  • Uli Fahrer
  • Vasily Lazarev
  • Vicente Salvador Cubedo
  • Vidru Eduard
  • Viktor Aschenbrenner
  • Viktor Ekmark
  • Vincent Chov
  • Vlad Sterescu
  • Volker Gropp
  • Walli
  • Walter_Ego
  • William Wenzel
  • Will Froning
  • William Hooper
  • William Perrin
  • William Stearns
  • Woojin Son
  • xMdb
  • Yanko Kaneti
  • Yaroslav Kulikovskikh
  • Yethal
  • Yevgeniy Kuksenko
  • Yew Kay Yan
  • Yogi
  • Yigal Dar
  • YURI LEE
  • Yurii Ostapchuk
  • Zeljko
  • zgen
  • Zoltan Magyari
  • Zsombor Vari

pi-builder's People

Contributors

alansill avatar bbeaudoin avatar etkaar avatar faddat avatar goblinqueen avatar intelfx avatar jakicoll avatar lopromise avatar mdevaev avatar msumulong avatar reedy avatar squoimote avatar xornet-sl avatar yura80 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pi-builder's Issues

Producing a .img file instead of flashing

I am testing out this system for use in Starport, a rapid blockchain development tool that allows developers and entrepreneurs to ship blockchain products as hardware.

I am trying to figure out if there is a way to get output as an image file.

Currently, I have been using packer-builder-arm but the docker-based approach used here looks interesting.

Goal

Basically, I just need to know how to build an image file instead of flashing to SD card, and we could use pi-builder and significantly lower our build times and increase development speed by reducing the length of CI cycles.

Trying to make os for Pi Zero W

Hello,
I've tried many times to get pikvm to get past the "make os" step but I always get the below output, it looks to trying to get the image from a URL that no longer exists.

===== Ensuring base rootfs =====
if [ ! -e ./.cache/base-rootfs-zerow.tar.gz ]; then
mkdir -p ./.cache
&& echo "Signature: 8a477f597d28d172789f06886806bc55" > "./.cache/CACHEDIR.TAG"
&& curl -L -f http://de3.mirror.archlinuxarm.org/os/ArchLinuxARM-rpi-latest.tar.gz -z ./.cache/base-rootfs-zerow.tar.gz -o ./.cache/base-rootfs-zerow.tar.gz
; fi
Warning: Illegal date format for -z, --time-cond (and not a file name).
Warning: Disabling time condition. See curl_getdate(3) for valid date syntax.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404 Not Found
make[1]: *** [Makefile:276: _rpi_base_rootfs_tgz] Error 22
make[1]: Leaving directory '/home/shawn/os/.pi-builder/v2-hdmi-zerow'
make: *** [Makefile:52: os] Error 2

Build Fails in Podman using non-Root

Running on an updated Fedora 33

$ uname -a
Linux localhost.localdomain 5.9.13-200.fc33.x86_64 #1 SMP Tue Dec 8 15:42:52 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Using alias to podman,

$ ls -lsa `which docker`
0 lrwxrwxrwx. 1 root root 15 Dec 10 18:49 /usr/bin/docker -> /usr/bin/podman

$ podman --version
podman version 2.2.1

As non-root use, I can access /tools/install-binfmt

$ ls /proc/sys/fs/binfmt_misc/
arm  register  status

During a build of PiKVM (make os), Podman is unable to mount:

===== Toolbox image is ready =====
===== Ensuring arm binfmt =====
docker run \
		--rm \
		--tty \
		--privileged \
	pi-builder-toolbox /tools/install-binfmt \
		--mount \
		arm \
		/usr/bin/qemu-arm-static
:: Mounting binfmt_misc to '/proc/sys/fs/binfmt_misc' ...
mount: /proc/sys/fs/binfmt_misc: permission denied.
Traceback (most recent call last):
  File "/tools/install-binfmt", line 139, in <module>
    main()
  File "/tools/install-binfmt", line 128, in main
    _mount_binfmt(options.binfmt_misc)
  File "/tools/install-binfmt", line 68, in _mount_binfmt
    subprocess.check_output(["mount", "binfmt_misc", "-t", "binfmt_misc", mount_path])
  File "/usr/lib/python3.9/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib/python3.9/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['mount', 'binfmt_misc', '-t', 'binfmt_misc', '/proc/sys/fs/binfmt_misc']' returned non-zero exit status 32.
make[1]: *** [Makefile:186: binfmt] Error 1
make[1]: Leaving directory '/home/cduplantis/code/pikv/os/.pi-builder'
make: *** [Makefile:54: os] Error 2

This is successful with sudo/rooted user however.

Potentially related to containers/podman#4725

pacman-key --populate fails because current time is not accessible in the build container

I'm trying to make os BOARD=rpi STAGES="__init__ os __cleanup__".

It fails at the pacman-key --populate archlinuxarm step:

Step 22/42 : RUN echo "Server = $REPO_URL/$arch/$repo" > /etc/pacman.d/mirrorlist && pacman-key --init && pacman-key --populate archlinuxarm
[...]
==> Appending keys from archlinuxarm.gpg...
gpg: key B55C5315DCD9EE1A was created 16088 days in the future (time warp or clock problem)

So I got a shell in the intermediate build container and found:

[root@d5d472cce578 /]# date
Thu 01 Jan 1970 03:00:00 AM MSK

While in the toolbox, things are fine:

$ docker run -it pi-builder-arm-toolbox
[root@c93584cae3b0 /]# date
Wed Dec 30 04:52:35 PM UTC 2020

I'll try to investigate further, but thought I'd let you know at this point. :)

Thank you for this bright idea on how to automate building these images. I'm currently evaluation different ways and this looks very promising. 👍

docker-extract

JSON Parse issue

But the thing is I haven't been able to figure out what is different about these two images as compared to the other ones.

I'll keep looking into it :)

Possible solutions?

error: failed to initialize alpm library

I cloned the repo, dropped the example config.mk in, and get the below result. I originally hit this issue while using pikvm to build a pi zero image.

And I have attempted to build this on several computers all running Ubuntu 20.04.

ubuntu@UbuntuDev:~/pi-builder$ make rpi3
===== Ensuring toolbox image =====
docker build
--rm
--tag pi-builder-arm-toolbox

--file toolbox/Dockerfile.root
toolbox
Sending build context to Docker daemon 15.36kB
Step 1/6 : FROM archlinux/archlinux:base-devel
---> bb1d16ed17e2
Step 2/6 : RUN mkdir -p /etc/pacman.d/hooks && ln -s /dev/null /etc/pacman.d /hooks/30-systemd-tmpfiles.hook
---> Using cache
---> 1ef62ba9ddd3
Step 3/6 : RUN pacman --noconfirm -Syu && pacman --needed --noconfirm -S a rp-scan python parted dosfstools r sync && (pacman --noconfirm -Sc || true) && rm -rf /var/cache/pacman/pkg/ *
---> Running in 9446a1e948e0
error: failed to initialize alpm library
(could not find or read directory: /var/lib/pacman/)
The command '/bin/sh -c pacman --noconfirm -Syu && pacman --needed --noc onfirm -S arp-scan python parted d osfstools rsync && (pacman --noconfirm -Sc || true) && rm -r f /var/cache/pacman/pkg/*' returned a non-zero code: 255
make: *** [Makefile:197: toolbox] Error 255

Running on Ubuntu 20.04 with the latest docker-ce.

"404 Not Found" when trying to build - ArchLinux naming scheme changed

I'm trying to build PiKVM for my Raspberry Pi 3.
But the following error shows up:

...
===== Binfmt arm is ready =====
===== Ensuring base rootfs =====
if [ ! -e ./.cache/base-rootfs-rpi3.tar.gz ]; then \
        mkdir -p ./.cache \
        && echo "Signature: 8a477f597d28d172789f06886806bc55" > "./.cache/CACHEDIR.TAG" \
        && curl -L -f http://de3.mirror.archlinuxarm.org/os/ArchLinuxARM-rpi-2-latest.tar.gz -z ./.cache/base-rootfs-rpi3.tar.gz -o ./.cache/base-rootfs-rpi3.tar.gz \
; fi
Warning: Illegal date format for -z, --time-cond (and not a file name).
Warning: Disabling time condition. See curl_getdate(3) for valid date syntax.
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 404 Not Found
make[1]: *** [Makefile:277: _rpi_base_rootfs_tgz] Error 22

The URL to the Arch image (in this case http://de3.mirror.archlinuxarm.org/os/ArchLinuxARM-rpi-2-latest.tar.gz) is not available anymore. I think they may have changed their naming-scheme.

If i'm correct then http://de3.mirror.archlinuxarm.org/os/ArchLinuxARM-rpi-aarch64-latest.tar.gz should be the right image for the Raspi3.

I believe that this just needs a simple fix in the Makefile.

error: target not found: openssl-1.0

I'm trying to build PiKVM for my Raspberry Pi 3.
But the following error shows up:

Step 2/7 : RUN mkdir -p /etc/pacman.d/hooks 	&& ln -s /dev/null /etc/pacman.d/hooks/30-systemd-tmpfiles.hook
 ---> Running in 4fbcda98e049
Removing intermediate container 4fbcda98e049
 ---> b0d8d04fe8f2
Step 3/7 : RUN pacman --noconfirm --ask=4 -Syy 	&& pacman --needed --noconfirm --ask=4 -S 		glibc 		openssl 		openssl-1.0 		openssl-1.1 	&& pacman --needed --noconfirm --ask=4 -S 		pacman 	&& pacman-db-upgrade 	&& pacman-key --init 	&& pacman --needed --noconfirm --ask=4 -S 		p11-kit 		archlinux-keyring 		ca-certificates 		ca-certificates-mozilla 		ca-certificates-utils 	&& pacman --noconfirm --ask=4 -Syu 	&& pacman --needed --noconfirm --ask=4 -S 		arp-scan 		python 		parted 		dosfstools 		rsync 	&& rm -rf /var/cache/pacman/pkg/*
 ---> Running in 6be9e605b91a
:: Synchronizing package databases...
 core downloading...
 extra downloading...
 community downloading...
warning: glibc-2.36-6 is up to date -- skipping
warning: openssl-3.0.7-4 is up to date -- skipping
error: target not found: openssl-1.0

I've removed manually openssl-1.0 and everything rest was correct

I believe that this just needs a simple fix in the os stage dockerfile .

Dirmngr issue with systemd-resolved

Docker does not work perfectly with systemd-resolved and dirmngr seems to suffer from this.

PR #15 submitted to fix this issue.

[root@62cd68d26d6f pacman.d]# pacman-key --keyserver hkps://hkps.pool.sks-keyservers.net -r 912C773ABBD1B584
gpg: keyserver receive failed: Server indicated a failure
==> ERROR: Remote key not fetched correctly from keyserver.
[root@62cd68d26d6f pacman.d]# echo standard-resolver >> /etc/gnupg/dirmngr.conf
[root@62cd68d26d6f pacman.d]# killall dirmngr
[root@62cd68d26d6f pacman.d]# pacman-key --keyserver hkps://hkps.pool.sks-keyservers.net -r 912C773ABBD1B584
gpg: key 912C773ABBD1B584: "Maxim Devaev <[email protected]>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
[root@62cd68d26d6f pacman.d]#

Try to use it pi-builder with raspberry pi os

I'am customizing pi-builder to build raspberry pi os instead of arch.
Currently I'am doing this in the makefile:

_rpi_base_rootfs_tgz: # 3rd step in make - downloads base root fs as tgz file not an image
	$(call say,"Ensuring base rootfs")
	if [ ! -e $(_RPI_BASE_ROOTFS_TGZ) ]; then \
		mkdir -p $(_CACHE_DIR) \
		&& cd $(_CACHE_DIR) \
		&& curl -L -f $(_RPI_ROOTFS_URL) -o $(_PIOS_IMAGE_NAME) \
		&& unxz -f $(_PIOS_IMAGE_NAME) \
		&& 7z x -aoa pi_image.img 1.img \
		&& mkdir -p root_fs \
		&& cd root_fs \
		&& 7z x -aoa ../1.img \
		&& rm -rf lib \
		&& ln -s usr/lib lib \
		&& cd ../.. \
		&& chmod -R +x .cache/root_fs/usr/bin/bash \
		&& cd .cache/root_fs \
		&& tar -czf ../.$(_RPI_BASE_ROOTFS_TGZ) * \
		&& echo "Signature: 8a477f597d28d172789f06886806bc55" > "../.$(_CACHE_DIR)/CACHEDIR.TAG" \
	; fi

But during OS building I hit a wall with:

qemu-arm-static: /lib/ld-linux-armhf.so.3: Invalid ELF image for this architecture

Dockerfile inside .build looks like this:

FROM scratch

ADD "common-pi-builder-arm-base-rootfs-rpi4.tgz" /
COPY "qemu-arm-static-orig" "/usr/bin/qemu-arm-static-orig"
COPY "qemu-arm-static" "/usr/bin/qemu-arm-static"
#RUN mkdir -p /etc/pacman.d/hooks \
#	&& ln -s /dev/null /etc/pacman.d/hooks/30-systemd-tmpfiles.hook

ARG BOARD
ENV BOARD $BOARD

ARG ARCH
ENV ARCH $ARCH

ARG LOCALE
ENV LOCALE $LOCALE

ARG TIMEZONE
ENV TIMEZONE $TIMEZONE

ARG REPO_URL
ENV REPO_URL $REPO_URL

ARG PIKVM_REPO_URL
ENV PIKVM_REPO_URL $PIKVM_REPO_URL

ARG PIKVM_REPO_KEY
ENV PIKVM_REPO_KEY $PIKVM_REPO_KEY
# https://lists.gnu.org/archive/html/qemu-devel/2017-10/msg03681.html
SHELL ["/usr/bin/bash"]


RUN echo "foobar---------------------"

The error messages appears when I try to run an echo in the last line.

[Essay of sorts] Commentary on Pi Image Generation

Hey so for some time I have been working on raspberry pi image generation.

Probably what I end up doing with pi-builder looks more like a fork or use of your design pattern.

I guess that one of the things that I'm looking for here is criticism of my proposed architecture because obviously you have spent a lot of time on this as well.

So far I have been using packer-builder-arm. Everything that would happen inside the darker container here happens inside a chroot over there.

The chroot doesn't allow for some things, like systemctl enable, and if you're using ubuntu, apt upgrade, If that upgrade will trigger kernel changes.

With pi-builder, what I found is that there's probably too many options although I think that this is the closest to ideal solution that I found by a wide margin. So what I'm going to do is I am going to take all of the Dockerfile.part files and make them into one Dockerfile. There are a number of reasons for this but probably the biggest one is that this will be fed into a code generator and spent out the other side and I think it's going to end up being a lot easier to manage as a single Dockerfile, single Makefile and a folder of dependencies.

I could be wrong and that's part of why am making this issue actually. I think that this is a really badly needed product. Currently it's much too hard to create images for embedded devices. It's that difficulty that actually drove me to try and bundle this into starport, our blockchain scaffolding tool.

I might break this out into an independent repository as kind of like a framework similar to pi builder but with fewer options and only in a 64-bit mode.

Ideally, I'd like my tool to be totally separate from the app that runs on top of it.

I just wanted to say thanks so much for your work, there are a lot of things in here that I had been having a difficult time figuring out and I'm just delighted that you made this open source and allowed me to learn from it. I'll link you to what I come up with later tonight. So far it's just been some basic customization and toying around but I think that in the next eight hours or so I'll actually have my setup in a pretty decent spot, owing in large part to your work.

Cheers!
-Jacob

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.