facebookincubator / go2chef Goto Github PK
View Code? Open in Web Editor NEWA Golang tool to bootstrap a system from zero so that it's able to run Chef to be managed
License: Apache License 2.0
go2chef's People
Contributors
Stargazers
Watchers
Forkers
oko ryanmoon timrots kcbraunschweig gbatye natewalck steelcowboy sanketmore ifel chenry3 davide125 stanchan ekmixon dafyddcrosby williamtheaker diegodisant nstrauss isabella232 yee-unit chermehdi ayoub-aknoun-edu aleksandrmeta skywalk7 bilyyygo2chef's Issues
[Issue] `go2chef.step.install.linux.dnf` doesn't treat Version consistently
I am presently using go2chef.step.install.linux.dnf to install a package and version and relying on dnf to figure out the repo. But I am seeing inconsistency with how go2chef is honoring the "version" value.
go2chef doesn't honor version when the package is not installed at all.
go2chef does honor version when the package is already installed
go2chef doesn't downgrade the package if the version is installed
Repro:
########## Go2chef config ##########
[centos@ip-172-16-1-244 ~]$ cat /etc/go2chef.config-broken.json
{
"steps": [
{
"type": "go2chef.step.install.linux.dnf",
"name": "install chef-13",
"version": "13.10.0-1.el7",
"package_name": "chef"
}
]
}
########## Remove installed chef ##########
[centos@ip-172-16-1-244 ~]$ sudo dnf remove chef -y
Dependencies resolved.
================================================================================================================================================================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
================================================================================================================================================================================================================================================================================================================================================================================================================================
Removing:
chef x86_64 14.15.6-1.el7 @fb-site-packages 108 M
Removing dependent packages:
fb-chef-solo-tools noarch 20220503-100441 @any-noarch 131 k
Transaction Summary
================================================================================================================================================================================================================================================================================================================================================================================================================================
Remove 2 Packages
Freed space: 108 M
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Erasing : fb-chef-solo-tools-20220503-100441.noarch 1/2
Erasing : chef-14.15.6-1.el7.x86_64 2/2
Running scriptlet: chef-14.15.6-1.el7.x86_64 2/2
Verifying : chef-14.15.6-1.el7.x86_64 1/2
Verifying : fb-chef-solo-tools-20220503-100441.noarch 2/2
Removed:
chef-14.15.6-1.el7.x86_64 fb-chef-solo-tools-20220503-100441.noarch
Complete!
[centos@ip-172-16-1-244 ~]$ rpm -qa | grep ^chef-
[centos@ip-172-16-1-244 ~]$
########## Run go2chef - doesn't honour version ##########
[centos@ip-172-16-1-244 ~]$
[centos@ip-172-16-1-244 ~]$ sudo /usr/local/bin/go2chef --local-config /etc/go2chef.config-broken.json
___ _ __
__ _ ___|_ )__| |_ ___ / _|
/ _` / _ \/ // _| ' \/ -_) _|
\__, \___/___\__|_||_\___|_|
|___/
GO2CHEF 2022/06/08 21:27:34 loading config from source go2chef.config_source.local
GO2CHEF 2022/06/08 21:27:34 EVENT: LOGGING_INITIALIZED in go2chef.cli -
GO2CHEF 2022/06/08 21:27:34 EVENT: STEP_0_START go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli -
Last metadata expiration check: 0:42:16 ago on Wed 08 Jun 2022 08:45:19 PM UTC.
Dependencies resolved.
================================================================================================================================================================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
================================================================================================================================================================================================================================================================================================================================================================================================================================
Installing:
chef x86_64 14.15.6-1.el7 fb-site-packages 28 M
Transaction Summary
================================================================================================================================================================================================================================================================================================================================================================================================================================
Install 1 Package
Total download size: 28 M
Installed size: 108 M
Downloading Packages:
chef-14.15.6-1.el7.x86_64.rpm 30 MB/s | 28 MB 00:00
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 30 MB/s | 28 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Installing : chef-14.15.6-1.el7.x86_64 1/1
Running scriptlet: chef-14.15.6-1.el7.x86_64 1/1
Thank you for installing Chef Infra Client! For help getting started visit https://learn.chef.io
Verifying : chef-14.15.6-1.el7.x86_64 1/1
Installed:
chef-14.15.6-1.el7.x86_64
Complete!
GO2CHEF 2022/06/08 21:27:43 EVENT: STEP_0_COMPLETE go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli - completed successfully in 8 second(s)
GO2CHEF 2022/06/08 21:27:43 EVENT: ALL_STEPS_COMPLETE in go2chef.cli - 1 step(s) completed successfully in 8 second(s)
2022/06/08 21:27:43 temp dirs cleanup completed
[centos@ip-172-16-1-244 ~]$ rpm -qa | grep ^chef-
chef-14.15.6-1.el7.x86_64
########## Install the desired chef version ##########
[centos@ip-172-16-1-244 ~]$ sudo dnf install chef-13.10.0-1.el7 -y
Last metadata expiration check: 0:44:26 ago on Wed 08 Jun 2022 08:45:19 PM UTC.
Dependencies resolved.
================================================================================================================================================================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
================================================================================================================================================================================================================================================================================================================================================================================================================================
Downgrading:
chef x86_64 13.10.0-1.el7 cloud-common-64 49 M
Transaction Summary
================================================================================================================================================================================================================================================================================================================================================================================================================================
Downgrade 1 Package
Total download size: 49 M
Downloading Packages:
chef-13.10.0-1.el7.x86_64.rpm 31 MB/s | 49 MB 00:01
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 31 MB/s | 49 MB 00:01
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Downgrading : chef-13.10.0-1.el7.x86_64 1/2
Running scriptlet: chef-13.10.0-1.el7.x86_64 1/2
Thank you for installing Chef!
Cleanup : chef-14.15.6-1.el7.x86_64 2/2
Running scriptlet: chef-14.15.6-1.el7.x86_64 2/2
Verifying : chef-13.10.0-1.el7.x86_64 1/2
Verifying : chef-14.15.6-1.el7.x86_64 2/2
Downgraded:
chef-13.10.0-1.el7.x86_64
Complete!
########## Run go2chef again - this time it parses the right version and honors it! ##########
[centos@ip-172-16-1-244 ~]$ sudo /usr/local/bin/go2chef --local-config /etc/go2chef.config-broken.json
___ _ __
__ _ ___|_ )__| |_ ___ / _|
/ _` / _ \/ // _| ' \/ -_) _|
\__, \___/___\__|_||_\___|_|
|___/
GO2CHEF 2022/06/08 21:30:30 loading config from source go2chef.config_source.local
GO2CHEF 2022/06/08 21:30:30 EVENT: LOGGING_INITIALIZED in go2chef.cli -
GO2CHEF 2022/06/08 21:30:30 EVENT: STEP_0_START go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli -
GO2CHEF 2022/06/08 21:30:30 INFO: third-party-source/go/github.com/facebookincubator/go2chef/plugin/step/install/linux/dnf/dnf.go:204::Package is already installed: chef-13.10.0-1.el7.x86_64, requested ^chef-13.10.0-1.el7.*
GO2CHEF 2022/06/08 21:30:30 INFO: third-party-source/go/github.com/facebookincubator/go2chef/plugin/step/install/linux/dnf/dnf.go:132::chef specified is already installed, not reinstalling
GO2CHEF 2022/06/08 21:30:30 EVENT: STEP_0_COMPLETE go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli - completed successfully in 0 second(s)
GO2CHEF 2022/06/08 21:30:30 EVENT: ALL_STEPS_COMPLETE in go2chef.cli - 1 step(s) completed successfully in 0 second(s)
2022/06/08 21:30:30 temp dirs cleanup completed
[Feature Request] Support specifying client key/cert when installing package using `go2chef.step.install.linux.dnf`
There is no way to provide sslclientcert/sslclientkey inside go2chef.step.install.linux.dnf - hence you can't authenticate with a yum endpoint while downloading a package.
GO2CHEF 2022/06/08 19:40:32 EVENT: STEP_3_FAILURE go2chef.step.install.linux.dnf:'install chef-13' in go2chef.cli - Get "https://<redacted_uri>/yum/centos/8.x/site-packages/x86_64/Packages/chef-13.10.0-1.el7.x86_64.rpm": remote error: tls: certificate required
Allow downgrading packages
Because Go2Chef does not allow downgrading, it alone does not help in a scenario where it might install a stable version of Chef and you'd want it to recover if a later version of Chef caused an issue.
For instance, Chef 15.6 works with our current code and Chef 16.2 doesn't. We can't use Go2Chef to get back to Chef 15.6 without first uninstalling Chef 16.2. There should be some kind of option, such as "allow_downgrade", in the config that enables this kind of functionality.
[Feature Request] `go2chef.step.install.linux.dnf` should allow specifying the repo path as source
The typical usage of installing a package in dnf is like the following
{
"steps": [
{
"type": "go2chef.step.install.linux.dnf",
"name": "install chef",
"version": "15.2.20-1.el7.x86_64",
"source": {
"type": "go2chef.source.http",
"url": "https://packages.chef.io/files/stable/chef/15.2.20/el/8/chef-15.2.20-1.el7.x86_64.rpm"
}
}
]
}
This is awesome, because it doesn't depend on any repo config on the const and make the bootstrap truly independent.
This works great for packages here the version doesn't change frequently - so the config is reasonably static.
However it might not be ideal in cases where the versions are changing quickly because of CI/CD. In that case it might be desirable to specify the path of the repo instead. Something like this:
{
"steps": [
{
"type": "go2chef.step.install.linux.dnf",
"name": "install chef-solo-tools",
"version": "15.2.20-1.el7.x86_64",
"source": {
"type": "go2chef.source.repo",
"baseurl": "https://packages.chef.io/centos/8.x/x86_64/"
}
}
]
}
[feature request] command step allow output redirection
Because of how commands are executed you can't do output redirection but you may want to. Request is to provide options to the command step to allow sending stdout and/or stderr to /dev/null (or I guess a file if someone wants?)
PackageName option to install step ineffective
If using the PackageName option to override the package to install from the default ("chef"), log messages will indicate the specified package is being installed but in reality it still tries to install a package called "chef" anyway. The option is not being respected. (tested on a host using DNF)
Support caching for HTTP source
Add caching support for the HTTP source. This allows us to:
- Avoid redownloading potentially very large files repeatedly on failures
- Preserve bootstrap files locally for re-bootstrapping in limited-access scenarios (i.e. roaming client machines)
[Issue] Add Makefile entry for Apple Silicon
It would be nice to have an easy Makefile entry for M1 and newer Macs.
I'm happy to do this and test it. Should we call it darwin-apple-silicon or darwin-arm or darwin-arm64?
Support `bundle.rb` as an entrypoint for `go2chef.step.bundle`
MSI Exit Codes that are non-zero
Right now we hunt for the 0 exit code by simply checking if err is non-nil. If you install an MSI that is already installed it will throw 1603 which is valid... kinda sorta.
If this should be idempotent we should be able to handle either known-"good" errors or let the user of the application specify the exit codes they want to accept.
Support Git repositories as a Source plugin
Git repositories are a common small-scale mechanism for distributing configs, etc. Add a Source plugin to support this in go2chef.
Clarify requirements
Trying to build this on Debian Stable, I get:
$ make
go test ./...
# github.com/facebookincubator/go2chef/plugin/step/install/darwin/pkg
plugin/step/install/darwin/pkg/pkg.go:127:61: exit.ExitCode undefined (type *exec.ExitError has no field or method ExitCode)
note: module requires Go 1.12# github.com/facebookincubator/go2chef/plugin/step/install/linux/apt
plugin/step/install/linux/apt/apt.go:111:75: err.(*exec.ExitError).ExitCode undefined (type *exec.ExitError has no field or method ExitCode)
note: module requires Go 1.12ok github.com/facebookincubator/go2chef (cached)
ok github.com/facebookincubator/go2chef/plugin/config/embed (cached)
ok github.com/facebookincubator/go2chef/plugin/config/http (cached)
ok github.com/facebookincubator/go2chef/plugin/config/local 0.002s
ok github.com/facebookincubator/go2chef/plugin/lib/certs (cached)
ok github.com/facebookincubator/go2chef/plugin/source/http 0.039s
ok github.com/facebookincubator/go2chef/plugin/step/sanitycheck (cached)
ok github.com/facebookincubator/go2chef/plugin/step/winsanitycheck (cached)
ok github.com/facebookincubator/go2chef/util/plugconf (cached)
make: *** [Makefile:7: test] Error 2
$ go version
go version go1.11.6 linux/amd64
Either fix the module to work with go 1.11, or update the README to clarify the dependency on go 1.12
Support aws sources defaulting to derived local region
AWS-specific sources will often be used from an aws instance with access to the aws metadata service. This is already used by the sdk to provide credentials if they aren't in the config. However the sdk doesn't provide a way to automatically derive the current region before creating a session even though the metadata service will tell you where you are.
Instead, when using a source like s3 or secretsmanager, if region isn't specified, manually query the metadata to find our current region and use it if available. To do this, basically:
curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone
Then strip off the last character and that will be your region.
Build failure on 32 bit architectures
_build/src/github.com/facebookincubator/go2chef/multilogger.go:26:3: constant 9223372036854775807 overflows int
See https://bugzilla.redhat.com/show_bug.cgi?id=1912666 for details. This came up while working on packaging go2chef for Fedora in https://bugzilla.redhat.com/show_bug.cgi?id=1912668
[Feature request] Support alternate install filenames on Darwin
Package names are currently hardcoded, which requires patching go2chef to use it with a free/libre Chef distribution like Cinc.
Allowing passing arbitrary filenames would make this tool more broadly useful if someone wants to use it as go2puppet or such.
Support checksum validation for HTTP source
Support checksum validation for go2chef.source.http downloads.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.