GooGet (Googet's Obviously Only a Goofy Experimental Title) is a modular package repository solution primarily designed for Windows.
GooGet is shipped with the official Google Cloud Platform Windows images and is used to maintain the guest environment.
To build and package googet run
go run goopack/goopack.go googet.goospec
This will result in googet.exe and googet.x86_64.VERSION.goo which can be installed on a
machine with the googet install
command (assuming googet is already
installed).
To install on a fresh machine copy both googet.exe and the googet package over and run:
googet -root 'c:/ProgramData/GooGet' install googet googet.x86_64.VERSION.goo
GooGet has the ability to use a conf file to change a few of the default settings.
Place a file named googet.conf in the googet root, which by default is
C:\ProgramData\GooGet
and configurable by the -root
flag.
proxyserver: http://address_to_proxy:port
archs: [noarch, x86_64]
cachelife: 10m
GooGet has the ability to use a repo file to change some repo specific settings. The file is named your-repo-name.repo by default. It can be used to set the source URL and indicate that the client should pass authorization headers in requests to the source URL.
URL: https://foo.com/googet/bar
useoauth: true
Googet supports using Google Cloud Storage as its server.
set GOOREPO=%TEMP%\googet-repo
set REPONAME=my_repo
mkdir %GOOREPO%\%REPONAME%
mkdir %GOOREPO%\packages
go run goopack/goopack.go googet.goospec
copy *.goo %GOOREPO%\packages
go run server\gooserve.go -root %GOOREPO% -save_index %GOOREPO%\%REPONAME%\index
gsutil mb --project my-project my-googet-server
gsutil rsync -r %GOOREPO% gs://my-googet-server
googet addrepo gcs gs://my-googet-server
rem This command should print 'gcs: gs://my-googet-server'
googet listrepos
rem This command should list the googet package and any other packages in your repo
googet available -sources gs://my-googet-server/
Note that you must regenerate the index and re-upload it to your bucket each time you add or change packges.
googet's People
Forkers
davegalos kleopatra999 beepmill adjackura cjgenevi itsmattl tomlanyon adamcarheden mbernhardt6 bamsammich bagarber assafrahav mdlglobal-atlassian-net gaohannk neotim tomoe bkatyl wutijat artithw igorpeshansky davgit manninglucas nanditajaiswal mjoliver isabella232 cricket-org minotrez karnvadaliya vigneshchandra atetubou asx lkwiatek jjerger jm2 ghas-results gitgerby nguyen-phillip sbrito85 graham-m-dunn mcsaucygooget's Issues
Googet should interpret the version string "1" as "1.0.0" instead of "0.0.1"
ParseVersion and fixVer in goospec.go have the unexpected behavior of taking version strings with less than 3 components and converting them into 3-component versions by prepending leading 0s rather than appending 0s. This means that "1" becomes "0.0.1" and "1.2" becomes "0.1.2" which results in comparisons like
- "1" < "1.0" < "1.0.0"
- "2" < "1.1"
- "3" < "1.0.0"
- "1.5" < "1.2.0"
This is contrary to the behavior of semver.ParseTolerant and common interpretations of version string comparisons.
Googet install will fail if driver is already installed
Observed behavior:
Sideload latest vioscsi driver with DISM
VERBOSE: Successfully added driver C:\builder\components\drivers\win81\vioscsi.inf
Install Googet package with vioscsi driver
Running C:\ProgramData\GooGet\googet.exe with arguments -root C:\ProgramData\GooGet -noconfirm install google-compute-engine-driver-vioscsi.
googet_install.go:165: Error installing google-compute-engine-driver-vioscsi.x86_64.16.1.2@11: command exited with error code 1
Expected behavior:
Googet will not error if driver is already installed
Googet on x86 displays AddressWidth exception
Observed Behavior
Built Googet using build.sh on Debian 9, set environment variable GOARCH=386
Saved executable to GCS bucket and DL to local Windows 7 Enterprise x86
.\googet -root 'C:\ProgramData\Googet' installed
ERROR: Logging before logger.Init.
ERROR: 2019/03/07 05:45:37.165601 system_windows.go:179: Error getting AddressWidth: Exception occurred. (Invalid query
)
No packages installed.
Expected Behavior
No error messages displayed.
Message is displayed for all googet commands
Caches get mixed up if two repo URLs use the same final path component.
Here's the culprit:
Line 151 in b08ef8d
Example:
googet addrepo first http://myserver.com/googet/myrepo
googet addrepo second gs://my-bucket/googet/myrepo
This will result in googet using C:\ProgramData\googet\cache\myrepo.rs as the cache for both repos. googet available
will list both repos but list the set of packages from whichever repo was read most recently for both of them.
The client has a unique name for each repository ('first' and 'second' in the example above). It should use that instead of the final path component.
Note that the fix is easy and I'm glad to submit a PR. However, the rollout could be tricky due to the potential of mixed-up caches. Doing something like giving the cache file a new extension (.rs2 or something) seems like the best way to avoid that. Please let me know if I should:
a) Please submit a PR!
b) No PR necessary, we'll fix it.
c) Too risky -- Let's leave it as-is and maybe put a warning in the docs about it.
Running goopack on Windows requires backslashes in package files
Running goopack
on Windows does not recognize forward slashes in the files
clause of the goospec
. Replacing forward slashes with backslashes makes it work. This is due to the glob
function returning paths with backslashes (it being Windows).
Repro case:
PS C:\src> mkdir a; mkdir b; $null > a/x; $null > b/y
Directory: C:\src
Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 2/2/2021 11:45 PM a
d----- 2/2/2021 11:45 PM b
PS C:\src> @"
>> {
>> "name": "goopack-test",
>> "version": "0.0.0",
>> "arch": "noarch",
>> "files": {
>> "a/x": "<ProgramFiles>/GoopackTest/x",
>> "b/y": "<ProgramFiles>/GoopackTest/y"
>> },
>> "sources": [{
>> "include": [
>> "a/x",
>> "b/y"
>> ]
>> }]
>> }
>> "@ | Out-File -encoding ASCII goopack-test.goospec
PS C:\src> goopack goopack-test.goospec
2021/02/02 23:59:18 requested files [a/x b/y] not in package
PS C:\src>
Changing the files to use backslashes makes it work:
PS C:\src> @"
>> {
>> "name": "goopack-test",
>> "version": "0.0.0",
>> "arch": "noarch",
>> "files": {
>> "a\\x": "<ProgramFiles>/GoopackTest/x",
>> "b\\y": "<ProgramFiles>/GoopackTest/y"
>> },
>> "sources": [{
>> "include": [
>> "a/x",
>> "b/y"
>> ]
>> }]
>> }
>> "@ | Out-File -encoding ASCII goopack-test.goospec
PS C:\src> goopack goopack-test.goospec
PS C:\src> dir *.goo
Directory: C:\src
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 2/3/2021 12:03 AM 319 goopack-test.noarch.0.0.0.goo
PS C:\src>
Another minor issue is that there is no indication why the forward slashes produced an error. It would be great to have a debug mode where the globbed file set is printed out.
/cc @adjackura
Windows - Installation scripts not successfully executing
I am having an issue with googet on a single server where it is not executing the installation script defined in the pkgspec file. As far as I can tell, it's not successfully calling powershell. If the install script is executed manually, it runs successfully.
I can repeat the issue with the following command (installation package does not matter, experience is the same regardless of package):
googet.exe -verbose -root c:\temp\jcl\Googet install -sources https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable google-osconfig-agent
It will extract and copy all of the files and then log:
INFO : 2021/06/29 15:37:12.722922 system_windows.go:76: Running install command: "packaging/googet/install.ps1"
ERROR: 2021/06/29 15:37:12.729706 googet_install.go:165: Error installing google-osconfig-agent.x86_64.20210607.00.0+win@1: open NUL: The system cannot find the file specified.
Again, powershell works by hand, the installation script is in the cache folder where it was extracted.
I pulled down the latest googet sources and liberally sprinkled some logging statements around the install.go and goolib.go files to see if paths, filenames, etc were being passed successfully. Here's a truncated excerpt from logging statements output from a modified googet version:
INFO : 2021/06/30 12:26:32.621186 system_windows.go:76: Running install command: "packaging/googet/install.ps1"
INFO : 2021/06/30 12:26:32.640718 goolib.go:59: JCL - Cleaned File Path: "c:\\temp\\jcl\\Googet\\cache\\google-osconfig-agent.x86_64.20210607.00.0+win@1\\packaging\\googet\\install.ps1"
INFO : 2021/06/30 12:26:32.656348 goolib.go:61: JCL - scriptInterpreter: "powershell"
INFO : 2021/06/30 12:26:32.668047 goolib.go:70: JCL - Powershell args: ["-ExecutionPolicy" "Bypass" "-NonInteractive" "-NoProfile" "-Command" "c:\\temp\\jcl\\Googet\\cache\\google-osconfig-agent.x86_64.20210607.00.0+win@1\\packaging\\googet\\install.ps1"]
INFO : 2021/06/30 12:26:32.682688 goolib.go:91: JCL - run c: "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe -ExecutionPolicy Bypass -NonInteractive -NoProfile -Command c:\\temp\\jcl\\Googet\\cache\\google-osconfig-agent.x86_64.20210607.00.0+win@1\\packaging\\googet\\install.ps1"
INFO : 2021/06/30 12:26:32.698977 goolib.go:94: JCL - run ok: false
INFO : 2021/06/30 12:26:32.702705 goolib.go:95: JCL - run err: "open NUL: The system cannot find the file specified."
INFO : 2021/06/30 12:26:33.529111 install.go:157: JCL - installPkg: dst: "c:\\temp\\jcl\\Googet\\cache\\[email protected]", rs.PackageSpec: "google-osconfig-agent.x86_64.20210607.00.0+win@1", dbOnly: false
ERROR: 2021/06/30 12:26:33.534309 googet_install.go:165: Error installing google-osconfig-agent.x86_64.20210607.00.0+win@1: open NUL: The system cannot find the file specified.
I strongly suspect that the issue is within the OS and/or its configuration but it only appears that googet and other google services installed on the OS are impacted by this (e.g., OSConfig Agent service is not able to run powershell to determine installed updates). It appears to successfully hand execution to os/exec func Run which then generates the error. I am somewhat at a loss as to how to troubleshoot this further. Any direction you can give would be greatly appreciated.
Architecture: x86_64
KernelRelease: 10.0.17763.1999
KernelVersion: 10.0.17763.1999 (WinBuild.160101.0800)
LongName: Microsoft Windows Server 2019 Datacenter
ShortName: windows
Version: 10.0.17763
$PSVersionTable
Name Value
---- -----
PSVersion 5.1.17763.1971
PSEdition Desktop
PSCompatibleVersions {1.0, 2.0, 3.0, 4.0...}
BuildVersion 10.0.17763.1971
CLRVersion 4.0.30319.42000
WSManStackVersion 3.0
PSRemotingProtocolVersion 2.3
SerializationVersion 1.1.0.1
Upgrading in-use files breaks if %TEMP% is not on C:
When updating a package, the client.RemoveOrRename()
function first attempts to delete a file, and if that fails (e.g. if it is in use) it instead moves it to a temp file returned by ioutil.TempFile()
.
This fails if the system has been configured to point %TEMP%
/ %TMP%
to a different drive to the source file.
For example:
googet_install.go:165: Error installing googet.x86_64.1.0.0@1234: rename \\?\C:\ProgramData\GooGet\googet.exe \\?\D:\TEMP\921630347: The system cannot move the file to a different disk drive.
In this case, it is trying to move C:\ProgramData\GooGet\googet.exe
to D:\TEMP\921630347
which is unsupported by os.Rename()
.
This is a known issue, closed as wontfix:
golang/go#13766
Speculation as to how Golang is meant to succeed as a general purpose programming language if it can't even move files across drives in Windows is left as an exercise to the reader.
My suggestion for how to fix is to instead rename the file in-place (in the same directory), and add an entry to the following MULTI_SZ
registry value to instruct Windows to delete the file on next reboot:
HKLM\System\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations
There's another alternative that involves directly calling MoveFileEx()
with the MOVEFILE_DELAY_UNTIL_REBOOT
flag, which avoids having to manipulate a MULTI_SZ
structure in the registry.
It goes roughly like this:
(1) Attempt to delete file, which fails
(2) Rename file in-place, to avoid renaming across drives
(3) Rename file to tmp dir with MoveFileEx()
syscall with the MOVEFILE_DELAY_UNTIL_REBOOT
flag set
googet installation failed without internet access
We are using Daisy workflow control with Google cloud build to create Windows images in Google cloud.
After we removed Internet access from our project, cloud build failed with error as blow:
Installing GooGet and GooGet packages.
ERROR: 2023/11/01 22:49:05.756131 client.go:104: error reading repo "https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable": Get "https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable/index.gz": net/http: TLS handshake timeout
ERROR: 2023/11/01 22:49:05.768357 googet_install.go:125: Can't resolve version for package "googet": no versions of package googet found in any repo
GooGet install failed from "D:\ProgramData\GooGet\googet.exe", retrying...
ERROR: 2023/11/01 22:49:36.223287 client.go:104: error reading repo "https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable": Get "https://packages.cloud.google.com/yuck/repos/google-compute-engine-stable/index.gz": net/http: TLS handshake timeout
ERROR: 2023/11/01 22:49:36.242315 googet_install.go:125: Can't resolve version for package "googet": no versions of package googet found in any repo
'C:\ProgramData\GooGet\googet.exe' is not recognized as an internal or external command,
operable program or batch file.
We would like to be able to extract googet packages into local filesystem so that we can install googet offline.
Add --extract to googet download params
Allows the end user to download and extract the contents of a Googet package to a local filesystem. Would also enable offline installations.
googet download ignores return codes
googet's download library, specifically the packageHTTP function, makes no attempt to validate that it received an expected return code (eg 200) from the server prior to saving the body of the response to disk. This can lead to goo files that contain the body of a failure page rather than the goo binary itself. Although the checksum verification may detect a problem in the resulting file before googet tries to use it, the failure mode of a checksum mismatch obscures the underlying problem with the network connection from the user. Ideally, a bad return code should never get written to disk, a more specific error message should be generated, and a corresponding exit code should be returned.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.