Comments (10)
14 problems on Ubuntu 17 due to bash/mksh/etc. version mismatches :-(
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=17.10
DISTRIB_CODENAME=artful
DISTRIB_DESCRIPTION="Ubuntu 17.10"
'_tmp/web' -> '/home/andy/git/oilshell/oil/web'
alias failed with status 1
array failed with status 1
assoc failed with status 1
blog2 failed with status 1
builtin-io failed with status 1
builtins failed with status 1
builtin-test failed with status 1
glob failed with status 1
sh-options failed with status 1
var-op-other failed with status 1
var-op-strip failed with status 1
var-ref failed with status 1
var-sub failed with status 1
xtrace failed with status 1
*** 14 tests FAILED
from oil.
Labelling this help wanted because maybe someone can write a shell script to make a chroot image?
I can provide more color on this.
from oil.
@Yorwba Let's continue the discussion here.
I said Alpine might be harder and Ubuntu is probably easier. The reason I think that is:
- The tests are already written against Ubuntu
- Alpine is busybox-based while Ubuntu is coreutils-based.
So potentially every test has to be tweaked. But maybe this is not as big a deal as I thought.
Intermediate task (which I may pick up): Provide an alias to run a single spec test in a chroot. Then we can migrate it test-by-test.
You know I think the main issue is: what shells do we test against?
- Binary Ubuntu packages (which we do now essentially)
- Binary Alpine packages
- Compile shells from source on Ubuntu or Alpine
I was thinking #3 because it gives us the most control over what shell version. Although honestly I think "whatever shell version the latest LTS version of Ubuntu uses" is not bad.
I was thinking "the latest version of every shell", but in theory that is more work (though it shouldn't be THAT much work.)
I have to go back and remember how debootstrap works. If I can debootstrap Ubuntu 18.04 from my 16.04 machine, that would be nice. But I think Ubuntu doesn't necessarily do that. In other words, I don't want the version of shells we test against to be tied to "did Andy upgrade his personal machine, or does he have access to an 18.04 machine?" Right now I do not.
Alpine is nice because they provide a small chroot image to download. And it seems to run fine on Ubuntu. So there's less possibility of "hidden" dependencies between guest and host.
from oil.
Thinking aloud here, another reason I have put this is off is because Ubuntu doesn't provide a nice root file system download like Alpine:
https://alpinelinux.org/downloads/
On Ubuntu you have to run debootstrap yourself and do some other nontrivial setup steps, as far as I remember.
I also have to check out what versions of dash/mksh/zsh/busybox Alpine has, if any.
from oil.
It seems like using Alpine's edge branch should keep you pretty close to "the latest version of every shell".
E.g. the repository currently has bash 4.4.19, which was packaged the day after the upstream release. The highest patchlevel would be 4.4.23, but judging by the upload date in GNU's FTP, those patches are barely over a week old.
Looking through the packages for the other shells, the delay never seems to be more than a few weeks.
from oil.
Small update on this:
-
I didn't want every developer to have to run commands as root, which
chroot
requires. So I investigated "rootless containers" withrunc
. I got a "hello world" to work, although it required more wrestling with Go's build system than I liked. -
Along the way I realized that for the spec tests, the only thing you really care about is having dash/bash/mksh/zsh in
$PATH
, as well as coreutils/busybox in$PATH
. So actually we should be able to have reproducible test results on a reasonable Linux machine without any kind of chroot or container. The test driver already appends to$PATH
, so we could make it so it resets$PATH
completely.
The downside is that this method might require everyone to build 4 or 5 shells from scratch. Or maybe I can build some "portable" binaries. I think I read about a blog post about that, which basically amounted to using an old libc (?)
https://www.cyphar.com/blog/post/20160627-rootless-containers-with-runc
from oil.
@granttrec found out that while the shell binaries are now isolated, the environment isn't! The unicode tests depend on certain environment variables.
I honestly forget all the details, but I know that both libc
and I think bash itself respect a bunch of locale environment variables.
We could probably just set the environment variables in test/spec-runner.sh
itself ?? Not sure.
from oil.
@andychu setting the environment variables will work, however we will have to check for a correct locale on the system first, then set LC_ALL
to the locale of choice, I assume en_US.UTF-8
; en_CA.UTF-8
worked for me.
from oil.
OK, I'll leave it up to you if you want to submit a patch. If someone else runs into then it may become higher priority. If you do, please mention the tests that depend on the locale in the comments, and point to any relevant libc or bash documentation.
from oil.
Done with "toil" on Travis CI!
https://github.com/oilshell/oil/tree/master/services
from oil.
Related Issues (20)
- command -v "$emptyvar" returns zero HOT 3
- Oils 0.23.0 TODO
- Missing "Str=>lower()"
- Abort with += on missing dict key
- intermittent crash running amd-test script -- reproducible in dbg, opt HOT 4
- OpenBSD doesn't have RLIMIT_AS
- Add pp [x + 42] to print an expression and its value - like Rust dbg!() HOT 2
- github URL pastebin HOT 8
- bash allows pasting multi-line code snippets, but OSH/YSH don't HOT 3
- allow backspace from multiline expressions HOT 1
- allow any custom prompts HOT 2
- typeof / reflection / assign results of '=' HOT 3
- emoji/unicode identifiers
- Setting up GNU readline history HOT 1
- 'view latest' released version in documentation - because old links are often surfaced HOT 1
- crash in parsing return HOT 1
- BashArray could have a sparse representation (ble.sh)
- bash SHELLOPTS and BASHOPTS support
- pitfall - IFS= read x doesn't set x (read --raw-line or for <> loop instead) HOT 6
- Fatal error when closing stderr on a non-fatal error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from oil.