Seg Fault running dotnet restore Linux about dotnet-docker

[TheRealPiotrP]

Nevermind, I see the issue now. @dstpierre we aren't seeing this on CLI CI because we build on pre-4.6.2 kernels. To build a workaround CLI we'd need a base OS with pre-4.6.2 which would use the affected preview2 CLI in order to build a CLI with an updated CoreCLR dependency.

Once @gkhanna79 shares the packages which need to be updated I can create a branch that includes the fix. As I mentioned earlier, however, we don't yet have a release channel for this fix so folks would need to build this branch independently for the time being.

[renkun-ken]

I'm also using GitLab CI but docker image microsoft/dotnet:latest and microsoft/dotnet-preview produce the same segment fault at dotnet restore

/bin/bash: line 42:    11 Segmentation fault      (core dumped) dotnet restore
ERROR: Build failed: exit code 1

[renkun-ken]

@naamunds Great work! I also verified that the docker images do work. Looks great now :) Thanks!

[bbrowning]

@MichaelSimons The latest dotnet-preview docker image fixed the segfault issue for me. I can now run 'dotnet restore' and 'dotnet run' on a simple hello world project without any errors. Thanks!

[janvorli]

It is very likely that the issue is fixed by dotnet/coreclr#6027

[bbrowning]

I'm also seeing a segmentation fault when running dotnet restore inside the docker image microsoft/dotnet:latest for a simple hello_world project created via dotnet new.

[MichaelSimons]

Can you please share what docker version you are using as well as what OS/version?

[bbrowning]

My host machine is Fedora 24 running Docker version 1.10.3, build ee81b72/1.10.3. The Docker image I'm running the dotnet commands inside of is:

root@afa06c87ce06:/hello_world# dotnet --info --version --verbose
.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     debian
 OS Version:  8
 OS Platform: Linux
 RID:         debian.8-x64

[bsell93]

Here is my system info as well @MichaelSimons:

dotnet --info --version --verbose
.NET Command Line Tools (1.0.0-preview1-002702)

Product Information:
 Version:     1.0.0-preview1-002702
 Commit Sha:  6cde21225e

Runtime Environment:
 OS Name:     debian
 OS Version:  8
 OS Platform: Linux
 RID:         debian.8-x64

[MichaelSimons]

@bsell93, could you please share what your Docker host OS/version is along with what version of Docker you are using?

Thanks

[bsell93]

@MichaelSimons I am running the Dotnet container on a ci machine that is hosted by GitLab so I am unable to access it directly, and when I attempted to use the docker command it was not found. But I did get the system info with the following output:

$ cat /etc/*-release || cat /proc/version
PRETTY_NAME="Debian GNU/Linux 8 (jessie)"
NAME="Debian GNU/Linux"
VERSION_ID="8"
VERSION="8 (jessie)"
ID=debian
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux runner-8a2f473d-project-921303-concurrent-0 4.6.0-coreos #2 SMP Fri Jun 17 00:54:17 UTC 2016 x86_64 GNU/Linux

[MichaelSimons]

@bsell93, @bbrowning - would you be willing to try the latest builds? You can easily try these out via https://hub.docker.com/r/microsoft/dotnet-preview/.

[bsell93]

@MichaelSimons I just tried using microsoft/dotnet-preview docker image and it still experienced a seg fault, and it seemed to fail earlier than the other docker image.

Full readout from CI:

gitlab-ci-multi-runner 1.2.0 (3a4fcd4)
Using Docker executor with image microsoft/dotnet-preview ...
Pulling docker image microsoft/dotnet-preview ...
Running on runner-8a2f473d-project-921303-concurrent-0 via runner-8a2f473d-machine-1466659914-247380c4-digital-ocean-4gb...
Cloning repository...
Cloning into '/builds'...
Checking out 21908d39 as hotfix...
$ cd $CI_PROJECT_DIR/server
$ dotnet restore
log  : Restoring packages for /builds/project.json...
log  : Restoring packages for /builds/project.json...
/bin/bash: line 35:     9 Segmentation fault      (core dumped) dotnet restore

ERROR: Build failed: exit code 1

[MichaelSimons]

This seems related to https://github.com/dotnet/cli/issues/3681.

[bsell93]

@MichaelSimons is it possible to go to a specific dotnet docker version because it was working roughly 13 days ago?

[MichaelSimons]

@bsell93, Docker supports pulling an image via its sha - docker pull microsoft/dotnet@sha256:5235a3980a8bfff70ad7a3b2ea5dfdf5225f733d021cbf1eb3bd4151ddded2c6'. It is my understanding that once an image is uploaded to a registry (e.g. Docker Hub) it is always available to download via it's sha. Images are never deleted from the registry even if a new one is uploaded with the same tag. The problem being you must know the sha. If you have an image you can rundocker inspect --format='{{.RepoDigests}}' microsoft/dotnet:latest
` to get the sha. I do not know how to query a registry to get all of the images that exist therefore I am not sure my answer is going to be much help for you.

[bsell93]

@MichaelSimons I tried pulling a specific image found in the docker image details, but I was unable to overcome the segmentation fault. I am certain it was working on the specific images that I tested, so there most be some other issue. I remember seeing an issue with libcurl3 updating, and others had to downgrade. Would something like that make sense?

[MichaelSimons]

@bsell93, Downgrading libcurl3 was mentioned in https://github.com/dotnet/cli/issues/3185 by one user. Did your host machine change in any way (e.g. did the kernel get upgraded)?

[bsell93]

@MichaelSimons as far as I know it did not get changed.

[bsell93]

That's great new @janvorli! I am curious as to how we can utilize this fix, or is it just a matter of waiting until everything gets updated?

[bsell93]

Anymore movement on this?

[janvorli]

@bsell93 I am not sure how the latest changes in coreclr are propagated to cli and dotnet-docker. @gkhanna79, is the cli bound to the coreclr release 1.0.0 or is it picking the latest changes?

[gkhanna79]

@piotrpMSFT Has CLI started to consume master branch of core-setup?

[chrisumbel]

This is affecting me in wercker pipelines. I've tried a few tags, but mainly 1.0.0-preview2-sdk. The pipelines run fine locally via the wercker CLI on Linux ubuntu-xenial 4.4.0-28-generic, but fail up in wercker.

[RobinDavisNotts]

any idea if theres a docker image with the fix in it?

or a workaround

[MichaelSimons]

@EolAncalimon, There isn't a docker image with the fix because there isn't a CLI with the fix.

@piotrpMSFT, @blackdwarf - when is there going to be a CLI that contains the updated dependencies to address this issue?

[dstpierre]

we wanted to give dotnet a test, I'm on Arch and docker version is:

Client:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.6.2
 Git commit:   b9f10c9
 Built:        Tue Jun 21 00:43:14 2016
 OS/Arch:      linux/amd64

Just did a docker -it microsoft/dotnet:latest and running dotnet new returns the following:

A command is running to initially populate your local package cache, to improve restore speed and enable offline access. This command will take up to a minute to complete and will only happen once.
Decompressing 100% 1774 ms
Expanding 4%Segmentation fault (core dumped)

Anything I can do, other than wait for the CLI to have a fix [if I understand it correctly]? Does the preview1 image would have this problem as well?

Thanks

[TheRealPiotrP]

We are working on a plan for how we might provide a nightly build with a fix in place. Until then, you should be able to do a local build of CLI that pulls in the right version of dependencies to get this unblocked. @gkhanna79 do we know which packages need to be updated, and to which versions?

[dstpierre]

@piotrpMSFT thanks for your reply, unfortunately I was not successful building the source.

I built it from a Ubuntu 16.04 docker image. I opened an issue on the cli project, although I'm not exactly clear if it was the right place to post that. Just trying to get started.

This is the error I'm getting:

Restoring Build Script projects...
log  : Restoring packages for /dl/cli/build_projects/shared-build-targets-utils/project.json...
log  : Restoring packages for /dl/cli/build_projects/Microsoft.DotNet.Cli.Build.Framework/project.json...
log  : Restoring packages for /dl/cli/build_projects/update-dependencies/project.json...
log  : Restoring packages for /dl/cli/build_projects/dotnet-cli-build/project.json...
/dl/cli/build_projects/dotnet-cli-build/build.sh: line 116:  4908 Segmentation fault    

I picked the 1.0.0 branch of the cli. Any idea, is there anyway to have it working at all? Is it just me doing something wrong?

For reference: https://github.com/dotnet/cli/issues/3861

Thanks for you time.

[TheRealPiotrP]

This is strange. The rel/1.0.0 branch is green on Ubuntu 14.04 and 16.04, so if you're running a stock ./build.sh then I don't see why you would have a different behavior. Can you tell us more about the machine configuration that is exhibiting the failure?

[TheRealPiotrP]

@livarcocc can you help ?

[naamunds]

I've verified that this is fixed on Ubuntu 14.04 with kernel 4.6.2 in the .NET Core 1.0.1 update released today (SDK version 1.0.0-preview2-003131). The microsoft/dotnet Docker images have been updated to use this .NET version.

[naamunds]

@renkun-ken The dotnet-preview Docker Hub repo has been replaced by the dotnet-nightly repo, so please use dotnet-nightly instead for last-known-good builds of .NET Core.

(This issue is fixed in both microsoft/dotnet and microsoft/dotnet-nightly.)