intel / stacks Goto Github PK

I am trying to follow the indications from https://github.com/intel/stacks/tree/master/dars/clearlinux, however, I am getting an error when building the dockerfile

...
Step 30/38 : COPY dars.ld.so.conf /etc/ld.so.conf
 ---> 9b616b1315e3
Step 31/38 : COPY silent.cfg /mkl
 ---> a60c4463df0a
Step 32/38 : RUN /mkl/install.sh -s /mkl/silent.cfg &&     ldconfig
 ---> Running in cb1a8747cb89
Missing optional prerequisites
-- Unsupported OS
Detected operating system is not supported. Supported operating systems for this
release are:
    - CentOS* 6 (Intel(R) 64), 7 (Intel(R) 64);
    - Debian* 8 (Intel(R) 64), 9 (Intel(R) 64);
    - Fedora* 27 (Intel(R) 64), 28 (Intel(R) 64);
    - Red Hat Enterprise Linux* 6 (Intel(R) 64), 7 (Intel(R) 64);
    - SUSE Linux Enterprise Server* 12 (Intel(R) 64), 15 (Intel(R) 64);
    - Ubuntu* 16.04 (Intel(R) 64), 18.04 (Intel(R) 64);

--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Removing intermediate container cb1a8747cb89
 ---> d01b71d1cffb
Step 33/38 : COPY --from=builder /install_root/mkl_wrapper/mkl_wrapper.* /opt/intel/mkl/wrapper/
 ---> 767822cdfce0
Step 34/38 : RUN mkdir -p /etc/spark /etc/hadoop &&     cp /usr/share/defaults/hadoop/log4j.properties /etc/hadoop &&     cp /usr/share/apache-spark/conf/log4j.properties.template /etc/spark/log4j.properties
 ---> Running in e5bd91544b6e
cp: cannot stat '/usr/share/defaults/hadoop/log4j.properties': No such file or directory
The command '/bin/sh -c mkdir -p /etc/spark /etc/hadoop &&     cp /usr/share/defaults/hadoop/log4j.properties /etc/hadoop &&     cp /usr/share/apache-spark/conf/log4j.properties.template /etc/spark/log4j.properties' returned a non-zero code: 1

The command that I used to build the dockerfile was

$ docker build --no-cache  --build-arg swupd_args= -t clearlinux/stacks-dars-mkl .

The following query if more of a request than an issue.
I pulled the clearlinux/stacks-pytorch-oss stack and tested AlexNet benchmarks as suggested here (https://docs.01.org/clearlinux/latest/guides/stacks/dlrs.html) with caffe2 obtained from the following repository
https://github.com/pytorch/pytorch.git
I noted the following message while running the benchmark “CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU”.
I would like to know if caffe2 with avx support could be placed within your clearlinux containers for pytorch.

Thanks,
Bala

Hello,
I tried to use your code d2s.py. It works properly and recognize both the installation of docker and singularity in my VM to produce a singularity image file. I noticed however some issues that I want to refer you:

1) The code works with python 3.6 but not with python 3.4.

2) The transformation from docker to singularity do not pass the correct environment variables to the final container.

I spoke with your colleague Unnikrishnan Nair, Rahul and he suggested to send you this comments.

Thanks

I just discovered stacks and thought it might solve some issues for me. Namely, I'd like a Docker image where I can run Intel Fortran. I did try building my own using the Intel Apt instructions and, well, I ended up with a 21 GB image. Obviously, I don't know what I'm doing in re Docker, so I thought maybe Stacks might be able to use it soon?

I did try pulling the sysstacks/hpcrs-centos:latest image but:

[root@f06e452db89e workspace]# ifort

Error: A license for Comp-FL could not be obtained.  (-1,359,2).

Is your license file in the right location and readable?
The location of your license file should be specified via
the $INTEL_LICENSE_FILE environment variable.
...

The nice thing with oneAPI is it's free-ness :)

Dear Intel team,
I tested latest mkl based stacks for pytorch and tensorflow on a system running with CentOS-7 and x86-64 architecture. While the pulling of the container went fine, attempt to import pytorch or tensorflow resulted in import error as follows

"import failure : Illegal instruction (core dumped)” error. However, I noted that import of other python modules (eg. numpy) was successful.

Could you help me to resolve the issue,
Bala

With the support of different baseOS for Stacks introduced by DBRSv2 in 94c8f39 would be good to restructure the way stacks are organized following the same repo naming structure as systacks dockerhub – stack/framework/OS

ie:
https://hub.docker.com/r/sysstacks/dbrs-memcached-centos8
https://hub.docker.com/r/sysstacks/dbrs-memcached-clearlinux

So the change restructure should change from:
stacks/<stack>/<OS>/<library>
To:
stacks/<stack>/<library>/<OS>

Users will likely be coming in looking by framework, not OS, and would let you keep your -specific release notes at that level, if needed in the future. Also then matches systacks dockerhub

We are only building dlrs image now in travis, all the other images (DBRS, DARS..) etc have to be added to travis

There may be an issue in the dars/clearlinux/mkl/README.md Dockerfile example in the Deploy DARS on Kubernetes section. It specifies:

RUN mkdir -p /etc/passwd /etc/pam.d /opt/spark/conf /opt/spark/work-dir

Shouldn't /etc/passwd be a file? So, perhaps it should read:

RUN mkdir -p /etc/pam.d /opt/spark/conf /opt/spark/work-dir
RUN touch /etc/passwd