ibm-datapower / datapower-labs

datapower-labs

Sections

DataPower on Docker

datapower-labs's Issues

Docker image won't start

Hi,
I am running docker-engine 1.11.1 and trying to get datapower to run in a docker container. I downloaded the .deb files from passport and tried using the Dockerfile from the quickstart and this git repository. The build succeeded however when I issued the following run command

docker run -it -h datapower --privileged datapower /bin/bash
I then issue the

/opt/ibm/datapower/datapower-launch

and get the following results. Running non interactively with the -d option and looking at the logs after leads to the same error messages.

Thu Apr 28 2016 10:50:09 ERR dpControl [pre-start][20] Cannot run losetup --show --find /var/opt/ibm/datapower/datapower.img
Thu Apr 28 2016 10:50:09 ERR dpControl [pre-start][20] losetup: losetup: Could not find any loop device. Maybe this kernel does not know
Thu Apr 28 2016 10:50:09 ERR dpControl [pre-start][20] losetup: about the loop device? (If so, recompile or `modprobe loop'.)
Thu Apr 28 2016 10:50:09 ERR dpControl [post-stop][20] Cannot open lockfile '/var/opt/ibm/datapower/datapower.img.lck': No such file or directory
Thu Apr 28 2016 10:50:09 ERR dpControl [post-stop][20] Cannot close LUKS partition 'var_opt_ibm_datapower_datapower_img': No such device (error 19)
Thu Apr 28 2016 10:50:09 ERR dpControl [post-stop][20] Cannot open lockfile '/var/opt/ibm/datapower/datapower.img.lck': No such file or directory
Thu Apr 28 2016 10:50:10 ERR dpControl [post-stop][20] Cannot open lockfile '/var/opt/ibm/datapower/datapower.img.lck': No such file or directory
Thu Apr 28 2016 10:50:10 ERR dpControl [post-stop][20] Cannot open lockfile '/var/opt/ibm/datapower/datapower.img.lck': No such file or directory

Any idea what is going on? Should I open a pmr?

Here are the relevant parts of the build process:

Setting up telnet (0.17-36build2) ...
update-alternatives: using /usr/bin/telnet.netkit to provide /usr/bin/telnet (telnet) in auto mode
Setting up schroot-common (1.6.8-1ubuntu1.1) ...
Setting up schroot (1.6.8-1ubuntu1.1) ...
Setting up kpartx (0.4.9-3ubuntu7.11) ...
Processing triggers for libc-bin (2.19-0ubuntu6.7) ...
Processing triggers for ureadahead (0.100.0-16) ...

• dpkg -i /tmp/ibm-datapower-common.deb /tmp/ibm-datapower-image.deb
  Selecting previously unselected package ibm-datapower-common.
  (Reading database ... 11714 files and directories currently installed.)
  Preparing to unpack /tmp/ibm-datapower-common.deb ...
  Unpacking ibm-datapower-common (7.5.0.0.274960) ...
  Selecting previously unselected package ibm-datapower-agoradco-image.
  Preparing to unpack /tmp/ibm-datapower-image.deb ...
  Unpacking ibm-datapower-agoradco-image (7.5.0.0.274960) ...
  Setting up ibm-datapower-agoradco-image (7.5.0.0.274960) ...
  Setting up ibm-datapower-common (7.5.0.0.274960) ...
  Processing triggers for ureadahead (0.100.0-16) ...
• sed -i -e /^web-mgmt/,/^exit/s/admin-state.*/admin-state "enabled"/g /opt/ibm/datapower/datapower-external.cfg
• rm /tmp/ibm-datapower-common.deb /tmp/ibm-datapower-image.deb
• /opt/ibm/datapower/datapower-docker-build.sh
• mkdir -p /datapower/config /datapower/local
• echo DataPowerConfigDir=/datapower/config
• echo DataPowerLocalDir=/datapower/local
• echo DataPowerCpuCount=4
• echo DataPowerMemoryLimit=8000
  ---> c415e4c3d13a
  Removing intermediate container 5c3d2ce88efe
  Step 5 : EXPOSE 9090
  ---> Running in c0b9ce7f4781
  ---> 8d57b851db74
  Removing intermediate container c0b9ce7f4781
  Step 6 : CMD /opt/ibm/datapower/datapower-launch
  ---> Running in e9432923bcaf
  ---> d0bca646f75a
  Removing intermediate container e9432923bcaf
  Successfully built d0bca646f75a

schroot rpm does not use EPEL

In several places, the schroot rpm is pulled directly from a url. It should not work this way, we should first install EPEL then let "yum install" pull in the schroot dependency.

running into issues while creating the docker image

Hi Team,

I am getting the below error while creating the docker file:

Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
There are no enabled repos.
Run "yum repolist all" to see the repos you have.
You can enable repos with yum-config-manager --enable

After successful start and stop container can not be started again.

Hi,
After successfully starting the datapower container on docker 1.11 I decided to stop the container and start it up again.
I ran the following:

docker run -d -h datapower --name=datapower --privileged -p 9090:9090 datapower
docker stop datapower
docker rm -v datapower
This last command resulted in the following error message:

Error response from daemon: Driver devicemapper failed to remove root filesystem 64636c8d9dfc842933d4ad95917d5d0394993c006f8bcc0a2ee5b5a445afad1a: Device is Busy

A restart of the docker service was needed to clear this message.

Then I tried to start the container again and ended up with the following message

systemctl restart docker
[root@testbed2 ~]# docker rm datapower
Error response from daemon: No such container: datapower
[root@testbed2 ~]# docker run -d -h datapower --name=datapower --privileged -p 9090:9090 datapower

5a48e3ef22f0b00dcc492980f4c8736b47d821d601ac23325cd438a489e6c376

[root@testbed2 ~]# docker logs 5a48e3ef22f0b00dcc492980f4c8736b47d821d601ac23325cd438a489e6c376
Sat Apr 30 2016 03:52:50 ERR dpControl [pre-start][1] Cannot unlock LUKS partition 'var_opt_ibm_datapower_datapower_img': File exists (error 17)
Sat Apr 30 2016 03:52:50 ERR dpControl [pre-start][1] Cannot open lockfile '/var/opt/ibm/datapower/datapower.img.lck': No such file or directory

Which probably revolves around something not being freed as a result of running in privileged mode.

At this point a restart of the operating system was needed to be able to start this container again.

Just as an FYI I had 8 free loop devices defined to the system.

If datapower was the only container running on this system a OS restart is not such a big thing however that is not an acceptable way of running long term.

This might be a docker issue however given that this container requires elevated privileges I suspect a combination a docker bug and some cleanup error might be to blame.

A friendly suggestion.

Rather than running a container within a container which is what you seem to be doing here. You might want to think of a way of packaging a busybox container and running your code natively.

You seem to start ubuntu or redhat as the driving OS for your container then you schroot into busybox (at least on the surface you seem to be doing that).

I really don't think this methodology is sustainable long term given the way docker is going.

Just a thought.

Ted

Docker toolbox windows environment Datapower starts but never listens on port 9090

After building the debian image in the windows toolbox environment I successfully started datapower however when I exec into the system I see that the environment has started but a listen for port 9090 doesn't ever happen.

Here are the log messages:

$ docker logs datapower
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/drouter/internalconfig/datapower-external.cfg': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /var/opt/ibm/datapower/datapower-external.cfg /tmp/ibm-dp-container-1-1-1462374806/primary/drouter/internalconfig/datapower-external.cfg || cp -f /var/opt/ibm/datapower/datapower-external.cfg /tmp/ibm-dp-container-1-1-1462374806/primary/drouter/internalconfig/datapower-external.cfg' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/etc/hosts.host': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /etc/hosts /tmp/ibm-dp-container-1-1-1462374806/primary/etc/hosts.host || cp -f /etc/hosts /tmp/ibm-dp-container-1-1-1462374806/primary/etc/hosts.host' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/etc/resolv.conf.host': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /etc/resolv.conf /tmp/ibm-dp-container-1-1-1462374806/primary/etc/resolv.conf.host || cp -f /etc/resolv.conf /tmp/ibm-dp-container-1-1-1462374806/primary/etc/resolv.conf.host' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/etc/os-release': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /etc/os-release /tmp/ibm-dp-container-1-1-1462374806/primary/etc/os-release || cp -f /etc/os-release /tmp/ibm-dp-container-1-1-1462374806/primary/etc/os-release' failed: Operation not permitted
cp: cannot stat '/etc/redhat-release': No such file or directory
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /etc/redhat-release /tmp/ibm-dp-container-1-1-1462374806/primary/etc/redhat-release || cp -f /etc/redhat-release /tmp/ibm-dp-container-1-1-1462374806/primary/etc/redhat-release' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/.dockerenv': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s //.dockerenv /tmp/ibm-dp-container-1-1-1462374806/primary/.dockerenv || cp -f //.dockerenv /tmp/ibm-dp-container-1-1-1462374806/primary/.dockerenv' failed: Operation not permitted
cp: cannot stat '//.dockerinit': No such file or directory
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s //.dockerinit /tmp/ibm-dp-container-1-1-1462374806/primary/.dockerinit || cp -f //.dockerinit /tmp/ibm-dp-container-1-1-1462374806/primary/.dockerinit' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/root/product_uuid': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /sys/devices/virtual/dmi/id/product_uuid /tmp/ibm-dp-container-1-1-1462374806/primary/root/product_uuid || cp -f /sys/devices/virtual/dmi/id/product_uuid /tmp/ibm-dp-container-1-1-1462374806/primary/root/product_uuid' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/root/uuid.rand': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /proc/sys/kernel/random/uuid /tmp/ibm-dp-container-1-1-1462374806/primary/root/uuid.rand || cp -f /proc/sys/kernel/random/uuid /tmp/ibm-dp-container-1-1-1462374806/primary/root/uuid.rand' failed: Operation not permitted
cp: cannot create regular file '/tmp/ibm-dp-container-1-1-1462374806/primary/etc/datapower.conf': Read-only file system
Wed May 04 2016 15:13:27 ERR dpControl [pre-start][1] Execution of 'cmp -s /var/opt/ibm/datapower/datapower.conf /tmp/ibm-dp-container-1-1-1462374806/primary/etc/datapower.conf || cp -f /var/opt/ibm/datapower/datapower.conf /tmp/ibm-dp-container-1-1-1462374806/primary/etc/datapower.conf' failed: Operation not permitted
Wed May 04 2016 15:13:27 INF dpControl [pre-start][1] Processing /tmp/ibm-dp-container-1-1-1462374806/primary/etc/datapower.conf
Wed May 04 2016 15:13:27 INF dpControl [pre-start][1] Standby Control Not Enabled
Wed May 04 2016 15:13:27 INF dpControl [pre-start][1] RaidBlockDeviceName is not specified
Wed May 04 2016 15:13:29 INF dpControl [session][1] Starting DataPower service ...

$ docker exec -it datapower /bin/bash
root@datapower:/# netstat -tan
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:2200 0.0.0.0:* LISTEN

 0:00 ps ax

root@datapower:/# ps axww
PID TTY STAT TIME COMMAND
1 ? SLs 0:01 /opt/ibm/datapower/datapower-launch
74 ? S 0:00 sh -c schroot --run-session --chroot var_opt_ibm_datapower_datapower_img-1af0d4d3-f7d2-4776-806b-67e698eb9d63 -- /bin/supervisor -m /dev/mapper/var_opt_ibm_datapower_datapower_img
75 ? S 0:00 schroot --run-session --chroot var_opt_ibm_datapower_datapower_img-1af0d4d3-f7d2-4776-806b-67e698eb9d63 -- /bin/supervisor -m /dev/mapper/var_opt_ibm_datapower_datapower_img
76 ? S 0:00 [init] -m /dev/mapper/var_opt_ibm_datapower_datapower_img
77 ? Ssl 0:00 [main] -m /dev/mapper/var_opt_ibm_datapower_datapower_img
11259 ? Ss 0:00 /bin/bash

Running the latest docker toolbox 1.11.1

Running datapower on OpenShift

Hi, was just wondering if anyone had managed to get datapower running on OpenShift?

I can run my image via docker on my openshift nodes:
docker run --privileged -p 9290:9090 ibm-datapower-licensed
Fri Aug 05 2016 10:28:55 INF dpControl [pre-start][1] Processing /tmp/ibm-dp-container-1-1-1470407332/primary/etc/datapower.conf
Fri Aug 05 2016 10:28:55 INF dpControl [pre-start][1] Standby Control Not Enabled
Fri Aug 05 2016 10:28:55 INF dpControl [pre-start][1] RaidBlockDeviceName is not specified
Fri Aug 05 2016 10:28:56 INF dpControl [session][1] Starting DataPower service ...
and I can connect to the WEBUI via localhost:9290

but get the following in the OpenShift logs when I try and launch it that way:

mount: permission denied
Fri Aug 05 2016 10:26:37 ERR dpControl [pre-start][1] Execution of 'mount --make-rprivate /' failed: Broken pipe
Fri Aug 05 2016 10:26:38 ERR dpControl [pre-start][1] Cannot run losetup --show --find /var/opt/ibm/datapower/ibm-datapower-licensed-1-7b3sh.datapower.img
Fri Aug 05 2016 10:26:38 ERR dpControl [pre-start][1] losetup: losetup: /var/opt/ibm/datapower/ibm-datapower-licensed-1-7b3sh.datapower.img: failed to set up loop device: No such file or directory
Fri Aug 05 2016 10:26:38 ERR dpControl [post-stop][1] Cannot open lockfile '/var/opt/ibm/datapower/ibm-datapower-licensed-1-7b3sh.datapower.img.lck': No such file or directory
/dev/mapper/control: mknod failed: Operation not permitted
Failure to communicate with kernel device-mapper driver.
Check that device-mapper is available in the kernel.
Fri Aug 05 2016 10:26:38 ERR dpControl [post-stop][1] Cannot close LUKS partition 'var_opt_ibm_datapower_ibm-datapower-licensed-1-7b3sh_datapower_img': No such device (error 19)
Fri Aug 05 2016 10:26:38 ERR dpControl [post-stop][1] Cannot open lockfile '/var/opt/ibm/datapower/ibm-datapower-licensed-1-7b3sh.datapower.img.lck': No such file or directory
Fri Aug 05 2016 10:26:39 ERR dpControl [post-stop][1] Cannot open lockfile '/var/opt/ibm/datapower/ibm-datapower-licensed-1-7b3sh.datapower.img.lck': No such file or directory
Fri Aug 05 2016 10:26:39 ERR dpControl [post-stop][1] Cannot open lockfile '/var/opt/ibm/datapower/ibm-datapower-licensed-1-7b3sh.datapower.img.lck': No such file or directory

Any thoughts?

Cheers

Sean

rpmimg fails `make build1

Installing dependencies

• yum -y update
  Loaded plugins: ovl, product-id, search-disabled-repos, subscription-manager
  This system is not registered to Red Hat Subscription Management. You can use subscription-manager to register.
  There are no enabled repos.
  Run "yum repolist all" to see the repos you have.
  You can enable repos with yum-config-manager --enable

and then the build fails

No port exposed

When I run "make evolve"

it perform this command:

sudo docker run -d --name datapower --restart=on-failure --privileged -P root/datapower-factory:0.1

but no port exposed. I have to manually add parameter:

-p xxx.xxx.xxx.xxx:9090:9090

That's work for me.