mlockall still necessary (containers) about open-iscsi

[Anjingkegui]

can you check the version of iscsiadm in your container and the version of iscsid on your host?

[gonzoleeman]

I believe others have worked on containerizing open-iscsi. I believe there is also an issue with the kernel communications when putting iscsid in a container. And I believe there is value in making open-iscsi work in containers.

[anusha-ragunathan]

I came across this issue while looking for something related. Thought I should update it with my experience.

Docker container running iscsid on an Ubuntu base image is possible. On Docker, there're options to run privileged containers ( that access that essentially adds a bunch of CAPS) as well as access the host network namespace. I have a Docker image on https://hub.docker.com/r/aragunathan/open-iscsi with details.

My question is, if its possible to run iscsid on the host and have iscsiadm running on the container. I've tried this approach and found that in spite of providing the container access to host's network ns and running it privileged, iscsiadm block on the network. I've captured strace from a sample call and it looks like so:

brk(NULL)                               = 0x55dadf6cf000                                                      
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)                 
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)                        
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3                                         
fstat(3, {st_mode=S_IFREG|0644, st_size=20480, ...}) = 0                                                  
mmap(NULL, 20480, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f532d5a2000                            
close(3)                                = 0                    
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)       
open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/tls/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls/x86_64", 0x7fff4f836410) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/tls/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/lib/x86_64-linux-gnu/tls", 0x7fff4f836410) = -1 ENOENT (No such file or directory)                           
open("/lib/x86_64-linux-gnu/x86_64/libc.so.6", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)    
stat("/lib/x86_64-linux-gnu/x86_64", 0x7fff4f836410) = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3                          
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832   
fstat(3, {st_mode=S_IFREG|0755, st_size=1868984, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f532d5a1000
mmap(NULL, 3971488, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f532cfb8000
mprotect(0x7f532d178000, 2097152, PROT_NONE) = 0
mmap(0x7f532d378000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1c0000) = 0x7f532d378000
mmap(0x7f532d37e000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f532d37e000
close(3)                                = 0  
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f532d5a0000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f532d59f000
arch_prctl(ARCH_SET_FS, 0x7f532d5a0700) = 0
mprotect(0x7f532d378000, 16384, PROT_READ) = 0                                             
mprotect(0x55dade274000, 8192, PROT_READ) = 0
mprotect(0x7f532d5a7000, 4096, PROT_READ) = 0                                    
munmap(0x7f532d5a2000, 20480)           = 0                                                   
rt_sigaction(SIGINT, {0x55dade03b270, [], SA_RESTORER, 0x7f532cfed4b0}, {SIG_DFL, [], 0}, 8) = 0                 
umask(0177)                             = 022          
getrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
setrlimit(RLIMIT_NOFILE, {rlim_cur=1024*1024, rlim_max=1024*1024}) = 0
access("/etc/iscsi/", F_OK)             = 0
mmap(NULL, 512000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f532d522000
socket(PF_LOCAL, SOCK_STREAM, 0)        = 3                                                                     
connect(3, {sa_family=AF_LOCAL, sun_path=@"ISCSIADM_ABSTRACT_NAMESPACE"}, 30) = 0                             
write(3, "\n\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 13624) = 13624                              
recvfrom(3,                                           ```

Any thoughts on what else is needed on the container running iscsiadm to connect with iscsid running on the host? Anything special about the iscsid socket and it's visibility from a container? I've also tried bind mounting several of the host's system directories such as /run, /dev, etc. and it doesnt help.

[anusha-ragunathan]

Could my problem be due to iscsid exposing its listen socket as an Abstract namespace socket? If it was a regular UDS, then it would be available on the filesystem and I would be able to bind mount the socket into the container running iscsiadm.

How does a container access the Abstract namespace socket running on the host?

[gonzoleeman]

Try it with regular sockets and see!

[anusha-ragunathan]

I tried setting the ListenStream in the iscsid.service file to a regular socket file on the filesystem. It doesnt work, the socket file is not created. Does the code allow to do that?

[anusha-ragunathan]

The issue I was facing was indeed due to mismatched versions between iscsid on the host and iscsiadm on the container. I resolved it a while ago. Thanks for chiming in!

[gonzoleeman]

Is this still an issue with anyone, or can I close it?

[Anjingkegui]

@gonzoleeman yeah, i think this issue can be resolved by just using one unified version.

[gonzoleeman]

Marking as closed. Please re-open if there is still an issue.