ahmedzerouali / conpan

ConPan: Analyze your Docker container in peace

ConPan: Analyze your Docker container in peace

ConPan inspect Docker containers and extract their installed packages to analyze them.

ConPan analyzes packages technical lag, vulnerabilities and other type of bugs. The output is a four pandas dataframes that contain general information about the analyzed DockerHub image, information about installed packages, how outdated they are, their vulnerabilities and other kind of bugs.

How it works

ConPan workflow is very simple:

• Pulls the Docker image.
• Runs it and extract installed packages.
• Tracks installed packages from their package managers.
• Inspects their technical lag: checks if they are outdated and how much they are lagging behind the latest available versions.
• Identifies vulnerable packages.
• Identifies other kind of bugs for installed packages.

Requirements

Python modules:

• pandas>=0.22.0
• requests>=2.18.2
• psycopg2-binary>=2.7.4
• matplotlib>=3.0.0
• tqdm>=>=4.36.0

Other:

• docker-ce (https://docs.docker.com/install/)

How to install/uninstall

ConPan is developed and tested mainly on GNU/Linux platforms. Thus it is very likely it will work out of the box on any Linux-like (or Unix-like) platform, upon providing the right requirements and version of Python (3.5, 3.6).

To install, run:

$> git clone https://github.com/neglectos/ConPan
$> python3 setup.py install

Or simply:

$> pip3 install git+https://github.com/neglectos/ConPan

To uninstall, run:

$> pip3 uninstall conpan

How to use

ConPan can be used from command line or directly from Python, both usages are described below.

You will need permission to use the Docker tool first.

From command line

Launching ConPan from command line does not require much effort.

$ conpan -p <package_type> -c <image> 

Example:

$ conpan -p debian -c 127labs/blog -d 

Output:

Results: 
General information about the Docker image:  127labs/blog
- pull_count: 12870
- star_count: 0
- description: 127Lab's blog powered by Ghost
- last_updated: 2017-04-29T16:34:03.485881Z
- full_size: 114209000

Results about installed packages in:  127labs/blog
# installed packages: 130
# tracked packages: 130
# vulnerabilities: 326
# bugs: 2047

From Python

ConPan can be embedded in your Python scripts. Again, the effort of using it is minimum.

#! /usr/bin/env python3
from conpan.conpan import ConPan

# With 2 parameters
image_community = '127labs/blog'
image_official = 'debian:buster-slim'

cp = ConPan(packages="debian", image=image_official)

# extracting all information
general_info, installed_packages, tracked_packages, vulnerabilities, bugs = cp.analyze()

Or

# Extracting some specific information

cp.general_info()
cp.installed_packages()
cp.tracked_packages()
cp.vulnerabilities()
cp.bugs()

A scientific publication reporting on the ConPan tool can be found here: https://doi.org/10.1109/MSR.2019.00089

You can cite the associated research paper using:

@inproceedings{zerouali2019conpan,
  title={ConPan: a tool to analyze packages in software containers},
  author={Zerouali, Ahmed and Cosentino, Valerio and Robles, Gregorio and Gonzalez-Barahona, Jesus M and Mens, Tom},
  booktitle={2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR)},
  pages={592--596},
  year={2019},
  organization={IEEE}
}

Or

A. Zerouali, V. Cosentino, G. Robles, J. M. Gonzalez-Barahona and T. Mens, "ConPan: A Tool to Analyze Packages in Software Containers", 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), 2019, pp. 592-596, doi: 10.1109/MSR.2019.00089

conpan's Issues

Add reference to the ConPan publication to the README file.

Ahmed, can you add the following information to the README file of the ConPan tool on GitHub?

A scientific publication reporting on the ConPan tool can be found here: https://doi.org/10.1109/MSR.2019.00089
You can cite this publication as follows:
A. Zerouali, V. Cosentino, G. Robles, J. M. Gonzalez-Barahona and T. Mens, "ConPan: A Tool to Analyze Packages in Software Containers," 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), 2019, pp. 592-596, doi: 10.1109/MSR.2019.00089

Executing Simple Conpan Command

Hi,

I am trying to execute the example provided in the README, sudo conpan -p debian -c 127labs/blog -d example, but I am not having any luck.
It is difficult for me to pinpoint the exact issue from the error message:
"
(docker) sc@sc-VirtualBox:~/Documents/docker/conpan/ConPan$ sudo conpan -p debian -c 127labs/blog -d example
[sudo] password for sc:
Connecting to DockerHub... Done
Pulling the Docker image... Done
Extracting installed packages... Done
Tracking installed packages... Done
65it [00:00, 31938.82it/s]s...
list index out of range
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/EGG-INFO/scripts/conpan", line 78, in main
general_info, installed_packages, tracked_packages, vulnerabilities, bugs = cp.analyze()
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/conpan.py", line 93, in analyze
vulnerabilities = self.vulnerabilities()
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/conpan.py", line 158, in vulnerabilities
return self.backend.get_vuls(self.trackedPackages)
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/backend/debian.py", line 338, in get_vuls
vuls = self.final_vuls(tracked_packages)
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/backend/debian.py", line 329, in final_vuls
df[col] = tcsv[index]
IndexError: list index out of range

""

I am running the latest version of Ubuntu in a virtual machine. Do anyone have any ideas as to why this may be occurring?

If I try to run conpan with images other than 127labs I get the following error:

sc@sc-VirtualBox:~/Documents/docker/conpan/ConPan$ sudo conpan -p debian -c apache2 -d example
Connecting to DockerHub... Done
Pulling the Docker image... Unable to find image 'apache2:latest' locally
docker: Error response from daemon: pull access denied for apache2, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
Unable to find image 'apache2:latest' locally
docker: Error response from daemon: pull access denied for apache2, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
Done
Extracting installed packages... Done
Tracking installed packages... Cannot set a DataFrame with multiple columns to the single column missing_updates
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/EGG-INFO/scripts/conpan", line 78, in main
general_info, installed_packages, tracked_packages, vulnerabilities, bugs = cp.analyze()
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/conpan.py", line 90, in analyze
tracked_packages = self.tracked_packages() ### TRACK THE INSTALLED PACKAGES
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/conpan.py", line 152, in tracked_packages
self.trackedPackages = self.backend.track_packages(installed_packages)
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/backend/debian.py", line 217, in track_packages
tracked = self.oudated_packages(tracked_packages[['package', 'version', 'first_seen']],
File "/usr/local/lib/python3.10/dist-packages/ConPan-1.0.0-py3.10.egg/conpan/backend/debian.py", line 189, in oudated_packages
tracked['missing_updates'] = tracked.apply(lambda d:
File "/usr/local/lib/python3.10/dist-packages/pandas-1.5.1-py3.10-linux-x86_64.egg/pandas/core/frame.py", line 3967, in setitem
self._set_item_frame_value(key, value)
File "/usr/local/lib/python3.10/dist-packages/pandas-1.5.1-py3.10-linux-x86_64.egg/pandas/core/frame.py", line 4122, in _set_item_frame_value
raise ValueError(
ValueError: Cannot set a DataFrame with multiple columns to the single column missing_updates

Thank you in advance.