Libcollector
Collect name, versions, and other information (e.g. license, author) of the used OSS libraries from github repositories.
Description
libcollector
is a python3 commandline tool to collect information of the library which used in specified github repositories.
This tool collects libraries from package management files like requirements.txt
with python or package.json
in nodejs. The available managers are below.
Optional information like license or author will collected from each package manager's web api or web site. If it can't find these information, the optional information will be empty.
You can specify repositories which you have Read permission.
Demo
Here is an execution sample. Target repository is vuejs/vue
, and output format is csv.
$ python libcollector.py
Installation
Requirements
- python3
Installation
Get this repository and install required modules.
$ git clone https://github.com/kusuwada/libcollector.git
$ cd libcollector
$ pip install -r requirements.txt
Usage
Prepare
Before use, you have to create your "github access token for commandline", if the target repository is private.
See GitHub Help: Authenticating to GitHub.
As it says, please select repo
in scope selection.
Basic usage
$ cd libcollector
$ python libcollector.py [--token {github_access_token}]
--token
is an optional. If the target includes private repository, you have to create "github access token" and set this option.
Settings
See and Edit libcollector/settings.yml
target_repositories
List of the target repositories. Format is below
{owner}/{repository}
e.g.
target_repositories:
- kusuwada/node-slack-log-exporter
- kusuwada/ruby-slack-log-exporter
- requests/requests
You can specify both public & private repository. If you specify private repository, see Prepare.
target_manager
List of the target library managers. Now, we support below managers.
- requirements.txt (python)
- Gemfile (ruby)
- package.json (javascript)
If you want to collect libraries from other manager file, you can add your implementation.
output
List of the export media/formats. Format is below.
{output_type}: {output_path}
Now, we support below formats.
- csv
- text
If you want to export other medias or db, you can add your implementation.
optional_information
You can specify output optional information below or not.
- info_license: library's license
- info_author: library's author
- info_homepage_url: library's homepage url
- info_code_url: library's source code url
If the tool can't get these information, output record becomes empty.
License
Please see the LICENSE file for details. [URL] or MIT
Author
Future Plan
- Support library collection from install command. e.g.
apt-get
,yum
- Add test