vuil / harbor

An enterprise-class container registry server based on Docker Distribution

Harbor

Project Harbor is an enterprise-class registry server that stores and distributes Docker images. Harbor extends the open source Docker Distribution by adding the functionalities usually required by an enterprise, such as security, identity and management. As an enterprise private registry, Harbor offers better performance and security. Having a registry closer to the build and run environment improves the image transfer efficiency. Harbor supports the setup of multiple registries and has images replicated between them. In addition, Harbor offers advanced security features, such as user management, access control and activity auditing.

Features

• Role based access control: Users and repositories are organized via 'projects' and a user can have different permission for images under a project.
• Policy based image replication: Images can be replicated (synchronized) between multiple registry instances. Great for load balancing, high availability, multi-datacenter, hybrid and multi-cloud scenarios.
• LDAP/AD support: Harbor integrates with existing enterprise LDAP/AD for user authentication and management.
• Image deletion & garbage collection: Images can be deleted and their space can be recycled.
• Graphical user portal: User can easily browse, search repositories and manage projects.
• Auditing: All the operations to the repositories are tracked.
• RESTful API: RESTful APIs for most administrative operations, easy to integrate with external systems.
• Easy deployment: Provide both an online and offline installer. Besides, a virtual appliance for vSphere platform (OVA) is available.

Install & Run

System requirements:

On a Linux host: docker 1.10.0+ and docker-compose 1.6.0+ .

On vSphere: vCenter 5.5+ for deployment of Harbor's virtual appliance.

Download binaries of Harbor release and follow Installation & Configuration Guide to install Harbor.

Refer to User Guide for more details on how to use Harbor.

Community

Slack: Join Harbor's community here: VMware {code}, Channel: #harbor.
Email: harbor@ vmware.com .
WeChat Group: Add WeChat id connect1688 to join WeChat discussion group.
More info on partners and users.

Contribution

We welcome contributions from the community. If you wish to contribute code and you have not signed our contributor license agreement (CLA), our bot will update the issue when you open a pull request. For any questions about the CLA process, please refer to our FAQ. Contact us for any questions: harbor @vmware.com .

License

Harbor is available under the Apache 2 license.

This project uses open source components which have additional licensing terms. The official docker images and licensing terms for these open source components can be found at the following locations:

• Photon OS 1.0: docker image, license
• Docker Registry 2.5: docker image, license
• MySQL 5.6: docker image, license
• NGINX 1.11.5: docker image, license

harbor's Issues

Provide lightwave deployment to travis for integration testing

We need to provide a public lightwave endpoint (I suggest deploying photon VM AMI, set up lightwave and take a snapshot that we can revert to at start of every travis run), then set up integration tests after deploying harbor using auth=lw_auth

S3 storage driver configuration

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

Integrate data population script with verification test suite

Integrating data population script with the test suite to verify multiple harbor deployment

Harbor schema install

Figure a way to incrementally install the Harbor schema on a RAFT endpoint.

Bug: misleading error when pulling private repo without proper auth

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)
Something about "cannot find repository"

Bug: start-harbor.sh not configuring nginx certs correctly

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

Verify if that is still an issue

HA: enable restarting of another harbor instance

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

Provide a means to take over the management of the lwraft root_dir / and s3 image bits from a failed harbor instance

Add some testing of roles in loadgen test

As followup to #12, please look into creating some users with guest roles to projects and verify that image pushing is not allowed for them

Implement authorization via lw oidc

Read project membership and role from oidc token.
Populate said data in etcd if not present already
Sync back updated role information on-demand to the db cache, when a user logs in.

Deploy release on Photon platform

Refer to Photon Platform KB on how to set up Harbor using the cluster API.

Create a tool which uses the Harbor REST api to populate data

https://github.com/vmware/harbor/blob/4e52b13ea9450491a09f4387b5c1a45b744c2910/contrib/sdk/harbor-py/harborclient/harborclient.py can be used as a model for the tool, or you can just use the sdk directly as well. Don't need anything too fancy here.
Something that can populate X users, Y projects, Z repos deterministically would be good.

Populate local db store from ETCD

Problem to solve: populate the local db store on first start or restart of the harbor instance

Bug: Unable to add user from browser

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

user-related view breaks if there isn't at least one non-admin user in system
(have not tried if reproduceable on sqlite db backend)

Upstream: notary/clair deep dive

Deep dive into upstream code, in particular the notary/clair features.
Investigate into the impact of supporting them in PC

UI updates and changes

sync changes from vmware/harbor, and build new version of UI and see if new features warrant changes to our implementation.
modify UI build to conditionally

• disable user creation (since this goes through LW now)
• disable repo replication

Restart the tests in the CI pipeline

Image push/pull tests
Loadgen tests

Bug: job service constantly restarting

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

probably more dao queries that need to be stubbed out.

Tighter Photon integration

Discuss PP-Harbor project/group mapping.

Add harbor RPM build target

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

rpm need to handle config, start, update/restart
will be used to install/update on photon OS VM on which harbor is deployed
also

• bundle schema in rpm
• include the schema update script

Implement authentication via LW oidc

• Set up lightwave endpoint
• Implement lw oidc based authenticator

photon service show not displaying harbor endpoint

Looks like some service properties are not updated

Improve test coverage of lwraft-related code

UT : sql and lwraft querybuilders
IT : processing of queries from lwraft testdb

Bug: Log container does not come up

In the AWS environment, the Harbor log container fails to startup which fails the entire deployment. It is a transient issue and is solved by rerunning docker-compose up

Clean up all lwraft accessors to use QueryBuilder

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

Update the Raft endpoint in the CI pipeline

Update the Raft endpoint in the CI pipeline, to version 1.3.1

QB In support

Implementation of IN for QueryBuilder, SQL and LWRaft

Sync persisted data from ETCD to local sqlite on a Harbor instance startup

Problem to solve: populate the local db store on first start or restart of the harbor instance

Poll ETCD for persisted data and create corresponding records in the Harbor local sqlite store.

Update implementation for QB

Implement the Update routine for the Query Builder interface. It is to be used to replace the mutators for updating data.

LDIF generator change to incorporate timestamps

Modify the LDIF generator to create a backward compatible schema to generate string & integer LDAP attributes for the timestamp based struct members of all Harbor models.

Implement parsing/validation of lw tokens with lw certs.

jwt-go needs to be configured to use the certs from lightwave

Check that Raft endpoint is reachable/compatible

On Harbor startup, check if the Raft endpoint is reachable and error out on startup.

Move Access Logs to Raft

Port AccessLog accessors to QueryBuilder

Create etcd interface

for get/set/delete operations.

Bug: Issue during harbor restart

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)
How was the restart accomplished?
Need more info on this.

Add support for lightwave group creation

the lightwave package (lwoidc) is missing this feature that we need to set up project membership for new projects.

Update to use CAS

Use CAS to do conditional write of gid. Check if feature is in lwraft 1.3.0 already.

Map raft responses to proper error types

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

Certain responses (e.g. 400 when object exists) should be considered harmless

Implement SortBy & Offset for LWRaft QB

QueryBuilder implementation for SQL and LWRaft.

Bug: need better error-check of lwraft results

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

especially for empty lists, null entries, etc

Deploy harbor using local sqlite instead of mysql

Enable Photon Harbor on ESX

• OVA builder
• Startup script, configuration fix
• CAL changes?

-------------------------

Not an issue, but a schedule delimiter

Retry Raft requests

Implement LWRaft request retrying and add timeout to the requests.

implement sync from etcd

set up etcd recursive watches, translate to SQL mutations.

Setup an ETCD container

Upstream: sync changes

Sync changes with upstream

Bug: RaftGetProjectByName crashes due to long query

(if issue is a bug, include steps to repro: env settings, branch/commit, etc, where relevant)

Bug: Need to handle GET query param length limit

Join and In calls are susceptible to overflow, while we discuss with lwraft team about relaxing the param length, we need to still have a means to break up any long filters into reasonable chunks

Set up Travis CI for this repo

Implement update admin password

After porting to raft, the update initial admin password functionality doesnt work yet. It needs to be ported to Raft.