BOSH release for Cloud Foundry Bits-Service

License: Apache License 2.0

Shell 11.63% HTML 15.85% Ruby 69.22% Go 2.58% Dockerfile 0.71%

bits-service-release's Introduction

Bits Service Release

Please note: the Bits-Service is not actively maintained anymore. More information.

A BOSH release for deploying the bits-service.

Deployment

Deploy a CF on BOSH Lite with bits-service enabled

Simply follow instructions in cf-deployment and the Ops-file README for experimental to add bits-service.yml and other necessary Ops-files (documented there as well) to enable the Bits-Service.

Run Tests

To run bits-service tests, you need to deploy it with upload size limits set to lower values.

Generate deployment manifest with --size-limits and deploy.

./scripts/generate-cf-with-bits-service-enabled-bosh-lite-manifest local --size-limits

Be advised that the deployment should have succeeded before at least once, otherwise CloudFoundry post-install scripts will fail to run with this stricter limits.

Configure test execution:

export BITS_SERVICE_PRIVATE_ENDPOINT_IP=10.244.0.74
export BITS_SERVICE_MANIFEST=./deployments/cf-with-bits-service-enabled.yml
export CC_API=https://api.bosh-lite.com
export CC_PASSWORD=xxx
export CC_USER=admin

./scripts/add-route

The following two lines need to be present in your /etc/hosts to run the tests:

10.244.0.74 bits-service.service.cf.internal bits-service.bosh-lite.com
10.244.0.130 blobstore.service.cf.internal

Then run:

bundle install
bundle exec rake

CI Pipeline

The pipeline is publicly visible at https://ci.flintstone.cf.cloud.ibm.com. The sources are located at bits-service-ci.

bits-service-release's People

Stargazers

Watchers

Forkers

jamesclonk mudler edwardstudy eirini-forks madamkiwi eric-cartman homedepot rakutentech giner emilyemily jimmykarily troytop suse petergtz

bits-service-release's Issues

Golang update

Hello,

I see you are using an old version of Go and our vulnerability scan complains about it.

Even though most CVEs won't apply to this specific implementation, would still be nice to have it updated to 1.11.5

Thanks!

nginx job complains about `server_names_hash_bucket_size`

I deployed CF on a bosh-lite VM using cf-deployment (SHA 36738ba9a4fc407a3eaef60604ca8e935efcb562) and the following ops files:

operations/experimental/bits-service.yml
operations/experimental/enable-bits-service-https.yml
operations/experimental/bits-service-webdav.yml
operations/experimental/enable-bits-service-consul.yml

The nginx_bits job on the api instance group was failing, reporting that server_names_hash_bucket_size needed to be increased. Changing this value to 128 in nginx.conf and restarting all the jobs allowed the deployment to succeed.

Even after this however, the bits service was not externally accessible, e.g.:

$ curl -kv https://bits-service.<system domain>/
*   Trying <ip>...
* Connected to bits-service.<system domain> (IP) port 443 (#0)
* found 148 certificates in /etc/ssl/certs/ca-certificates.crt
* found 592 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
*        server certificate verification SKIPPED
*        server certificate status verification SKIPPED
*        common name: routerSSL (matched)
*        server certificate expiration date OK
*        server certificate activation date OK
*        certificate public key: RSA
*        certificate version: #3
*        subject: C=USA,O=Cloud Foundry,CN=routerSSL
*        start date: Tue, 12 Dec 2017 11:57:50 GMT
*        expire date: Wed, 12 Dec 2018 11:57:50 GMT
*        issuer: C=USA,O=Cloud Foundry,CN=routerCA
*        compression: NULL
* ALPN, server did not agree to a protocol
> GET / HTTP/1.1
> Host: bits-service.<system domain>
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Content-Length: 264
< Content-Type: text/html
< Date: Tue, 12 Dec 2017 15:05:00 GMT
< Server: nginx
< X-Vcap-Request-Id: d4090290-9d51-4a61-450b-e6dbfa992241
<
<html>
<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<center>The plain HTTP request was sent to HTTPS port</center>
<hr><center>nginx</center>
</body>
</html>
* Connection #0 to host bits-service.<system domain> left intact

Fix containerPort for bits-service

@edwardstudy commented on Mon May 06 2019

Hi,

I just found that bats-service exposed as 8888 but containerPort used 4443 which was not matched. It that correct?

@cfdreddbot commented on Mon May 06 2019

✅ Hey edwardstudy! The commit authors and yourself have already signed the CLA.

@cf-gitbot commented on Mon May 06 2019

We have created an issue in Pivotal Tracker to manage this:

https://www.pivotaltracker.com/story/show/165818442

The labels on this github issue will be updated when the story is started.

@mnitchev commented on Fri May 10 2019

Hi,

The containerPort is "primarily informational" as per the Kubernetes docs, but we see why this might be confusing. We'd recommend just removing it or just list all the open ports but since this is the bits-service yaml it's their call

Maybe @kiranjain2 has more info

@edwardstudy commented on Mon May 13 2019

@nimakaviani Hi, I would like to give some of my backgrounds. I wanted to use istio to track scf+eirini. Then I found an issue when enabling istio. The sidecars(istio-proxy) of bits pods could not be ready because they tried to check ports 4443 and failed to get responses: envoy missing listener for inbound application port: 4443 So I changed pod template because this port 8888 is hardcoded in bis config.

So it's better to correct port about what bits want to expose.

How do you think about it?

Another question: Did you know how to change code from bits-service side?

Thank you!

@eric-cartman commented on Mon May 13 2019

@mnitchev / @edwardstudy
Hi Mario,
here are Bits-Team thoughts on this:

The above PR is against a helm file located in eirini-release not in bits-service - by nature we don't have any stakes in this
NOTICE: We, Bits-Service, have no authority to accept PRs for eirini-release
Q: Another question: Did you know how to change code from bits-service side?
Short answer: Open a PR in for bits-service (https://github.com/cloudfoundry-incubator/bits-service) but please keep in mind - your current PR for adjusting the bits.yaml is against eirini-release which is the right place to request a change for any helm related adjustment of bits-service.

Please feel free to chat us up on CF Slack #bits-service in case you need more information.
Regards,
Bits-Service

@mnitchev commented on Tue May 14 2019

Hello @edwardstudy,

Since we will be moving to an external bits-service helm chart located here, it would be better if you move your PR there. You can track when we move to the external helm chart in the GitHub PR or Tracker story.

BR,
@alex-slynko && @mnitchev

Create a release with attached tgz to make inclusion in pipeline easier

CAPI team has started testing the bits-service integration with CC. Creating a GitHub release with an attached BOSH release tarball would make it easier to include into our pipeline. This release could also be included in this ops file to avoid creating a dev release out of band.

Missing instructions to download dependencies locally of bitsgo

Not sure if I missed something, but I had to run glide install in bits-service to download the dependencies.

That's not mentioned in the Readme at all afaik

Support external dns and cert manager

Hi there,

We got a PR cloudfoundry/eirini-release#81 which simplifies certificate management. Since you have extracted bits service helm chart, it makes sense for you to implement this feature. This significantly helps operators to use the bits service as a registry.

cc original PR author @paurczar

spec.containers.image with tags and sha256 checksum breaks CRI-O download

My KubeCF + Eirini deployment to a Kubernetes system running crio version 1.16.1 failed with Failed to pull image / unknown transport errors:

Failed to pull image "eirini/rootfs-downloader:2.32.0@sha256:6ae511688a27a453dcf31bf5a3bd7287ba99233e1586c7aeb78c87a18c68dbe4": rpc error: code = Unknown desc = Invalid image name "docker.io/eirini/rootfs-downloader:2.32.0@sha256:6ae511688a27a453dcf31bf5a3bd7287ba99233e1586c7aeb78c87a18c68dbe4", unknown transport "docker.io/eirini/rootfs-downloader"

I tracked this down to this CRI-O issue: cri-o/cri-o#2351

The image specified for bits-service and rootfs-downloader in templates/bits.yaml has the tag and sha256 appended, which doesn't seem to work with CRI-O. That's set here:

bits-service-release/helm/bits/templates/bits.yaml

Line 118 in bb0e1f4

 image: eirini/bits-service:2.36.0@sha256:4cf84e13890890f5d8443a5e6e129b701d524f51d35c9c4295a0562ed8bb1bb2 

and here:

bits-service-release/helm/bits/templates/bits.yaml

Line 143 in bb0e1f4

 image: eirini/rootfs-downloader:2.32.0@sha256:6ae511688a27a453dcf31bf5a3bd7287ba99233e1586c7aeb78c87a18c68dbe4 

Removing the tags from my copy of the chart and updating allowed the deployment to complete successfully.

Could these values be made configurable from a values.yaml rather than being hard coded in the template? This would provide an easy work-around without end users having to change the chart.

cloudfoundry-incubator / bits-service-release Goto Github PK