vmware-archive / pcf-pipelines Goto Github PK

The OM upload-product command has a a default timeout setting of 1800 seconds. This timeout was not long enough for one of my test environments to upload the ERT causing the pipeline to fail. I hit this issue for the upgrade to 1.9.13 and 1.9.14. I had to manually restart the upgrade-tile job which completed successfully.

https://github.com/pivotal-cf/om/blob/master/docs/upload-product/README.md

Hey, thanks for sharing the pipeline setup! We are currently trying to automate a lot of the infrastructure around CF with Concourse. Do you plan to add a step for om configure-product for the CF ERT? If not, what objections do you see?

looking at manual tasks once could track and quantify the time/effort it takes to upgrade services, application containers (tomcat for instance) and upgrade the OS. are there any capture details as part of these pipelines that could be used to quantify across tile, platform and stemcell upgrades? possibly output in each report as a stepping stone to storing externally to capture.

install-ert pipelines should support mysql scp backup configuration

install-ert pipelines should support mysql s3 backup configuration

When running the azure opsman upgrade (v0.8.0) pipeline we get the following error:

line 2: cannot unmarshal !!map into string
  line 3: cannot unmarshal !!map into string
  line 4: cannot unmarshal !!map into string
  line 5: cannot unmarshal !!map into string
  line 6: cannot unmarshal !!map into string
  line 7: cannot unmarshal !!map into string
  line 8: cannot unmarshal !!map into string
  line 9: cannot unmarshal !!map into string
  line 10: cannot unmarshal !!map into string

When hijacking into the replace-vm container we can see the config.yml seems to contain extraneous curly brackets like this:

azure:
  vhd_image_url: {{}}
  subscription_id: {{longid}}
  ...

Getting below during opsman install running install-pcf/vsphere/pipeline

./pivnet-opsman-product/pcf-vsphere-1.10.3.ova
jq: error: ip0/0 is not defined at , line 6:
(.PropertyMapping[] | select(.Key == ip0)).Value = $ip0 |
jq: error: netmask0/0 is not defined at , line 7:
(.PropertyMapping[] | select(.Key == netmask0)).Value = $netmask0 |
jq: error: gateway/0 is not defined at , line 8:
(.PropertyMapping[] | select(.Key == gateway)).Value = $gateway |
jq: error: DNS/0 is not defined at , line 9:
(.PropertyMapping[] | select(.Key == DNS)).Value = $dns |
jq: error: ntp_servers/0 is not defined at , line 10:
(.PropertyMapping[] | select(.Key == ntp_servers)).Value = $ntpServers |
jq: error: admin_password/0 is not defined at , line 11:
(.PropertyMapping[] | select(.Key == admin_password)).Value = $adminPassword |
jq: error: custom_hostname/0 is not defined at , line 12:
(.PropertyMapping[] | select(.Key == custom_hostname)).Value = $customHostname
jq: 7 compile errors

It looks like the pcf-pipelines/tasks/import-opsman/task.sh file was refactored for this current v0.7.0 tar release.

If you leave ssl cert blank in the params file it attempts to use a nonexistent parameter $OPS_MGR_GENERATE_SSL_ENDPOINT to generate self signed certs.

This fails as it's not defined in the parameters.

When tcp_routing value is set to disable, the tcp_routing_ports field is still required and configuration of the ER tile fails. See screenshot. Ports should not be required when tcp routing is disabled.

## TCP routing and routing services
tcp_routing: disable                   # Enable/disable TCP routing (enable|disable)
tcp_routing_ports:                   # A comma-separated list of ports and hyphen-separated port ranges, e.g. 52135,34000-35000,23478

When disabling tcp routing, left tcp ports blank. Job failed at config-elastic-runtime-tile stage config-er-tile with error:

staging cf 1.11.0-build.49
finished staging
Status: 200 OK
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Tue, 09 May 2017 20:10:08 GMT
Etag: W/"072de635532b82c376c8c599b299e176"
Server: nginx/1.4.6 (Ubuntu)
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: ea690d92-d7f4-483a-ab86-e478f443ad82
X-Runtime: 0.193366
X-Xss-Protection: 1; mode=block
Using self signed certificates generated using Ops Manager...
configuring product...
setting properties
could not execute "configure-product": failed to configure product: request failed: unexpected response:
HTTP/1.1 422 Unprocessable Entity
Transfer-Encoding: chunked
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Tue, 09 May 2017 20:10:10 GMT
Server: nginx/1.4.6 (Ubuntu)
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 39a7affc-fb61-4955-8c67-bbaec5abbb0a
X-Runtime: 0.493735
X-Xss-Protection: 1; mode=block

57
{"errors":{".properties.tcp_routing.enable.reservable_ports":["Value can't be blank"]}}
0

Since the ops mgr settings are not persisted/backed up as part of the pipeline, if the concourse worker container goes away during the deploy for any reason, then there is the risk that you will not be able to put the settings back into ops mgr.

Can we maybe have a backup done as part of the pipeline to s3? or an option for something like that?

Using 0.14.1 release of pcf-pipelines. Running on GCP, attempting to upgrade from Ops Manager 1.10.3 to 1.10.8

The replace-opsman-vm task executes the following command:
cliaas-linux replace-vm --config cliaas-config/config.yml --identifier "ops-manager-1-10-3"

After a couple of minutes, the Ops Manager VM shuts down:

However, the cliaas-linux remains unresponsive until the end of the 10-minute timeout period, and then errors out with:

error: waitforstatus after stopvm failed: polling for status timed out

Getting following error on upload-elastic-runtime-tile

chmod: cannot access 'tool-om/om-linux': No such file or directory

Believe this is due to the folder being changed to om-cli instead of tool-om.

Hijacking the build container shows these directories:
root@25c4a38e-76d9-4945-658e-8916f27ca8cf:/tmp/build/8ac8d33d# ll
total 0
drwxr-xr-x 1 root root 86 Apr 20 02:22 ./
drwxr-xr-x 1 root root 16 Apr 20 02:08 ../
drwxr-xr-x 1 root root 44 Apr 18 18:27 om-cli/
drwxr-xr-x 1 root root 218 Apr 19 23:58 pcf-pipelines/
drwxr-xr-x 1 root root 20 Apr 20 01:25 pivnet-cli/
drwxr-xr-x 1 root root 116 Apr 20 01:22 pivnet-product/

Doing a search of the repo several tasks still call out the old name it appears.

In pcf-pipelines/tasks/config-ert/task.sh

  local azs_csv=$1
  echo $azs_csv | awk -F "," -v braceopen='{' -v braceclose='}' -v name='"name":' -v quote='"' -v OFS='"},{"name":"' '$1=$1 {print braceopen name quote $0 quote braceclose}'
}```

Only gets the first word and not the full az name as vSphere allows spaces.

When using the deploy-director task on GCP it fails with the following:

####################################################################
GETTING JSON FOR: DIRECTOR -> networks <- file ...
####################################################################
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key}[] 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}[]
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key}[] 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}[]
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key}[] 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}[]
jq: 2 compile errors
jq: error: syntax error, unexpected '$' (Unix shell quoting issues?) at <top-level>, line 1:
.${key} 
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.${key}
jq: 2 compile errors

thus the networking and the az tab is not filled in correctly. It fails as well for the following:

Applying changes on Ops Manager @ gcp.domain.com
could not execute "apply-changes": could not check for any already running installation: could not make api request to installations endpoint: token could not be retrieved from target url: Post https://gcp.domain.com/uaa/oauth/token: dial tcp: lookup gcp.domain.com on 8.8.8.8:53: no such host

The opsman Url should be opsman.gcp.domain.com, not gcp.domain.com.

When running the create-infrastructure part of the pipeline, this fails with

The module root could not be found. There is nothing to output.

Changing the terraform output command to

output_json=$(/opt/terraform/terraform output -json -state=terraform-$version.tfstate)

like in https://github.com/pivotal-cf/pcf-pipelines/blob/ae469a2e305871f9735a70b4f73f437a937fb67d/tasks/install-pcf-gcp/create-infrastructure/task.sh#L51

it works.

With commit 3da8645 the upstream aws-concourse repository was deprecated. The majority of the tasks were brought in but the README references pcfaws_terraform_pipeline.yml which does not exist.

Either the README.md is wrong or the file was missed.

From @cholick, originally in gcp-concourse:

GCP installation instructions don't restrict traffic flow to tags, see:
http://docs.pivotal.io/pivotalcf/1-9/customizing/gcp-prepare-env.html#firewall_rules

But the terraformed firewall rule for internal traffic does:
https://github.com/pivotal-cf/gcp-concourse/blob/master/terraform/firewalls.tf#L76-#L77

This creates traffic issues on bosh-deployed / odb vms, which don't get tagged in the same way.

cc @JaredGordon @miyer-pivotal

The Configure-ert job for AWS CF install(tasks/install-ert/configure-ert.sh) is very different from a more generic one here (tasks/config-ert/task.sh). Ideally the latter one should be used. This will also as a side effect fix up #53 as there is a PR in works for the VSphere one.

See ref: https://github.com/pivotal-cf/aws-concourse/issues/12 (that repository is now deprecated).

This is just a copy of the issue.

@bijukunjummen
I noticed that some of the pipeline steps refer to this docker image virtmerlin/c0-worker, wouldn't hosting and using docker images from a team dockerhub a better idea.

@krishicks
We have chores to replace any non-cflinuxfs2 images with either cflinuxfs2 or one based on
cflinuxfs2 that we maintain.

They're here:

https://www.pivotaltracker.com/story/show/144251129
https://www.pivotaltracker.com/story/show/144251109
https://www.pivotaltracker.com/story/show/144251095

The pipeline should first check which ERT version is in use with PCF prior to downloading the latest from PivNet and doing any work. If the in-use ERT version is equal to latest on PivNet, do nothing.

Install pipeline failing with following message.

missing input concourse-vsphere

Looking through the repo this folder path is called out several times but doesn't appear to be present in the repo.

The network name in the params.yml file for property om_vm_network does not support space characters in the value (e.g. VLAN 26). The resulting network name is just VLAN.

The root cause appears to be this line in tasks/import-opsman/task.sh:

  setNetworkMapping $OM_VM_NETWORK

The $OM_VM_NETWORK variable should be quoted.

Ran the build pack pipeline and it only updates to 3.16, not to 4.0, even though 4.0 is out.

https://network.pivotal.io/products/buildpacks#/releases/5169

Not sure if this is intentional or not?

https://github.com/pivotal-cf/pcf-pipelines/tree/master/tasks/upload-product-and-stemcell/task.yml

Still points to :
run:
path: pcf-pipelines/tasks/upload-product/task.sh

needs to be updated to point to :
run:
path: pcf-pipelines/tasks/upload-product-and-stemcell/task.sh

I downloaded the Concourse Automation Pipeline from PivNet to update an instance of PCF. I came across a minor error where the downloaded pcf-pipeline-tarball from pivnet did not match the filename present in the run command of the tarball-unpack task within upgrad-tile job. Same issue with upgrade-buildpack pipeline where the tarball-unpack task in for each buildpack has a mismatch in the pcf-pipeline tarball and the product_version define in the resource section.

• name: pcf-pipelines-tarball
  type: pivnet
  source:
  api_token: {{pivnet_token}}
  product_slug: pcf-automation
  product_version: v0.1.0

  • task: tarball-unpack
    config:
    inputs: [ { name: pcf-pipelines-tarball } ]
    outputs: [ { name: pcf-pipelines } ]
    platform: linux
    image_resource: { type: docker-image, source: {repository: cloudfoundry/cflinuxfs2} }
    run: { path: tar, args: [ "-xvzf", "pcf-pipelines-tarball/pcf-pipelines.tgz" ] }

The last line should be:
run:` { path: tar, args: [ "-xvzf", "pcf-pipelines-tarball/pcf-pipelines-v0.1.0.tgz" ] }

To make it resilient, it would be better if chainging the product_version of pcf-pipeline-tarball does not require changing the run task in the tarball-unpack task.

access to bosh.io is a required firewall rule

https://github.com/pivotal-cf/pcf-pipelines/blob/master/FIREWALL.md

See ref: https://github.com/pivotal-cf/aws-concourse/issues/13

This is just a copy of that issue.

@bijukunjummen
The OPSMAN_URI parameter is likely to be redundant - in some of the pipeline steps it is derived as https://opsman.$ERT_DOMAIN, if by any chance a different OPSMAN_URI is provided the import-stemcell step fails as the url's don't match up. We should either use OPSMAN_URI at all places or remove OPSMAN_URI and derive the url.

@ryanpei
@bijukunjummen so we would replace OPSMAN_URI with ERT_DOMAIN in all places where it is > currently used?

@bijukunjummen
Yes, I think so @ryanpei, OR make use of the OPSMAN_URI everywhere a opsman's url is required instead of deriving it as https://opsman.$ERT_DOMAIN. Former may be a good start though.

Wrong glob reference being used in install-pcf/vsphere/pipeline.yml.

- name: upload-elastic-runtime-tile
  plan:
...
    - get: pivnet-cli
      params: {globs: ["*linux_amd64*"]}

Should be:

- name: upload-elastic-runtime-tile
  plan:
...
    - get: pivnet-cli
      params: {globs: ["*linux-amd64*"]}

Note the dash usage instead of underscore in globs.

This is due to the pivnet cli not being present in the fetch directory.

upload-er-tile:
chmod: missing operand after '+x'
Try 'chmod --help' for more information.

I'm currently investigating this too as it's a blocker for me.

The reference architecture is 3 networks and AZs. Make the scripts smarter to allow for non-reference architectures. Our current need is for 1 network and 1 AZ.

Please take this as a suggestion / constructive feedback after using these for a few months on AWS, Azure, and vSphere

The most easy to debug and customize IaaS for pcf-upgrades and installs is vSphere because of the use of govc. Whereas the other clouds Ops Man pipelines have not once done what I'd expect, and have required many rounds of debugging to tweak my setup to the expectations of cliaas.

While golang as a CLI is great, using golang effectively as a scripting language (as it is here, since there are so many variations in the task) has major maintainability problems in the wild.

These pipelines need to be customized for specific customer environments, and unless cliaas is going to handle almost every permutation of variation, it's a black box to system administrators who are never going to learn golang , so people will wind up writing their own scripted CLI , or IaaS Template (Azure ARM template, AWS CF, etc.) or terraform-based version of the task. I'd much rather have something easily tinker-able like this.

that said if the goal is to have cliaas "just handle it all", kudos, it's just not there for a while.

Examples include

• use of private vs public IPs, #51
• customized disk settings (it uses the VMDK/VHD/AMI defaults which are usually far too low)
• reserved IOPS for disks
• multi-homed NICs for Ops Man
• common post-processing for Ops Man including: TLS certs, LDAP PAM setup

Can we please make the upload-tile task "waiting for opsmgr" message not scroll forever while waiting for a response. It takes ten minutes to time out, so it gets quite long. Thanks

Last night while trying to upgrade to Ops Manager 1.10.8, my pipelines (0.14.1, da072c7) failed in the download-stemcells task with the following error:

Logged-in successfully
pcf-pipelines/tasks/download-pivnet-stemcells/task.sh: line 55: pivnet: unbound variable
pcf-pipelines/tasks/download-pivnet-stemcells/task.sh: line 60: pivnet: unbound variable
Could not find stemcell 3363.24 for vsphere. Did you specify a supported IaaS type for this stemcell version?

(for example if the network failed to be updated in the previous step)

This should fail out, however job shows green.

Configuring networks...
Status: 422 Unprocessable Entity
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Thu, 27 Apr 2017 16:07:57 GMT
Server: nginx/1.4.6 (Ubuntu)
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: 043b0400-32af-478a-91cf-569714e40c44
X-Runtime: 0.175094
X-Xss-Protection: 1; mode=block
{
  "errors": [
    "Networks There is an error on a network.",
    "Networks DEPLOYMENT: Subnets : [0] Availability zone references can't be blank",
    "Availability zones This infrastructure requires an availability zone",
    "Availability zones cannot find availability zone with name PCF"
  ]
}```

I'm unsure if this is specific to the pipeline, om, or Ops Mgr. But I set haproxy static ip, and got the message "errors":["IP '192.168.144.11' is already taken, cannot be taken for 'cf-f7fc90f1f948b5ffffb9_ha_proxy-f7a7297e6fc566c528ea-partition-21a67ea37bbec67c3833'"]} during apply.

deployment_network_name: "DEPLOYMENT"
deployment_vsphere_network: "VM Network"   # vCenter Deployment network name
deployment_nw_cidr: 192.168.128.0/17          # Deployment network CIDR, ex: 10.0.0.0/22
deployment_excluded_range: "192.168.128.1-192.168.144.9, 192.168.159.250-192.168.255.255"    # Deployment network exclusion range
deployment_nw_dns: 192.168.3.14           # Deployment network DNS
deployment_nw_gateway: 192.168.128.1       # Deployment network Gateway
deployment_nw_azs: PCF Cluster           # Comma seperated list of AZ's to be associated with this network

ha_proxy_ips: 192.168.144.111, 192.168.144.112          # Comma-separated list of static IPs

Tried out this project as you deprecated the older gcp-concourse project. However when using the pipeline running the create-infrastructure bit, this fails not finding the terraform command.

pcf-pipelines/tasks/install-pcf-gcp/upload-opsman.sh: line 30: terraform: command not found

I got it to work by changing this value:
https://github.com/pivotal-cf/pcf-pipelines/blob/master/install-pcf/gcp/params.yml#L9

Should that be a CHANGEME?

From @bijukunjummen:

The terraform scripts for provisioning an opsman vm should associate an elastic
ip address with the opsman vm, otherwise the upgrade scripts for opsman tends
to fail as it tries to associate the non-elastic ip to the new instance and
fails.

Moved from https://github.com/pivotal-cf/aws-concourse/issues/14.

Currently, looking at the parameters available in vsphere/upgrade-ops-manager-params.yml, the list seems much smaller than what the OpsManager UI exposes. I'd expect it to be much larger (to recreate the original configuration) or much smaller (configuration imported from previously exported installation).

Is the Upgrade Ops Manager pipeline ready for use on vSphere installations?

Looks like the pipe is downloading something from Rahul Jain's GIT:

Status: Downloaded newer image for rjain/buildbox@sha256:

This should probably only reference materials from C[0] or R&D.

govc import.ova is failing with below message

`Running` govc import.ova -options=opsman_settings.json -name=OpsManager-1.9.3-201702240222 -k=true -u=VCENTER -ds=DS -dc=DC -pool=POOL  -folder=FOLDER /tmp/build/39dbd1bf/pivnet-opsmgr/pcf-vsphere-1.9.3.ova
./govc/govc_linux_amd64: no file specified

I think that something is conflicting. As when I remove all but the below options deploy begins to work correctly.

Running govc import.ova -options=opsman_settings.json -name=OpsManager-1.9.3-201702240222 -k=true  -folder=FOLDER /tmp/build/39dbd1bf/pivnet-opsmgr/pcf-vsphere-1.9.3.ova

[24-02-17 02:56:48] Uploading pivotal-ops-manager-disk1.vmdk... (28%, 16.5MiB/s)

my params.yml

## TCP routing and routing services
tcp_routing: disable                    # Enable/disable TCP routing (enable|disable)
tcp_routing_ports:                  # A comma-separated list of ports and hyphen-separated port ranges, e.g. 52135,34000-35000,23478
route_services: disable                 # Enable/disable route services (enable|disable)

Error during configure-elastic-runtime/config-er-tile:

 staging cf 1.10.7-build.1
finished staging
SSL Configuration:...
Setting CF resources and properties...
configuring product...
setting properties
could not execute "configure-product": failed to configure product: request failed: unexpected response:
HTTP/1.1 422 Unprocessable Entity
Transfer-Encoding: chunked
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Tue, 02 May 2017 01:49:08 GMT
Server: nginx/1.4.6 (Ubuntu)
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Request-Id: f46b568e-4b9c-49b6-ae05-2bc98f493b65
X-Runtime: 0.716472
X-Xss-Protection: 1; mode=block

132
{"errors":{".properties.tcp_routing.enable.reservable_ports":["Value can't be blank"],".properties.security_acknowledgement":["Value can't be blank"],".properties.mysql_backups":["Value 'enable' does not match the select_value of a known option"],".mysql_monitor.recipient_email":["Value can't be blank"]}}
0 

download-stemcell tool should be able to handle mix case like vSphere instead of only requiring lowercase. Notes in the params file do state to look up the iaas name from the repo but it would be nice if it is not case sensitive

This problem happened in two distinct PCF 1.9.2 environments of a customer that deployed the pcf-pipelines (tested with v0.8, v0.11 and v0.13.2) :
the tasks for Apply-Change and Wait-for-Opsmgr of the Upgrade-Tile pipeline both return that a task is already running in OpsMgr even though there is no one started in the Ops Mgr UI. That return code prevents the pipeline from proceeding to the Apply-Changes phase of the upgrade, requiring the customer to use the OpsMgr UI to continue.

The root cause:
We found out that in the OpsMgr's API "installations" output contained a task a couple of months old that was still in "running" state and with no finished_at date (see example below).
That entry caused the tasks mentioned above to incorrectly return that a task is already running (even the apply-changes command of the om tool v0.23 fails because of it) because their methods simply check for the existence of an entry with "running" state.
According to the customer, what seems to have caused that situation was the reboot of the OpsMgr VM after the corresponding running action got stuck. Apparently OpsMgr left that entry unchanged in its installs table after the reboot and never updated it to failed state.

{
"user_name": "admin",
"finished_at": null,
"started_at": "2017-02-01T17:38:42.941Z",
"status": "running",
"additions": [],
"deletions": [],
"updates": [
{
  "identifier": "p-rabbitmq",
   ...
},
...
],
"id": 17
},

Potential solutions:
A) Update both wait-for-opsmgr task.sh and the om tool to parse the recent OpsMgr events json file instead of just searching for an entry with "running" status; OR

B) Provide a Known-Issues readme in the pcf-pipelines package describing the issue above and the workaround below to fix those event entries in the OpsMgr DB:
1) Make a backup copy of OpsMgr settings (add link to docs)
2) SSH to the OpsMgr VM and become root (sudo su -)
3) Switch to postgres user (sudo su postgres)
4) Execute command psql (no password required)
5) Connect to the DB:
> \connect tempest_production
6) Find the id of the task in "running" state
SELECT from installs WHERE status='running';
7) Change the status of the corresponding entry:
UPDATE installs SET status='failed', finished_at='2017-05-04T13:58:39.620Z', finished=true WHERE id='<ID-NUMBER-FROM-PREVIOUS-STEP>';

See ref: https://github.com/pivotal-cf/aws-concourse/issues/8

This is just a copy of the issue.

@wjjk940
https://github.com/c0-ops/om (0.8.1) https://github.com/pivotal-cf/om/releases (0.22.0) — I don’t know why the script isn’t using om [options] configure-bosh []

@ryanpei
this is a good point. we should refactor the pipelines to use om configure-bosh now that this is supported in om master.

It looks like configure-ert step with 1.9.21 of ERT is breaking as there is a new required field in the Authentication and Enterprise SSO of configuration. Likely also with the latest patch version of 1.10.

In params.yml file's trusted_certs property (and ssl_cert property), multi-line values are allowed by YAML. Example:

trusted_certs: |
  -----BEGIN CERTIFICATE-----
  ...
  -----END CERTIFICATE-----

This doesn't translate correctly into JSON during the configure-ops-director/config-opsdir task. JSON does not allow multiline values. Using a multiline yaml value for a certificate causes a 400 Bad Request when configuring security.

The workaround is to make the certificate value a single line value in the yaml file. We used \r\n in place of hard return characters. However, it would be a nice fix to be able to specify a certificate as a multiline value as yaml syntax allows.