[JENKINS-13230] Not able to use Jenkins Environment Variables and variables injected through Env Inject plugin when executing commands over ssh in Publish over SSH plugin about publish-over-ssh-plugin

[jira-importer]

bap:

The following is the output (on a windows master, Jenkins 1.456) from a job that injects variables from "Prepare an environment for the run", "Build Environment" and from an "Inject environment variables" build step.
All of the variables are set using the "Properties Content" textarea.
The variables are all replaced when the "Exec command" is run after the variable has been injected.
When the command is run before the variable has been injected, the variable is not replaced, and can be seen in the command with the dollar sign in front.
This all looks exactly as expected. I can test in a linux environment in a week or so, but I have no reason to expect a different result.
Can you please test using the "Properties Content" textarea and check that the variables are replaced?

Started by user anonymous
[EnvInject] - Preparing an environment for the build.
[EnvInject] - Keep Jenkins system variables.
[EnvInject] - Keep Jenkins build variables.
[EnvInject] - Injecting as environment variables the properties content
injectedFromPrepare=Injected from Prepare

[EnvInject] - Variables injected successfully.
[EnvInject] - Unset unresolved 'PROMPT' variable.
Building in workspace C:\Users\bap\jenkins\1.456\JENKINS_HOME\jobs\test inject ssh\workspace
SSH: Connecting from host [essien]
SSH: Connecting with configuration [localhost] ...
SSH: EXEC: STDOUT/STDERR from command [echo Before build Injected from Prepare && echo Before build $injectedFromBuildEnv && echo Before build $injectedFromBuilder] ...
Before build Injected from Prepare
Before build $injectedFromBuildEnv
Before build $injectedFromBuilder
SSH: EXEC: completed after 200 ms
SSH: Disconnecting configuration [localhost] ...
SSH: Transferred 0 file(s)
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
injectedFromBuildEnv=Injected from Build Environment

[EnvInject] - Variables injected successfully.
[EnvInject] - Unset unresolved 'PROMPT' variable.
SSH: Connecting from host [essien]
SSH: Connecting with configuration [localhost] ...
SSH: EXEC: STDOUT/STDERR from command [echo A build step before inject build step Injected from Prepare && echo A build step before inject build step Injected from Build Environment && echo A build step before inject build step $injectedFromBuilder] ...
A build step before inject build step Injected from Prepare
A build step before inject build step Injected from Build Environment
A build step before inject build step $injectedFromBuilder
SSH: EXEC: completed after 401 ms
SSH: Disconnecting configuration [localhost] ...
SSH: Transferred 0 file(s)
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content
injectedFromBuilder=Injected from Builder

[EnvInject] - Variables injected successfully.
[EnvInject] - Unset unresolved 'PROMPT' variable.
SSH: Connecting from host [essien]
SSH: Connecting with configuration [localhost] ...
SSH: EXEC: STDOUT/STDERR from command [echo A build step after inject build step Injected from Prepare && echo A build step after inject build step Injected from Build Environment && echo A build step after inject build step Injected from Builder] ...
A build step after inject build step Injected from Prepare
A build step after inject build step Injected from Build Environment
A build step after inject build step Injected from Builder
SSH: EXEC: completed after 201 ms
SSH: Disconnecting configuration [localhost] ...
SSH: Transferred 0 file(s)
SSH: Connecting from host [essien]
SSH: Connecting with configuration [localhost] ...
SSH: EXEC: STDOUT/STDERR from command [echo After build Injected from Prepare && echo After build Injected from Build Environment && echo After build Injected from Builder] ...
After build Injected from Prepare
After build Injected from Build Environment
After build Injected from Builder
SSH: EXEC: completed after 200 ms
SSH: Disconnecting configuration [localhost] ...
SSH: Transferred 0 file(s)
SSH: Connecting from host [essien]
SSH: Connecting with configuration [localhost] ...
SSH: EXEC: STDOUT/STDERR from command [echo Publisher Injected from Prepare && echo Publisher Injected from Build Environment && echo Publisher Injected from Builder] ...
Publisher Injected from Prepare
Publisher Injected from Build Environment
Publisher Injected from Builder
SSH: EXEC: completed after 400 ms
SSH: Disconnecting configuration [localhost] ...
SSH: Transferred 0 file(s)
Finished: SUCCESS

[jira-importer]

skumar:

I tested by setting variables using the "Properties Content" textarea. It did not work for me. I am using Jenkins 1.456 on a RHEL Linux machine.

You can see from below that MODULE_TEST and DEPLOY_DIRECTORY are injected during build time but not replaced during promotion.

Build Log:

Started by user anonymous
[EnvInject] - Preparing an environment for the build.
[EnvInject] - Keep Jenkins system variables.
[EnvInject] - Keep Jenkins build variables.
[EnvInject] - Evaluation the following Groovy script content:
if (MODULE_NAME==null)

{ return null; }

if ("ACCT".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-AM"] ; return map }

if ("ATTCH".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "Attachments"] ; return map }

if ("CCM".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-CCM"] ; return map }

if ("DNG".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-DNG"] ; return map }

if ("DR".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-DR"] ; return map }

if ("FPD".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-FPD"] ; return map }

if ("FPD_PRINT".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-FPD"] ; return map }

if ("RDM".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-RDM"] ; return map }

if ("REPORTS".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-Reports"] ; return map }

if ("UMS".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-UMS"] ; return map }

if ("WARRANT".equals(MODULE_NAME))

{ def map = [MODULE_HOME: "SSP-Warrant"] ; return map }

Building in workspace /usr/AutomatedBuilds/SSP-AM
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
MODULE_TEST=SSP-AM
DEPLOY_DIRECTORY=/usr/J2EE/CAS/deployment

[EnvInject] - Variables injected successfully.
[SSP-AM] $ /bin/bash /tmp/hudson27958.sh
MODULE_NAME: ACCT
SVN_REVISION: HEAD
BUILD_TYPE: DB_SCRIPTS
DBCHANGELOG_FILES: dbchangelog-0-sspbl-0000.xml

Archiving artifacts
Finished: SUCCESS

Promotion Log:

build BapSshPromotionPublisherPlugin[delegate=BapSshPublisherPlugin[consolePrefix=SSH: ,delegate=BPInstanceConfig[publishers=[BapSshPublisher[configName=DEV BIG-IP,verbose=false,transfers=[BapSshTransfer[sourceFiles=scripts/Environments/SSPDEV/*,excludes=/_jdbc.properties,removePrefix=scripts/Environments/,remoteDirectory=,remoteDirectorySDF=false,flatten=false,cleanRemote=false,execCommand=chmod 755 SSPDEV -R
./bigip_jenkins_predeployment.sh SSPDEV/sspdev_module_pools.meta 10.0.90.36 $MODULE_NAME,execTimeout=120000,pseudoTty=false]],useWorkspaceInPromotion=false,usePromotionTimestamp=true,retry=,label=,credentials=]],continueOnError=false,failOnError=true,masterNodeName=,alwaysPublishFromMaster=false,paramPublish=]]] SUCCESS
SSH: Connecting from host [SSP-DEV-BUILDS.gce2000.com]
SSH: Connecting with configuration [SSP Dev Appserver] ...
SSH: EXEC: STDOUT/STDERR from command [echo "MODULE_TEST:$MODULE_TEST"
echo "DEPLOY_DIRECTORY:$DEPLOY_DIRECTORY"

cd /J2EE/CAS/deploy/ACCT
chmod 775 scripts source Distribution -R
sudo -u appsuser -i /J2EE/CAS/deploy/ACCT/scripts/jenkins_deploy.sh ACCT DB_SCRIPTS SSP dbchangelog-0-sspbl-0000.xml] ...
MODULE_TEST:
DEPLOY_DIRECTORY:

SSH: Disconnecting configuration [SSP Dev Appserver] ...
SSH: Transferred 3 file(s)
build BapSshPromotionPublisherPlugin[delegate=BapSshPublisherPlugin[consolePrefix=SSH: ,delegate=BPInstanceConfig[publishers=[BapSshPublisher[configName=SSP Dev Appserver,verbose=false,transfers=[BapSshTransfer[sourceFiles=Distribution/BizServices/applications/.ear, scripts/jenkins_deploy.sh, scripts/Environments/SSPDEV/, source/db/*/,excludes=,removePrefix=,remoteDirectory=$MODULE_NAME,remoteDirectorySDF=false,flatten=false,cleanRemote=false,execCommand=echo "MODULE_TEST:$MODULE_TEST"
echo "DEPLOY_DIRECTORY:$DEPLOY_DIRECTORY"

cd /J2EE/CAS/deploy/$MODULE_NAME
chmod 775 scripts source Distribution -R
sudo -u appsuser -i /J2EE/CAS/deploy/$MODULE_NAME/scripts/jenkins_deploy.sh $MODULE_NAME $BUILD_TYPE SSP $DBCHANGELOG_FILES,execTimeout=240000,pseudoTty=true]],useWorkspaceInPromotion=false,usePromotionTimestamp=true,retry=,label=,credentials=]],continueOnError=false,failOnError=true,masterNodeName=,alwaysPublishFromMaster=false,paramPublish=]]] SUCCESS

[jira-importer]

bap:

Ah! It's in a promotion.
Try prefixing the variable names with promotion_, eg. $promotion_MODULE_NAME

[jira-importer]

skumar:

Bap,
Tried $promotion_MODULE_TEST and $promotion_DEPLOY_DIRECTORY. No luck! It did not replace the values.

[jira-importer]

bap:

The injected variables are not replaced when the promotion executes.
You need to either execute the script again during the promotion, or store the variables in a file, archive it, and then inject again during the promotion.
When the variables are injected during the promotion, they will need the promotion_ prefix.

[jira-importer]

skumar:

It would be nice if the injected variables are also replaced like the build parameters. Not sure why this facility is given for only build parameters and not other environment variables.

[jira-importer]

bap:

All environment variables are restored - the variables that envInject provides are build variables, not environment variables.
The publish over plugins are not aware of the EnvInject plugin, so do not know how to read and restore the variables that it had added to a build.

[jira-importer]

gbois:

@bap
EnvInject variables are build process environment variables, jenkins variables and build variables (parameterized, jenkins build variables, all contributions).

publih-over-ssh has to be aware of EnvInject with the following code.

• Get EnvInjetAction
  EnvInjectActionRetriever#getEnvInjectAction(build)
• Get Environment variables in the EnvInjectAction
  EnvInjectAction#getEnvMap()
• Use these variables to resolve your values

[jira-importer]

bap:

@Gbois
I was just explaining to Suresh why the variables were not restored - the publish over plugins would need to be "EnvInject aware" and have code written to specifically access the stored variables.
Thanks for the code - but it is the publish over plugins that don't know how to read and restore the variables not me

[jira-importer]

skumar:

Greg and Bap,
Just wanted to follow up with you to know if the publish-over-ssh plugin has been made "EnvInject aware". With this feature all the variables injected by EnvInject can be used during promotion which enhances the ability of EnvInject Plugin.

[jira-importer]

bap:

IMO this is a fundamental design problem

Without EnvInject, the build environment can be accessed directly.

If someone installs EnvInject then it "hijacks" the environment variables and moves them to its own action - they are no longer available directly from the build object.

This requires every plugin to code for the existence of an optional plugin if it wants to be able to access the environment variables that the EnvInject plugin has moved.

@Gbois is there some method in core that can be called to access the variables that EnvInject would contribute to so that this dependency can be broken?

[jira-importer]

chantivlad:

Kind-of related issue: https://issues.jenkins-ci.org/browse/JENKINS-14340.

[jira-importer]

bap:

@chanti I have no idea what you mean by "natural" environment variables

Your output example looks like it is entirely to do with recursive resolution (JENKINS-14340) and I can't see the connection with this issue which is to do with the EnvInject plugin.

[jira-importer]

chantivlad:

@bap by "natural", i meant standard behaviour of vanilla jenkins. i would have expected the publish-over-ssh plugin to replicate that behaviour. i am surprised that an extra token plugin might be needed for that purpose.
but you are right, after rereading this issue, it might not have anything to do with it.
i will remove my comment and post it on my issue.