Ampersands in Terraform Outputs are being Encoded about terraform-mixin

[sgettys]

The reason for the switch from -raw to -json is because -raw doesn't support complex object types for the output, only string, bool, and number. I've moved over to doing proper JSON unmarshaling for the output string to essentially rip off the json encoding put on there by Terraform. I'll be adding in some more tests to make sure we're not missing another regression for this, thanks for finding and detailing this bug!

[carolynvs]

I am fixing a bug in the mixins today and then cutting new release candidates of everything. You can always build the mixin locally with mage build and then install it into your PORTER_HOME directory with mage install until an updated release candidate is available.

[martinpeck]

A conversation about this issue has taken place on the Terraform repo.

The following are the most useful comments:
hashicorp/terraform#31756 (comment)
hashicorp/terraform#31756 (comment)

The point made in these comments is that it's not a valid use of the -json format switch to take JSON output from Terraform and then parse it as a string, rather than as JSON.

A suggested (untested) code fix has been given that might allow the outputs to be inspected, for true JSON output to then be returned as JSON, but for other output (regular strings) to be returned as their raw value.

[martinpeck]

The use of the -raw switch within this mixin means that JSON encoded strings are being passed as outputs from the bundle, but they are being treated as regular strings. In the case where that string has an ampersand it results in an encoded ampersand being passed back.

[carolynvs]

@martinpeck Thanks for the the background on the problem and apologies for the regression. It's a bit to digest but I'm sure we can get this fixed in the mixin with all the great info from the other issue's comments.

If you are up for trying a PR, great! Otherwise I'll keep it at the top of our backlog and will try to get it fixed in the 1.0 release of the mixin.

[martinpeck]

Thanks @carolynvs. I'll take a look next week and see if I can PR a fix for this.

I'll also take a look at the reasoning behind the switch from -raw to -json, but if you have some context as to why that change was made that might he helpful when it comes to testing.

[carolynvs]

@sgettys can provide more detail, here is the original issue and PR that prompted the switch

[martinpeck]

That's awesome @sgettys !
Thanks!

[martinpeck]

Thanks @sgettys , @VinozzZ , @carolynvs ! Really appreciate the quick turn around.

Can you let me know roughly when this might appear in a release? I'd like to pick this change up for a project I'm working on right now, if possible.

[martinpeck]

Hey there. I see that an RC with this fix is now released. What do we need to do in order to pick up this mixin?
Do we need to upgrade to the latest version of porter, or will older versions of porter install this newer mixin?

[carolynvs]

I recommend installing porter v1.0.0-rc.2 and then install the terraform mixin with porter mixins install terraform --version v1.0.0-rc.1.

The new versions of the mixins probably work with v0 still but really I have never tried. They are intended to work with porter v1.

[martinpeck]

Out of interest...for a fresh install of everything, how does porter determine which mixin versions to install?

update: ignore me....it was a dumb question

[carolynvs]

Just in case anyone else is wondering, here's how it works.

Previously Porter v0 installed a bunch of extra mixins. Now Porter v1's install script only installs porter and the exec mixin (the same version as porter, since they are in the same repo). All other mixins must be explicitly installed.

Also the porter mixins install command installs only the latest stable version which excludes these pre-release versions of the mixins. So you have to explicitly pick the version when installing a pre-release mixin with the --version flag.