fastly / go-fastly Goto Github PK

I'm trying to add GET /wafs/rules to the waf functionality. The response returns and ID as an integer, and jsonapi does not seem to want to unmarshal it and consistently returns:
json: cannot unmarshal number into Go struct field Node.id of type string
I tried editing the payload from Fastly so that the top level ID was a string rather than an int, and jsonapi was then able to unmarshal it fine. I'm not sure the desired way to handle this behavior?
I can provide a more in-depth example if needed.

I have a cache condition with a name containing a character which needs to be urlencoded. (ex: a text/html condition). When I execute the following code which simply lists and tries to delete conditions which are found, it will fail:

        existingCacheSettings, err := client.ListCacheSettings(&fastly.ListCacheSettingsInput{Service: "abcd", Version: 10})
        for _, setting := range existingCacheSettings {
                err := client.DeleteCacheSetting(&fastly.DeleteCacheSettingInput{Service: "abcd", Name: setting.Name, Version: 10})
                if err != nil {
                        return err
                }
        }

The reason it is failing is because it ends up calling the following URL:

/service/abcd/version/10/condition/text/html

(note the unescaped forward-slash)

Any parameters which go into forming an API call URL should be urlescaped to prevent issues such as this. These object names are perfectly valid from Fastly's POV, but will cause go-fastly to malfunction.

I tried to address this in go-fastly by wrapping all URL components which could contain these characters in an escape method from net/url. Unfortunately, name components in the Fastly API may contain not only forward slashes but also whitespace, and net/url has no method to deal with both possible cases. Fixing this in go-fastly may require that we do a bit of a hacky strings.Replace().

The SDK for working with S3 logging appears to be missing support for some properties of the API. Specifically:

• acl is not currently a supported value
• redundancy is missing support for Infrequent Access and One Zone Infrequent Access

This looks like the main file that needs to be updated, but also I am not a Go developer. 😅

In cases where I'd like to disable Fastly caching by default, I would set the default TTL to 0. However, the use of omitempty in the UpdateSettingsInput struct means that the TTL field is omitted entirely if the value is set to 0. A possible solution is to make the field required by removing the omitempty tag or turning DefaultTTL into a pointer. I'm glad to make a PR for either approach but wanted to hear your thoughts.

It would be super useful to support shield option in https://docs.fastly.com/api/config#backend

That would be great, because it could then be exposed to the terraform provider.

The client doesn't support Azure Blob Storage logging. There is a feature request for the terraform-provider-fastly #117 to support Azure Blob Storage logging. As always the functionality needs to be added to the client first.

clonedVersion, err := client.CloneVersion(&fastly.CloneVersionInput{
    Service: service.ID,
    Version: 0,
})

Error: Missing required field 'Version'

I'm sure this is probably systematic..

syslog logging is missing message_type parameter. It's similar to s3 or gce logging.

Since client is a struct, it is impossible to inject and test using the client.
as the goism says accept interfaces, return structs

👋the client doesn't support the override_host field of the backend definition. The initial feature request #163 was open in the Fastly Terraform provider repository. However, the client needs to support the field before it can be added to the provider.

Does this repo support image optimizer API? Thanks!

also relates to https://github.com/terraform-providers/terraform-provider-fastly/issues/239

When I use ListWAFs:

	var serviceID = "derp"

	latest, err := client.ListWAFs(&fastly.ListWAFsInput{
	 	Service: serviceID,
		Version: 19,
	})
	if err != nil {
		log.Fatal(err)
	 	log.Fatal(latest)
	}

I'm getting the following error:

2017/10/03 10:43:52 Only numbers can be parsed as dates, unix timestamps

It seems to be erroring in the ListWAFs unmarshal, I believe because this UTC date format ("2016-09-28 22:52:33 UTC") that is returned in the field last_push is trying to be unmarshalled into *time.Time.

add the ability to batch update a dictionary with duplicates.
so that duplicate already existing key/value pairs are ignored and update a keys value if the key already exists but the value is different.

So essentially a batch upsert dictionary items
https://docs.fastly.com/en/guides/working-with-dictionary-items-using-the-api#upserting-dictionary-items

would this be possible?

It is not filtering by certificate id since there is a typo in the pairing on custom_tls_domain.go line 34

Fastly are introducing a cloud WAF service, currently on limited availability. It would be good for go-fastly to support this.

API docs: https://docs.fastly.com/api/waf

When attempting to request page 0 of ListPrivateKeys, I get the error:

400 - Bad Request:
Title:  Can't fetch keys
Detail: offset must be an integer

Using PageNumber 1 works as expected. Here is a script to reproduce:

import (
	"github.com/fastly/go-fastly/v2/fastly"
)

func main() {
	client, err := fastly.NewClient("apiToken")
	if err != nil {
		// handle error
	}
	var privateKeyList []*fastly.PrivateKey
	pageSize := uint(100)
	page := uint(0)
	for {
		privateKeys, err := client.ListPrivateKeys(&fastly.ListPrivateKeysInput{PageSize: &pageSize, PageNumber: &page})
		if err != nil {
			// Error occurs here
		}
		if len(privateKeys) == 0 {
			break
		}
		for _, cert := range privateKeys {
			privateKeyList = append(privateKeyList, cert)
		}
		page++
	}
	return
}

I'm happy to contribute to fixing this, but wanted to raise an issue to get a consensus on how trivial it actually is.

Would getting the typo fixed in the name cause the need for a major revision change? At least the mapstructure tag is spelled correctly, so its not a big deal, found it when I spelled it correctly and got an error though.

https://github.com/fastly/go-fastly/blob/main/fastly/datacenters.go#L6

The Fastly API supports creating a syslog endpoint with mutual tls by specifying a client cert and key. https://docs.fastly.com/api/logging#api-section-logging_syslog

These fields haven't been implemented in the go-fastly api client yet, which is used by the fastly terraform provider, and therefore can not currently deploy syslog with mutual tls through terraform.

Additionally, upstream docs don't have any information on listing services AFAICT.

Is the assumption to make here that /services always returns all services if no parameters are given?

Using v6.0.1

To reproduce:

• Create a new condition with name abc 123
• Call client.DeleteCondition

I think that there's some double encoding going on - the URL that's eventually hit looks like

https://api.fastly.com/service/<service ID>/version/1/condition/abc%2520123

LatestVersion for me is reporting as 99 when in fact it's 401.

I think this is because when I print out the listed versions I see:

&{Number:4 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}
&{Number:40 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}
&{Number:400 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}
&{Number:401 Comment: ServiceID:123 Active:true Locked:true Deployed:false Staging:false Testing:false}
&{Number:41 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}

The last few iterations are:

&{Number:9 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}
&{Number:90 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}
...others...
&{Number:99 Comment: ServiceID:123 Active:false Locked:true Deployed:false Staging:false Testing:false}

So looks like the Number isn't sorted correctly. Maybe the latestVersion call isn't sorting correctly?

Most responses from Fastly API return the fields CreatedAt, UpdatedAt, DeleteAt however the go-fastly client is not consistently converting them to one type.
Most of the responses are using *time.Time however there are still many just mapping to string. It would be very good to have some consistency around these fields.

This is list of fields with type string.

fastly/acl_entry.go:	DeletedAt string `mapstructure:"deleted_at"`
fastly/acl_entry.go:	CreatedAt string `mapstructure:"created_at"`
fastly/acl_entry.go:	UpdatedAt string `mapstructure:"updated_at"`
fastly/service.go:	CreatedAt     string     `mapstructure:"created_at"`
fastly/service.go:	DeletedAt     string     `mapstructure:"deleted_at"`
fastly/dictionary_item.go:	CreatedAt string `mapstructure:"created_at"`
fastly/dictionary_item.go:	DeletedAt string `mapstructure:"deleted_at"`
fastly/dictionary_item.go:	UpdatedAt string `mapstructure:"updated_at"`
fastly/bigquery.go:	CreatedAt         string `mapstructure:"created_at"`
fastly/bigquery.go:	DeletedAt         string `mapstructure:"deleted_at"`
fastly/event_logs.go:	CreatedAt   string                 `jsonapi:"attr,created_at"`
fastly/acl.go:	DeletedAt string `mapstructure:"deleted_at"`
fastly/acl.go:	CreatedAt string `mapstructure:"created_at"`
fastly/acl.go:	UpdatedAt string `mapstructure:"updated_at"`
fastly/vcl_snippets.go:	DeletedAt string `mapstructure:"deleted_at"`
fastly/vcl_snippets.go:	CreatedAt string `mapstructure:"created_at"`
fastly/vcl_snippets.go:	UpdatedAt string `mapstructure:"updated_at"`
fastly/vcl_snippets.go:	CreatedAt string `mapstructure:"created_at"`
fastly/vcl_snippets.go:	UpdatedAt string `mapstructure:"updated_at"`
fastly/director.go:	DeletedAt string `mapstructure:"deleted_at"`
fastly/director.go:	CreatedAt string `mapstructure:"created_at"`
fastly/director.go:	UpdatedAt string `mapstructure:"updated_at"`
fastly/dictionary.go:	CreatedAt string `mapstructure:"created_at"`
fastly/dictionary.go:	DeletedAt string `mapstructure:"deleted_at"`
fastly/dictionary.go:	UpdatedAt string `mapstructure:"updated_at"`

The format_version field is currently supported by:

• logentries
• s3
• sumologic
• syslog

The go-fastly client also supports the following logging providers:

• ftp
• gcs
• papertrail
• bigquery

All logging providers support the format_version field according to the Fastly API Docs. It would be great if the ftp, gcs, papertrail and the bigquery providers would also support the format_version field in order to use Version 1 log format.

The fastly api provides the http response header Fastly-RateLimit-Remaining
( https://developer.fastly.com/reference/api/ ), which can be used to monitor
how well an app is dealing with api throttling, or as an input to the app's own api call rate limiting.
go-fastly does not currently return this information to the caller.

I will submit a proof of concept pull request showing one low-impact way of exposing this information to apps which need it.

The Fastly API currently supports server side encryption when setting up s3logging, including providing a specific KMS key ID that can be used by the IAM user.

This can be seen in the UI

The fastly client's GetService() method returns a Service struct which includes an ActiveVersion field, however the Fastly API docs for the GET /service/{service_id} endpoint (which the method in question utilizes) does not include an active_version field in its response:

HTTP/1.1 200 OK
Content-Type: application/json

{
  "comment": "",
  "created_at": "2020-04-27T19:40:49.000Z",
  "customer_id": "x4xCwxxJxGCx123Rx5xTx",
  "deleted_at": null,
  "id": "SU1Z0isxPaozGVKXdv0eY",
  "name": "test-service",
  "publish_key": "3c18bd0f5ada8f0cf54724d86c514a8eac4c9b75",
  "type": "vcl",
  "updated_at": "2020-04-27T19:40:49.000Z",
  "versions": [
    {
      "created_at": "2020-04-09T18:14:30.000Z",
      "updated_at": "2020-04-09T18:15:30.000Z",
      "deleted_at": null,
      "active": true,
      "comment": "",
      "deployed": true,
      "locked": false,
      "number": 1,
      "staging": false,
      "testing": false,
      "service_id": "SU1Z0isxPaozGVKXdv0eY"
    }
  ]
}

The consequence of this is that the Service struct advertises an ActiveVersion in the docs (e.g., godoc) although it will never be initialized and will always be equal to the underlying type's zero-value (i.e., 0). The fact that this unused public field is part of the Service struct can be misleading.

(I stumbled upon this issue by using the ActiveVersion in a Service struct which led to downstream errors. The GetServiceDetails() method and its returned ServiceDetail struct worked as expected.)

This all leads to following question:

Should the Service struct's ActiveVersion field be removed?

Removing it does introduce a breaking change. However, any end users rely on the ActiveVersion field from a Service would likely be dealing with unexpected behavior in any case. Note that none of the API's service endpoints return a active_version key in their response. I'd advocate for the field's removal (I did open this issue after all) but I'm curious what the maintainers/community have to say given the breaking change that would be introduced.

👋 the name of a condition can't be updated. Here is an example to prove the behavior of the Fastly API:

GET https://ww.api.fastly.com/service/{{service_id}}/version/{{version_no}}/condition

curl -X GET \
  https://api.fastly.com/service/{{service_id}}/version/{{version_no}}/condition \
  -H 'Fastly-Key: {{token}}'

[
    {
        "statement": "resp.status >= 500 && resp.status < 600",
        "updated_at": "2019-05-08T18:31:45Z",
        "name": "error_response",
        "version": "{{version_no}}",
        "comment": "",
        "type": "RESPONSE",
        "deleted_at": null,
        "priority": "10",
        "service_id": "{{service_id}}",
        "created_at": "2019-05-08T18:31:45Z"
    }
]

PUT https://www.fastly.com/service/{{service_id}}/version/{{version_no}}/condition/error_response

curl -X PUT \
  https://api.fastly.com/service/{{service_id}/version/{{version_no}}/condition/error_response \
  -H 'Fastly-Key: {{token}}' \
  -d '{
	"name": "error_response_updated"
}'

{
    "msg": "Bad request",
    "detail": "Cannot change name"
}

The omitempty in UpdateBackendInput appears to prevent callers from clearing OverrideHost and other fields.

Purge with soft=true is broken

// Purge instantly purges an individual URL.
func (c *Client) Purge(i *PurgeInput) (*Purge, error) {
	if i.URL == "" {
		return nil, ErrMissingURL
	}

	ro := new(RequestOptions)
	ro.Parallel = true
	if i.Soft {
		ro.Headers["Fastly-Soft-Purge"] = "1"
	}

	resp, err := c.Post("purge/"+i.URL, ro)
	if err != nil {
		return nil, err
	}

	var r *Purge
	if err := decodeBodyMap(resp.Body, &r); err != nil {
		return nil, err
	}
	return r, nil
}

this line access a non initialized map maybe! ro.Headers["Fastly-Soft-Purge"] = "1"

The client doesn't support to add a comment to the version creation payload. There is a feature request for the terraform-provider-fastly #126 to support the comment attribute in the request payload. As always the functionality needs to be added to the client first.

Fastly supports 'deliver' in its UI for cache settings, so this provider should add the option as well as cache, pass and restart.

https://docs.fastly.com/en/guides/log-streaming-https is available via the API but has no client interface I can see.

Hi Seth,

I've created an S3 logging endpoint named prefix/xyz. At some point I decided it is no longer needed, so wanted to delete it. According to current implementation DELETE request will be send to /service/SERVICE_ID/version/VERSION/logging/s3/ENDPOINT_NAME, which translates to: /service/SERVICE_ID/version/VERSION/logging/s3/prefix/xyz in this particualr case. The request ends with 404.

To handle this case properly the name has to be url-encoded: /service/SERVICE_ID/version/VERSION/logging/s3/prefix%2xyz

The issue looks similar to #65.

We use glide for dependency management. It would be ideal to have a release tag that we can pin against.

Client.GetService returns Service and according to the documentation it should have ActiveVersion but that is always null. This is because the underlying API for service does not return the version. There is a similar method GetServiceDetails that returns ServiceDetail which will have both ActiveVersion and Version fields. This is confusing as the readers of the documentation would expect Service returned by GetService to have a proper Version set.

The client doesn't support Splunk logging. There is a feature request for the terraform-provider-fastly #130 to support Splunk logging. As always the functionality needs to be added to the client first.

This function attempts to workaround a quirk of the form library where cases of . characters are replaced with %5C..

https://github.com/sethvargo/go-fastly/blob/master/client.go#L164

Problem is when it does this, it replaces intentional cases of \.. For example, a VCL I'm uploading has a \. within a regex, which this code clobbers and replaces with a bare ..

In the README

// Get the latest active version
latest, err := client.LatestVersion(&fastly.LatestVersionInput{
  ServiceID: serviceID,
})
if err != nil {
  log.Fatal(err)
}

The comment states Get the latest active version

Not sure if the expectation is that it would fetch the latest active version, but client.LatestVersion will fetch the latest version, regardless of its state (active or draft).

We should consider providing a helper to get the latest active version - or an update to README to demonstrate the correct way to fetch the active version.

Trying to fetch historical stats without specifying a service fails with source data must be an array or slice, got map.

e.g.

     stats, err := client.GetStats(&fastly.GetStatsInput{
         Field:  "bandwidth",
         From:   "2020-06-01",
         To:     "2020-07-01",
         By:     "day",
         Region: "all",
     })
     if err != nil {
         log.Fatal(err)
     }

I've been toying around with the following modification to stats.go:

diff --git a/fastly/stats.go b/fastly/stats.go
index 26698ef..088147e 100644
--- a/fastly/stats.go
+++ b/fastly/stats.go
@@ -96,10 +96,10 @@ type GetStatsInput struct {
 
 // StatsResponse is a response from the service stats API endpoint
 type StatsResponse struct {
-       Status  string            `mapstructure:"status"`
-       Meta    map[string]string `mapstructure:"meta"`
-       Message string            `mapstructure:"msg"`
-       Data    []*Stats          `mapstructure:"data"`
+       Status  string              `mapstructure:"status"`
+       Meta    map[string]string   `mapstructure:"meta"`
+       Message string              `mapstructure:"msg"`
+       Data    map[string][]*Stats `mapstructure:"data"`
 }
 
 // GetStats returns stats data based on GetStatsInput

But I'm not going to raise that in a pull request as it fundamentally changes the response structure and obviously needs prior discussion.

For this API:

POST:

{
"data": {
"type": "tls_certificate",
"attributes": {
"cert_blob": "-----BEGIN CERTIFICATE-----\n...\n-----END CERTIFICATE-----\n",
"name": "My certificate"
}
}
}

👋The example in README.md assumes that at least one backend has already been added to the service. At least one domain and one backend definition are required in order to be able to pass the validation check and to activate a version. The example code won't work as expected if someone uses a service without any backend definitions.

This leads to too many open files error.

According to the net/http package docs, the client must close the response body when finished with it.

Sorry if this is not the right way to go about asking, but could I ask you to please tag a release version?

I have a simple app which instantiates a single fastly.Client and performs actions over a period of time. As time goes on, the app fills up with more and more dead socket connections to the Fastly API. Eventually the app hits the FD limit and can no longer perform actions.

So, it would seem that something in go-fastly is leaking sockets. I'm digging around to see if I can find the cause.

It would be nice to add the ability to make a fake client for tests and mocking, so that we can use go-fastly in code that has mocked testing.

This line alters keys that begin with a slash or contain more than one slash in a row.

E.g.1: A call to
/service/yourservice/purge//keyStartingWithASlash
becomes
/service/yourservice/purge/keyStartingWithASlash

E.g.2: A call to
/service/yourservice/purge/keyWithDouble//Slash
becomes
/service/yourservice/purge/keyWithDouble/Slash

It appears the intended behavior at this line is something more along the lines of "add or remove slashes as needed at the joined location when merging the host path with the api path" or in code: 57dedaf

Tangentially related:
Due to #14, it's worth noting that Fastly explicitly states that keys should not be URL encoded. If/when a fix for #14 does URL encode data parameters to the Fastly API, that fix must not URL encode cache keys.

NewRealtimeStatsClient pulls the API key out of an environment variable, whereas NewClient gets it from a function param. It also panics instead of returning an error (unlike NewClient), but perhaps there's some reason for that.

Since this module has very recently changed hands, this might be something worth considering breaking while everyone is already going through and updating import paths.