iron-io / iron_go

Iron.io API libraries

Iron.io Go Client Library

IronMQ

IronMQ is an elastic message queue for managing data and event flow within cloud applications and between systems.

The full API documentation is here and this client tries to stick to the API as much as possible so if you see an option in the API docs, you can use it in the methods below.

You can find Go docs here.

Getting Started

Get credentials

To start using iron_go, you need to sign up and get an oauth token.

1. Go to http://iron.io/ and sign up.
2. Create new project at http://hud.iron.io/dashboard
3. Download the iron.json file from "Credentials" block of project

--

Configure

1. Reference the library:

import "github.com/iron-io/iron_go/mq"

2. Setup your Iron.io credentials

3. Create an IronMQ client object:

queue := mq.New("test_queue");

The Basics

Get Queues List

queues, err := mq.ListQueues(0, 100);
for _, element := range queues {
	fmt.Println(element.Name);
}

--

Get a Queue Object

You can have as many queues as you want, each with their own unique set of messages.

queue := mq.New("test_queue");

Now you can use it.

--

Post a Message on a Queue

Messages are placed on the queue in a FIFO arrangement. If a queue does not exist, it will be created upon the first posting of a message.

id, err := q.PushString("Hello, World!")

--

Retrieve Queue Information

info, err := q.Info()
fmt.Println(info.Name);

--

Get a Message off a Queue

msg, err := q.Get()
fmt.Printf("The message says: %q\n", msg.Body)

--

Delete a Message from a Queue

msg, _ := q.Get()
// perform some actions with a message here
msg.Delete()

Be sure to delete a message from the queue when you're done with it.

--

Queues

Retrieve Queue Information

info, err := q.Info()
fmt.Println(info.Name);
fmt.Println(info.Size);

QueueInfo struct consists of the following fields:

type QueueInfo struct {
	Id            string            `json:"id,omitempty"`
	Name          string            `json:"name,omitempty"`
	PushType      string            `json:"push_type,omitempty"`
	Reserved      int               `json:"reserved,omitempty"`
	RetriesDelay  int               `json:"retries,omitempty"`
	Retries       int               `json:"retries_delay,omitempty"`
	Size          int               `json:"size,omitempty"`
	Subscribers   []QueueSubscriber `json:"subscribers,omitempty"`
	TotalMessages int               `json:"total_messages,omitempty"`
	ErrorQueue    string            `json:"error_queue,omitempty"`
}

--

Delete a Message Queue

deleted, err := q.Delete()
if(deleted) { 
  fmt.Println("Successfully deleted")
} else {
  fmt.Println("Cannot delete, because of error: ", err)
}

--

Post Messages to a Queue

Single message:

id, err := q.PushString("Hello, World!")
// To control parameters like timeout and delay, construct your own message.
id, err := q.PushMessage(&mq.Message{Timeout: 60, Delay: 0, Body: "Hi there"})

Multiple messages:

You can also pass multiple messages in a single call.

ids, err := q.PushStrings("Message 1", "Message 2")

To control parameters like timeout and delay, construct your own message.

ids, err = q.PushMessages(
	&mq.Message{Timeout: 60, Delay: 0,  Body: "The first"},
	&mq.Message{Timeout: 60, Delay: 10, Body: "The second"},
	&mq.Message{Timeout: 60, Delay: 10, Body: "The third"},
	&mq.Message{Timeout: 60, Delay: 0,  Body: "The fifth"},
)

Parameters:

• Timeout: After timeout (in seconds), item will be placed back onto queue. You must delete the message from the queue to ensure it does not go back onto the queue. Default is 60 seconds. Minimum is 30 seconds. Maximum is 86,400 seconds (24 hours).

• Delay: The item will not be available on the queue until this many seconds have passed. Default is 0 seconds. Maximum is 604,800 seconds (7 days).

--

Get Messages from a Queue

msg, err := q.Get()
fmt.Printf("The message says: %q\n", msg.Body)

When you pop/get a message from the queue, it is no longer on the queue but it still exists within the system. You have to explicitly delete the message or else it will go back onto the queue after the timeout. The default timeout is 60 seconds. Minimal timeout is 30 seconds.

You also can get several messages at a time:

// get 5 messages
msgs, err := q.GetN(5)

And with timeout param:

messages, err := q.GetNWithTimeout(4, 600)

Touch a Message on a Queue

Touching a reserved message extends its timeout by the duration specified when the message was created, which is 60 seconds by default.

msg, _ := q.Get()
err := msg.Touch()

There is another way to touch a message without getting it:

err := q.TouchMessage("5987586196292186572")

--

Release Message

msg, _ := q.Get()
delay  := 30
err := msg.release(delay)

Or another way to release a message without creation of message object:

delay := 30
err := q.ReleaseMessage("5987586196292186572", delay)

Optional parameters:

• delay: The item will not be available on the queue until this many seconds have passed. Default is 0 seconds. Maximum is 604,800 seconds (7 days).

--

Delete a Message from a Queue

msg, _ := q.Get()
// perform some actions with a message here
err := msg.Delete()

Or

err := q.DeleteMessage("5987586196292186572")

Be sure to delete a message from the queue when you're done with it.

--

Peek Messages from a Queue

Peeking at a queue returns the next messages on the queue, but it does not reserve them.

message, err := q.Peek()

There is a way to get several messages not reserving them:

messages, err := q.PeekN(50)
for _, m := range messages {
  fmt.Println(m.Body)
}

And with timeout param:

messages, err := q.PeekNWithTimeout(4, 600)

--

Clear a Queue

err := q.Clear()

Add an Alert to a Queue

Check out our Blog Post on Queue Alerts.

Alerts have now been incorporated into IronMQ. This feature lets developers control actions based on the activity within a queue. With alerts, actions can be triggered when the number of messages in a queue reach a certain threshold. These actions can include things like auto-scaling, failure detection, load-monitoring, and system health.

You may add up to 5 alerts per queue.

Required parameters:

• type: required - "fixed" or "progressive". In case of alert's type set to "fixed", alert will be triggered when queue size pass value set by trigger parameter. When type set to "progressive", alert will be triggered when queue size pass any of values, calculated by trigger * N where N >= 1. For example, if trigger set to 10, alert will be triggered at queue sizes 10, 20, 30, etc.
• direction: required - "asc" or "desc". Set direction in which queue size must be changed when pass trigger value. If direction set to "asc" queue size must growing to trigger alert. When direction is "desc" queue size must decreasing to trigger alert.
• trigger: required. It will be used to calculate actual values of queue size when alert must be triggered. See type field description. Trigger must be integer value greater than 0.
• queue: required. Name of queue which will be used to post alert messages.

err := q.AddAlerts(
  &mq.Alert{Queue: "new_milestone_queue", Trigger: 10, Direction: "asc",  Type: "progressive"},
  &mq.Alert{Queue: "low_level_queue",     Trigger: 5,  Direction: "desc", Type: "fixed" })

Update alerts in a queue

err := q.AddAlerts(
  &mq.Alert{Queue: "milestone_queue", Trigger: 100, Direction: "asc",  Type: "progressive"})

Remove alerts from a queue

You can delete an alert from a queue by id:

err := q.RemoveAlert("532fdf593663ed6afa06ed16")

Or delete several alerts by ids:

err := q.RemoveAlerts("532f59663ed6afed16483052", "559663ed6af6483399b3400a")

Also you can delete all alerts

err := q.RemoveAllAlerts()

Please, remember, that passing zero of alerts while update process will lead to deleating of all previously added alerts.

q.AddAlerts(
  &mq.Alert{Queue: "alert1", Trigger: 10, Direction: "asc", Type: "progressive"},
  &mq.Alert{Queue: "alert2", Trigger: 5,  Direction: "desc", Type: "fixed" })
info, _ := q.Info() // 2

q.UpdateAlerts()
info, _ = q.Info()  // 0

--

Push Queues

IronMQ push queues allow you to setup a queue that will push to an endpoint, rather than having to poll the endpoint. Here's the announcement for an overview.

Update a Message Queue

queueInfo := mq.QueueInfo{
	//...
} 
info, err := q.Update(queueInfo);

QueueInfo struct consists of following fields:

type QueueInfo struct {
	PushType      string            `json:"push_type,omitempty"`
	RetriesDelay  int               `json:"retries,omitempty"`
	Retries       int               `json:"retries_delay,omitempty"`
	Subscribers   []QueueSubscriber `json:"subscribers,omitempty"`
	// and some other fields not related to push queues
}

The following parameters are all related to Push Queues:

• push_type: Either multicast to push to all subscribers or unicast to push to one and only one subscriber. Default is multicast.
• retries: How many times to retry on failure. Default is 3. Maximum is 100.
• retries_delay: Delay between each retry in seconds. Default is 60.
• subscribers: An array of QueueSubscriber This set of subscribers will replace the existing subscribers. To add or remove subscribers, see the add subscribers endpoint or the remove subscribers endpoint.

QueueSubscriber has the following structure:

type QueueSubscriber struct {
	URL     string            `json:"url"`
	Headers map[string]string `json:"headers,omitempty"`
}

--

Set Subscribers on a Queue

Subscribers can be any HTTP endpoint. push_type is one of:

• multicast: will push to all endpoints/subscribers
• unicast: will push to one and only one endpoint/subscriber

err := q.AddSubscribers(
	"http://mysterious-brook-1807.herokuapp.com/ironmq_push_3", 
	"http://mysterious-brook-1807.herokuapp.com/ironmq_push_4")

--

Get Message Push Status

After pushing a message:

subscribers, err := message.Subscribers()

Returns an array of subscribers with status.

--

Revert Queue Back to Pull Queue

If you want to revert you queue just update push_type to 'pull'.

q.Update(mq.QueueInfo{
	PushType: "pull",	
});

--

Further Links

• IronMQ Overview
• IronMQ REST/HTTP API
• Push Queues
• Other Client Libraries
• Live Chat, Support & Fun

---

© 2011 - 2014 Iron.io Inc. All Rights Reserved.

iron_go's Issues

darwin: config throws `Error getting home directory: exec: "sh": executable file not found in $PATH`

I've been experiencing this now for several months and a co-worker of mine sat down and traced this error all the way back to this line:

https://github.com/iron-io/iron_go/blob/master/config/homedir.go#L57

When building on Darwin it throws the error:

Error getting home directory: exec: "sh": executable file not found in $PATH

I found that if we modified the line to use an absolute path to the sh executable it works like following:

cmd := exec.Command("/bin/sh", "-c", "eval echo ~$USER")

I realize that as that function is Unix specific /bin/sh is not likely going to be the same across all platforms as such I have not submitted a PR.

api client has no timeouts

The iron_go api uses the default http client (and hence the default http.Transport), which has no timeouts.

It would be nice to have at least a Transport.Dial with a timeout, and Transport.ResponseHeaderTimeout (I haven't checked if the latter is fully compatible with the api). When go1.3 can be targeted, an http.Client.Timeout might be a useful option as well.

Increment call doesn't return the new value

Any reason why the Increment call doesn't return the new value?

Relatively easy fix but wanted to reach out before submitting a pull request.

Also might make sense to create a new call as to not break current implementations. Thoughts?

methods with non-pointer as receiver

Is there any reason why methods signature in this library are like:
func (q Queue) ListQueues(page, perPage int)
instead of:
func (q *Queue) ListQueues(page, perPage int)
as widely used in golang world?

"use of closed network connection"

When long polling a queue using queue.GetNWithTimeoutAndWait() i'm consistently seeing "use of closed network connection" errors.

Is there something that i might be doing incorrectly to create these issues?

Thanks!
Joe

How to pass credentials?

How to pass credentials, you have mentioned to download credential.json. But there is no documentation on how to use it. Can you please point to right documentation.

Support for golang.org/x/net/context ?

I think there is a opportunity to fix some of the smell with the methods by letting people specify timeouts, etc. in a standard way.

Also, using this method to handle optional parameters can avoid having to create named overloads like this.

Manually setting IronCache project_id and token?

If I'm switching my cache based on environment, how can I configure iron caches cache.New() to use changing credentials?

undefined field in worker.go: undefined: sleepBetweenRetries

/iron_go/worker/worker.go:145: undefined: sleepBetweenRetries

Change Subscribe method to SetSubscribers and add an AddSubscribers method

AddSubscribers should use this endpoint: http://dev.iron.io/mq/reference/api/#add_subscribers_to_a_queue

Does it make sense to iron_go api to return an error when no messages in the queue?

Hey guys,

Wonderful api and platform you have here. I just wanted to see if you are okay with modifying the iron_go api to not return an error if no messages are in the queue. The logic is simple, if an error is returned than something went wrong, but having 0 messages in the queue is not technically an error, it should just simply result in an empty read with the payload being nil.

Also, wanted to ask if you supported any kind of blocking get when fetching a message in the queue? This way we don't have to poll in a tight loop.

Please add a LICENSE file

I believe it is either BSD like Go and many of its community's packages (and like the JS IronMQ client), but an explicit LICENSE file is much needed.