AMQP compatible backend for the KrakenD framework
krakendio / krakend-amqp Goto Github PK
View Code? Open in Web Editor NEWAMQP compatible backend for the KrakenD framework
Home Page: https://www.krakend.io
License: Apache License 2.0
AMQP compatible backend for the KrakenD framework
Home Page: https://www.krakend.io
License: Apache License 2.0
Hi everyone,
We're using KrakenD as our API Gateway and we're using krakend-amqp to directly publish in our RabbitMq cluster and it keeps losing the connection/channel, and it only recovers rebooting KrakenD.
Our configuration:
"extra_config": {
"github.com/devopsfaith/krakend-amqp/produce": {
"routing_key": "Route",
"exchange": "search",
"durable": true,
"delete": false,
"exclusive": false,
"no_wait": true,
"mandatory": true,
"immediate": false
}
}
I'm trying to understand how to implement an api gateway that deals with microservices that expose rest and rpc endpoints.
We chose Krakend, I implemented the endpoints and configured rabbitmq, and it works, but I don't know how to give to the correct client from the queue. Can I give it back with some key?
@kpacha
first way
here i try set username with a queue and return the new username
{
"endpoint": "/api/v1.0/setUserName/{reply_to}/{mess_id}/{route}",
"method": "POST",
"output_encoding": "json",
"extra_config": {
"github.com/devopsfaith/krakend/proxy": {
"sequential": false
}
},
"backend": [
{
"host": [
"amqp://guest:guest@rabbitMQ:5672/"
],
"url_pattern": "/",
"encoding": "json",
"sd": "static",
"method": "GET",
"disable_host_sanitize": true,
"extra_config": {
"github.com/devopsfaith/krakend-amqp/consume": {
"name": "queue",
"exchange": "exchange",
"durable": true,
"delete": false,
"exclusive": false,
"no_wait": true,
"no_local": false,
"routing_key": [
"#"
],
"prefetch_count": 10
},
"github.com/devopsfaith/krakend-amqp/produce": {
"exchange": "exchange2",
"durable": true,
"delete": false,
"exclusive": false,
"no_wait": true,
"mandatory": true,
"immediate": false,
"name": "queue2",
"reply_to_key":"Reply_to",
"msg_id_key":"Mess_id",
"routing_key":"Route"
}
}
}
]
},
second way
producer
here i try set username with a queue
{
"endpoint": "/api/v1.0/setUserName/{reply_to}/{mess_id}/{route}",
"method": "POST",
"headers_to_pass": ["Content-Type"],
"backend": [
{
"host": [
"amqp://guest:guest@rabbitMQ:5672/"
],
"sd": "static",
"encoding": "json",
"disable_host_sanitize": true,
"extra_config": {
"github.com/devopsfaith/krakend-amqp/produce": {
"name": "queue",
"exchange": "exchange",
"durable": true,
"delete": false,
"exclusive": false,
"no_wait": true,
"mandatory": true,
"immediate": false,
"reply_to_key":"Reply_to",
"msg_id_key":"Mess_id",
"routing_key":"Route"
}
}
}
]
},
consumer
here return the new username
{
"endpoint": "/api/v1.0/consumer/{queue_name}/{route}",
"method": "GET",
"backend": [
{
"host": [
"amqp://guest:guest@rabbitMQ:5672/"
],
"sd": "static",
"encoding": "json",
"disable_host_sanitize": true,
"extra_config": {
"github.com/devopsfaith/krakend-amqp/consume": {
"name": "Queue_name",
"exchange": "exchange2",
"durable": true,
"delete": false,
"exclusive": false,
"internal": false,
"no_wait": true,
"no_local": false,
"routing_key": "Route",
"prefetch_count": 10
}
}
}
]
}
thanks for help)
Hello,
I would like to be able to customize the type of exchange used. In hard, the value is "topic" and I would like to use "direct".
I made the modification on a local branch but I do not have the authorization to push it.
It would be enough simply to add a key "type" in the configuration :
# customer.go
# [...]
err = ch.ExchangeDeclare(
cfg.Exchange, // name
cfg.Type, // type
# [...]
# producer.go
# [...]
err = ch.ExchangeDeclare(
cfg.Exchange, // name
cfg.Type, // type
# [...]
# client.go
# [...]
type queueCfg struct {
Name string `json:"name"`
Type string `json:"type"`
# [...]
# client_test.go
# [...]
consumerNamespace: map[string]interface{}{
"name": "queue-1",
"type": "topic",
"exchange": "some-exchange",
# [...]
producerNamespace: map[string]interface{}{
"name": "queue-1",
"type": "topic",
"exchange": "some-exchange",
When do you think ?
PS : Excuse me for my English, I speak French.
Unfortunately I can't send messages to an Exchange configured as type fanout ... inequivalent arg 'type' for exchange 'user.events' in vhost '/': received 'topic' but current is 'fanout'
. In the documentation I can't find a parameter where I can configure the type. Is fanout not supported or am I missing something?
krakend.json
{
"version":3,
"port":8080,
"endpoints":[
{
"endpoint":"/api/user/event/signedUp",
"method":"POST",
"backend":[
{
"extra_config": {
"backend/amqp/producer": {
"name": "user.event.UserSignedUp",
"exchange": "user.events",
"durable": true,
"delete": false,
"no_wait": true,
"no_local": false,
"routing_key": "userSignedUp",
"prefetch_count": 10,
"mandatory": false,
"immediate": false
}
},
"host":[
"amqp://guest:[email protected]:5672/"
],
"disable_host_sanitize":true
}
]
}
]
}
KrakenD
krakend_ce_1 | 2022/04/25 14:15:52 KRAKEND ERROR: [ENDPOINT: /api/user/event/signedUp] Exception (504) Reason: "channel/connection is not open"
krakend_ce_1 | [GIN] 2022/04/25 - 14:15:52 | 500 | 293.667µs | 172.26.0.1 | POST "/api/user/event/signedUp"
rabbitMq
rabbitmq | 2022-04-25 14:15:52.230932+00:00 [error] <0.4687.0> Channel error on connection <0.4678.0> (172.28.0.1:64784 -> 172.28.0.2:5672, vhost: '/', user: 'guest'), channel 1:
rabbitmq | 2022-04-25 14:15:52.230932+00:00 [error] <0.4687.0> operation exchange.declare caused a channel exception precondition_failed: inequivalent arg 'type' for exchange 'user.events' in vhost '/': received 'topic' but current is 'fanout'
When a host is not defined for and endpoint, the middleware will throw a panic since it's assuming all endpoints will have a host backend defined (and it should).
Config example:
{
"version": 2,
"extra_config": {},
"timeout": "3000ms",
"cache_ttl": "300s",
"output_encoding": "json",
"name": "testing",
"endpoints": [
{
"endpoint": "/new2",
"method": "GET",
"extra_config": {},
"output_encoding": "json",
"concurrent_calls": 1,
"backend": [
{
"url_pattern": "/v1/employees",
"encoding": "json",
"sd": "static",
"extra_config": {}
}
]
}
]
}
Error:
[GIN-debug] GET /new1 --> github.com/devopsfaith/krakend/router/gin.CustomErrorEndpointHandler.func1 (3 handlers)
panic: runtime error: index out of range
goroutine 1 [running]:
github.com/devopsfaith/krakend-amqp.backendFactory.initConsumer(0x24c9d80, 0xc0000aa040, 0x24d7080, 0xc000331a40, 0xc0001adbf0, 0xc0004a9530, 0xc000570790, 0x24c9d80, 0xc0000aa040, 0xc0003e1300, ...)
/go/pkg/mod/github.com/devopsfaith/[email protected]/consumer.go:28 +0x159d
github.com/devopsfaith/krakend-amqp.backendFactory.New(0x24c9d80, 0xc0000aa040, 0x24d7080, 0xc000331a40, 0xc0001adbf0, 0xc0004a9530, 0xc000570790, 0xc0003e1300, 0x100)
/go/pkg/mod/github.com/devopsfaith/[email protected]/client.go:35 +0x7a
Hello,
Following the documentation here https://www.krakend.io/docs/backends/amqp-producer/#routing_key the routing key is expected to work with static values.
Unfortunately, this functionality is not supported by the code.
I am willing to open a Pull Request to make the code supporting this. Are you ok with including this functionality?
@kpacha @alombarte is there any alternative to make use of static routing values?
Add ability to override exchange type for the producer, because it is hardcoded to topic
Hi everyone! first of all, awesome products you've been building.
So we're thinking on using KrakenD as our API Gateway, and since we have a message-driven API built on top of RabbitMQ for our internal communication between services, we've been looking into using this plugin to allow external parties to call a rest endpoint and Kraken to transport to AMQP and so on..
Everything looks good, we've installed Kraken as a standalone daemon and the plugin works fine, but as soon as we put together a Docker image (we have everything containerized) we started hitting this error with the exact same configurations for everything:
gateway_1 | [GIN] 2021/01/13 - 13:55:38 | 500 | 170.544µs | 192.168.32.1 | GET /testpub2
gateway_1 | Error #01: Get amqp://minpoc:***@messagebus:5672/: unsupported protocol scheme "amqp"
Running kraken with --debug throws a more descriptive error. Still unsure what's causing it though
[GIN] 2021/01/13 - 14:06:17 | 500 | 66.303µs | 192.168.32.1 | GET /testpub2
Error #01: Exception (504) Reason: "channel/connection is not open"
I haven't been able to find anything anywhere about this, any ideas??
Thanks a lot!
Hello,
I declare this endpoint, with an amqp producer backend:
"endpoint": "/chamallows",
"method": "POST",
"input_headers": ["*", "Cookie"],
"backend": [
{
"host": ["amqp://guest:guest@rabbitmq:5672"],
"disable_host_sanitize": true,
"url_pattern": "/",
"extra_config": {
"backend/amqp/producer": {
"name": "",
"exchange": "sugar-entrance",
"durable": true,
"delete": false,
"no_wait": false,
"no_local": false,
"routing_key": "haribo",
"mandatory": false,
"immediate": false
}
}
}
]
The routing_key: "haribo"
isn't transmitted to rabbitmq. And it seems that only routing_key
from url parameters (with capitalization) is working. But in my case I can't use this feature. I don't see this distinctive characteristic in the doc
And I think the reason of that come from this line: https://github.com/krakendio/krakend-amqp/blob/1948b9d0c1465fddb67a88dfe715816666e1a7f6/producer.go#L124
Am I missing something? Is this behavior intentional? If not, is it a problem that can be quickly solved on your side?
Thanks for your work
Hi everyone! Are there any plans to have this plugin also able to make RPC calls?
As our current PoC is being develop, we're looking into a way for having a REST API exposed to clients (eg: web app or third party devs) while having all our backends behind rabbitmq.
In order to do so, and because the plugin right now only supports fire n forget, we're looking into enabling RPC inside this plugin so we can leverage all Rabbit's flavors (such as autoscaling and so on) instead of plugin directly our services to kraken.
Hope this makes sense, i'll start working on a fork of this repo, but wanted to first check out if you guys had any plans for this.
Thanks again!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.