GithubHelp home page GithubHelp logo

xuxueli / xxl-mq Goto Github PK

View Code? Open in Web Editor NEW
431.0 47.0 228.0 9.25 MB

A lightweight distributed message queue framework.(分布式消息队列XXL-MQ)

Home Page: http://www.xuxueli.com/xxl-mq/

License: GNU General Public License v3.0

Java 99.94% Dockerfile 0.06%
java message-queue broker queue broadcast distributed

xxl-mq's People

Contributors

xuxueli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

xxl-mq's Issues

关于部署的一个小白问题

在server端部署时,是不是除了部署xxl-mq-broker这个子项目,还应该部署xxl-mq-client子项目下的broker类,因为只有这样XxlMqBroker才能通过Spring的依赖注入获得xxlMqMessageDao,并且执行init方法?

初次接触分布式CS架构,所以有些细节不是很熟悉,见笑了

建议手动设定线程池

看了代码里面留有大量的Executors,还是建议手动设定方式。毕竟中间件还是要求更好的质量。

Topic下的待处理消息数据在1W条左右,xxl-rpc会有invoke error

2022-05-28 11:11:33,694 sak-mq-comsumer: INFO -- rpc.remoting.invoker.reference.XxlRpcReferenceBean -- >>>>>>>>>>> xxl-rpc, invoke error, address:192.168.0.150:7080, XxlRpcRequestXxlRpcRequest{requestId='348c4076-2f68-46e0-acf1-1cd760cdb846', createMillisTime=1653707483694, accessToken='null', className='com.xxl.mq.client.broker.IXxlMqBroker', methodName='pullNewMessage', parameterTypes=[class java.lang.String, class java.lang.String, int, int, int], parameters=[site_group_push_product_batch, DEFAULT, 1, 3, 100], version='null'}

目前的xxl-rpc-core: 1.3.0

尝试升级到1.5.0,XxlRpcReferenceBean的构造方法已被移除 netType参数也被移除了,

XxlMqConsumer的代码似乎有问题

我在阅读XxlMqConsumer类的代码时,发现如果注入一个TopicComsumer,XxlMqConsumer类在初始化时会watchTopic,但是在initTopicConsumer方法的136行中,似乎执行了ZK注册consumer的逻辑,而且用的是queueConsumerRespository,这里似乎不需要再注册consumer了吧,因为之前已经watchTopic了,即使需要注册,也应该使用topicConsumerRespository吧?
PS:现在代码的逻辑中,key使用的是consumer的topic名称,所以如果在一台机器上,启动两个监听同一Topic的TopicConsumer是不是会有问题?

broker部署报错

org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /xxl-mq/rpc
at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
at com.xxl.mq.client.rpc.util.ZkServiceRegistry.registerServices(ZkServiceRegistry.java:80)
at com.xxl.mq.client.rpc.netcom.NetComServerFactory$1.run(NetComServerFactory.java:50)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)

为什么基于zk来进行广播呢?

基于zk广播的确简单也容易理解,它的qps大吗? 性能应该不太好吧 ,为什么不在有多个client,让那个server发送client呢 接受到数据处理呢????

XxlMqClient 中代码的一个疑问,似乎同步没有起作用

您好,我在安装并阅读代码时,在xx-mq-client包的XxlMqClient 类中发现getXxlMqService的方法中使用CountDownlatch时,countdown方法在await方法之前,这似乎违背了countDownLatch的使用方法,同步会失效。可以劳烦您解释一下吗?

consumer group 和provider group设计在一起了,是有问题的

  1. provider端发送消息时,需要知道consumer的group,否则消息查询不到,group设计耦合了
  2. 测试用例中可以用,是因为都是DEFAULT
        List<XxlMqMessage> list = xxlMqMessageDao.pullNewMessage(XxlMqMessageStatus.NEW.name(), topic, group, consumerRank, consumerTotal, pagesize);

对应sql

SELECT <include refid="Base_Column_List" />
		FROM xxl_mq_message AS t
		WHERE 	t.topic = #{topic}
			AND t.group = #{group}
			AND t.status = #{newStatus}
			AND t.effectTime <![CDATA[ < ]]> NOW()
			<if test="consumerTotal > 1">
				AND (
					(
						t.shardingId <![CDATA[ = ]]> 0
						AND
						MOD(t.id, #{consumerTotal}) = #{consumerRank}
					)
					OR
					(
						t.shardingId <![CDATA[ > ]]> 0
						AND
						MOD(t.shardingId, #{consumerTotal}) = #{consumerRank}
					)
				)
			</if>
		ORDER BY t.id ASC
		LIMIT #{pagesize}

PR请求Review

加了消息过期机制,发了一个PR,帮忙review下吧

单机部署可以运行,多网卡多机部署失败

多机多网卡部署,Brocker提示注册成功
“>>>>>>>>>>> xxl-rpc registe service item, ...........”
但Service端提示:
“>>>>>>>>>>> xxl-rpc,no address from service:........”
跟作者沟通过,怀疑时zk没注册成功。但把它换到同一个环境下也不行,应该是多网卡问题。希望得到解决。

配置目录问题

/**
     * zk config file
     */
    private static final String ZK_ADDRESS_FILE = "/data/webapps/xxl-conf.properties";

配置地址可否开放出来,可以自定义

在XxlProducer中增加新方法后无法找到该方法

你好,我在com.xxl.mq.client.XxlMqProducer类中增添了新的方法produceCachedMsg,然后在example工程中增加一个controller action调用该方法,但是会报以下错误:
for servlet [springmvc] in context with path [/xxl-mq-example] threw exception [Handler processing failed; nested exception is java.lang.NoSuchMethodError: com.xxl.mq.client.XxlMqProducer.produceCachedMsg(Ljava/lang/String;Ljava/util/Map;)V] with root cause
java.lang.NoSuchMethodError: com.xxl.mq.client.XxlMqProducer.produceCachedMsg(Ljava/lang/String;Ljava/util/Map;)V

然而奇怪的是,如果我在一个tomcat上同时启动broker和example两个项目,那么就可以成功的调用该方法。如果在server1中部署了broker,在server2上部署example,就会报上述错误

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.