Comments (5)
from funboost.
使用eta也存在该报错
from funboost.
不知道你的代码是怎么写的,没有贴出你的代码来。
from funboost.
已经解决了,顺带问一下,为什么启动消费者后,网络流出速率非常高
消费者代码如下:
import asyncio
import os
import sys
from fastapi_libs.ctx.app_ctx import app_context
from fastapi_libs.script_tool.script_alive import script_alive
from fastapi_libs.asyncmsg import MsgEngine
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
from wk_schedule import app
from wk_schedule.constants.enums import AsyncMsg
@script_alive(__file__.split('/')[-1])
async def run():
# 心跳检测
await MsgEngine().send(AsyncMsg.heartbeat, delay=30, script_name="batch_create_task")
async def work():
async with app_context(app):
from wk_schedule.logic.notify_service_logic import delay_notify_service_logic
delay_notify_service_logic.consume()
await run()
if __name__ == "__main__":
loop = asyncio.get_event_loop()
task = asyncio.ensure_future(work())
loop.run_until_complete(task)
任务代码如下:
@boost("notify_service_queue", broker_kind=BrokerEnum.REDIS_ACK_ABLE, concurrent_mode=ConcurrentModeEnum.ASYNC, max_retry_times=1)
async def notify_service_logic(data_block: dict):
if await is_time_out(data_block.get("current_overtime")):
logger.error(f"process data[{data_block.get('data_block')} - {data_block.get('block_sn')}] timeout, current_overtime {data_block.get('current_overtime')}")
return
client = await DrainerClient.get(data_block.get('data_type'))
if client is None:
logger.error(f"process data[{data_block.get('data_type')} - {data_block.get('block_sn')}] error, not box process")
return
# 通知下游服务时记录本次任务触发事件(last_try_time)
data_block['last_try_time'] = datetime.strftime(datetime.now(), '%Y-%m-%d %H:%M:%S')
bean = DataBlock()
await bean.init_bean(data_block)
state, info = await client.execute(bean)
if state not in DATA_BLOCK_STATE_RESULT:
raise ReturnResultValueError(state)
# 注册更新任务状态
update_data_block_logic.apply_async({"data_block": data_block, "state": state})
大致逻辑是延迟通知下游服务器
from funboost.
boost装饰器可以加个qps
from funboost.
Related Issues (20)
- [用户自己环境问题]linux 上运行报错 _sqlite3.so: undefined symbol: PyUnicode_FromSTring HOT 4
- 啥时候能支持redis 用户名参数啊 HOT 4
- 在mac 嘛 HOT 1
- 在M1芯片的Mac操作系统中会出现如下问题 HOT 3
- [BUG] 队列名中出现 '/' 符号时,本地日志创建报错 HOT 2
- 新人咨询 HOT 3
- 【bug】redis 参数读取提前了 HOT 5
- [用户自己问题]【bug】工程下的funboost_config.py 经常被覆盖成默认的配置 HOT 3
- qps设置了5但是好像并不精准 HOT 15
- win平台multi_process_consume()消费日志缺少 HOT 45
- windows10在线安装错误,python版本----3.1.0 [没有使用国内源或者翻墙访问pypi]] HOT 2
- ModuleNotFoundError: No module named 'decorator_libs' HOT 4
- 定时任务如何实现分布式, 方式任务重复执行 HOT 39
- [BUG] run_consumer_with_multi_process HOT 1
- 远程的问题 HOT 9
- 建议:项目工程可以搞的正式一点,readme和文档默认成英语,这样能更好的推广到全球 HOT 4
- sqlite作为中间件例子在mac/linux运行,报错 read-only [用户文件夹权限问题,自己修改个文件夹] HOT 4
- 如何快捷的停止multi_process_consume 这个消费函数的所有进程 HOT 3
- base_publisher的publish函数好像有点问题[用户没了解python字典是可变对象] HOT 8
- 发现一个有趣的问题 HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from funboost.