maaxyz / maaframework Goto Github PK

View Code? Open in Web Editor NEW

965.0 965.0 116.0 6.18 MB

基于图像识别的自动化黑盒测试框架 | An automation black-box testing framework based on image recognition

License: GNU Lesser General Public License v3.0

CMake 2.47% C 5.62% Python 14.77% C++ 70.17% Thrift 0.10% Batchfile 0.02% Shell 0.11% Dockerfile 0.02% CSS 6.72%

black-box-testing computer-vision

maaframework's People

Contributors

Stargazers

Watchers

maaframework's Issues

MaaFW應用於其他遊戲開發

我的設計需求流程是這樣
1.遊戲大廳頁面，點擊畫面上TAP字樣收取資源

2.點擊X關閉資源頁面

根據說明，我寫了一個Training.json ↓
training.zip

Roi的範圍是用CropRoi工具得到的，嘗試了幾次後都是這樣的錯誤訊息
maa.txt

撰寫的遊戲是idolypride，是一款直向的手遊
前端用的是MaaCommonJS

[Feature Request] "key" 字段支持 key name string

https://github.com/MaaAssistantArknights/MaaFramework/blob/5cbd70f65043de3cd7020d41575d1b1e07df916d/docs/zh_cn/3.3-%E4%BB%BB%E5%8A%A1%E6%B5%81%E6%B0%B4%E7%BA%BF%E5%8D%8F%E8%AE%AE.md?plain=1#L236-L237

要按的键 没理解错的话指的是 adb shell input keyevent 后面的值 <key code number or name>。

PS > adb shell input -h
Usage: input [<source>] [-d DISPLAY_ID] <command> [<arg>...]
...
The commands and default sources are:
...
      keyevent [--longpress|--doubletap] <key code number or name> ... (Default: keyboard)
...

但 fw 将 string 解析为 list<int, > 而不是 <key name>。

测试用例

以下命令应该是等价的

adb shell input keyevent 4
adb shell input keyevent BACK （以此为例用于fw中）
adb shell input keyevent KEYCODE_BACK

日志

 [2023-08-01 20:51:28.926][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | enter 
 [2023-08-01 20:51:29.324][TRC][Px8924][Tx33707][UnitBase.cpp][L76][MaaNS::ControllerNS::Unit::UnitBase::command] [scmd=["adb","-s","192.168.229.218:33445","shell","input keyevent 66"]] [ret=0] [pipe_data.size()=0] [sock_data.size()=0] [duration=397ms] 
 [2023-08-01 20:51:29.325][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | leave, 398ms 
 [2023-08-01 20:51:29.325][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | enter 
 [2023-08-01 20:51:29.636][TRC][Px8924][Tx33707][UnitBase.cpp][L76][MaaNS::ControllerNS::Unit::UnitBase::command] [scmd=["adb","-s","192.168.229.218:33445","shell","input keyevent 65"]] [ret=0] [pipe_data.size()=0] [sock_data.size()=0] [duration=310ms] 
 [2023-08-01 20:51:29.637][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | leave, 311ms 
 [2023-08-01 20:51:29.637][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | enter 
 [2023-08-01 20:51:30.307][TRC][Px8924][Tx33707][UnitBase.cpp][L76][MaaNS::ControllerNS::Unit::UnitBase::command] [scmd=["adb","-s","192.168.229.218:33445","shell","input keyevent 67"]] [ret=0] [pipe_data.size()=0] [sock_data.size()=0] [duration=669ms] 
 [2023-08-01 20:51:30.308][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | leave, 670ms 
 [2023-08-01 20:51:30.308][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | enter 
 [2023-08-01 20:51:31.014][TRC][Px8924][Tx33707][UnitBase.cpp][L76][MaaNS::ControllerNS::Unit::UnitBase::command] [scmd=["adb","-s","192.168.229.218:33445","shell","input keyevent 75"]] [ret=0] [pipe_data.size()=0] [sock_data.size()=0] [duration=705ms] 
 [2023-08-01 20:51:31.014][TRC][Px8924][Tx33707][TapInput.cpp][L64][MaaNS::ControllerNS::Unit::TapKeyInput::press_key] | leave, 706ms

[TODO] 轮廓匹配

[TODO] Win32Controller

[TODO] 单次 Load 中不允许同名 task

[Bug] MaaControllerSetOption部分实现遗漏

MaaCtrlOption_DefaultAppPackageEntry 和 MaaCtrlOption_DefaultAppPackage没有实现

[RFC] 通用资源包定义

主要是考虑做一个通用的GUI而不是一大堆CLI工具

一个资源包是一个zip
文件结构
- /resource/XXX 加载的资源
- /resource.json 资源配置文件
- /custom.json 自定义识别器/动作配置
- /control.json 控制配置文件

`resource.json`

{
  "resource": {
    "[资源ID]": {
      "name": "[显示名称]",
      "description": "[完整描述]",
      "extends": [ "[资源ID]" ],
      "path": "[资源路径]"
    }
  },
  "app": {
    "start": "[启动入口]",
    "stop": "[停止入口]",
    "orientation": "portrait|landscape",
    "size": {
      "long": "[长边长度]",
      "short": "[短边长度]"
    }
  }
}

`custom.json`

待定

`control.json`

{
  "option": {
    "[选项ID]": {
      "name": "[选项名称]",
      "type": "checkbox",
      "inject": [ "[JSON路径]" ],
      "case": {
        "true": {},
        "false": {}
      }
    },
    "[选项ID]": {
      "name": "[选项名称]",
      "type": "select_string|select_number",
      "default": "[默认值]",
      "inject": [ "[JSON路径]" ],
      "case": [
        {
          "name": "[选项名称]",
          "value": "[选项值]",
          "provide": {}
        }
      ]
    },
    "[选项ID]": {
      "name": "[选项名称]",
      "type": "input_string|input_number",
      "default": "[默认值]",
      "inject": [ "[JSON路径]" ]
    },
  },
  "entry": [
    {
      "name": "[入口名称]",
      "task": "[入口任务ID]",
      "option": [
        "[选项ID]"
      ]
    }
  ]
}

[TODO] 正则替换使用 wstring

framework的快速开始文档里链接失效

Pipeline JSON 文件部分的demo 与json schema 链接 404

[TODO] AdbControlUnit config 各字段添加默认参数

[TODO] 乱七八糟的整理

产物包添加 binding
产物包添加 sample
cmake 为 RPC CLI / test 添加选项
单独的冒烟及各种测试

[Enhancement] Debug 模式下 ocr 识别出错时，也保存截图至 Vision

现在没保存截图，识别出错了也不知道卡在哪里

 [2023-09-19 16:04:56.376][INF][Px21696][Tx31628][PipelineTask.cpp][L109][MaaNS::TaskNS::PipelineTask::find_first_and_run] Task hit: Click_BaseDefense_Partial [result.rec_result.box=[273 x 36 from (946, 224)]] 
 [2023-09-19 16:04:56.376][DBG][Px21696][Tx31628][Actuator.cpp][L16][MaaNS::TaskNS::Actuator::run] [task_data.name=Click_BaseDefense_Partial] | enter 
 [2023-09-19 16:04:56.376][DBG][Px21696][Tx31628][Actuator.cpp][L230][MaaNS::TaskNS::Actuator::sleep] ready to sleep 200ms [interval=200ms] 
 [2023-09-19 16:04:56.577][DBG][Px21696][Tx31628][Actuator.cpp][L239][MaaNS::TaskNS::Actuator::sleep] end of sleep 200ms [interval=200ms] 
 [2023-09-19 16:04:56.578][DBG][Px21696][Tx50175][TapInput.cpp][L29][MaaNS::ControllerNS::UnitNs::TapTouchInput::click] | enter 
 [2023-09-19 16:04:56.578][DBG][Px21696][Tx50175][TapInput.cpp][L33][MaaNS::ControllerNS::UnitNs::TapTouchInput::click] [x=1087] [y=241] 
 [2023-09-19 16:04:56.640][DBG][Px21696][Tx50175][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","-s","127.0.0.1:7555","shell","input tap 1087 241"]] [ret=0] [pipe_data.size()=0] [sock_data.size()=0] [duration=61ms] 
 [2023-09-19 16:04:56.640][DBG][Px21696][Tx50175][TapInput.cpp][L29][MaaNS::ControllerNS::UnitNs::TapTouchInput::click] | leave, 62ms 
 [2023-09-19 16:04:56.640][DBG][Px21696][Tx31628][Actuator.cpp][L230][MaaNS::TaskNS::Actuator::sleep] ready to sleep 500ms [interval=500ms] 
 [2023-09-19 16:04:57.146][DBG][Px21696][Tx31628][Actuator.cpp][L239][MaaNS::TaskNS::Actuator::sleep] end of sleep 500ms [interval=500ms] 
 [2023-09-19 16:04:57.146][DBG][Px21696][Tx31628][Actuator.cpp][L16][MaaNS::TaskNS::Actuator::run] | leave, 770ms 
 [2023-09-19 16:04:57.147][DBG][Px21696][Tx31628][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] [cur_task_name_=Click_BaseDefense_Partial] [list=[
    "Start_SweepMission_Partial",
    "Sub_Click_EnterButton_OnClearedMission_Partial",
    "Stop"
]] | enter 
 [2023-09-19 16:04:57.597][DBG][Px21696][Tx50175][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","-s","127.0.0.1:7555","exec-out","screencap | gzip -1"]] [ret=0] [pipe_data.size()=1456620] [sock_data.size()=0] [duration=449ms] 
 [2023-09-19 16:04:57.623][DBG][Px21696][Tx31628][PipelineTask.cpp][L170][MaaNS::TaskNS::PipelineTask::find_first] recognize: Start_SweepMission_Partial 
 [2023-09-19 16:04:57.709][ERR][Px21696][Tx31628][OCRer.cpp][L74][MaaNS::VisionNS::OCRer::predict_det_and_rec] wrong ocr_result size [ocr_result.boxes=null] [ocr_result.boxes.size()=0] [ocr_result.text=[
    "SF"
]] [ocr_result.text.size()=1] [ocr_result.rec_scores=[
    0.357468
]] [ocr_result.rec_scores.size()=1] 
 [2023-09-19 16:04:57.709][DBG][Px21696][Tx31628][OCRer.cpp][L25][MaaNS::VisionNS::OCRer::analyze] Start_SweepMission_Partial Raw: [results=[]] [param_.model=] [costs=85ms] 
 [2023-09-19 16:04:57.709][DBG][Px21696][Tx31628][OCRer.cpp][L31][MaaNS::VisionNS::OCRer::analyze] Start_SweepMission_Partial Proc: [results=[]] [expected=[
    "Mission Info",
    "MissionInfo"
]] [param_.model=] [costs=86ms] 
 [2023-09-19 16:04:57.710][DBG][Px21696][Tx31628][PipelineTask.cpp][L170][MaaNS::TaskNS::PipelineTask::find_first] recognize: Sub_Click_EnterButton_OnClearedMission_Partial 
 [2023-09-19 16:04:57.957][ERR][Px21696][Tx31628][OCRer.cpp][L74][MaaNS::VisionNS::OCRer::predict_det_and_rec] wrong ocr_result size [ocr_result.boxes=null] [ocr_result.boxes.size()=0] [ocr_result.text=[
    ""
]] [ocr_result.text.size()=1] [ocr_result.rec_scores=[
    0.000000
]] [ocr_result.rec_scores.size()=1] 
 [2023-09-19 16:04:57.958][DBG][Px21696][Tx31628][OCRer.cpp][L25][MaaNS::VisionNS::OCRer::analyze] Sub_Click_EnterButton_OnClearedMission_Partial Raw: [results=[]] [param_.model=] [costs=248ms] 
 [2023-09-19 16:04:57.958][DBG][Px21696][Tx31628][OCRer.cpp][L31][MaaNS::VisionNS::OCRer::analyze] Sub_Click_EnterButton_OnClearedMission_Partial Proc: [results=[]] [expected=[
    "Enter"
]] [param_.model=] [costs=248ms] 
 [2023-09-19 16:04:57.958][DBG][Px21696][Tx31628][PipelineTask.cpp][L170][MaaNS::TaskNS::PipelineTask::find_first] recognize: Stop 
 [2023-09-19 16:04:57.958][DBG][Px21696][Tx31628][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] | leave, 811ms 
 [2023-09-19 16:04:57.958][INF][Px21696][Tx31628][PipelineTask.cpp][L109][MaaNS::TaskNS::PipelineTask::find_first_and_run] Task hit: Stop [result.rec_result.box=[0 x 0 from (0, 0)]]

这个技术是否可应用到网页进行识别

抱歉在这里发了不相关的话题，但是我很想知道这个是否能支持excel读取并进行网页识别填写个人因为公司需要刷工单所以有这个想法

[TODO] 连接断开后的重试等操作

[Bug] 非 base 资源加载后 "template" 字段未能 fallback

现在的Base里的json如果写"template": "Buttons/Home.png"，加载了EN后，EN找不到 template 不会 fallback 去 Base 中找。

python api 无法加载 ControlUnit 动态库

大佬您好，我在尝试看您sample/python下的代码，遇到了一些问题，如果您有空的话，可以解答一下吗：

自己尝试时，这里在建立连接报错：RuntimeError: Failed to create ADB controller（实际端口port是从bluestacks.conf中读取的，且检查过没问题）
https://github.com/MaaAssistantArknights/MaaFramework/blob/5cd4cc945e395edfca47a7b4096b1885fcb807da/sample/python/__main__.py#L14
debug发现，似乎是self._handle为空：
https://github.com/MaaAssistantArknights/MaaFramework/blob/5cd4cc945e395edfca47a7b4096b1885fcb807da/source/binding/Python/maa/controller.py#L313-L324
3.而在library.py中，确实没有看到函数Library.framework.MaaAdbControllerCreateV2的定义。Library.framework是类型WinDLL : ...， Library应该是正常加载了（Library.open(...)），请问是我哪里理解错了吗

目前是想先用python和Bluestacks 5建立连接，再去看流程怎么搭建，windows 10平台。然而卡在连接上了，希望大佬能看一下，谢谢！

[Bug] StopTask 停不下来

不要停下来啊（

测试样本：json 如下，Sub_Start 和 End 的 times_limit 均设置为0

    "TacticalChallenge": {
        "next": [
            "Sub_Start_TacticalChallenge_Partial",
            "Sub_Goto_TacticalChallenge"
        ]
    },
    "Sub_Start_TacticalChallenge_Partial": {
        "doc": "times_limit setted in code",
        "is_sub": true,
        "times_limit": 5,
        "recognition": "OCR",
        "text": "TacticalChallenge",
        "roi": [ 0, 0, 450, 150 ],
        "next": [
            "End_TacticalChallenge_Partial",
            "Click_RankByTheFront_Partial"
        ],
        "runout_next": [
            "End_TacticalChallenge_Partial"
        ]
    },
    "End_TacticalChallenge_Partial": {
        "doc": "times_limit setted in code, 0 -> TacticalChallenge0",
        "recognition": "OCR",
        "text": "0/5",
        "roi": [ 167, 427, 128, 121 ],
        "next": [
            "Sub_End_TacticalChallenge_Partial"
        ],
        "runout_next": [
            "TacticalChallenge0"
        ]
    },
    "TacticalChallenge0": {
        "times_limit": 1,
        "next": [
            "Click_ClaimButton_ForCreditPoints_Partial",
            "Sub_Goto_TacticalChallenge"
        ],
        "runout_next": [
            "StopTask"
        ]
    },
    "StopTask": {
        "action": "StopTask"
    }

 [2023-08-15 18:41:06.432][TRC][Px13472][Tx62517][PipelineTask.cpp][L222][MaaNS::TaskNS::PipelineTask::find_first] [cur_task_name_=TacticalChallenge0] [list=[
    "StopTask"
]] | enter 
 [2023-08-15 18:41:07.248][TRC][Px13472][Tx2772][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","-s","192.168.1.110:33445","exec-out","screencap | gzip -1"]] [ret=0] [pipe_data.size()=2435087] [sock_data.size()=0] [duration=815ms] 
 [2023-08-15 18:41:07.290][TRC][Px13472][Tx62517][PipelineTask.cpp][L227][MaaNS::TaskNS::PipelineTask::find_first] recognize: StopTask 
 [2023-08-15 18:41:07.290][TRC][Px13472][Tx62517][PipelineTask.cpp][L222][MaaNS::TaskNS::PipelineTask::find_first] | leave, 858ms 
 [2023-08-15 18:41:07.290][INF][Px13472][Tx62517][PipelineTask.cpp][L197][MaaNS::TaskNS::PipelineTask::find_first_and_run] Task hit: StopTask [result.rec.box=Rect(0, 0, 0, 0)] 
 [2023-08-15 18:41:07.290][TRC][Px13472][Tx62517][PipelineTask.cpp][L372][MaaNS::TaskNS::PipelineTask::start_to_act] [act.task_data.name=StopTask] | enter 
 [2023-08-15 18:41:07.290][TRC][Px13472][Tx62517][PipelineTask.cpp][L594][MaaNS::TaskNS::PipelineTask::sleep] ready to sleep 200ms [interval=200ms] 
 [2023-08-15 18:41:07.500][TRC][Px13472][Tx62517][PipelineTask.cpp][L603][MaaNS::TaskNS::PipelineTask::sleep] end of sleep 200ms [interval=200ms] 
 [2023-08-15 18:41:07.500][INF][Px13472][Tx62517][PipelineTask.cpp][L400][MaaNS::TaskNS::PipelineTask::start_to_act] Action: StopTask 
 [2023-08-15 18:41:07.500][TRC][Px13472][Tx62517][PipelineTask.cpp][L372][MaaNS::TaskNS::PipelineTask::start_to_act] | leave, 210ms 
 [2023-08-15 18:41:07.501][INF][Px13472][Tx62517][PipelineTask.cpp][L68][MaaNS::TaskNS::PipelineTask::run] breakpoints pop [top_bp=TacticalChallenge] [next_list=[
    "Sub_Start_TacticalChallenge_Partial",
    "Sub_Goto_TacticalChallenge"
]] 
 [2023-08-15 18:41:07.501][TRC][Px13472][Tx62517][PipelineTask.cpp][L222][MaaNS::TaskNS::PipelineTask::find_first] [cur_task_name_=StopTask] [list=[
    "Sub_Start_TacticalChallenge_Partial",
    "Sub_Goto_TacticalChallenge"
]] | enter

[breaking change] 重写 swipe 相关 APIs （不影响 pipeline）

MaaFW功能詢問!

有這樣的功能嗎
指定區域內不含有a.png 跟 b.png的選項隨機挑選他

範例 :
我想要選擇區域內，亮著的人

於是我的判斷想要設計成，不去選上面有「仕事中」跟「元氣最大」的人
或者循序點擊每張圖，點滿三個人就好(但是有灰底這要怎麼判斷?)

還是瑪麗有什麼推薦的設計作法?

[TODO] Python binding

[TODO] cv imshow

[TODO] namespace for pipeline

[TODO] Testing 里检查运行结果、以及更多的测试！

TemplateMatch 匹配圖片得分偏低

模擬器解析度 : 720 x 1280
圖片使用的是BS5自帶的截圖功能

參考圖片 : 原圖.zip

前端用的是nekosu寫的MaaCommonJS
寫了一個training.json去執行
MaaFW目前使用的是fix: vcxproj #526 版本

預期設計 :

在大廳點擊Tap的按鈕收取資源
點X回到大廳

目前問題 :
使用新的CropRoi工具裁切出了Tap.png 跟 tapexit.png 兩張圖
但是使用TemplateMatch 的方式，得分都會偏低，只有0.2~0.3，無法判別

log請參考
log.zip

[TODO] TaskObserver

[BUG] MaaTouch not work

[TODO] 文档

[TODO] 分离 test 和 sample，在 CI 中利用 test 进行一些测试

depends on #79

[BUG] ControlUnit 动态库不能正常卸载

[TODO] DebugController

[Enhancement] 在竖屏状态下启动横屏软件

目前版本遇到竖屏，框架会直接退出。
相关日志：

[TRC][Px28952][Tx8280][UnitBase.cpp][L76][MaaNS::ControllerNS::Unit::UnitBase::command] [scmd=["adb","-s","192.168.1.110:33445","exec-out","screencap | gzip -1"]] [ret=0] [pipe_data.size()=1397107] [sock_data.size()=0] [duration=777ms] 
[ERR][Px28952][Tx8280][ScreencapHelper.cpp][L88][MaaNS::ControllerNS::Unit::ScreencapHelper::decode_raw] screencap size image [im_width=1080] [im_height=1920] don't match [width_=1920] [height_=1080] 
[INF][Px28952][Tx8280][ScreencapHelper.cpp][L46][MaaNS::ControllerNS::Unit::ScreencapHelper::process_data] data is not empty, but image is empty 
[INF][Px28952][Tx8280][ScreencapHelper.cpp][L52][MaaNS::ControllerNS::Unit::ScreencapHelper::process_data] try to cvt lf 
break

预期

能配置待测软件宽高比
遇到不匹配的宽高比，尝试从先启动 DefaultAppPackageEntry 再重新匹配宽高比

[Problem] 文档为空

2.1-回调协议.md
文档没有内容

[Bug] 当入口点任务 enabled 为 false 时，会死循环

 [2023-09-26 17:55:14.801][DBG][Px16184][Tx13697][MaaAPI.cpp][L723][MaaPostTask] [inst=000001A380047DB0] [entry=Scrimmage] [param={}] | enter 
 [2023-09-26 17:55:14.801][INF][Px16184][Tx13697][InstanceMgr.cpp][L89][MaaNS::InstanceMgr::post_task] [entry=Scrimmage] [param={}] 
 [2023-09-26 17:55:14.801][DBG][Px16184][Tx13697][TaskDataMgr.cpp][L30][MaaNS::TaskNS::TaskDataMgr::set_param] [param={
}] | enter 
 [2023-09-26 17:55:14.801][DBG][Px16184][Tx13697][TaskDataMgr.cpp][L30][MaaNS::TaskNS::TaskDataMgr::set_param] | leave, 0ms 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx13697][InstanceMgr.cpp][L110][MaaNS::InstanceMgr::post_task] [id=1] 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx47649][InstanceMgr.cpp][L284][MaaNS::InstanceMgr::run_task] [id=1] [task_ptr=000001A3824391E0] | enter 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx13697][MaaAPI.cpp][L723][MaaPostTask] | leave, 0ms 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx47649][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] [callback_=00007FFB128632A4] [callback_arg_=0000000000000000] [msg=Task.Started] [details={
    "entry": "Scrimmage",
    "hash": "",
    "id": 1,
    "name": "Scrimmage",
    "uuid": "d149b2a589ca5dd1"
}] | enter 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx47649][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] | leave, 0ms 
 [2023-09-26 17:55:14.802][INF][Px16184][Tx47649][InstanceMgr.cpp][L301][MaaNS::InstanceMgr::run_task] task start: [details={
    "entry": "Scrimmage",
    "hash": "",
    "id": 1,
    "name": "Scrimmage",
    "uuid": "d149b2a589ca5dd1"
}] 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx47649][PipelineTask.cpp][L21][MaaNS::TaskNS::PipelineTask::run] [entry_=Scrimmage] | enter 
 [2023-09-26 17:55:14.802][DBG][Px16184][Tx47649][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] [cur_task_name_=Scrimmage] [list=[
    "Scrimmage"
]] | enter 
 [2023-09-26 17:55:15.158][DBG][Px16184][Tx3182][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","-s","127.0.0.1:7555","exec-out","screencap | gzip -1"]] [ret=0] [pipe_data.size()=385398] [sock_data.size()=0] [duration=355ms] 
 [2023-09-26 17:55:15.174][INF][Px16184][Tx3182][ScreencapHelper.cpp][L58][MaaNS::ControllerNS::UnitNs::ScreencapHelper::process_data] end_of_line is LF 
 [2023-09-26 17:55:15.174][DBG][Px16184][Tx3182][ControllerMgr.cpp][L375][MaaNS::ControllerNS::ControllerMgr::check_and_calc_target_image_size] Re-calc image target size: [image_target_long_side_=0] [image_target_short_side_=720] [cur_width=688] [cur_height=387] 
 [2023-09-26 17:55:15.174][INF][Px16184][Tx3182][ControllerMgr.cpp][L401][MaaNS::ControllerNS::ControllerMgr::check_and_calc_target_image_size] [image_target_width_=1280] [image_target_height_=720] 
 [2023-09-26 17:55:15.177][DBG][Px16184][Tx47649][PipelineTask.cpp][L170][MaaNS::TaskNS::PipelineTask::find_first] recognize: Scrimmage 
 [2023-09-26 17:55:15.177][DBG][Px16184][Tx47649][PipelineTask.cpp][L173][MaaNS::TaskNS::PipelineTask::find_first] Task disabled: Scrimmage 
 [2023-09-26 17:55:15.177][DBG][Px16184][Tx47649][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] | leave, 374ms 
 [2023-09-26 17:55:15.177][DBG][Px16184][Tx47649][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] [cur_task_name_=Scrimmage] [list=[
    "Scrimmage"
]] | enter 
 [2023-09-26 17:55:15.386][DBG][Px16184][Tx3182][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","-s","127.0.0.1:7555","exec-out","screencap | gzip -1"]] [ret=0] [pipe_data.size()=385396] [sock_data.size()=0] [duration=209ms] 
 [2023-09-26 17:55:15.395][DBG][Px16184][Tx47649][PipelineTask.cpp][L170][MaaNS::TaskNS::PipelineTask::find_first] recognize: Scrimmage 
 [2023-09-26 17:55:15.395][DBG][Px16184][Tx47649][PipelineTask.cpp][L173][MaaNS::TaskNS::PipelineTask::find_first] Task disabled: Scrimmage 
 [2023-09-26 17:55:15.395][DBG][Px16184][Tx47649][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] | leave, 218ms 
 [2023-09-26 17:55:15.395][DBG][Px16184][Tx47649][PipelineTask.cpp][L165][MaaNS::TaskNS::PipelineTask::find_first] [cur_task_name_=Scrimmage] [list=[
    "Scrimmage"
]] | enter

[Bug] SyncContext run_recognizer 貌似不执行？

[TODO] 通过命令行子进程的方式启动 Custom 相关任务

[TODO] Set log level by API

[Enhancement] 让流水线能够从子任务中结束

子任务需要一个 is_done 属性字段，让流水线直接结束。

is_done 默认值如下：

if is_sub == true:
    is_done = false;

if is_sub == false:
    is_done = true;

例如以下任务需要 is_done：

graph TD;
    任务-->正常运行-->完成;
    任务-->Sub_遇到游戏报错;
    Sub_遇到游戏报错-->Warning_继续运行-->任务;
    Sub_遇到游戏报错-->Done_Error-->notify-->完成;

适用于待测软件经常报错且错误类型多种多样，能够将报错检测或某一类任务整合起来。

目前版本是这样处理报错的，需要将每个 Warning 和 Error 的检测任务添加到 next 字段:

graph TD;
    任务-->正常运行-->完成;
    任务-->Sub_Warning1-->任务;
    任务-->Sub_Warning234-->任务;
    任务-->Error1-->notify;
    任务-->Error234-->notify;
    notify-->完成;

[TODO] 动态加载 ControlUnit

版本一致性检查
Load 的原子性

想要問問萌新如何參與這個項目

純開發方面是個新手有什麼可以做的?

另外這個項目未來會有什麼low code的開發方式嗎，例如

簡單的gui介面
簡單的圖像辨識
簡單的連接裝置

[Feature request] 需要一个字段用来指示任务链运行到终点时，是否失败

当前返回 MaaStatus_Failed 的貌似只有任务超时

[TODO] 软件声音识别

获取声音
声音转光谱模板匹配
其他声音识别算法

[Bug] Maa Touch read info timeout

maa.log
不懂捏

[TODO] GPU ORT

[Feature request] Matcher 详细识别信息

[TODO] xfeatures2d

[Bug] times_limit 字段需要重置行为

如果一个入口task到结束时，某个Task达到了times_limit，然后cli post 了另一个入口task，那这个Task的times_limit也不会被重置？

[Feature Request] 双指缩放/放大

[Bug] timeout_next没有正常工作

简要说明：
MiHoYoLogo 应当在 timeout 之后执行 timeout_next 里的 RestartBH3，但没有正常进行

[Feature Request] Color Match

放个想法在这：）
或者上彩色直方图匹配（

`ColorMatch`

颜色匹配，即“找色”。

该任务属性需额外部分字段：

roi: array<int, 4> | list<array<int, 4>>
识别区域坐标。可选，默认 [0, 0, 0, 0]，即全屏。
四个值分别为 [x, y, w, h]。
color: string | list<string, >
颜色的 hex 代码，如 #000000。必选。
threshold: double | list<double, >
颜色占比阈值，即 color 在 roi 中的占比。可选，默认 1.0 。
若为数组，长度需和 template 数组长度相同。
total_threshold: double
总颜色占比阈值，即 colors 在 roi 中的占比。可选，默认 1.0 。
method: int
模板匹配算法。可选。可参考 https://www.compuphase.com/cmetric.htm
method_threshold: double
模板匹配算法的颜色匹配阈值，即颜色的相近程度。可选，默认 1.0 。

Originally posted by @moomiji in #33

[Bug] 无法通过数据线连接实体设备

 [2023-10-06 01:19:12.523][DBG][Px25668][Tx15577][MaaToolKitAPI.cpp][L43][MaaToolKitFindDeviceWithAdb] | enter 
 [2023-10-06 01:19:12.524][DBG][Px25668][Tx15577][DeviceMgr.cpp][L31][MaaNS::ToolKitNS::DeviceNS::DeviceMgr::find_device_with_adb] [adb_path=adb] | enter 
 [2023-10-06 01:19:12.524][DBG][Px25668][Tx15577][DeviceMgr.cpp][L40][MaaNS::ToolKitNS::DeviceNS::DeviceMgr::request_adb_serials] [adb_path=adb] | enter 
 [2023-10-06 01:19:12.525][DBG][Px25668][Tx15577][ControlUnitMgr.cpp][L86][MaaNS::ControllerNS::UnitNs::create_adb_device_list_obj] [adb_path=adb] [config={"command":{"Abilist":["{ADB}","-s","{ADB_SERIAL}","shell","getprop ro.product.cpu.abilist | tr -d '\n\r'"],"ChmodBin":["{ADB}","-s","{ADB_SERIAL}","shell","chmod 700 \"/data/local/tmp/{BIN_WORKING_FILE}\""],"Click":["{ADB}","-s","{ADB_SERIAL}","shell","input tap {X} {Y}"],"Connect":["{ADB}","connect","{ADB_SERIAL}"],"Devices":["{ADB}","devices"],"ForwardSocket":["{ADB}","-s","{ADB_SERIAL}","forward","tcp:{FOWARD_PORT}","localabstract:{LOCAL_SOCKET}"],"InvokeApp":["{ADB}","-s","{ADB_SERIAL}","shell","export CLASSPATH=\"/data/local/tmp/{APP_WORKING_FILE}\"; app_process /data/local/tmp {PACKAGE_NAME}"],"InvokeBin":["{ADB}","-s","{ADB_SERIAL}","shell","export LD_LIBRARY_PATH=/data/local/tmp/; \"/data/local/tmp/{BIN_WORKING_FILE}\" {BIN_EXTRA_PARAMS}"],"KillServer":["{ADB}","kill-server"],"NetcatAddress":["{ADB}","-s","{ADB_SERIAL}","shell","cat /proc/net/arp | grep : "],"Orientation":["{ADB}","-s","{ADB_SERIAL}","shell","dumpsys input | grep SurfaceOrientation | grep -m 1 -o -E [0-9]"],"PressKey":["{ADB}","-s","{ADB_SERIAL}","shell","input keyevent {KEY}"],"PullFile":["{ADB}","-s","{ADB_SERIAL}","pull","/data/local/tmp/{TEMP_FILE}","{DST_PATH}"],"PushBin":["{ADB}","-s","{ADB_SERIAL}","push","{BIN_PATH}","/data/local/tmp/{BIN_WORKING_FILE}"],"Resolution":["{ADB}","-s","{ADB_SERIAL}","shell","dumpsys window displays | grep -o -E cur=+[^\\ ]+ | grep -o -E [0-9]+"],"SDK":["{ADB}","-s","{ADB_SERIAL}","shell","getprop ro.build.version.sdk | tr -d '\n\r'"],"ScreencapEncode":["{ADB}","-s","{ADB_SERIAL}","exec-out","screencap -p"],"ScreencapEncodeToFile":["{ADB}","-s","{ADB_SERIAL}","shell","screencap -p > \"/data/local/tmp/{TEMP_FILE}\""],"ScreencapRawByNetcat":["{ADB}","-s","{ADB_SERIAL}","exec-out","screencap | nc -w 3 {NETCAT_ADDRESS} {NETCAT_PORT}"],"ScreencapRawWithGzip":["{ADB}","-s","{ADB_SERIAL}","exec-out","screencap | gzip -1"],"StartApp":["{ADB}","-s","{ADB_SERIAL}","shell","am start -n {INTENT}"],"StopApp":["{ADB}","-s","{ADB_SERIAL}","shell","am force-stop {INTENT}"],"Swipe":["{ADB}","-s","{ADB_SERIAL}","shell","input swipe {X1} {Y1} {X2} {Y2} {DURATION}"],"UUID":["{ADB}","-s","{ADB_SERIAL}","shell","settings get secure android_id"]},"prebuilt":{"maatouch":{"package":"com.shxyke.MaaTouch.App","root":"./MaaAgentBinary/maatouch"},"minicap":{"arch":["x86","armeabi-v7a","armeabi"],"root":"./MaaAgentBinary/minicap","sdk":[31,29,28,27,26,25,24,23,22,21,19,18,17,16,15,14]},"minitouch":{"arch":["x86_64","x86","arm64-v8a","armeabi-v7a","armeabi"],"root":"./MaaAgentBinary/minitouch"}}}] | enter 
 [2023-10-06 01:19:12.525][DBG][Px25668][Tx15577][ControlUnitMgr.cpp][L86][MaaNS::ControllerNS::UnitNs::create_adb_device_list_obj] | leave, 1ms 
 [2023-10-06 01:19:12.525][DBG][Px25668][Tx15577][DeviceList.cpp][L16][MaaNS::ControllerNS::UnitNs::DeviceList::request_devices] | enter 
 [2023-10-06 01:19:12.570][DBG][Px25668][Tx15577][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","devices"]] [ret=0] [pipe_data.size()=69] [sock_data.size()=0] [duration=44ms] 
 [2023-10-06 01:19:12.570][DBG][Px25668][Tx15577][UnitBase.cpp][L79][MaaNS::ControllerNS::UnitNs::UnitBase::command] stdout output:
List of devices attached

11451419	device

127.0.0.1:16384	device





 [2023-10-06 01:19:12.570][INF][Px25668][Tx15577][DeviceList.cpp][L43][MaaNS::ControllerNS::UnitNs::DeviceList::request_devices] [devices_=[
    "11451419",
    "127.0.0.1:16384"
]] 
 [2023-10-06 01:19:12.570][DBG][Px25668][Tx15577][DeviceList.cpp][L16][MaaNS::ControllerNS::UnitNs::DeviceList::request_devices] | leave, 45ms 
 [2023-10-06 01:19:12.570][DBG][Px25668][Tx15577][DeviceMgr.cpp][L40][MaaNS::ToolKitNS::DeviceNS::DeviceMgr::request_adb_serials] | leave, 46ms 
 [2023-10-06 01:19:12.571][DBG][Px25668][Tx15577][DeviceMgr.cpp][L31][MaaNS::ToolKitNS::DeviceNS::DeviceMgr::find_device_with_adb] | leave, 47ms 
 [2023-10-06 01:19:12.571][DBG][Px25668][Tx15577][MaaToolKitAPI.cpp][L43][MaaToolKitFindDeviceWithAdb] | leave, 48ms 
 [2023-10-06 01:19:21.116][DBG][Px25668][Tx15577][AdbController.cpp][L17][MaaNS::ControllerNS::AdbController::AdbController] [adb_path_=adb] [address_=11451419] [unit_mgr_=0000026A82A25E80] [unit_mgr_->activity_obj()=0000026A82A69710] [unit_mgr_->connection_obj()=0000026A82A68FC0] [unit_mgr_->device_info_obj()=0000026A8298D9F0] [unit_mgr_->key_input_obj()=0000026A82A25680] [unit_mgr_->screencap_obj()=0000026A82A68C60] [unit_mgr_->touch_input_obj()=0000026A82A2CA30] 
 [2023-10-06 01:19:21.116][DBG][Px25668][Tx15577][MaaAPI.cpp][L340][MaaAdbControllerCreate] | leave, 2ms 
 [2023-10-06 01:19:21.117][DBG][Px25668][Tx15577][MaaAPI.cpp][L417][MaaControllerPostConnection] [ctrl=0000026A82894BB0] | enter 
 [2023-10-06 01:19:21.117][DBG][Px25668][Tx15577][MaaAPI.cpp][L417][MaaControllerPostConnection] | leave, 0ms 
 [2023-10-06 01:19:21.117][DBG][Px25668][Tx35654][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] [callback_=00007FF9BAC33324] [callback_arg_=0000000000000000] [msg=Controller.Action.Started] [details={
    "id": 1,
    "uuid": ""
}] | enter 
 [2023-10-06 01:19:21.120][DBG][Px25668][Tx35654][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] | leave, 3ms 
 [2023-10-06 01:19:21.121][DBG][Px25668][Tx35654][Connection.cpp][L14][MaaNS::ControllerNS::UnitNs::Connection::connect] | enter 
 [2023-10-06 01:19:23.817][DBG][Px25668][Tx35654][UnitBase.cpp][L76][MaaNS::ControllerNS::UnitNs::UnitBase::command] [scmd=["adb","connect","11451419"]] [ret=0] [pipe_data.size()=83] [sock_data.size()=0] [duration=2696ms] 
 [2023-10-06 01:19:23.817][DBG][Px25668][Tx35654][UnitBase.cpp][L79][MaaNS::ControllerNS::UnitNs::UnitBase::command] stdout output:
cannot resolve host '11451419' and port 5555: 不知道这样的主机。 (11001)



 [2023-10-06 01:19:23.818][DBG][Px25668][Tx35654][Connection.cpp][L14][MaaNS::ControllerNS::UnitNs::Connection::connect] | leave, 2697ms 
 [2023-10-06 01:19:23.818][DBG][Px25668][Tx35654][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] [callback_=00007FF9BAC33324] [callback_arg_=0000000000000000] [msg=Controller.ConnectFailed] [details={
    "adb": "adb",
    "address": "11451419",
    "why": "ConnectFailed"
}] | enter 
 [2023-10-06 01:19:23.818][DBG][Px25668][Tx35654][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] | leave, 0ms 
 [2023-10-06 01:19:23.818][ERR][Px25668][Tx35654][AdbController.cpp][L65][MaaNS::ControllerNS::AdbController::_connect] failed to connect 
 [2023-10-06 01:19:23.819][DBG][Px25668][Tx35654][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] [callback_=00007FF9BAC33324] [callback_arg_=0000000000000000] [msg=Controller.Action.Failed] [details={
    "id": 1,
    "uuid": ""
}] | enter 
 [2023-10-06 01:19:23.819][DBG][Px25668][Tx35654][Base/MessageNotifier.hpp][L18][MaaNS::MessageNotifier<void (__cdecl*)(char const *,char const *,void *),void *>::notify] | leave, 0ms

maaxyz / maaframework Goto Github PK

maaframework's People

Contributors

Stargazers

Watchers

Forkers

maaframework's Issues

测试用例

resource.json

custom.json

control.json

预期

ColorMatch

Recommend Projects

Recommend Topics

Recommend Org

Jobs

`resource.json`

`custom.json`

`control.json`

`ColorMatch`