joyycom / vnn Goto Github PK
View Code? Open in Web Editor NEWVNN是由欢聚集团(Joyy Inc.)推出的高性能、轻量级神经网络部署框架。目前已为Hago、VOO、VFly、马克相机等App提供20余种AI能力的支持,覆盖直播、短视频、视频编辑等泛娱乐场景和工程场景
License: Other
VNN是由欢聚集团(Joyy Inc.)推出的高性能、轻量级神经网络部署框架。目前已为Hago、VOO、VFly、马克相机等App提供20余种AI能力的支持,覆盖直播、短视频、视频编辑等泛娱乐场景和工程场景
License: Other
public static class VNN_Image {
public long ori_fmt; /* orientation format enum of img*/
public int pix_fmt; /* pixel format enum of img /
public int mode_fmt; / mode format enum of detection, can use video/picture/default /
public int width; //image width
public int height; //image height
public int channels; //image channel
public byte[] data; //cpu data
public float[] rect;
public int[] texture; / if is cpu-backend, texture is NULL. But if is gpu-backend, texture may be a VNN_Texture(android|ios) or a CVPixelbuffer(ios). */
}
SDK支持美颜吗?
怎么保存视频?
调用VNN_Apply_Face_CPU返回0个人脸,但随后调用VNN_Get_Face_Attr(_detection_data)返回一个人脸
这样就非常方便嵌入了
value里面是什么,jacobian里面是使用的什么数据计算得到的,感觉value像是表情参数,jacobian是关键点检测得到的点位数据求jacobian,我猜的对吗
mac M1虚拟机win11上调用vnn库,运行的时候报异常,跑不起来,求大佬解决?
或者SDK代码有开源的计划吗?
hello,请教一下,disney face 和 3D face 若想绘制在bitmap上可以怎么做呢?求指点~
沙发、、Very Good~~
性能很棒,求模型转换工具链以及模型调用接口
Demo也不行,有解决方法吗
同意的+1,希望官方放出源码
disney和3dgame,对输入的图片或者视频帧,有最小尺寸限制吗?
Hi..great app!! Would love to know how to get models into .vnnmodel format so i can use custom trained models..that would be a great extra..as any search into .vnnmodel yields a search result of..u.
Android Studio Flamingo | 2022.2.1 Patch 2
ndk: 21.1.635245
cmake:3.6.4111459
Execution failed for task ':vnnlib:generateJsonModelDebug'.
Build command failed.
Error while executing process .../cmake/3.6.4111459/bin/cmake with arguments {-H/.../VNN-master/demos/Android/vnn_android_demo/vnnlib
请问Linux X86上有性能优化吗
你好,我在使用vnn进行facemask检测时遇到了一些native崩溃,这是我的检测代码:
``
public static Bitmap vnnFaceMask(Bitmap bitmap, boolean recycle, float[] rect) {
Bitmap result = Bitmap.createBitmap(128, 128, Bitmap.Config.ARGB_8888);
result.eraseColor(Color.BLACK);
try {
VNNHelper vnnHelper = new VNNHelper(App.appContext);
vnnHelper.createModels(VNNHelper.VNN_EFFECT_MODE.VNN_FACE_MASK);
VNN.VNN_Image inputImage = new VNN.VNN_Image();
inputImage.width = bitmap.getWidth();
inputImage.height = bitmap.getHeight();
inputImage.data = BitmapUtil.getRGBAFromBitmap(bitmap);
inputImage.ori_fmt = VNN.VNN_OrientationFormat.VNN_ORIENT_FMT_DEFAULT;
inputImage.pix_fmt = VNN.VNN_PixelFormat.VNN_PIX_FMT_RGBA8888;
inputImage.mode_fmt = VNN.VNN_MODE_FMT.VNN_MODE_FMT_PICTURE;
vnnHelper.apply(VNNHelper.VNN_EFFECT_MODE.VNN_FACE_MASK, inputImage, null);
VNN.VNN_ImageArr imageArr = vnnHelper.imageArr;
if (recycle) {
BitmapUtil.recycleBitmap(bitmap);
}
if (imageArr.imgsNum == 0) {
return result;
}
int width = imageArr.imgsArr[0].width;
int height = imageArr.imgsArr[0].height;
int[] pix = new int[width * height];
for (int j = 0; j < pix.length; j++) {
pix[j] = imageArr.imgsArr[0].data[j] == 0 ? Color.BLACK : Color.WHITE;
}
result.setPixels(pix, 0, width, 0, 0, width, height);
if (rect != null) {
System.arraycopy(imageArr.imgsArr[0].rect, 0, rect, 0, 4);
}
vnnHelper.destroyVNN(VNNHelper.VNN_EFFECT_MODE.VNN_FACE_MASK);
} catch (Error e) {
e.printStackTrace();
}
return result;
}
测试环境和崩溃信息如下:
1、
测试机型:oppo A37m
崩溃日志:
E/<<<Native_Capturer>>>: handle signal: signal=11 sigCode=1 errno=0
E/xunwind: #00 pc 00000000000171a8 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #1 pc 000000000000afc0 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #2 pc 0000000000007b44 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #3 pc 0000000000007974 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #4 pc 0000000000005620 /data/app/应用包名-2/lib/arm64/libvnn_face.so (VNN_Apply_Face_CPU+144)
E/xunwind: #5 pc 0000000000003abc /data/app/应用包名-2/lib/arm64/libvnnjni.so (Java_com_duowan_vnnlib_VNN_applyFaceCpu+580)
E/xunwind: #6 pc 000000000217dbd4 /data/dalvik-cache/arm64/data@app@应用包名-2@[email protected]
E/NativeCapturer: crash:
####
handleThread=Thread-3177
crashThread=GLHandlerThread
####
2、这个问题比较奇怪,在某次编译出来的apk包出现这个问题,在未修改检测代码的情况下,重新编译出来的apk未复现这个问题
测试机型:小米11、红米6A
崩溃日志:
2022-02-14 15:20:47.875 30880-32060/? A/libc: Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 32060 (GLHandlerThread), pid 30880 (cordion.prettyo)
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Build fingerprint: 'Xiaomi/venus/venus:11/RKQ1.200928.002/V12.5.13.0.RKBCNXM:user/release-keys'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Revision: '0'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: ABI: 'arm64'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Timestamp: 2022-02-14 15:20:48+0800
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: pid: 30880, tid: 32060, name: GLHandlerThread >>> 应用包名 <<<
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: uid: 14387
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Abort message: 'JNI DETECTED ERROR IN APPLICATION: fid == null
in call to GetLongField
from int com.duowan.vnnlib.VNN.applyFaceCpu(int, com.duowan.vnnlib.VNN$i, com.duowan.vnnlib.VNN$f)'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x0 0000000000000000 x1 0000000000007d3c x2 0000000000000006 x3 0000007d0ce792d0
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x4 000024393f4f282a x5 000024393f4f282a x6 000024393f4f282a x7 2a284f3f3924ffff
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x8 00000000000000f0 x9 a4bd9e266b09fda5 x10 0000000000000000 x11 ffffffc0fffffbdf
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x12 0000000000000001 x13 00000000000000e2 x14 ffff000000000000 x15 ffffffffffffffff
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x16 0000007d9fbd6948 x17 0000007d9fbb5350 x18 0000000000000001 x19 00000000000078a0
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x20 0000000000007d3c x21 00000000ffffffff x22 0000000000000058 x23 0000000000000058
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x24 0000007d1bbd5157 x25 0000000000000001 x26 0000007d1bbec171 x27 0000007d1c1f2000
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x28 b400007d1c6d1c00 x29 0000007d0ce79350
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: lr 0000007d9fb68a60 sp 0000007d0ce792b0 pc 0000007d9fb68a8c pst 0000000000001000
2022-02-14 15:20:48.018 32149-32149/? A/DEBUG: backtrace:
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: NOTE: Function names and BuildId information is missing for some frames due
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: NOTE: to unreadable libraries. For unwinds of apps, only shared libraries
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: NOTE: found under the lib/ directory are readable.
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #00 pc 0000000000089a8c /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: 5f57d25b37c043ed36c0e4147dcc8b3f)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #1 pc 000000000055de88 /apex/com.android.art/lib64/libart.so (art::Runtime::Abort(char const*)+2308) (BuildId: 5b103c304a50c13c7fbeaacc0e0df496)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #2 pc 0000000000013978 /system/lib64/libbase.so (android::base::SetAborter(std::__1::function<void (char const*)>&&)::$_3::__invoke(char const*)+76) (BuildId: 7f81cc97d60164bfae3b52a2cd822f20)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #3 pc 0000000000012fa4 /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+320) (BuildId: 7f81cc97d60164bfae3b52a2cd822f20)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #4 pc 0000000000386344 /apex/com.android.art/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+2572) (BuildId: 5b103c304a50c13c7fbeaacc0e0df496)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #5 pc 00000000003becf4 /apex/com.android.art/lib64/libart.so (art::JNI::GetLongField(_JNIEnv*, _jobject*, _jfieldID*)+1624) (BuildId: 5b103c304a50c13c7fbeaacc0e0df496)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #6 pc 0000000000003904 /data/app/~~tkynBF6I6wYFbvaZ834Qpg==/应用包名-Dgei18UFSwJBh-34YYpIig==/lib/arm64/libvnnjni.so (Java_com_duowan_vnnlib_VNN_applyFaceCpu+140) (BuildId: 7ad79b2d6a5b50e2a8bba41a5833ceae7ecdcb3d)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #7 pc 0000000000050360 /data/app/~~tkynBF6I6wYFbvaZ834Qpg==/应用包名-Dgei18UFSwJBh-34YYpIig==/oat/arm64/base.odex (deleted)
2022-02-14 15:20:48.024 1676-2446/? E/NsdService: id 364996 for SERVICE_GET_ADDR_SUCCESS has no client mapping
2022-02-14 15:20:48.024 1676-2446/? E/NsdService: Unhandled { when=0 what=393242 obj=com.android.server.NsdService$NativeEvent@a2708df target=com.android.internal.util.StateMachine$SmHandler }
由于没有开源代码,我无法定位和修改问题,希望能够解决,谢谢!
/src/main/cpp/vnnjni.cpp:59: undefined reference to `VNN_Destroy_Face'
/src/main/cpp/vnnjni.cpp:1197: undefined reference to `VNN_Destroy_ObjCount'
./../src/main/cpp/vnnjni.cpp:1313: undefined reference to `VNN_Destroy_DocRect'
Can I know a specific version of the CMake tool and NDK version which is used at the time of compiling demos
崩溃堆栈如下:
vnn_core!venus::runtime::Serializable::~Serializable+0x19ca
vnn_core!venus::runtime::Conv2dInfo::VisitAttr+0x210
vnn_core!venus::runtime::Serializable::ToFile+0x1927
vnn_core!venus::runtime::VNModelInfo::VisitAttr+0x1480
vnn_core!venus::runtime::Serializable::ToFile+0x1927
vnn_core!venus::runtime::VNModelInfoArr::VisitAttr+0x2e2
vnn_core!venus::runtime::Serializable::ToFile+0x1927
vnn_core!venus::runtime::VNNModel::VisitAttr+0xa2
vnn_core!venus::runtime::Serializable::FromFile+0x4ac
vnn_core!venus::utility::load_model+0x244
vnn_general!venus::runtime::AlignedPool::default constructor closure'+0x153b vnn_general!venus::runtime::AlignedPool::
default constructor closure'+0x13f4
vnn_general!VNN_Create_General+0x23c
使用3D Game效果人物脸上有黑色框...这个怎么能去掉
two age "3-12", three "13-30",
Hope to see more application scenarios in the future!
源码会完全开源吗,最近还会有更新吗
在小米8SE安卓设备上,3D、卡通等效果,明显感觉卡顿
如题,
下载下来后,在 \VNN\libs\Android\arm64-v8a 中的so文件,总共大概6.6M
1、使用迪士尼和3Dgame的效果时会裁掉一部分头发的区域,怎么扩展处理的区域?如果使用人脸mask,整体会不太协调,因为只处理的人脸的部分,头发、脖子等未处理。
2、怎么提升输出效果的清晰度?
请问,人脸额外状态(比如:闭眼情况,眨眼情况,张嘴情况)检测没有生效,是要做什么额外设置吗?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.