GithubHelp home page GithubHelp logo

echamudi / opencv-wasm Goto Github PK

View Code? Open in Web Editor NEW
299.0 299.0 34.0 5.71 MB

Precompiled OpenCV 4.3.0 to JavaScript + WebAssembly for node and deno. ๐Ÿฆ•

Home Page: https://www.npmjs.com/package/opencv-wasm

License: BSD 3-Clause "New" or "Revised" License

Shell 9.62% JavaScript 73.83% TypeScript 16.55%
deno javascript node opencv typescript

opencv-wasm's Introduction

OpenCV-Wasm

Build Status

Precompiled OpenCV to JavaScript + WebAssembly for node.js and deno environment. ๐Ÿฆ•

In this Wasm-compiled OpenCV, there's no need to have OpenCV installed in the machine. The entire OpenCV library is already inside this package (opencv.js and opencv.wasm).

This module has zero dependencies.

Examples

Code Input Output
dilation.js (node) image sample 1 dilation
templateMatching.js (node) source:
image sample 2
template:
image sample 2 template
template matching

Installation

node

npm install opencv-wasm

Code example:

const { cv, cvTranslateError } = require('opencv-wasm');

let mat = cv.matFromArray(2, 3, cv.CV_8UC1, [1, 2, 3, 4, 5, 6]);
console.log('cols =', mat.cols, '; rows =', mat.rows);
console.log(mat.data8S);

cv.transpose(mat, mat);
console.log('cols =', mat.cols, '; rows =', mat.rows);
console.log(mat.data8S);

/*
cols = 3 ; rows = 2
Int8Array(6) [ 1, 2, 3, 4, 5, 6 ]
cols = 2 ; rows = 3
Int8Array(6) [ 1, 4, 2, 5, 3, 6 ]
*/

deno

import { cv, cvTranslateError } from 'https://deno.land/x/[email protected]/mod.ts';
// Change the @<version> with the latest or any version you desire.
// Check the available versions here: https://deno.land/x/opencv.

Code example:

import { cv, cvTranslateError } from 'https://deno.land/x/[email protected]/mod.ts';

let mat = cv.matFromArray(2, 3, cv.CV_8UC1, [1, 2, 3, 4, 5, 6]);
console.log('cols =', mat.cols, '; rows =', mat.rows);
console.log(mat.data8S);

cv.transpose(mat, mat);
console.log('cols =', mat.cols, '; rows =', mat.rows);
console.log(mat.data8S);

/*
cols = 3 ; rows = 2
Int8Array(6) [ 1, 2, 3, 4, 5, 6 ]
cols = 2 ; rows = 3
Int8Array(6) [ 1, 4, 2, 5, 3, 6 ]
*/

Usage

Because this module is using the same code as the official OpenCV.js for the web, you can use the same documentation at the web: https://docs.opencv.org/4.3.0/d5/d10/tutorial_js_root.html

There are some minor initialization changes, because this module will be loaded synchronously instead of the OpenCV's default (asynchronously).

You can check the files inside examples folder as reference on how to initialize, loading images, and saving images.

Error Handling

By default, mistakes in code will produce error code. You can use the following snippet to translate the error code into meaningful statement from OpenCV.

const { cv, cvTranslateError } = require('opencv-wasm');

try {
    // Your OpenCV code
} catch (err) {
    console.log(cvTranslateError(cv, err));
}

Versioning

This npm module uses the following versioning number:

<opencv version>-<this module version>

For Example

4.3.0-9
OpenCV version 4.3.0
OpenCV-Wasm Module version 9

Development

Building

Run the following script on macOS or Linux (tested on Ubuntu). You need docker on the system.

npm install
(cd ./utils && sh ./build.sh)
(cd utils && node generateCvProps.js)

Testing

After completing the build script, you can run the test provided by OpenCV, and the test from this repo.

# OpenCV's test
(cd ./build_wasm_test/bin && npm install)
(cd ./build_wasm_test/bin && node tests.js)

# This repo's test
npm test

Authors

See also the list of contributors who participated in this project.

License

Copyright ยฉ 2020 Ezzat Chamudi and OpenCV-Wasm Project Authors

OpenCV-Wasm code is licensed under BSD-3-Clause. Images, logos, docs, and articles in this project are released under CC-BY-SA-4.0.

OpenCV License.

Libraries, dependencies, and tools used in this project are tied with their licenses.

opencv-wasm's People

Contributors

echamudi avatar rwv avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

opencv-wasm's Issues

Opencv.js - findPerspective returns wrong corners coordinates

Hi,

I'm trying to implement AKAZE with opencv.js and opencv-wasm.

Here is my code:

const { cv, cvTranslateError } = require('opencv-wasm');
const Jimp = require('jimp');
const util = require('./util.js');

 (async () => {
     try {
         console.log("Enter 1");
         const jimpSrc = await Jimp.read("./images/how_to_play.png");
         const jimpSrcToMatch = await Jimp.read("./images/scene.png");

    console.log("Enter 2");
    // initiate ORB detector, default number of features to retain is 500, bit too low
    let akaze = new cv.AKAZE();
    let img1 = await cv.matFromImageData(jimpSrc.bitmap);
    let img2 = await cv.matFromImageData(jimpSrcToMatch.bitmap);
    let res = new cv.Mat();
    let mask1 = new cv.Mat();
    let mask2 = new cv.Mat();
    console.log("Enter 3");

    cv.cvtColor(img1, img1, cv.COLOR_BGRA2GRAY, 0);
    cv.cvtColor(img2, img2, cv.COLOR_BGRA2GRAY, 0);

    // find descriptors and keypoints with AKAZE detector
    let kp1 = new cv.KeyPointVector();
    let kp2 = new cv.KeyPointVector();
    let des1 = new cv.Mat();
    let des2 = new cv.Mat();

    akaze.detectAndCompute(img1, mask1, kp1, des1);
    akaze.detectAndCompute(img2, mask2, kp2, des2);

    // create BFMatcher object and and match descriptors
    let matcher = new cv.BFMatcher();
    let matches = new cv.DMatchVectorVector();
    let filteredMatches = new cv.DMatchVector();

    matcher.knnMatch(des1, des2, matches, 2);

    // filter matches
    for (let i = 0; i < matches.size(); ++i) {
        let match = matches.get(i);
        let dMatch1 = match.get(0);
        let dMatch2 = match.get(1);
        if (dMatch1.distance <= dMatch2.distance * 0.7) {
            filteredMatches.push_back(dMatch1);
        }
    }

    cv.drawMatches(img1, kp1, img2, kp2, filteredMatches, res);
    util.saveMat(res, "matcher-output.png");

    if (filteredMatches.size() > 13) {
        let objectPoints = [];
        let scenePoints = [];

        for (let i = 0; i < filteredMatches.size(); ++i) {
            let dMatch = filteredMatches.get(i);
            objectPoints.push(
                new cv.Point(kp1.get(dMatch.queryIdx).pt.x, kp1.get(dMatch.queryIdx).pt.y));
            scenePoints.push(
                new cv.Point(kp2.get(dMatch.trainIdx).pt.x, kp2.get(dMatch.trainIdx).pt.y));
        }

        let homography = cv.findHomography(
            cv.matFromArray(objectPoints.length, 3, cv.CV_32F, objectPoints),
            cv.matFromArray(scenePoints.length, 3, cv.CV_32F, scenePoints), cv.RANSAC, 3);
        let sceneCorners = new cv.Mat(4, 1, cv.CV_32FC2);
        //let objCorners = cv.matFromArray(4, 1, cv.CV_32FC2, [0, 0, img1.cols - 1, 0, img1.cols - 1, img1.rows - 1, 0, img1.rows - 1]);
        let objCorners = new cv.Mat(4, 1, cv.CV_32FC2);

        console.log("Transforming object corners to scene corners...");
        // cv.perspectiveTransform(objCorners, sceneCorners, homography);

        var view = objCorners.data32F;
        view[0] = 0;
        view[1] = 0;
        view[2] = img1.cols;
        view[3] = 0;
        view[4] = img1.cols;
        view[5] = img1.rows;
        view[6] = 0;
        view[7] = img1.rows;

        cv.perspectiveTransform(objCorners, sceneCorners, homography);

        var dataAns = sceneCorners.data32F;

        var x1 = new cv.Point(dataAns[0], dataAns[1]);
        var x2 = new cv.Point(dataAns[2], dataAns[3]);
        var x3 = new cv.Point(dataAns[4], dataAns[5]);
        var x4 = new cv.Point(dataAns[6], dataAns[7]);

        cv.line(res, x1, x2,
            new cv.Scalar(0, 0, 0), 10, cv.LINE_AA, 0);
        cv.line(res, x2, x3,
            new cv.Scalar(0, 0, 0), 10, cv.LINE_AA, 0);
        cv.line(res, x3, x4,
            new cv.Scalar(0, 0, 0), 10, cv.LINE_AA, 0);
        cv.line(res, x4, x1,
            new cv.Scalar(0, 0, 0), 10, cv.LINE_AA, 0);
    }

} catch (err) {
    console.log(cvTranslateError(cv, err));
}
})();

The problem is, that final corners coordinates x1, x2, x3, x4 are always like 1.00007678, 0.9999767623, -0.9878787.... Any ideas why?

P.S I'm well familiar with opencv for Java and Python.

How to set Module['wasmBinary']?

I'm trying to use it in a project that uses vite, but I'm trying to call it with an import statement because the require statement cannot be used. However, I get the following error and cannot read it. (A similar problem also occurred when using Webpack)

main.ts

import { cv } from 'opencv-wasm'; // replace with require('opencv-wasm'); in prod

console.log(cv.getBuildInformation());

Outputs:

ncaught RuntimeError: abort(sync fetching of the wasm failed: you can preload it to Module['wasmBinary'] manually, or emcc.py will do that for you when generating HTML (but not JS)). Build with -s ASSERTIONS=1 for more info.
    at abort (opencv.js:775:27)
    at getBinary (opencv.js:802:25)
    at instantiateSync (opencv.js:848:38)
    at Object.createWasm [as asm] (opencv.js:870:21)
    at opencv.js:7177:34
    at opencv.js:7887:12
    at opencv.js:14:26
    at node_modules/opencv-wasm/opencv.js (opencv.js:23:19)
    at __require (chunk-BQAOEP2F.js?v=df6f9452:12:50)
    at node_modules/opencv-wasm/index.js (index.js:2:9)

It says Module ['wasmBinary'] manually. How should I set it here?

Although it is out of the scope of this project, I tried to build opencv wasm by myself, but I could not confirm it well because the wasm file was not output even if I changed the setting.

Error in cvtColor

Hi, just wondering is this is a bug or something on my code.

Running the below code will throw exception below:

TypeError [ERR_INVALID_ARG_TYPE]: The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received null
    let jimpSrc = await Jimp.read(__dirname + '/image.jpg');
    let src = cv.matFromImageData(jimpSrc.bitmap);
    let dst = new cv.Mat()
    cv.cvtColor(src, dst, cv.COLOR_BGR2GRAY)

However, calling dilate as per your example works fine. Perhaps something that I missed?

Thanks in advance.

HTMLVideoElement is not defined

Hi guys,

Above all, thank you for the work done on the subject, and for sharing through this repo.
I've been trying for several days to use openCV on a node environment, in an electron application.
I had some problems before now... but for the last one, I didn't find a solution...

I'am trying to use : cv.VideoCapture()
and I've get this error : ReferenceError: HTMLVideoElement is not defined

At the beginning of my app, I have the installDom method :

installDOM() {
    const dom = new JSDOM();
    global.document = dom.window.document;
    // The rest enables DOM image and canvas and is provided by node-canvas
    global.Image = Image;
    global.HTMLCanvasElement = Canvas;
    global.ImageData = ImageData;
    global.HTMLImageElement = Image;
  } 

And effectively, I don't have HTMLVideoElement inside...
So : I don't understand how it can works...

Do you have an idea ?

API of opencv.js

Hey Ezzat,
me again ;-)
Have been googling a couple of hours to find an API of all functionality in OpenCv.js
But I didn't manage to find anything decent.

But in the other issue, you mentioned this:

And in my code playground.js, I can console.log the cv and see the open-cv functionalities.

Is that a complete list of all functionality offered by OpenCv.js, or do I still need to look at other places?

Thanks!
Bart

Is web environment supported?

The README says "Precompiled OpenCV to JavaScript + WebAssembly for node.js and deno environment", but is web environment supported?

If not, will opencv-wasm also add support for web soon (and is it hard to do)?

Property 'CAP_PROP_FRAME_HEIGHT' does not exist on type 'typeof ..

Hi,

I was testing your module and found this issue:
"Property 'CAP_PROP_FRAME_WIDTH' does not exist on type 'typeof ..node_modules/opencv-wasm/types/opencv"

From normal reference code below:
"cv.CAP_PROP_FRAME_HEIGHT"

Is this variable intentionally missing or is the type just not defined? I had a quick look at the original js port and got the impression it should exist at cv.CAP_PROP_FRAME_HEIGHT. Not sure if i can get the same/correct value from the video element (scaled pixels value). Same goes with fps, frame count etc properties.

Kind regards

compiler version

Hi!

It seems that you've used emscripten to compile OpenCV.js to wasm. Is that right? Could you please tell me the version of emcc that you used for this (if possible)?

Question - matFromImageData returning empty Object

Hi, I've started using this lib on node.js and I'm facing the following issue:

let srcImage = await Jimp.read('./image.png') let src = cv.matFromImageData(srcImage.bitmap)
src is returning an empty Mat object

the srcImage.bitmap values are:
{ width: 278, height: 155, depth: 8, interlace: false, palette: false, color: true, alpha: true, bpp: 4, colorType: 6, data: <Buffer eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb eb ff eb eb ... 172310 more bytes>, gamma: 0 }

Detect when OpenCv.js is loaded

Hi Ezzat,

It seems it takes quite some time to load the wasm.
Is it possible somehow to apply an event handler, which is called when the module is completely loaded?

Thanks !!!
Bart

Question: Has anyone made opencv-wasm work with webpack?

Hi @echamudi,
thanks for providing this package, I really appreciate that.

I have read the other issues so I know that you are not working with webpack. But maybe someone else made it work and can give some input?

I have done some research, but failed to succeed.

For completeness sake, this is the error I am getting:

sync fetching of the wasm failed: you can preload it to Module['wasmBinary'] manually, or emcc.py will do that for you when generating HTML (but not JS)
Full error
sync fetching of the wasm failed: you can preload it to Module['wasmBinary'] manually, or emcc.py will do that for you when generating HTML (but not JS)
opencv.js:760 sync fetching of the wasm failed: you can preload it to Module['wasmBinary'] manually, or emcc.py will do that for you when generating HTML (but not JS)
abort @ opencv.js:760
getBinary @ opencv.js:791
instantiateSync @ opencv.js:837
createWasm @ opencv.js:859
eval @ opencv.js:7166
eval @ opencv.js:7871
eval @ opencv.js:10
eval @ opencv.js:11
eval @ opencv.js:14
./node_modules/opencv-wasm/opencv.js @ main.bundle.js:1321
__webpack_require__ @ main.bundle.js:2196
eval @ index.js:2
./node_modules/opencv-wasm/index.js @ main.bundle.js:1311
__webpack_require__ @ main.bundle.js:2196
eval @ index.js:15
./app/index.js @ main.bundle.js:205
__webpack_require__ @ main.bundle.js:2196
(anonymous) @ main.bundle.js:2272
(anonymous) @ main.bundle.js:2275
opencv.js:842 failed to compile wasm module: RuntimeError: abort(sync fetching of the wasm failed: you can preload it to Module['wasmBinary'] manually, or emcc.py will do that for you when generating HTML (but not JS)). Build with -s ASSERTIONS=1 for more info.
instantiateSync @ opencv.js:842
createWasm @ opencv.js:859
eval @ opencv.js:7166
eval @ opencv.js:7871
eval @ opencv.js:10
eval @ opencv.js:11
eval @ opencv.js:14
./node_modules/opencv-wasm/opencv.js @ main.bundle.js:1321
__webpack_require__ @ main.bundle.js:2196
eval @ index.js:2
./node_modules/opencv-wasm/index.js @ main.bundle.js:1311
__webpack_require__ @ main.bundle.js:2196
eval @ index.js:15
./app/index.js @ main.bundle.js:205
__webpack_require__ @ main.bundle.js:2196
(anonymous) @ main.bundle.js:2272
(anonymous) @ main.bundle.js:2275
opencv.js:764 Uncaught RuntimeError: abort(sync fetching of the wasm failed: you can preload it to Module['wasmBinary'] manually, or emcc.py will do that for you when generating HTML (but not JS)). Build with -s ASSERTIONS=1 for more info.
    at abort (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:764:27)
    at getBinary (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:791:25)
    at instantiateSync (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:837:38)
    at Object.createWasm [as asm] (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:859:21)
    at eval (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:7166:40)
    at eval (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:7871:12)
    at Object.eval (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:10:31)
    at eval (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:11:12)
    at eval (webpack://webpack-boilerplate/./node_modules/opencv-wasm/opencv.js?:14:2)
    at Object../node_modules/opencv-wasm/opencv.js (http://localhost:8080/main.bundle.js:1321:1)

I have also tried to build opencv-wasm with the -s "SINGLE_FILE=1" option to avoid having a wasm file, but a .js file instead.
Described here Though, I must have done something wrong, as I still got the .wasm file.

I have also tried copying the wasm file into an asset folder and editing the path in opencv.js (opencvWasmBinaryFile) to point there. Also no success.

So maybe someone else has another idea. I would love to get it to work. Previously I was using opencv4nodejs, but would like to move opencv completely into the browser :-)

solvePnP not available

Hi :)
When I install opencv-wasm from npm, everything works fine. I can call cv.solvePnP. But when I use the same exact code to build an own version - then solvePnP is not available.

Is that possible npm version is dirty, with some configs updated?

Error when loading wasm in NodeJs version 12.16.1

Hi Ezzat,
Thanks for sharing this library!

When executing this statement:

const { cv, cvTranslateError } = require('opencv-wasm');

I get this error:

failed to compile wasm module: LinkError: WebAssembly.Instance(): Import #79 module="env" function="memory" error: memory import must be a WebAssembly.Memory object

When looking e.g. at this article, it seems to me that the code to load wasm depends on the NodeJs version?

I have tested this in NodeJs version 12.16.1 (running on a Raspberry Pi).

Would appreciate if you could give me advice about this!

Thanks in advance!
Bart

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.