[Meta] Bulding opencv.js to integrate a couple of functions from a specific module for use in Node and browser apps

[aptlin]

Hi @huningxin and the team, thank you for your extensive work!

TLDR

1. Is installing opencv.js from NPM the recommended way to use in Node/browsers?

2. Is there any way to compile a very strict subset of opencv.js (down to several functions, described below) in order to achieve a small bundle size, other than forking opencv, tracing and removing the unneeded files from imgproc, and then compiling with build flags like here?

Explanation

I am a fellow GSoC participant, growing the TF.js model garden. The current model I am working on is text detection based on PSENet, which relies heavily on the opencv imgproc module for post-processing (in particular, the progressive scale expansion algorithm uses ConnectedComponents, and then the results are processed using minAreaRect and boxPoints).

Now I am trying to reproduce these steps in combination with TF.js inference, having stuck in the conundrum how to integrate opencv.js into the pipeline.

There are other implementations like the one from UC Irvine, while the official docs only provide some guidance on how to compile opencv.js locally, so I feel a bit confused about what to do next. The reason why there is no official npm package seems to be just that: opencv is huge and use-cases are often too specific in scope to justify the use of vanilla opencv.js, which blows up the bundle size.

I would love to hear your ideas on how you would achieve the above!

Thank you for your time and effort.

[Wenzhao-Xiang]

Hi, @sdll do you mean you want to build a specific subset of opencv.js by yourself?

[aptlin]

@Wenzhao-Xiang, I was wondering about that as one of the solutions, yes, since PSENet only requires the imgproc module of opencv, and whether it was the preferred solution in my case, given the size of vanilla opencv.js (~9 mb). Or would you recommend using the opencv.js NPM module, even though it has not been updated for two years?

[huningxin]

@sdll , thanks for your comments.

2. Is there any way to compile a very strict subset of opencv.js (down to several functions, described below) in order to achieve a small bundle size, other than forking opencv, tracing and removing the unneeded files from imgproc, and then compiling with build flags like here?

You may need to tweak the function white-list of OpenCV.js to achieve that. The white-list is defined in embindgen.py. You may remove modules except imgproc and core, and unneeded functions of the imgproc module.

1. Is installing opencv.js from NPM the recommended way to use in Node/browsers?

Regarding to the npm module, it was maintained by @sajjadt . Sajjad, do you have any plans to update the OpenCV.js NPM module?

[aptlin]

@huningxin, thank you so much for your help!

@sajjadt, it would be nice to revive the npm package, and probably borrow TypeScript typings from opencv4nodejs. It might also be nice to configure CI for keeping the build up to date with opencv master. I would be more than happy to help with those!