Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exporting the convert methods #1

Open
VerioN1 opened this issue Jan 24, 2023 · 32 comments
Open

exporting the convert methods #1

VerioN1 opened this issue Jan 24, 2023 · 32 comments
Assignees
Labels
enhancement New feature or request

Comments

@VerioN1
Copy link

VerioN1 commented Jan 24, 2023

I noticed that you need to convert the image to an array of 8bit luma data, or rgb before passing it to decode_barcode. why not also expose the methods that do that?
Maybe I'm wrong and you can do this in vanilla js, but I couldn't find any example that do that without installing zxing (for js).
If there are other ways to pass image from js to the decoding function without installing zxing, I will be happy to know .

@hschimke
Copy link
Collaborator

I'll have to look up what JS image data looks like (canvas, image, I don't remember?). If it's a byte stream with predictable layout I can expose a method to handle it no problem.

I have some enhancements I want to make to this library, including the ability to set encode and decode hints for JS requests.

Would it be find for the function to be "yet another decode" function, where you hand it the JS image data directly and it handles the conversion in the background, or would a two step process be better, where one method generates the byte stream, and another decodes it?

@hschimke hschimke self-assigned this Jan 24, 2023
@hschimke hschimke added the enhancement New feature or request label Jan 24, 2023
@VerioN1
Copy link
Author

VerioN1 commented Jan 24, 2023

I guess exposing a function that handles the conversion of the image to what ever decode you use, should be preferable since you are decoupling it from the decoding function. exposing multiple such conversion methods can be even better.
IMO most use cases of this wasm, would be file uploading - so maybe knowing how to handle canvas or event base64, and exposing a method that can convert it to the data type of the barcode decoder - would be nice.

you can contact me and I can help you test and improve it.

@VerioN1
Copy link
Author

VerioN1 commented Jan 24, 2023

this is somehting I wrote to manipulate the Image data so I can pass it to zxing in my js app :

reader.onload = async (e) => {
const canvas = document.getElementById('imageCanvas') as HTMLCanvasElement;
if (!canvas) return;
const img = new Image();
img.src = e?.target?.result as string;
img.onload = async () => {
canvas.width = img.width;
canvas.height = img.height;
const ctx = canvas.getContext('2d');
if (!ctx) return;
ctx.drawImage(img, 0, 0);
const result = await zxingDecodeCanvas(canvas);
console.log(result);
if (result?.error) reject(result as BarcodeScannerResult);
resolve(result as BarcodeScannerResult);
};
};

@hschimke
Copy link
Collaborator

Got it, I'll get familiar with this and get it implemented. I'm finishing up adding maxicode detection to rxing (zxing and rxing currently only support it in a "pure barcode" image) and then I'll get to this. Probably by the end of the week.

@hschimke
Copy link
Collaborator

please check out version 0.1.3 on npm https://www.npmjs.com/package/rxing-wasm and let me know if that meets your requirements.

@VerioN1
Copy link
Author

VerioN1 commented Jan 26, 2023

tested it and indeed the function converts the image data properly (since im not getting not found error).
What is weird is that the return value of the decode_barcode isn't right. adding a picture of a successful decode:
image

@VerioN1
Copy link
Author

VerioN1 commented Jan 26, 2023

as you can see the in the console log - the result only has ptr in it (which I guess is a memory pointer address maybe?)

@VerioN1
Copy link
Author

VerioN1 commented Jan 26, 2023

I did tried to make it async and await those functions (just to see if anything changes) and it returned the same result

@hschimke
Copy link
Collaborator

I pushed another version, but it shouldn't make a huge difference here.

The result should be an object, could you try using the text() method on the object and seeing if you get what you're looking for?

@VerioN1
Copy link
Author

VerioN1 commented Jan 26, 2023

ohh my bad. It worked!
Only thing left is to let us configure the hints :)

@hschimke
Copy link
Collaborator

It's there! Check out the new decode_barcode_with_hints function. It takes a DecodeHintDictionary object, which you can create as you would any object, and which has a set_hint method.

I haven't written any documentation for it yet, so let me know if you have issues.

@VerioN1
Copy link
Author

VerioN1 commented Jan 27, 2023

great! will you make a NPM lib with it?, or , if you don't have the time to handle it, I can make a lib out of it.

@hschimke
Copy link
Collaborator

@hschimke
Copy link
Collaborator

I'm going to close this, but feel free to re-open or open a new one if you have additional issues. The NPM package is currently at 0.1.4, which tracks one version back from the current rxing release. I should have 0.1.5 our later tonight or tomorrow.

@VerioN1
Copy link
Author

VerioN1 commented Jan 28, 2023

Im still not sure how to implement the decode hints map
example:
image

now also tried to make the possible formats in new Set or dict, didn't work.

@hschimke hschimke reopened this Jan 29, 2023
@hschimke
Copy link
Collaborator

I plan to document all this next week. For that particular field it is looking for a string with a comma separated list of barcode format names.

@VerioN1
Copy link
Author

VerioN1 commented Jan 29, 2023

apparently, im getting this error of out of memory bounds because of try harder set to true

@hschimke
Copy link
Collaborator

If possible, please attach a sample image that generates the error, I'll look into it.

@VerioN1
Copy link
Author

VerioN1 commented Jan 29, 2023

that's the image:
360_F_455480661_B1ndlageM3kplzg1NRPFUgYj2iWXvDQS

@VerioN1
Copy link
Author

VerioN1 commented Jan 29, 2023

literally turning on the TryHarder option - throws the error

@hschimke
Copy link
Collaborator

Just to verify, that should be:
(upc a) 123456789012

@VerioN1
Copy link
Author

VerioN1 commented Jan 29, 2023

yes

@hschimke
Copy link
Collaborator

I haven't gotten a test bed up, so I cannot really debug this well.

Could you try passing "true" to the TryHarder set_hint and seeing if that work?

So far, I haven't gotten my own library to work without throwing an "unreachable" exception.

@hschimke
Copy link
Collaborator

Oh wow, I figured it out, I was passing in very wrong height and width data (does it show I haven't worked in node for almost a year?)

I get a successful working decode of that image, passing in the hint with hints.set_hint(DecodeHintTypes.TryHarder, "true");

@hschimke
Copy link
Collaborator

hschimke commented Jan 30, 2023

Here is the code I used, I think your issue is the "true" vs true.

import { convert_js_image_to_luma, decode_barcode_with_hints, DecodeHintDictionary, DecodeHintTypes } from "rxing-wasm";


const input = document.getElementById('image_file_input');
input.addEventListener('change', handleFiles);

function handleFiles(e) {
    const canvas = document.getElementById('cvs');
    const ctx = canvas.getContext('2d');
    const img = new Image;
    img.src = URL.createObjectURL(e.target.files[0]);
    img.onload = function () {
        canvas.width = img.width;
        canvas.height = img.height;
        ctx.drawImage(img, 0, 0, img.width, img.height);
    }
}

const scan_btn = document.getElementById('scan_btn');
scan_btn.addEventListener('click', onClickScan);

function onClickScan() {
    const canvas = document.getElementById('cvs');
    const context = canvas.getContext('2d');
    const imageData = context.getImageData(0, 0, canvas.width, canvas.height);
    const luma_data = convert_js_image_to_luma(imageData.data);
    const hints = new DecodeHintDictionary();
    hints.set_hint(DecodeHintTypes.TryHarder, "true");
    const result = decode_barcode_with_hints(luma_data, canvas.width, canvas.height, hints);
    alert(result.text());
}

@hschimke
Copy link
Collaborator

Alright, I added a demo application in the examples folder. It shows one way to handle basic hints and decoding.

Please let me know if your issue is resolved.

@VerioN1
Copy link
Author

VerioN1 commented Jan 31, 2023

looks great. tried to play with it and was going great. I do think you will need to eventually create a table that maps between properties and thier type ( and maybe even a column of an example value at the so it will be explicit) .
one last this is that with TryHarder true nad UpcEanExtension - I managed to make the browser totally freeze while trying to scan this particular image.
I will be more than happy if you could have a look why would that be .
WhatsApp Image 2023-01-19 at 18 12 21

@VerioN1
Copy link
Author

VerioN1 commented Jan 31, 2023

if you need a video of the browser behaver I can provide one. But try on your machine first with this code 👍:
`<script type="module">
import init, { BarcodeFormat, DecodeHintTypes, DecodeHintDictionary, decode_barcode_with_hints, convert_js_image_to_luma } from "./rxing_wasm.js";

  let imgFile = document.getElementById("image-input")
  let reader = new FileReader();

  imgFile.addEventListener('input', (e) => {
    reader.readAsDataURL(imgFile.files[0]);
  })
  reader.onload = function(event) {
    let img = new Image();
    img.src = event.target.result;
    img.onload = function() {
      let canvas = document.createElement('canvas');
      canvas.width = img.width;
      canvas.height = img.height;
      let ctx = canvas.getContext('2d');
      ctx.drawImage(img, 0, 0);
      let imgData = ctx.getImageData(0, 0, img.width, img.height);
      let data = imgData.data;
      let height = canvas.height;
      let width = canvas.width;
      init().then(() => {
        try{
          let luma8Data = convert_js_image_to_luma(data);
          const hintsDict = new DecodeHintDictionary()
          hintsDict.set_hint(DecodeHintTypes.TryHarder, "true");
          hintsDict.set_hint(DecodeHintTypes.POSSIBLE_FORMATS, 'UpcEanExtension')
          let parsedBarcode = decode_barcode_with_hints(luma8Data, width, height, hintsDict);
          console.log(parsedBarcode)
          console.log(parsedBarcode.text());
        }catch (e) {
          console.warn(e)
        }
      });
    }
  }



</script>`

@hschimke
Copy link
Collaborator

It breaks the cli app too (100% cpu in what looks like an infinite loop). I'll figure out what's going on with it.

@hschimke
Copy link
Collaborator

hschimke commented Feb 1, 2023

Two commits in the main repository fix the infinite loop. The image doesn't decode, but it no longer hangs.

rxing-core/rxing@6389d05
rxing-core/rxing@7526558

This update is pushed to npm as 0.1.6 (https://www.npmjs.com/package/rxing-wasm)

@hschimke
Copy link
Collaborator

hschimke commented Feb 2, 2023

Watch out for versions 0.1.7 - 0.1.11. A change in some dependencies has caused some havoc I'm still sorting out. I suggest pinning to 0.1.6 for now

@hschimke
Copy link
Collaborator

hschimke commented Feb 2, 2023

This is resolved with npm version 0.1.12

This also brings a lot more documentation, and the ability to get metadata from the result object. A javascript Map object is returned, containing parsed metadata.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants