Real user images dataset

[nliaudat]

Hi,

I ask if someone is ready to upload an image of your counter every 12 hours to a github repo?

For the past month, I've been trying to improve the accuracy of an AI model to process all the digits in a single step, without having to specify the position of each digit.

This process is already implemented by seedstudio (but doesn't work very well).

I'm getting average results due to a lack of good usable data.

Would you be willing to send the RAW images and crop positions of your counters at regular intervals in order to improve the model?

This would also be very useful for improving the current models, which are trained on a majority of images with a dark background.

My model doesn't deal with decimals, so I think detecting a 9 instead of a 6 is complicated enough without introducing 9.1. What's more, it doesn't provide much accuracy, since the aim is to have a minimum error rate with 1 decimal place (1 liter).

Ps. : I can write the api to upload to github, huggingface.co or as you want.

[uSlackr]

I’d be willing to feed data if it is automated \\Greg

…

________________________________ From: Nicolas Liaudat ***@***.***> Sent: Tuesday, February 11, 2025 2:30:49 PM To: jomjol/AI-on-the-edge-device ***@***.***> Cc: Subscribed ***@***.***> Subject: [jomjol/AI-on-the-edge-device] Real user images dataset (Discussion #3551) Hi, I ask if someone is ready to upload an image of your counter every 12 hours to a github repo? For the past month, I've been trying to improve the accuracy of an AI model to process all the digits in a single step, without having to specify the position of each digit. This process is already implemented by seedstudio<https://wiki.seeedstudio.com/Train-Water-Meter-Digits-Recognition-Model-with-SenseCAP-A1101/> (but doesn't work very well). I'm getting average results due to a lack of good usable data. Would you be willing to send the RAW images and crop positions of your counters at regular intervals in order to improve the model? This would also be very useful for improving the current models, which are trained on a majority of images with a dark background. My model doesn't deal with decimals, so I think detecting a 9 instead of a 6 is complicated enough without introducing 9.1. What's more, it doesn't provide much accuracy, since the aim is to have a minimum error rate with 1 decimal place (1 liter). Ps. : I can write the api to upload to github, huggingface.co or as you want. — Reply to this email directly, view it on GitHub<#3551>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGEO6Z66A3GQA4QW3Q2YZ3T2PJFWTAVCNFSM6AAAAABW53RVI6VHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZXHE2DQNBXGM>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jasaw]

For what it's worth, there are a few techniques that I usually employ to train a model to "ignore" dark/light background and just look for digits. I have only done this on a larger model, so I'm not sure whether this technique will produce a good result on small models that run on an ESP32.

1. Convert the training image to grey scale and normalize the image brightness to span the entire 0 to 255 value. There is no need for the model to be colour aware.
2. Invert the same training image that has been converted to grey scale and normalize the brightness. This will "teach" the model to ignore background colour, instead look for the digit. You could invert half the image, quarter of the image, and so on.
3. Do a few variations of the same grey scale training image, like vary the image brightness, add some noise to the image, rotate the image a little bit.

When we actually use the model, we want to convert the input image to grey scale and normalize the brightness before feeding it to the model.

[nliaudat]

Thank you for the advices.
What I need is a lot more of images and especially images with the whole eight digits of the counter.
I want to pass the image in one shot in the ocr AI engine.

For grayscale transformation, I works only on custom ocr model.
If using pretrained models with fine tuning, the result with grayscale images is degraded compared to RGB input.

PS. : The collect and send images part in ai-on-the-edge will be very difficult after deep analysis.
Some has already tried without success actually. (#3326, #1330, #3378)
I mean the best way to go is perhaps an home assistant component or a docker, but it will limit the number of users

[uSlackr]

I an reading an LCD panel. I feel like my picture might be important [cid:37fa6f04-ecb3-4e41-937f-9737cfa313ff] \\Greg

…

________________________________ From: Nicolas Liaudat ***@***.***> Sent: Thursday, February 13, 2025 12:35 To: jomjol/AI-on-the-edge-device ***@***.***> Cc: uSlackr ***@***.***>; Comment ***@***.***> Subject: Re: [jomjol/AI-on-the-edge-device] Real user images dataset (Discussion #3551) Thank you for the advices. What I need is a lot more of images and especially images with the whole eight digits of the counter. I want to pass the image in one shot in the ocr AI engine. For grayscale transformation, I works only on custom ocr model. If using pretrained models with fine tuning, the result with grayscale images is degraded compared to RGB input. PS. : The collect and send images part in ai-on-the-edge will be very difficult after deep analysis. Some has already tried without success actually. (#3326<#3326>, #1330<#1330>, #3378<#3378>) I mean the best way to go is perhaps an home assistant component or a docker, but it will limit the number of users — Reply to this email directly, view it on GitHub<#3551 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AGEO6Z2B4YDNQ26YEDKE26L2PTJWTAVCNFSM6AAAAABW53RVI6VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMJZGA4TCNA>. You are receiving this because you commented.Message ID: ***@***.***>

[caco3]

Are you aware of my demo data set (see https://jomjol.github.io/AI-on-the-edge-device-docs/Demo-Mode/)?
It contains over 800 images of my water meter. It is ok for me if you want to use it to train/improve your project.

The collect and send images part in ai-on-the-edge will be very difficult after deep analysis.
Some has already tried without success actually. (#3326, #1330, #3378)

Local storing of the image on the SD card should not be difficult. Just nobody was willing to implement it yet.
Uploading it to an external service could be more tricky. We already have a webhook functionality, but it would have to be extended. IMO it would be better to store on the SD card and extract them from externally.

[nliaudat]

Many thanks.
Your demo dataset is mainly for analog devices and my target is to enhance the digital recognition.
At that point, I mean the easyest way to go is to make an hassio component.
But what part of users use hassio ?
The actual recognition model works, but I mean it has to be fine tuned on more real user images and especially in case of errors.
The web has a lot of ready to go datasets for water meters, but the zoom factor do not correspond with ours.
Regards

[caco3]

If you want, I can also record such images for my gas meter:

Your demo dataset is mainly for analog devices and my target is to enhance the digital recognition

Please note that it should be called digit and not digital because there is nothing digital on those meters (we also used his term wrongly until a while ago).

[nliaudat]

Thanks.
For now, I have to do some more testing.
Regards

Actually, my custom model get 10441264 on your image cause it was trained on a large dataset of 8 digits water meters.

[caco3]

My meter also has a 3th red digit, how ever there is no point of using it as it counts to mooch between 2 rounds anyway.

Having it forced to 8 digits is IMO a disadvantage. Eg. on my water meter, I only use the relevant digits. There is no point of adding 3 leading zeros which will not change for the next couple of years.

[nliaudat]

I have done some testing on webhooks and it may works if the provided image where not AlgRoi, but Alg (Raw after alignement).

The define ALGROI_LOAD_FROM_MEM_AS_JPG takes memory and I do not want to implement an second ALG_LOAD_FROM_MEM_AS_JPG to have only the raw image as it could be excessif memory usage.

For memories , here is the python version of php webhook (https://jomjol.github.io/AI-on-the-edge-device-docs/Webhook/) :

from flask import Flask, request, jsonify
import csv
import os

app = Flask(__name__)

# List of allowed API keys
ALLOWED_API_KEYS = {
    '123',
    '456',
    '789'
}

@app.before_request
def check_api_key():
    # Get the API key from the request headers
    received_api_key = request.headers.get('APIKEY')
    
    # Check if the received API key is in the allowed list
    if received_api_key not in ALLOWED_API_KEYS:
        return jsonify({'status': 'error', 'message': 'Invalid API key'}), 403
    
    # Attach the API key to the request object for later use
    request.api_key = received_api_key

@app.route('/webhook', methods=['POST', 'PUT'])
def webhook():
    # Create a directory for the API key if it doesn't exist
    api_key_dir = os.path.join('data', request.api_key)
    os.makedirs(api_key_dir, exist_ok=True)

    if request.method == 'POST':
        # Handle POST request: Write data to CSV
        data = request.get_json()
        if not data or not isinstance(data, list):
            return jsonify({'status': 'error', 'message': 'Invalid JSON data'}), 400

        csv_file = os.path.join(api_key_dir, 'webhook_log.csv')
        try:
            with open(csv_file, 'a', newline='') as csvfile:
                csv_writer = csv.writer(csvfile)
                for item in data:
                    csv_writer.writerow([
                        item.get('timestampLong'),
                        item.get('name'),
                        item.get('rawValue'),
                        item.get('value'),
                        item.get('preValue'),
                        item.get('rate'),
                        item.get('changeAbsolute'),
                        item.get('error')
                    ])
            return jsonify({'status': 'success', 'message': 'Data written to CSV file'}), 200
        except Exception as e:
            return jsonify({'status': 'error', 'message': 'Unable to open CSV file'}), 500

    elif request.method == 'PUT':
        # Handle PUT request: Save image
        timestamp = request.args.get('timestamp')
        if not timestamp or not timestamp.isdigit() or int(timestamp) < 0:
            return jsonify({'status': 'error', 'message': 'Invalid timestamp'}), 400

        image_data = request.data
        if not image_data:
            return jsonify({'status': 'error', 'message': 'No image data received'}), 400

        image_file_path = os.path.join(api_key_dir, f'{timestamp}.jpg')
        try:
            with open(image_file_path, 'wb') as image_file:
                image_file.write(image_data)
            return jsonify({'status': 'success', 'message': 'Image uploaded successfully'}), 200
        except Exception as e:
            return jsonify({'status': 'error', 'message': 'Unable to save the image'}), 500

    else:
        # Handle unsupported HTTP methods
        return jsonify({'status': 'error', 'message': 'Method not allowed'}), 405

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5001)

[nliaudat]

final component : https://github.com/nliaudat/aioted_manager
Still draft but working.