Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Web] #23230

Open
abhaytechnoscore opened this issue Dec 31, 2024 · 0 comments
Open

[Web] #23230

abhaytechnoscore opened this issue Dec 31, 2024 · 0 comments
Labels
.NET Pull requests that update .net code platform:web issues related to ONNX Runtime web; typically submitted using template

Comments

@abhaytechnoscore
Copy link

Describe the issue

public class HomeController : Controller
{
private readonly MLContext _mlContext;
private readonly PredictionEngine<SignLanguageInput, SignLanguageOutput> _predictionEngine;

public HomeController()
{
    _mlContext = new MLContext();

    // Load ONNX model 
    var modelPath = Path.Combine(Directory.GetCurrentDirectory(), "wwwroot", "models", "hand_landmark_sparse_Nx3x224x224.onnx"); 

    var dataView = _mlContext.Data.LoadFromEnumerable(new List<SignLanguageInput>());
    var pipeline = _mlContext.Transforms.ApplyOnnxModel(modelPath);
      
    var trainedModel = pipeline.Fit(dataView);
    _predictionEngine = _mlContext.Model.CreatePredictionEngine<SignLanguageInput, SignLanguageOutput>(trainedModel);
}

public IActionResult Index()
{
    return View();
}
 
public IActionResult PredictGesture([FromBody] string base64Image)
{
    if (string.IsNullOrEmpty(base64Image) || base64Image == "data:,")
        return BadRequest("Image data is missing!");

    // Decode the base64 image and save it temporarily
    var imageBytes = Convert.FromBase64String(base64Image.Replace("data:image/png;base64,", ""));
    var tempImagePath = Path.Combine(Path.GetTempPath(), $"{Guid.NewGuid()}.png");
    System.IO.File.WriteAllBytes(tempImagePath, imageBytes);

    try
    {
        // Preprocess the image to create input tensor
        var inputTensor = PreprocessImage(tempImagePath, 92, 92);  
        float[] inputArray = inputTensor.ToArray();
        // Create input for prediction
        var input = new SignLanguageInput { input = inputArray };

        // Predict gesture
        var result = _predictionEngine.Predict(input);

        if (result != null)
        {
            return Ok(new
            {
                Prediction = result.Label,
                Confidence = result.Confidence,
            });
        }
        else
        {
            return StatusCode(500, "Prediction returned null");
        }
    }
    catch (Exception ex)
    {
        return StatusCode(500, $"Internal server error: {ex.Message}");
    }
    finally
    {
        // Clean up temporary file
        System.IO.File.Delete(tempImagePath);
    }
}


private DenseTensor<float> PreprocessImage(string imagePath, int width, int height)
{
    using var bitmap = new Bitmap(imagePath);
    using var resized = new Bitmap(bitmap, new Size(width, height));  

    int channels = 3; // RGB
    var tensor = new float[channels * width * height];  
    int index = 0;

    for (int y = 0; y < resized.Height; y++)
    {
        for (int x = 0; x < resized.Width; x++)
        {
            var pixel = resized.GetPixel(x, y);

            // Channel-first format (C, H, W)
            tensor[index + 0] = pixel.R / 255f; // Red
            tensor[index + 1] = pixel.G / 255f; // Green
            tensor[index + 2] = pixel.B / 255f; // Blue

            index += channels;
        }
    }

    // Create a DenseTensor directly from the preprocessed image
    var tensorShape = new[] { 1, 3, height, width }; // NCHW format
    return new DenseTensor<float>(tensor, tensorShape); // Return as DenseTensor
}

}

I have given my code (in mvc). In this code, i am getting error on line "var result = _predictionEngine.Predict(input);" and error is "System.ArgumentOutOfRangeException: 'Index was out of range. Must be non-negative and less than the size of the collection. (Parameter 'index')'"

Using package :

.Net Framework : net8.0
Microsoft.ML : Version="4.0.0"
Microsoft.ML.OnnxRuntime.Gpu Version="1.20.1"
Microsoft.ML.OnnxRuntime.Managed Version="1.20.1"
Microsoft.ML.OnnxTransformer Version="4.0.0"
SixLabors.ImageSharp Version="3.1.6"
System.Drawing.Common Version="9.0.0"

and using window is "Window 11" with 64bit OS

To reproduce

to reproduce it, plz run application and give sign image and then it will give error.

Urgency

No response

ONNX Runtime Installation

Built from Source

ONNX Runtime Version or Commit ID

1.20.1

Execution Provider

Other / Unknown

@abhaytechnoscore abhaytechnoscore added the platform:web issues related to ONNX Runtime web; typically submitted using template label Dec 31, 2024
@github-actions github-actions bot added the .NET Pull requests that update .net code label Dec 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
.NET Pull requests that update .net code platform:web issues related to ONNX Runtime web; typically submitted using template
Projects
None yet
Development

No branches or pull requests

1 participant