return descriptors?

[antithing]

Hi, thank you for this cuda update and the decouple! Question, how can i return the cv::point AND the descriptor from the feature extraction?

Ideally a std vector of cv Keypoints and a cv mat of matching descriptors.

Thanks!

[OroChippw]

Key points in decoupled/non-decoupled mode can be obtained through the function GetKeypointsResult. There is no relevant interface for matching descriptors, but you can add the Get function in the Extractor_PostProcess function in decoupled mode to obtain it.
line 222 :

float* desc = (float*)tensor[2].GetTensorMutableData();

[antithing]

@OroChippw Thank you. How are these formatted?
eg, how can I extract the descriptor for a specific point?

What i want to do is store the features and descriptors to bin files, for use in a structure from motion pipeline.

is float* desc the descriptors for every keypoint? How would I, for example, store data like:

std::vector<cv::Keypoint> features. (including the sigma and angle as well as cv::Point)
std::vector<cv::Mat> descriptors;

Thank you!

[antithing]

Okay I have been testing. I have extended an inference function like this:

std::pair<std::vector<cv::Point2f>, float*> LightGlueDecoupleOnnxRunner::InferenceSingleImage(Configuration cfg,
    const cv::Mat& srcImage)
{

    if (srcImage.empty())
    {
        throw  "[ERROR] ImageEmptyError ";
    }
    cv::Mat srcImage_copy = cv::Mat(srcImage);

    // Extract Keypoints
    std::cout << "[INFO] => PreProcess srcImage" << std::endl;
    cv::Mat src = Extractor_PreProcess(cfg, srcImage_copy, scales[0]);
  

    Extractor_Inference(cfg, src);

    auto src_extract = Extractor_PostProcess(cfg, std::move(extractor_outputtensors[0]));

    extractor_outputtensors.clear();

    return src_extract;
}

To return the point and descriptor data.

Does this look correct to push it into storage containers?

  auto kpts_result = FeatureMatcher->InferenceSingleImage(cfg, imgGray);

                auto regions = std::unique_ptr<sregions>(new sregions);

                for (int i = 0; i < kpts_result.first.size(); i++)
                {
                    PointFeature fp(kpts_result.first[i].x, kpts_result.first[i].y); //uses cv::Point
                    Descriptor<float, 256> descriptor;                                   // an Eigen Matrix
                    for (int k = 0; k < 256; ++k)
                        descriptor[k] = static_cast<float>(kpts_result.second[k]);

                    regions.Descriptors().push_back(descriptor);
                    regions.Features().push_back(fp);
                }