Return quality of fit for PointCloudLibrary RANSAC model fitting

I'm trying to use Point Cloud Library to fit a circle in 3d space to a collection of 3d points, with robust rejection of outliers. RANSAC seems to be a suitable way to accomplish this.

Along with the parameters of the detected circle, I'd like to report back a quality measure of how well the finally-selected points actually fit the model. Something ought to be available as a byproduct of the RANSAC algorithm, which iteratively fits candidate models to the supplied data. I want to be able to provide feedback to a user that their data is bad, and they should feel bad measure again to get better data; and I want to be able to prevent moving forwards in a calibration process if the input data isn't up to scratch, which could otherwise eventually lead to unusable results.

Here's my current code. vec and circle3d are basic structs for marshalling across a P/Invoke interface to a .NET application which consumes this API, and have no significant behaviour.

PCLCIRCLEFITTING_API bool FitCircle(const vec* const points, const UINT32 point_count, const double threshold,
                                    circle3d* circle, double* variance)
{
    const pcl::PointCloud<pcl::PointXYZ>::Ptr cloud (new pcl::PointCloud<pcl::PointXYZ>);

    cloud->width = point_count;
    cloud->height = 1;
    cloud->is_dense = true;
    cloud->points.resize(static_cast<size_t>(cloud->width) * cloud->height);

    // Copy point coordinates from marshallable structs to PCL cloud
    for (pcl::index_t i = 0; i < static_cast<pcl::index_t>(cloud->size ()); i++)
    {
        (*cloud)[i].x = points[i].x;
        (*cloud)[i].y = points[i].y;
        (*cloud)[i].z = points[i].z;
    }

    // Create RandomSampleConsensus object and compute the appropriate model
    const pcl::SampleConsensusModelCircle3D<pcl::PointXYZ>::Ptr circle_model (new pcl::SampleConsensusModelCircle3D<pcl::PointXYZ>(cloud));

    pcl::RandomSampleConsensus<pcl::PointXYZ> ransac(circle_model, threshold);
    // Enable parallel processing, selecting number of threads based on hardware
    ransac.setNumberOfThreads(0);
    const bool success = ransac.computeModel();

    if (success)
    {
        Eigen::VectorXf coefficients;
        ransac.getModelCoefficients(coefficients);

        circle->center.x = coefficients[0];
        circle->center.y = coefficients[1];
        circle->center.z = coefficients[2];
        circle->radius = coefficients[3];
        circle->normal.x = coefficients[4];
        circle->normal.y = coefficients[5];
        circle->normal.z = coefficients[6];

        // This is returning NaN, apparently because the model hasn't been computed yet?
        *variance = circle_model->computeVariance();
    }
    else
    {
        *variance = 0.0;
    }

    return success;
}

When the code runs, it successfully finds a suitable set of parameters for the RANSAC model based on the test data I'm feeding in, but variance is always coming out as NaN. Stepping into the computeVariance() method, I see the following logic in the header:

      /** \brief Compute the variance of the errors to the model from the internally
        * estimated vector of distances. The model must be computed first (or at least
        * selectWithinDistance must be called).
        */
      inline double
      computeVariance () const
      {
        if (error_sqr_dists_.empty ())
        {
          PCL_ERROR ("[pcl::SampleConsensusModel::computeVariance] The variance of the Sample Consensus model distances cannot be estimated, as the model has not been computed yet. Please compute the model first or at least run selectWithinDistance before continuing. Returning NAN!\n");
          return (std::numeric_limits<double>::quiet_NaN ());
        }
        return (computeVariance (error_sqr_dists_));
      }

The error_square_dists_.empty () expression is returning true, but I don't understand why this would be so, after I have successfully run the RANSAC computation and received valid results with correct model coefficients.

Am I using this computeVariance() method incorrectly? If so, what would be a better way to get a "quality score" for the RANSAC fitting?

I looked into the Circle3D model implementation, and it does not fill error_sqr_dists_ (in contrast to most other models).
You might also want to check the implementation of computeVariance, as the term "variance" might be misleading. It seems more like a median distance computation.

Regarding a quality score: RANSAC uses the number of inliers (number of points where distance to model is less than user specified threshold) to decide which model is best. You can get the inliers of the final model via getInliers and warn if there are too few inliers.

I hope this helps.