As soon as it comes to applying a blur with a large kernel size, then, unfortunately, an irreversible deformation of the object of interest is obtained. The object is outlined and outlined in contours, but at the same time in its size, shape (and sometimes its position) may have a significant discrepancy with the original. Needless to say, morphological operations (erosion, dilatation) superimposed on top only exacerbate the problem.
In the situation with the presented images, the difficulty arises from the fact that the conveyor belt has very distinguishable traces of industrial use. These are scratches, stains and other elements of this kind, which have a different position in the background image and the image with the object of interest, and generally differ in details in details. This moment becomes crucial when a pixel-by-pixel subtraction of one image from another is performed.
In general, the task is to ensure that without damaging the object itself, try to make it stand out against the general background. Let's try to go from the opposite, and instead of applying a blur, increase the sharpness of the edges.
void subtractPyr(const cv::Mat &src_mat, cv::Mat &dst_mat, int iterations) { std::vector<cv::Size> sizes(iterations); cv::Mat mat = src_mat.clone(); for(int i = 0; i < iterations; ++i) { sizes[iterations-(i+1)] = mat.size(); cv::pyrDown(mat, mat); } for(int i = 0; i < iterations; ++i) cv::pyrUp(mat, mat, sizes.at(i)); cv::subtract(mat, src_mat, dst_mat); } cv::Mat src1_mat = cv::imread("foreground.jpg"); cv::Mat src2_mat = cv::imread("background.jpg"); cv::Mat gry1_mat, gry2_mat; cv::cvtColor(src1_mat, gry1_mat, cv::COLOR_BGR2GRAY); cv::cvtColor(src2_mat, gry2_mat, cv::COLOR_BGR2GRAY); subtractPyr(gry1_mat, gry1_mat, 5); subtractPyr(gry2_mat, gry2_mat, 5);
Image with object of interest:
Background Image:
Despite the fact that the background image has its own set of noise, it nevertheless contains those details that are useful to subtract from the image with the object of interest. After that, binarization can be performed.
cv::Mat gry_mat = gry1_mat - gry2_mat; cv::Mat bin_mat; cv::threshold(gry_mat, bin_mat, 1, 255, cv::THRESH_BINARY | cv::THRESH_OTSU);
Get the following mask:
One of the problems associated with incorrectly selected lighting is clearly visible here. The upper left corner of the object of interest is partially dissolved, which does not allow searching for a contour encircling the object with the correct result.
One way or another, we will continue precisely through the search for contours and sifting them out by area with an empirically selected size. Their hierarchy is no longer required, which means we will merge all the points into one single large contour.
std::vector<std::vector<cv::Point> > contours; cv::findContours(bin_mat.clone(), contours , cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE); std::vector<cv::Point> big_contour; for(auto itr = contours.begin(); itr != contours.end(); ++itr) { if(cv::contourArea(*itr) > 300.) { const std::vector<cv::Point> &contour = *itr; for(auto itr2 = contour.begin(); itr2 != contour.end(); ++itr2) { big_contour.push_back(*itr2); } } }
If you draw a large outline, you can see that the extra details in the image are no longer captured, and he as a whole lies within the boundaries of the object of interest.
It remains to obtain a rectangle enclosing the object of interest.
cv::RotatedRect rc = cv::minAreaRect(big_contour); cv::Point2f points[4]; rc.points(points); // ΠΠ°ΡΠΈΡΡΠ΅ΠΌ Π΅Π³ΠΎ. for(int i = 0; i < 4; ++i) cv::line(dst_mat, points[i], points[(i+1)%4], cv::Scalar(0,0,255));
As is clearly seen in the following image, the resulting rectangle does not perfectly describe the object of interest. This problem is associated with the optical deformation of this object itself, as well as with the presence of a rather wide shadow present on its left side. Unfortunately, these problems can not be eliminated except for the adjustment of the camera lens and lighting.
