It is necessary to track the movement of one white dot on a black background. I use OpenCV and the CameraPreview class based on SurfaceView. How best to implement this?

My idea:

  1. We do a touch on the area where the point is, save the coordinates.
  2. In the onPreviewFrame method on call, we convert byte -> Mat object.
  3. Make a temp Mat object for comparison, cut it around the rectangle using the coordinates of the wheelbarrow. We will use this matrix to store the template.
  4. Call the matchtemplate () method
  5. Getting the point where the greatest comparison is is to draw a rectangle.

And so for each frame.

A few questions: What is the best way to transfer byte to Mat? What format is needed to create a matrix object from a byte array? What format is better for using match template?

And one more problem of mine: The resolution of my display is 1920 (height) 1080 (width), and the camera settings are vice versa 1920 (width) 1080 (height). At the output, I get an array of data with an inverted image. How to flip an array with bytes? Or is it better to do it with a Mat object?

Here is the code that I have at the moment. A pattern search is performed, but the results are not what you want.

public void onPreviewFrame(byte[] data, Camera camera) { if (scanSpace) { mainActivity = new MainActivity(); Log.d(TAG,"OnPreviewFRAME_START"); Camera.Parameters parameters = camera.getParameters(); int width = parameters.getPreviewSize().width; int height = parameters.getPreviewSize().height; Log.d(TAG, "Camera parameters: width: " + String.valueOf(width) + " height: " + String.valueOf(height) ); //convert the byte[] to Bitmap through YuvImage; //make sure the previewFormat is NV21 (I set it so somewhere before) YuvImage yuv = new YuvImage(data, parameters.getPreviewFormat(), width, height, null); ByteArrayOutputStream out = new ByteArrayOutputStream(); yuv.compressToJpeg(new Rect(0, 0, width, height), 100, out); Bitmap bmp = BitmapFactory.decodeByteArray(out.toByteArray(), 0, out.size()); Log.d(TAG, "Frame (picture) parameter: width: " + String.valueOf(bmp.getWidth()) + " height: " + String.valueOf(bmp.getHeight())); //convert Bitmap to Mat; note the bitmap config ARGB_8888 conversion that //allows you to use other image processing methods and still save at the end Log.d(TAG, "BMP size for search: HEIGHT: " + String.valueOf(bmp.getHeight()) + " WIDTH: " + String.valueOf(bmp.getWidth())); Mat orig = new Mat(bmp.getHeight(), bmp.getWidth(),CvType.CV_8UC3); bmp = bmp.copy(Bitmap.Config.ARGB_8888, true); Utils.bitmapToMat(bmp, orig); Imgproc.cvtColor(orig, orig, Imgproc.COLOR_BGR2GRAY,4); Log.d(TAG, "Old RES: " + String.valueOf(orig.rows()) + " " + String.valueOf(orig.cols())); Mat rotMat = new Mat(2, 3, CvType.CV_32FC1); Mat destination = new Mat(orig.rows(), orig.cols(), orig.type()); Point center = new Point(destination.cols() / 2, destination.rows() / 2); rotMat = Imgproc.getRotationMatrix2D(center, 90, 1); Imgproc.warpAffine(orig, destination, rotMat, destination.size()); Log.d(TAG, "NEW RES: " + String.valueOf(destination.rows()) + " " + String.valueOf(destination.cols())); Mat crop_orig = mainActivity.cropImage(orig); Log.d(TAG, "CROP IMAGE PARAMS: HEIGHT: " + String.valueOf(crop_orig.height()) + " WIDTH: " + String.valueOf(crop_orig.width())); Mat result = new Mat(); matchTemplate(orig, crop_orig, result, Imgproc.TM_SQDIFF); Core.MinMaxLocResult r = Core.minMaxLoc(result); System.out.println(r.minVal + " " + r.minLoc); Core.MinMaxLocResult mmr = Core.minMaxLoc(result); Point matchLoc = mmr.maxLoc; double tmp_x = matchLoc.x; double tmp_y = matchLoc.y; xPoint_1 = (int) tmp_x; yPoint_1 = (int) tmp_y; Log.d(TAG, "FINISH template"); scanSpace = false; camera.setPreviewCallback(this); } }

    0