Java中如果要流式处理帧数据，但又不想同步处理，只想根据CPU能力，能运算多少算多少

Tags: /Android/ Date Created:

如何按CPU运算能力抽帧运算数据，例如视频流中，CPU能运算多少，就抽多少帧进行运算？

例如在摄像头视频流OnPreviewFrame中，根据CPU的运算能力，动态进行人脸识别或者TensorFlow物体识别，并且不影响帧率？

public void onPreviewFrame(byte[] bytes, Camera camera) {
    // 从回调数据解码 YUV 图像
    mCamera.addCallbackBuffer(callbackBuffer);
    Bitmap bitmap;

    if (rotate) {
        Bitmap raw = NV21ToBitmap(bytes, previewSize.width, previewSize.height);
        Matrix matrix = new Matrix();
        matrix.postRotate(270);
        bitmap = Bitmap.createBitmap(raw, 0, 0, raw.getWidth(), raw.getHeight(), matrix, false);
        raw.recycle();
    } else {
        bitmap = NV21ToBitmap(bytes, previewSize.width, previewSize.height);
    }

    detectFace(bitmap);
    detectObject(bitmap);
    drawWatermark(bitmap);
    encodeVideo(bitmap);
    showPreview(bitmap);

    bitmap.recycle();
}

例如 detectFace()和detectObject很消耗CPU，该如何处理？

答案是利用线程和来处理：

首先定义类变量：

private final Handler handlerObjectDetect;

private final HandlerThread threadObjectDetect;
并初始化：
threadObjectDetect = new HandlerThread("YOLO Detect");
threadObjectDetect.start();
handlerObjectDetect = new Handler(threadObjectDetect.getLooper());

然后定义类变量，例如computingDetection，防止重入：

if (computingDetection) {
    Canvas canvas = new Canvas(bitmap);
    for (int i = 0; i < detectResult.size() - 1; i++) {
        RectF r = detectResult.get(i).getLocation();
        paint.setTextSize(bitmap.getWidth() / 30);
        canvas.drawText(detectResult.get(i).getTitle(), r.left, r.bottom, paint);
        canvas.drawRect(r, objectPaint);
    }
    return;
}

最后用handerObjectDetect.Post()来运行代码即可：

handlerObjectDetect.post(new Runnable() {
    @Override
    public void run() {
        computingDetection = true;

        final List<Classifier.Recognition> results = detector.recognizeImage(croppedBitmap);
        Canvas canvas = new Canvas(croppedBitmap);

        detectResult.clear();
        for (final Classifier.Recognition result : results) {
            final RectF location = result.getLocation();
            if (location != null && result.getConfidence() >= MINIMUM_CONFIDENCE_YOLO) {
                canvas.drawRect(location, objectPaint);
                location.left = location.left / YOLO_INPUT_SIZE * bitmap.getWidth();
                location.top = location.top / YOLO_INPUT_SIZE * bitmap.getHeight();
                location.right = location.right / YOLO_INPUT_SIZE * bitmap.getWidth();
                location.bottom = location.bottom / YOLO_INPUT_SIZE * bitmap.getHeight();
                result.setLocation(location);
                detectResult.add(result);
            }
        }
        croppedBitmap.recycle();
        computingDetection = false;
    }
});