A helper component class for efficiently obtaining camera frames from WebCamTexture and converting them to OpenCV Mat format in real-time using AsyncGPUReadback. More...

Public Member Functions
override bool	DidUpdateThisFrame ()
	Use this to check if the Mat has changed since the last frame. Since it would not make sense to do expensive video processing in each Update call, check this value before doing any processing.

override void	Dispose ()
	Releases all resource used by the WebCamTexture2MatAsyncGPUHelper object.

override Mat	GetMat ()
	Get the mat of the current frame.

virtual float	GetMatUpdateFPS ()
	Return the frame rate at which the Mat is updated. (interval at which the DidUpdateThisFrame() method becomes true).

override void	Pause ()
	Pause the active camera.

override void	Play ()
	Start the active camera.

override void	Stop ()
	Stop the active camera.

Public Member Functions inherited from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper
virtual Source2MatHelperColorFormat	GetBaseColorFormat ()
	Return the video base color format.

virtual Color32[]	GetBufferColors ()
	Get the buffer colors.

virtual Matrix4x4	GetCameraToWorldMatrix ()
	Return the camera to world matrix.

virtual string	GetDeviceName ()
	Return the active camera device name.

virtual float	GetFPS ()
	Return the active camera framerate.

virtual int	GetHeight ()
	Return the active camera height.

virtual Matrix4x4	GetProjectionMatrix ()
	Return the projection matrix matrix.

virtual WebCamDevice	GetWebCamDevice ()
	Return the active WebcamDevice.

virtual WebCamTexture	GetWebCamTexture ()
	Return the active WebcamTexture.

virtual int	GetWidth ()
	Return the active camera width.

virtual void	Initialize (bool autoPlay=true)
	Initialize this instance.

virtual void	Initialize (bool requestedIsFrontFacing, float requestedFPS=30f, bool rotate90Degree=false, bool autoPlay=true)
	Initialize this instance.

virtual void	Initialize (int requestedWidth, int requestedHeight, bool autoPlay=true)
	Initialize this instance.

virtual void	Initialize (string deviceName, int requestedWidth, int requestedHeight, bool requestedIsFrontFacing=false, float requestedFPS=30f, bool rotate90Degree=false, bool autoPlay=true)
	Initialize this instance.

virtual bool	IsFrontFacing ()
	Indicate whether the active camera device is currently front facng.

virtual bool	IsInitialized ()
	Indicate whether this instance has been initialized.

virtual bool	IsPaused ()
	Indicate whether the camera is paused.

virtual bool	IsPlaying ()
	Indicate whether the active camera is currently playing.

Public Member Functions inherited from OpenCVForUnity.UnityUtils.Helper.ICameraSource2MatHelper
Public Member Functions inherited from OpenCVForUnity.UnityUtils.Helper.ISource2MatHelper
Public Member Functions inherited from OpenCVForUnity.UnityUtils.Helper.IMatUpdateFPSProvider

Protected Member Functions
override IEnumerator	_Initialize ()
	Initialize this instance by coroutine.

virtual IEnumerator	_WaitForEndOfFrameCoroutine ()

virtual void	CallReadback ()

virtual void	CancelWaitForEndOfFrameCoroutine ()
	Cancel WaitForEndOfFrame Coroutine.

virtual void	OnCompleteReadback (AsyncGPUReadbackRequest request)

override void	OnValidate ()

override void	ReleaseResources ()
	To release the resources.

override void	Update ()

Protected Member Functions inherited from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper
virtual void	CancelInitCoroutine ()
	Cancel Init Coroutine.

virtual Mat	FlipAndRotateMat (Mat frameMat, Mat rotatedFrameMat, bool flipVertical, bool flipHorizontal, bool isFrontFacing, int videoRotationAngle, ScreenOrientation screenOrientation, bool isScreenOrientationCorrect, bool isVidevideoRotationAngleCorrect)

virtual void	FlipMat (Mat mat, bool flipVertical, bool flipHorizontal, bool isFrontFacing, int videoRotationAngle)
	Flip Mat.

virtual IEnumerator	hasUserAuthorizedCameraPermission ()
	Check camera permission state by coroutine.

virtual bool	IsScreenOrientationCorrect ()
	Is ScreenOrientation the correct value?

virtual bool	IsVideoRotationAngleCorrect ()
	Is VideoRotationAngle the correct value?

virtual bool	NeedsRotatedFrameMat (bool isScreenOrientationCorrect)
	Determines whether the RotatedFrameMat is needed.

virtual IEnumerator	OnApplicationFocus (bool hasFocus)

virtual void	OnDestroy ()
	Raises the destroy event.

Protected Attributes
float	_requestedMatUpdateFPS = 30f

AsyncGPUReadbackRequest	asyncGPUReadbackRequestBuffer
	asyncGPUReadbackRequestBuffer

bool	didUpdateThisFrame = false
	Whether the mat that can be obtained with the GetMat method has been updated in this frame. This flag is changed after waiting until WaitForEndOfFrame in the coroutine.

RenderTexture	renderTexture
	The RenderTexture.

IEnumerator	waitForEndOfFrameCoroutine
	The waitForEndOfFrameCoroutine.

Protected Attributes inherited from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper
bool	_flipHorizontal = false

bool	_flipVertical = false

UnityEvent	_onDisposed

Source2MatHelperErrorUnityEvent	_onErrorOccurred

UnityEvent	_onInitialized

Source2MatHelperColorFormat	_outputColorFormat = Source2MatHelperColorFormat.RGBA

string	_requestedDeviceName = null

float	_requestedFPS = 30f

int	_requestedHeight = 480

bool	_requestedIsFrontFacing = false

int	_requestedWidth = 640

bool	_rotate90Degree = false

int	_timeoutFrameCount = 1500

bool	autoPlayAfterInitialize
	If set to true play after completion of initialization.

Source2MatHelperColorFormat	baseColorFormat = Source2MatHelperColorFormat.RGBA
	The base color format.

Mat	baseMat
	The base mat.

Color32[]	colors
	The buffer colors.

Mat	frameMat
	The frame mat.

bool	hasInitDone = false
	Indicates whether this instance has been initialized.

IEnumerator	initCoroutine
	The initialization coroutine.

bool	isInitWaiting = false
	Indicates whether this instance is waiting for initialization to complete.

bool	isScreenOrientationCorrect
	Is ScreenOrientation the correct value?

bool	isScreenSizeChangeWaiting = false
	isScreenSizeChangeWaiting

bool	isVideoRotationAngleCorrect
	Is VideoRotationAngle the correct value?

Mat	rotatedFrameMat
	The rotated frame mat.

int	screenHeight
	The height of the screen.

ScreenOrientation	screenOrientation
	The orientation of the screen.

int	screenWidth
	The width of the screen.

WebCamDevice	webCamDevice
	The active WebcamDevice.

WebCamTexture	webCamTexture
	The active WebcamTexture.

Properties
virtual float	requestedMatUpdateFPS `[get, set]`
	Sets the frame rate at which the Mat is updated. (interval at which the DidUpdateThisFrame() method becomes true).

Properties inherited from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper
virtual bool	flipHorizontal `[get, set]`
	Set whether to flip horizontal.

virtual bool	flipVertical `[get, set]`
	Set whether to flip vertically.

UnityEvent	onDisposed `[get, set]`
	UnityEvent that is triggered when this instance is disposed.

Source2MatHelperErrorUnityEvent	onErrorOccurred `[get, set]`
	UnityEvent that is triggered when this instance is error Occurred.

UnityEvent	onInitialized `[get, set]`
	UnityEvent that is triggered when this instance is initialized.

virtual Source2MatHelperColorFormat	outputColorFormat `[get, set]`
	Select the output color format.

virtual string	requestedDeviceName `[get, set]`
	Set the name of the camera device to use. (or device index number)

virtual float	requestedFPS `[get, set]`
	Set the frame rate of camera.

virtual int	requestedHeight `[get, set]`
	Set the height of camera.

virtual bool	requestedIsFrontFacing `[get, set]`
	Set whether to use the front facing camera.

virtual int	requestedWidth `[get, set]`
	Set the width of camera.

virtual bool	rotate90Degree `[get, set]`
	Set whether to rotate camera frame 90 degrees. (clockwise)

virtual int	timeoutFrameCount `[get, set]`
	The number of frames before the initialization process times out.

Properties inherited from OpenCVForUnity.UnityUtils.Helper.ICameraSource2MatHelper
Properties inherited from OpenCVForUnity.UnityUtils.Helper.ISource2MatHelper
Properties inherited from OpenCVForUnity.UnityUtils.Helper.IMatUpdateFPSProvider

Detailed Description

A helper component class for efficiently obtaining camera frames from WebCamTexture and converting them to OpenCV Mat format in real-time using AsyncGPUReadback.

The WebCamTexture2MatAsyncGPUHelper class captures video frames from a device's camera using WebCamTexture and utilizes AsyncGPUReadback to efficiently transfer pixel data to the CPU. Each frame is then converted to an OpenCV Mat object. This component handles camera orientation, rotation, and necessary transformations to ensure the Mat output aligns correctly with the device's display orientation.

By leveraging AsyncGPUReadback, this component reduces CPU load and enhances performance, making it particularly suitable for applications requiring real-time image processing with high frame rates. It enables seamless integration of OpenCV-based image processing algorithms with Unity's camera input while optimizing resource usage.

Note: By setting outputColorFormat to RGBA, processing that does not include extra color conversion is performed.

Attach this component to a GameObject and call GetMat() to retrieve the latest camera frame in Mat format. The helper class manages camera start/stop operations, AsyncGPUReadback requests, and frame updates internally.

Member Function Documentation

◆ _Initialize()

override IEnumerator OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper._Initialize ( )

protectedvirtual

Initialize this instance by coroutine.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ _WaitForEndOfFrameCoroutine()

virtual IEnumerator OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper._WaitForEndOfFrameCoroutine ( )

protectedvirtual

◆ CallReadback()

virtual void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.CallReadback ( )

protectedvirtual

◆ CancelWaitForEndOfFrameCoroutine()

virtual void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.CancelWaitForEndOfFrameCoroutine ( )

protectedvirtual

Cancel WaitForEndOfFrame Coroutine.

◆ DidUpdateThisFrame()

override bool OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.DidUpdateThisFrame ( )

virtual

Use this to check if the Mat has changed since the last frame. Since it would not make sense to do expensive video processing in each Update call, check this value before doing any processing.

Returns: true, if the Mat has been updated false otherwise.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ Dispose()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.Dispose ( )

virtual

Releases all resource used by the WebCamTexture2MatAsyncGPUHelper object.

Call Dispose when you are finished using the WebCamTexture2MatAsyncGPUHelper. The Dispose method leaves the WebCamTexture2MatAsyncGPUHelper in an unusable state. After calling Dispose, you must release all references to the WebCamTexture2MatAsyncGPUHelper so the garbage collector can reclaim the memory that the WebCamTexture2MatAsyncGPUHelper was occupying.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ GetMat()

override Mat OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.GetMat ( )

virtual

Get the mat of the current frame.

The Mat object's type is 'CV_8UC4' or 'CV_8UC3' or 'CV_8UC1' (ColorFormat is determined by the outputColorFormat setting). Please do not dispose of the returned mat as it will be reused.

Returns: The mat of the current frame.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ GetMatUpdateFPS()

virtual float OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.GetMatUpdateFPS ( )

virtual

Return the frame rate at which the Mat is updated. (interval at which the DidUpdateThisFrame() method becomes true).

Returns: The active camera framerate.

Implements OpenCVForUnity.UnityUtils.Helper.IMatUpdateFPSProvider.

◆ OnCompleteReadback()

virtual void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.OnCompleteReadback ( AsyncGPUReadbackRequest request )

protectedvirtual

◆ OnValidate()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.OnValidate ( )

protectedvirtual

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ Pause()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.Pause ( )

virtual

Pause the active camera.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ Play()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.Play ( )

virtual

Start the active camera.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ ReleaseResources()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.ReleaseResources ( )

protectedvirtual

To release the resources.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ Stop()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.Stop ( )

virtual

Stop the active camera.

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

◆ Update()

override void OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.Update ( )

protectedvirtual

Reimplemented from OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatHelper.

Member Data Documentation

◆ _requestedMatUpdateFPS

float OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper._requestedMatUpdateFPS = 30f

protected

◆ asyncGPUReadbackRequestBuffer

AsyncGPUReadbackRequest OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.asyncGPUReadbackRequestBuffer

protected

asyncGPUReadbackRequestBuffer

◆ didUpdateThisFrame

bool OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.didUpdateThisFrame = false

protected

Whether the mat that can be obtained with the GetMat method has been updated in this frame. This flag is changed after waiting until WaitForEndOfFrame in the coroutine.

◆ renderTexture

RenderTexture OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.renderTexture

protected

The RenderTexture.

◆ waitForEndOfFrameCoroutine

IEnumerator OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.waitForEndOfFrameCoroutine

protected

The waitForEndOfFrameCoroutine.

Property Documentation

◆ requestedMatUpdateFPS

virtual float OpenCVForUnity.UnityUtils.Helper.WebCamTexture2MatAsyncGPUHelper.requestedMatUpdateFPS

getset

Sets the frame rate at which the Mat is updated. (interval at which the DidUpdateThisFrame() method becomes true).

Implements OpenCVForUnity.UnityUtils.Helper.IMatUpdateFPSProvider.

The documentation for this class was generated from the following file:

OpenCVForUnity/Assets/OpenCVForUnity/org/opencv/unity/Helper/Source2MatHelper/WebCamTexture2MatAsyncGPUHelper.cs

Public Member Functions

Protected Member Functions

Protected Attributes

Properties

Detailed Description

Member Function Documentation

◆ _Initialize()

◆ _WaitForEndOfFrameCoroutine()

◆ CallReadback()

◆ CancelWaitForEndOfFrameCoroutine()

◆ DidUpdateThisFrame()

◆ Dispose()

◆ GetMat()

◆ GetMatUpdateFPS()

◆ OnCompleteReadback()

◆ OnValidate()

◆ Pause()

◆ Play()

◆ ReleaseResources()

◆ Stop()

◆ Update()

Member Data Documentation

◆ _requestedMatUpdateFPS

◆ asyncGPUReadbackRequestBuffer

◆ didUpdateThisFrame

◆ renderTexture

◆ waitForEndOfFrameCoroutine

Property Documentation

◆ requestedMatUpdateFPS