|
OpenCV for Unity 3.0.4
Enox Software / Please refer to OpenCV official document ( http://docs.opencv.org/5.0/index.html ) for the details of the argument of the method.
|
This class allows to create and manipulate comprehensive artificial neural networks. More...
Public Member Functions | |
| Net () | |
| void | connect (string outPin, string inpPin) |
| Connects output of the first layer to input of the second layer. | |
| void | disableKVCache () |
| Disables KV-Cache for all AttentionOnnxI layers. | |
| string | dump () |
| Dump net to String. | |
| void | dumpToFile (string path) |
| Dump net structure, hyperparameters, backend, target and fusion to dot file. | |
| void | dumpToPbtxt (string path) |
| Dump net structure, hyperparameters, backend, target and fusion to pbtxt file. | |
| bool | empty () |
| void | enableFusion (bool fusion) |
| Enables or disables layer fusion in the network. | |
| void | enableKVCache () |
| Enables KV-Cache for all AttentionOnnxI layers. | |
| void | enableWinograd (bool useWinograd) |
| Enables or disables the Winograd compute branch. The Winograd compute branch can speed up 3x3 Convolution at a small loss of accuracy. | |
| void | finalizeNet () |
| Finalizes the network configuration and prepares it for inference. | |
| Mat | forward () |
Runs forward pass to compute output of layer with name outputName. | |
| void | forward (List< Mat > outputBlobs) |
Runs forward pass to compute output of layer with name outputName. | |
| void | forward (List< Mat > outputBlobs, List< string > outBlobNames) |
Runs forward pass to compute outputs of layers listed in outBlobNames. | |
| void | forward (List< Mat > outputBlobs, string outputName) |
Runs forward pass to compute output of layer with name outputName. | |
| Mat | forward (string outputName) |
Runs forward pass to compute output of layer with name outputName. | |
| void | forwardAndRetrieve (List< List< Mat > > outputBlobs, List< string > outBlobNames) |
Runs forward pass to compute outputs of layers listed in outBlobNames. | |
| long | getFLOPS (List< MatOfInt > netInputShapes, MatOfInt netInputTypes) |
| Computes FLOP for whole loaded model with specified input shapes. | |
| Layer | getLayer (DictValue layerId) |
| Layer | getLayer (int layerId) |
| Returns pointer to layer with specified id or name which the network use. | |
| Layer | getLayer (string layerName) |
| int | getLayerId (string layer) |
| Converts string name of the layer to the integer identifier. | |
| List< string > | getLayerNames () |
| int | getLayersCount (string layerType) |
| Returns count of layers of specified type. | |
| void | getLayerShapes (List< MatOfInt > netInputShapes, MatOfInt netInputTypes, int layerId, List< MatOfInt > inLayerShapes, List< MatOfInt > outLayerShapes) |
| void | getLayerTypes (List< string > layersTypes) |
| Returns list of types for layer used in model. | |
| void | getMemoryConsumption (List< MatOfInt > netInputShapes, MatOfInt netInputTypes, long[] weights, long[] blobs) |
| Computes bytes number which are required to store all weights and intermediate blobs for model. | |
| IntPtr | getNativeObjAddr () |
| Mat | getParam (int layer) |
| Returns parameter blob of the layer. | |
| Mat | getParam (int layer, int numParam) |
| Returns parameter blob of the layer. | |
| Mat | getParam (string layerName) |
| Mat | getParam (string layerName, int numParam) |
| void | getPerfProfile (List< string > names, List< string > timems, List< string > counts) |
| Returns profiling data captured during the last forward pass. | |
| long | getPerfProfile (MatOfDouble timings) |
| Returns overall time for inference and timings (in ticks) for layers. | |
| MatOfInt | getUnconnectedOutLayers () |
| Returns indexes of layers with unconnected outputs. | |
| List< string > | getUnconnectedOutLayersNames () |
| Returns names of layers with unconnected outputs. | |
| void | printPerfProfile () |
| Prints the profile captured during the last forward pass in a formatted table using CV_LOG_INFO. | |
| int | registerOutput (string outputName, int layerId, int outputPort) |
| Registers network output with name. | |
| void | resetKVCache () |
| Resets KV-Cache for all AttentionOnnxI layers. | |
| void | setInput (Mat blob) |
| Sets the new input value for the network. | |
| void | setInput (Mat blob, string name) |
| Sets the new input value for the network. | |
| void | setInput (Mat blob, string name, double scalefactor) |
| Sets the new input value for the network. | |
| void | setInput (Mat blob, string name, double scalefactor, in Vec4d mean) |
| Sets the new input value for the network. | |
| void | setInput (Mat blob, string name, double scalefactor, in(double v0, double v1, double v2, double v3) mean) |
| Sets the new input value for the network. | |
| void | setInput (Mat blob, string name, double scalefactor, Scalar mean) |
| Sets the new input value for the network. | |
| void | setInputShape (string inputName, MatOfInt shape) |
| Specify shape of network input. | |
| void | setInputsNames (List< string > inputBlobNames) |
| Sets outputs names of the network input pseudo layer. | |
| void | setParam (int layer, int numParam, Mat blob) |
| Sets the new value for the learned param of the layer. | |
| void | setParam (string layerName, int numParam, Mat blob) |
| Sets the parameter blob of a layer identified by its name or output tensor name. | |
| void | setPreferableBackend (int backendId) |
| Ask network to use specific computation backend where it supported. | |
| void | setPreferableTarget (int targetId) |
| Ask network to make computations on specific target device. | |
Public Member Functions inherited from OpenCVForUnity.DisposableObject | |
| void | Dispose () |
| Releases resources used by this object. | |
| void | ThrowIfDisposed () |
| Throws ObjectDisposedException if this object has been disposed. | |
Static Public Member Functions | |
| static Net | __fromPtr__ (IntPtr addr) |
| static Net | readFromModelOptimizer (MatOfByte bufferModelConfig, MatOfByte bufferWeights) |
| Create a network from Intel's Model Optimizer in-memory buffers with intermediate representation (IR). | |
| static Net | readFromModelOptimizer (string xml, string bin) |
| Create a network from Intel's Model Optimizer intermediate representation (IR). | |
Static Public Member Functions inherited from OpenCVForUnity.DisposableObject | |
| static IntPtr | ThrowIfNullIntPtr (IntPtr ptr) |
| Returns the native pointer, or throws CoreModule.CvException if it is zero. | |
Protected Member Functions | |
| override void | Dispose (bool disposing) |
Protected Member Functions inherited from OpenCVForUnity.DisposableOpenCVObject | |
| DisposableOpenCVObject () | |
| Initializes a new instance with a zero native pointer and dispose enabled. | |
| DisposableOpenCVObject (bool isEnabledDispose) | |
| Initializes a new instance with a zero native pointer. | |
| DisposableOpenCVObject (IntPtr ptr) | |
| Initializes a new instance with the specified native pointer and dispose enabled. | |
| DisposableOpenCVObject (IntPtr ptr, bool isEnabledDispose) | |
| Initializes a new instance with the specified native pointer. | |
Protected Member Functions inherited from OpenCVForUnity.DisposableObject | |
| DisposableObject () | |
| Initializes a new instance with dispose enabled. | |
| DisposableObject (bool isEnabledDispose) | |
| Initializes a new instance. | |
Additional Inherited Members | |
Package Attributes inherited from OpenCVForUnity.DisposableOpenCVObject | |
Properties inherited from OpenCVForUnity.DisposableObject | |
| bool | IsDisposed [get, protected set] |
| bool | IsEnabledDispose [get, set] |
This class allows to create and manipulate comprehensive artificial neural networks.
Neural network is presented as directed acyclic graph (DAG), where vertices are Layer instances, and edges specify relationships between layers inputs and outputs.
Each network layer has unique integer id and unique string name inside its network. LayerId can store either layer name or layer id.
This class supports reference counting of its instances, i. e. copies point to the same instance.
| OpenCVForUnity.DnnModule.Net.Net | ( | ) |
|
static |
| void OpenCVForUnity.DnnModule.Net.connect | ( | string | outPin, |
| string | inpPin ) |
Connects output of the first layer to input of the second layer.
| outPin | descriptor of the first layer output. |
| inpPin | descriptor of the second layer input. |
Descriptors have the following template
| void OpenCVForUnity.DnnModule.Net.disableKVCache | ( | ) |
Disables KV-Cache for all AttentionOnnxI layers.
|
protectedvirtual |
Reimplemented from OpenCVForUnity.DisposableOpenCVObject.
| string OpenCVForUnity.DnnModule.Net.dump | ( | ) |
Dump net to String.
| void OpenCVForUnity.DnnModule.Net.dumpToFile | ( | string | path | ) |
Dump net structure, hyperparameters, backend, target and fusion to dot file.
| path | path to output file with .dot extension |
| void OpenCVForUnity.DnnModule.Net.dumpToPbtxt | ( | string | path | ) |
Dump net structure, hyperparameters, backend, target and fusion to pbtxt file.
| path | path to output file with .pbtxt extension |
Use Netron (https://netron.app) to open the target file to visualize the model. Call method after setInput(). To see correct backend, target and fusion run after forward().
| bool OpenCVForUnity.DnnModule.Net.empty | ( | ) |
Returns true if there are no layers in the network.
| void OpenCVForUnity.DnnModule.Net.enableFusion | ( | bool | fusion | ) |
Enables or disables layer fusion in the network.
| fusion | true to enable the fusion, false to disable. The fusion is enabled by default. |
| void OpenCVForUnity.DnnModule.Net.enableKVCache | ( | ) |
Enables KV-Cache for all AttentionOnnxI layers.
| void OpenCVForUnity.DnnModule.Net.enableWinograd | ( | bool | useWinograd | ) |
Enables or disables the Winograd compute branch. The Winograd compute branch can speed up 3x3 Convolution at a small loss of accuracy.
| useWinograd | true to enable the Winograd compute branch. The default is true. |
| void OpenCVForUnity.DnnModule.Net.finalizeNet | ( | ) |
Finalizes the network configuration and prepares it for inference.
This method must be called after setting backend/target via setPreferableBackend() and setPreferableTarget(), and before the first forward() call. It creates the underlying execution session (e.g. ONNX Runtime session) on the configured backend/target. If not called explicitly, the first forward() will call it automatically.
Calling finalizeNet() early lets you pay the one-time setup cost at a predictable point and catch configuration errors before inference.
| Mat OpenCVForUnity.DnnModule.Net.forward | ( | ) |
Runs forward pass to compute output of layer with name outputName.
| outputName | name for layer which output is needed to get |
blob for first output of specified layer.
By default runs forward pass for the whole network.
| void OpenCVForUnity.DnnModule.Net.forward | ( | List< Mat > | outputBlobs | ) |
Runs forward pass to compute output of layer with name outputName.
| outputBlobs | contains all output blobs for specified layer. |
| outputName | name for layer which output is needed to get |
If outputName is empty, runs forward pass for the whole network.
| void OpenCVForUnity.DnnModule.Net.forward | ( | List< Mat > | outputBlobs, |
| List< string > | outBlobNames ) |
Runs forward pass to compute outputs of layers listed in outBlobNames.
| outputBlobs | contains blobs for first outputs of specified layers. |
| outBlobNames | names for layers which outputs are needed to get |
| void OpenCVForUnity.DnnModule.Net.forward | ( | List< Mat > | outputBlobs, |
| string | outputName ) |
Runs forward pass to compute output of layer with name outputName.
| outputBlobs | contains all output blobs for specified layer. |
| outputName | name for layer which output is needed to get |
If outputName is empty, runs forward pass for the whole network.
| Mat OpenCVForUnity.DnnModule.Net.forward | ( | string | outputName | ) |
Runs forward pass to compute output of layer with name outputName.
| outputName | name for layer which output is needed to get |
blob for first output of specified layer.
By default runs forward pass for the whole network.
| void OpenCVForUnity.DnnModule.Net.forwardAndRetrieve | ( | List< List< Mat > > | outputBlobs, |
| List< string > | outBlobNames ) |
Runs forward pass to compute outputs of layers listed in outBlobNames.
| outputBlobs | contains all output blobs for each layer specified in outBlobNames. |
| outBlobNames | names for layers which outputs are needed to get |
| long OpenCVForUnity.DnnModule.Net.getFLOPS | ( | List< MatOfInt > | netInputShapes, |
| MatOfInt | netInputTypes ) |
Computes FLOP for whole loaded model with specified input shapes.
| netInputShapes | vector of shapes for all net inputs. |
| netInputTypes | vector of types for all net inputs. |
This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
| Layer OpenCVForUnity.DnnModule.Net.getLayer | ( | int | layerId | ) |
Returns pointer to layer with specified id or name which the network use.
| Layer OpenCVForUnity.DnnModule.Net.getLayer | ( | string | layerName | ) |
This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
| int OpenCVForUnity.DnnModule.Net.getLayerId | ( | string | layer | ) |
Converts string name of the layer to the integer identifier.
| List< string > OpenCVForUnity.DnnModule.Net.getLayerNames | ( | ) |
| int OpenCVForUnity.DnnModule.Net.getLayersCount | ( | string | layerType | ) |
Returns count of layers of specified type.
| layerType | type. |
| void OpenCVForUnity.DnnModule.Net.getLayerShapes | ( | List< MatOfInt > | netInputShapes, |
| MatOfInt | netInputTypes, | ||
| int | layerId, | ||
| List< MatOfInt > | inLayerShapes, | ||
| List< MatOfInt > | outLayerShapes ) |
This is an overloaded member function, provided for convenience. It differs from the above function only in what argument(s) it accepts.
The only overload of getLayerShapes that should be kept in 5.x
| void OpenCVForUnity.DnnModule.Net.getLayerTypes | ( | List< string > | layersTypes | ) |
Returns list of types for layer used in model.
| layersTypes | output parameter for returning types. |
| void OpenCVForUnity.DnnModule.Net.getMemoryConsumption | ( | List< MatOfInt > | netInputShapes, |
| MatOfInt | netInputTypes, | ||
| long[] | weights, | ||
| long[] | blobs ) |
Computes bytes number which are required to store all weights and intermediate blobs for model.
| netInputShapes | vector of shapes for all net inputs. |
| netInputTypes | vector of types for all net inputs. |
| weights | output parameter to store resulting bytes for weights. |
| blobs | output parameter to store resulting bytes for intermediate blobs. |
| IntPtr OpenCVForUnity.DnnModule.Net.getNativeObjAddr | ( | ) |
| Mat OpenCVForUnity.DnnModule.Net.getParam | ( | int | layer | ) |
Returns parameter blob of the layer.
| layer | name or id of the layer. |
| numParam | index of the layer parameter in the Layer::blobs array. |
Layer::blobs
| Mat OpenCVForUnity.DnnModule.Net.getParam | ( | int | layer, |
| int | numParam ) |
Returns parameter blob of the layer.
| layer | name or id of the layer. |
| numParam | index of the layer parameter in the Layer::blobs array. |
Layer::blobs
| Mat OpenCVForUnity.DnnModule.Net.getParam | ( | string | layerName | ) |
| Mat OpenCVForUnity.DnnModule.Net.getParam | ( | string | layerName, |
| int | numParam ) |
| void OpenCVForUnity.DnnModule.Net.getPerfProfile | ( | List< string > | names, |
| List< string > | timems, | ||
| List< string > | counts ) |
Returns profiling data captured during the last forward pass.
Entries are sorted by time in descending order. Empty vectors are returned if profiling is disabled (DNN_PROFILE_NONE).
| long OpenCVForUnity.DnnModule.Net.getPerfProfile | ( | MatOfDouble | timings | ) |
Returns overall time for inference and timings (in ticks) for layers.
Indexes in returned vector correspond to layers ids. Some layers can be fused with others, in this case zero ticks count will be return for that skipped layers. Supported by DNN_BACKEND_OPENCV on DNN_TARGET_CPU only.
| timings | vector for tick timings for all layers. |
| MatOfInt OpenCVForUnity.DnnModule.Net.getUnconnectedOutLayers | ( | ) |
Returns indexes of layers with unconnected outputs.
FIXIT: Rework API to registerOutput() approach, deprecate this call
| List< string > OpenCVForUnity.DnnModule.Net.getUnconnectedOutLayersNames | ( | ) |
Returns names of layers with unconnected outputs.
FIXIT: Rework API to registerOutput() approach, deprecate this call
| void OpenCVForUnity.DnnModule.Net.printPerfProfile | ( | ) |
Prints the profile captured during the last forward pass in a formatted table using CV_LOG_INFO.
In DNN_PROFILE_DETAILED mode, prints per-layer label, time, and percentage. In DNN_PROFILE_SUMMARY mode, prints per-type count, time, and percentage. Does nothing if profiling is disabled (DNN_PROFILE_NONE) or all timings are zero.
|
static |
| int OpenCVForUnity.DnnModule.Net.registerOutput | ( | string | outputName, |
| int | layerId, | ||
| int | outputPort ) |
Registers network output with name.
Function may create additional 'Identity' layer.
| outputName | identifier of the output |
| layerId | identifier of the second layer |
| outputPort | number of the second layer input |
| void OpenCVForUnity.DnnModule.Net.resetKVCache | ( | ) |
Resets KV-Cache for all AttentionOnnxI layers.
| void OpenCVForUnity.DnnModule.Net.setInput | ( | Mat | blob | ) |
Sets the new input value for the network.
| blob | A new blob. Should have CV_32F or CV_8U depth. |
| name | A name of input layer. |
| scalefactor | An optional normalization scale. |
| mean | An optional mean subtraction values. |
connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
\[input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)\]
| void OpenCVForUnity.DnnModule.Net.setInput | ( | Mat | blob, |
| string | name ) |
Sets the new input value for the network.
| blob | A new blob. Should have CV_32F or CV_8U depth. |
| name | A name of input layer. |
| scalefactor | An optional normalization scale. |
| mean | An optional mean subtraction values. |
connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
\[input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)\]
| void OpenCVForUnity.DnnModule.Net.setInput | ( | Mat | blob, |
| string | name, | ||
| double | scalefactor ) |
Sets the new input value for the network.
| blob | A new blob. Should have CV_32F or CV_8U depth. |
| name | A name of input layer. |
| scalefactor | An optional normalization scale. |
| mean | An optional mean subtraction values. |
connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
\[input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)\]
| void OpenCVForUnity.DnnModule.Net.setInput | ( | Mat | blob, |
| string | name, | ||
| double | scalefactor, | ||
| in Vec4d | mean ) |
Sets the new input value for the network.
| blob | A new blob. Should have CV_32F or CV_8U depth. |
| name | A name of input layer. |
| scalefactor | An optional normalization scale. |
| mean | An optional mean subtraction values. |
connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
\[input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)\]
| void OpenCVForUnity.DnnModule.Net.setInput | ( | Mat | blob, |
| string | name, | ||
| double | scalefactor, | ||
| in(double v0, double v1, double v2, double v3) | mean ) |
Sets the new input value for the network.
| blob | A new blob. Should have CV_32F or CV_8U depth. |
| name | A name of input layer. |
| scalefactor | An optional normalization scale. |
| mean | An optional mean subtraction values. |
connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
\[input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)\]
| void OpenCVForUnity.DnnModule.Net.setInput | ( | Mat | blob, |
| string | name, | ||
| double | scalefactor, | ||
| Scalar | mean ) |
Sets the new input value for the network.
| blob | A new blob. Should have CV_32F or CV_8U depth. |
| name | A name of input layer. |
| scalefactor | An optional normalization scale. |
| mean | An optional mean subtraction values. |
connect(String, String) to know format of the descriptor.
If scale or mean values are specified, a final input blob is computed as:
\[input(n,c,h,w) = scalefactor \times (blob(n,c,h,w) - mean_c)\]
| void OpenCVForUnity.DnnModule.Net.setInputShape | ( | string | inputName, |
| MatOfInt | shape ) |
Specify shape of network input.
| void OpenCVForUnity.DnnModule.Net.setInputsNames | ( | List< string > | inputBlobNames | ) |
Sets outputs names of the network input pseudo layer.
Each net always has special own the network input pseudo layer with id=0. This layer stores the user blobs only and don't make any computations. In fact, this layer provides the only way to pass user data into the network. As any other layer, this layer can label its outputs and this function provides an easy way to do this.
| void OpenCVForUnity.DnnModule.Net.setParam | ( | int | layer, |
| int | numParam, | ||
| Mat | blob ) |
Sets the new value for the learned param of the layer.
| layer | name or id of the layer. |
| numParam | index of the layer parameter in the Layer::blobs array. |
| blob | the new value. |
Layer::blobs
| void OpenCVForUnity.DnnModule.Net.setParam | ( | string | layerName, |
| int | numParam, | ||
| Mat | blob ) |
Sets the parameter blob of a layer identified by its name or output tensor name.
| layerName | layer name (classic engine) or raw ONNX output tensor name (ENGINE_NEW). |
| numParam | index of the constant weight input to update (0 = kernel, 1 = bias, etc.). |
| blob | the new parameter value. |
| void OpenCVForUnity.DnnModule.Net.setPreferableBackend | ( | int | backendId | ) |
Ask network to use specific computation backend where it supported.
| backendId | backend identifier. |
Backend
| void OpenCVForUnity.DnnModule.Net.setPreferableTarget | ( | int | targetId | ) |
Ask network to make computations on specific target device.
| targetId | target identifier. |
Target
List of supported combinations backend / target:
| DNN_BACKEND_OPENCV | DNN_BACKEND_INFERENCE_ENGINE | DNN_BACKEND_CUDA | |
|---|---|---|---|
| DNN_TARGET_CPU | + | + | |
| DNN_TARGET_OPENCL | + | + | |
| DNN_TARGET_OPENCL_FP16 | + | + | |
| DNN_TARGET_MYRIAD | + | ||
| DNN_TARGET_FPGA | + | ||
| DNN_TARGET_CUDA | + | ||
| DNN_TARGET_CUDA_FP16 | + | ||
| DNN_TARGET_HDDL | + |