Library Usage

DVI library can be integrated in a very flexible way in your applications. The following APIs are available:

• API C++

• API Python (Python binding of the C++ API)

In this page you will find a general description of the library usage for both C++ and Python API.

Warning

Before using the library, it’s mandatory to accept the DVI User License Agreeement. At this point, you may have already accepted it before running the Demo Application. If not, you can accept it by running the following command on a terminal on the device:

$ dvi_accept_ula

This step is required only once for each device.

DVI Instance Lifecycle

The foundational element of the library is the DVI class (C++ DVI / Python DVI). All the features of the DVI library can be used by following these steps of the DVI class lifecycle:

1. Create

2. Setup

3. Train / Load

4. Test

5. Save

In the following paragraphs a description of these steps is provided.

Create

The first step to use the library is to create an instance of DVI class. An instance of DVI will hold a single, individual defect model. However, multiple DVI instances can be created and used at the same time.

C++

To instantiate a DVI object, call the create method:

std::shared_ptr<dvi::DVI> model = dvi::DVI::create()

All the DVI methods are accessible through the returned shared pointer. The resources allocated by the DVI instance will be automatically released when the shared pointer goes out of scope.

Python

To instantiate a DVI object, create an instance of DVI class:

model = dvi.DVI()

All the DVI methods are accessible through the returned instance. The resources allocated by the DVI instance will be automatically released when the instance goes out of scope.

Setup

Once created, a DVI instance must be initialized by providing the image size it should work on.

C++

Call the setup method with the image size:

int width = 300;
int height = 400;
cv::Size image_size{width, height};
auto ret = model->setup(image_size);

Python

Call the setup method with image size as argument:

width, height = 300, 400
ret = model.setup(image_size=(width, height))

This method sets working resolution of DVI instance. Setup method can be called again later to reset an already setup/trained/loaded DVI instance, for example for working with a different image size or to clear the model.

Tip

Instead of calling setup method again, it’s possibile to let the object go out of scope and create a new one.

Warning

As of now, multiple existing DVI instances must work at the same image resolution. So, it’s not possible to have two different DVI instances setup with different image sizes. Attempting to setup a DVI instance with resolution different from the others will return an error (ERROR_WRONG_SETUP_SIZE_FOR_MULTIPLE_MODELS in C++ / Python).

Train / Load

After setup, DVI instance is ready to get a model for defect inspection. Model’s acquisition can occur in two different ways:

• Train a model with a collection of nominal images. Make sure to provide a good variety of nominal images to the model. Check the User Guide for more details on how to prepare a good training set.

C++

Call the train method with a vector of training images as input:

std::vector<cv::Mat> train_imgs;
auto ret = model->train(train_imgs);

Python

Call the train method with a list of training images as input argument:

ret = model.train(train_images)

Images must have resolution provided during Setup step.

• Load a previously trained model.

C++

Call the load method with model path as input argument:

std::string modelPath = "/tmp/model_path";
auto ret = model->load(modelPath);

Python

Call the load method with model path as input argument:

model_path = "/tmp/model_path"
ret = model.load(model_path)

Load method can be called with the path to a model previously saved. See Save section for details.

Warning

• Train and Load Method cannot be directly called on a newly created DVI Instance. For training or loading a DVI Instance, it’s mandatory to initialize it (see Setup for details).

• Train and Load step cannot be directly performed on a DVI instance that already holds a model. For re-training or loading a new model on a DVI instance, it’s mandatory to re-setup it (see Setup for details).

Test

Once training/loading step is completed, DVI instance holds a valid model that can be used to classify new unseen images.

C++

Call the test method.

std::vector<cv::Mat> test_imgs;
std::vector<double> scores;
std::vector<cv::Mat1d> scores_maps;
auto ret = model->test(test_imgs, scores, scores_maps);

Vectors scores and scores_maps will contain test results, in the same order of input images.

Python

Call the test method:

ret, scores, scores_maps = model.test(test_imgs)

This method works on a test set of images. Test images must have resolution equal to the one provided during Setup.

During test step, an anomaly score is assigned at each test image. Higher the score, higher the probability for that image to be anomalous. The score_maps vector will contain the anomaly score map for each test image. Each score map is a 2D matrix with the same dimensions of the input image, where each pixel contains the anomaly score for that pixel. This can be useful for visualizing the anomaly score distribution over the image or to identify where the main anomaly is located.

This method can be called multiple times on the same DVI instance, with different test sets. It’s possible for example to call it for each frame of a live video stream, by just providing the current frame as the single element of input vector.

Note

To classify images, the scores should be compared against a threshold value. A suggestion for the reference threshold value can be obtained by using a specific method (C++ getThresholdReference / Python threshold_reference). Threshold value can also be selected and tuned by user. Please keep in mind that threshold reference value is just a suggestion and it’s not guaranteed to be the best threshold for your specific use case.

Save

In any moment, if your DVI Instance contain a valid model (for example after a Train / Load or a Test ), you might desire to save your model.

C++

Call the save method.

std::string modelpath = "/tmp/my_model.dat";
ret = model->save(modelpath);

Python

Call the save method:

modelpath = "/tmp/my_model.dat"
ret = model.save(modelpath)

The model will be saved to the provided path, any extension can be used. The model can be loaded again with the Load method.

Input Images

DVI Library can process 1-channel grayscale images or 3-channels BGR color images. Input images, for both training and test steps, must have the same resolution provided during setup step.

Multiple DVI Instances and Multithreading

In the same program, multiple DVI instances can be created and setup at the same time. This is useful for example when you need to work with multiple models at the same time, or when there are multiple streams of images to be processed.

All DVI instances must work at the same resolution. In order to change the working resolution, all DVI instances (excpet one at most) must be released and new ones must be created and setup at the new resolution.

The C++ library supports multi-threading for parallelization of tasks. Multiple DVI instances can run together holding different models on different threads, or a single DVI instance can perform parallel test tasks on different threads using the same valid model.

However, there are some limitations for usage of same DVI instance on different threads. If test method is running on a thread by a DVI Instance, it’s not possible for other threads to call a method that modify internal state of that DVI instance (e.g setup). A call to these methods would be wait for termination of all active test tasks in order to be executed.

Note

The Python binding of the library does not support real multi-threading because of the Python GIL.

The library exploits multi-threading for parallelization of internal tasks as well, so it’s possibile to observe high CPU usage when using the library. It’s possibile to limit the number of threads used by the library by setting the environment variable OMP_NUM_THREADS to the desired number of threads.

Error Handling

All the methods of the DVI class return a result code that can be used to check if the method has been executed successfully or not. The result code is an enumeration that can be found in the API reference (C++ dvi::DVI::Result / Python dvi.DVI.Result).

The result code can be used to understand the nature of the error and to provide a meaningful error message to the user.