MVTec HALCON Acquisition Interface

MVTec HALCON Acquisition Interface

MVTec HALCON acquisition interface is now available for Prophesee Event-Based cameras.

HALCON software provides a set of widely used capabilities that facilitate rapid development of machine vision applications. It features the availability of a large number of ready-to-use algorithms. Through the integration with Prophesee’s event-based Metavision® cameras, developers can implement machine learning-enabled applications using event-based data and operating at much higher speed, in challenging lighting conditions.

In addition to this page, a video training shows how to install and use the MVTec HALCON acquisition interface: https://www.youtube.com/watch?v=sQkqdbvDAbo

Installation

Requirements:
  1. Metavision SDK >=3.1.0 (no need to install all dependencies)
  2. MVTec HALCON SW
  3. OS
    1. Windows 10 or 11
    2. Linux Ubuntu 18.04 or 20.04
Install the interface:
  1. Download the interface from the attachment to this page
  2. Extract the interface from the archive and copy to the %HALCONROOT% folder.
Now, you are ready to stream and process data from event-based Prophesee cameras in HALCON SW.

We provide examples showing how to acquire and process data from Prophesee event-based cameras and also from RAW files with already recorded data. To be able to run the examples using RAW files, we suggest you to download RAW files from Metavision dataset.

Features

The interface enables asynchronous data acquisition in two main formats :
  1. frames format that can be acquired with grab_image_async operator and that can be processed with 2D image processing operators
  2. buffers of CD events in XYPT format (available in HDevelopXL only) that can be acquired with grab_data_async operator and that can be processed as a 3D model using time as depth axis
In all formats, the data are generated form CD (Contrast Detection) events acquired from Prophesee event-based sensors. CD events are triggered by temporal contrast (relative intensity) change. CD events have a polarity, negative or positive (CD ON or CD OFF), depending on decreasing or increasing light intensity. See more details on event-based concepts.

Data Acquisition Formats

Frames format (CDFrames and Timesurfaces)
Frames can be generated in two ways as CDFrames or Timesurfaces, and you can switch between them via an acquisition parameter (cdevents_vis_mode).
CDFrames are images generated from CD events (ON and OFF) accumulated over a given period of time. If several events are generated for the same pixel during the given accumulation time, then only the last event with its polarity is used to update the pixel value.
Please, note that the generated CDFrames are different from the frames acquired from a classical frame-based camera, and therefore, the result of applying 2D operators on such CDFrames can differ from results obtained on frame-based data.
CDFrame format is not the best format to demonstrate advantages of event-based technology.


Timesurfaces are images generated from the last event timestamp per pixel (for both ON and OFF events). Most recent events have larger pixels values and older events have smaller pixels values. Thus, this data format contains information about events history inside a 2D image. On the screenshot below, the timesurface is visualized using the 'jet' look-up table.



Buffers of events in XYPT
format
Buffers of events are formatted in 4 images:
  1. image with X sensor coordinate,
  2. image with Y sensor coordinate,
  3. image with Timestamp,
  4. image with Polarity.
These data can be used to generate a 3D object model using time as depth axis.



Interface Delivery

The interface includes:
  1. pre-compiled libraries
  2. documentation
  3. examples showing how to stream data from a camera and how to read data from RAW files using Metavision interface (examples folder)
  4. look-up-tables for data visualization - they define the transformation of a pixels values within the original image into a chosen color scheme
Windows delivery
Linux delivery
├── bin
│   └── x64-win64
│       ├── hAcqMetavision.dll
│       ├── hAcqMetavisionxl.dll
│       └── metavision_camera.dll
├── doc
│   └── html
│       └── reference
│           └── acquisition
│               └── hAcqMetavision.html
├── examples
│   └── hdevelop
│       └── Image
│           └── Acquisition
│               ├── hacqmetavision_cdframe.hdev
│               ├── hacqmetavision_cdframe_from_rawfile.hdev
│               ├── hacqmetavision_params.hdev
│               ├── hacqmetavision_timesurface.hdev
│               ├── hacqmetavision_xypt.hdev
│               ├── hacqmetavision_xypt_from_rawfile.hdev
│               ├── hacqmetavision_xypt_noise_removal.hdev
│               ├── hacqmetavision_xypt_polarity_filter.hdev
│               └── hacqmetavision_xypt_segmentation.hdev
└── lut
    ├── prophesee_dark.lut
    └── prophesee_light.lut
├── doc
│   └── html
│       └── reference
│           └── acquisition
│               └── hAcqMetavision.html
├── examples
│   └── hdevelop
│       └── Image
│           └── Acquisition
│               ├── hacqmetavision_cdframe.hdev
│               ├── hacqmetavision_cdframe_from_rawfile.hdev
│               ├── hacqmetavision_params.hdev
│               ├── hacqmetavision_timesurface.hdev
│               ├── hacqmetavision_xypt.hdev
│               ├── hacqmetavision_xypt_from_rawfile.hdev
│               ├── hacqmetavision_xypt_noise_removal.hdev
│               ├── hacqmetavision_xypt_polarity_filter.hdev
│               └── hacqmetavision_xypt_segmentation.hdev
├── lib
│   └── x64-linux
│       ├── hAcqMetavision.so
│       ├── hAcqMetavisionxl.so
│       ├── libmetavision_camera.so
│       ├── libmetavision_camera.so.1
│       └── libmetavision_camera.so.1.2.4
└── lut
    ├── prophesee_dark.lut
    └── prophesee_light.lut


Getting Started

At first, you need to connect Prophesee camera to USB3.0 port.
Then, open HDevelopXL  and either open one of the provided examples or Image Acquisition GUI to connect to Prophesee camera.

Image Acquisition GUI

Go to the Assistants menu and "Open New Image Acquisition". In the 'Source' tab, press "Auto-detect Interfaces" and select the "Metavision" interface from the dropdown list.


Now, you can connect to the camera and acquire data.

You will see data from the camera in the Graphics Window. By default, the data are acquired in the format of frames (CDFrame) containing ON and OFF events.


Parameters of Acquisition Control and Device-Specific Settings

Acquisition Control
The following acquisition parameters are available:
  1. for frame data, adjust:
    1. fps - frame rate
    2. accumulation_time - time duration used to generate a frame (by default, it's 40ms and it means that the events generated during the last 40ms are drawn in each frame)
  2. for XYPT data, you can use:
    1. time_slice - time duration used to generate XYPT buffer (by default, it's 40ms and it means that each XYPT buffer has the events generated during the last 40ms)


Once you tune the settings, you can generate the corresponding code easily.


Device-specific settings
There is a variety of device-specific settings available:
  1. Biases  - to tune the sensor behavior for a specific environment and application
  2. Region of Interest - to set a region of interest where events will be acquired (useful for readout)
  3. Digital Filters
    1. Anti-Flicker - to filter flickering (periodically repeating) events
    2. Event Rate Control - to control the maximal event rate (set to 20MEv/s by default)
    3. Event Trail Filter - to filter events with Trail or STC filters
Once you have explored the beginner settings, do not hesitate to switch to the "Guru" mode.

Examples

We provide HDevelop examples showing how to acquire event-based data with Metavision interface:
  • Examples with frame data format
    • hacqmetavision_cdframe.hdev - acquisition of CDFrames from a live camera
    • hacqmetavision_cdframe_from_rawfile.hdev - reading data in CDFrames format from a RAW file
    • hacqmetavision_timesurface.hdev - acquisition of timesurfaces from a live camera
  • Examples with buffers of CD events in XYPT format
    • hacqmetavision_xypt.hdev - acquisition of XYPT data from a live camera
    • hacqmetavision_xypt_from_rawfile.hdev - reading data in XYPT format from a RAW file
    • hacqmetavision_xypt_noise_removal.hdev - reading data in XYPT format from a RAW file and noise removal
    • hacqmetavision_xypt_polarity_filter.hdev - reading data in XYPT format from a RAW file and generating separate 3D models for events of ON and OFF polarities
    • hacqmetavision_xypt_segmentation.hdev - reading data in XYPT format from a RAW file and segmenting connected components
  • Generic:
    • hacqmetavision_params.hdev - Acquisition of CDFrames from a live camera and setting parameters of acquisition control and device-specific parameters (sensor biases, ERC, ROI, Anti-Flicker filter, STC/Trail filter)
To open an example, simply drag and drop it into Hdevelop program window.

Some of the provided examples are using RAW files, and you can try them without having a Prophesee camera.
These examples can be run on RAW files from Metavision dataset.

    As a Prophesee customer, get access to your personal ticketing tool, application notes product manuals and more resources. 
    Request your free access today.