OpenMV Cam

OpenMV Cam

OpenMV GENX320 Event Camera Module makes Prophesee GenX320 sensor compatible and ready to be used with compact, Python-powered, machine vision embedded platforms such as OpenMV Cam H7 Plus (recommended) and OpenMV Cam RT1062.

OpenMV Cam H7 Plus with GENX320 Event Camera Module

Data are transmitted from the sensor via a low-power and low latency CMOS Parallel Interface (CPI).

OpenMV platform is provided with an open-source driver and OpenMV IDE. However, it is not supported by Metavision SDK. 


Key Features

  • Cameras based on STM32 H7 Plus and NXP RT1062 micro-controllers 

    • Max event histogram rate on STM32 H7 Plus is 375 fps

    • Max event histogram rate on NXP RT1062 is 180 fps

  • Streams CD events or event histograms natively from GenX320 sensor

  • Includes a set of machine learning and image processing algorithms and examples

  • Small form factor camera expandable via shield add-ons


Supported SW


Supported sensors and settings

Supported sensor modules:

Supported sensor data formats:
  • Contrast Detection (CD) pixels events streamed in EVT2.0 format and output as NumPy ndarray arrays (ongoing work).
    • Output Ndarray holds
      • # 0: event type
      • # 1: seconds timestamp
      • # 2: milliseconds timestamp
      • # 3: microseconds timestamp
      • # 4: x coordinate (0-319)
      • # 5: y coordinate (0-319)
  • Event histograms built on-sensor in Diff3D format and output as images.
    • The histograms are built on-sensor by accumulating Contrast Detection (CD) pixels events over a chosen “integration/accumulation period”
    • Output data are formatted as 2D images with a resolution of 320×320 pixels, each represented as 8-bit values

Supported sensor settings:

  • Sensor biases

  • Event Signal Processing (ESP) filters

  • Low Power Mode (PM2) allowing to reduce consumption whilst maintaining quick wake-up based on scene dynamics can be activated by sensor.sleep function

  • Further sensor features can be configured by writing/reading registers (note that the read/write command accepts 16-bit values only)


Installation

  1. Install OpenMV IDE
  2. (optional) Clone and compile OpenMV driver if you want to do low-level changes in the driver and re-flash the board

Getting Started

  • Plug the OpenMV Cam to USB port on your PC

  • Start OpenMV IDE, connect to the camera and “Install the latest development release”

  • Run ready-to-use examples for GenX320




Examples

There are several examples that comes with the driver. They can be opened and started from OpenMV IDE.  

Minimal example of histogram streaming

This minimal example shows how to acquire event histograms from the sensor, visualize them and print the output histogram rate.
  1. import sensor
import time

sensor.reset() # Initializes the sensor.
sensor.set_pixformat(sensor.GRAYSCALE) # Must always be grayscale.
sensor.set_framesize(sensor.B320X320) # Must always be 320x320.
sensor.set_framerate(100)

clock = time.clock()

while True:
    clock.tick()
    img = sensor.snapshot()
    print(clock.fps())


Minimal example of event streaming

This example shows how to acquire CD events, visualize them as a histogram and print the output rate.
  1. import sensor
    import image
    import time
    from ulab import numpy as np

    # Surface to draw the histogram image on.
    img = image.Image(320, 320, sensor.GRAYSCALE)

    # ndarray to hold events from the camera
    # must be EVT_res events by 6 values
    #
    # 0: event type
    # 1: seconds timestamp
    # 2: milliseconds timestamp
    # 3: microseconds timestamp
    # 4: x coordinate (0-319 for the genx320)
    # 5: y coordinate (0-319 for the genx320)
    events = np.zeros((2048, 6), dtype=np.uint16)

    # Initialize the sensor.
    sensor.reset()
    sensor.set_pixformat(sensor.GRAYSCALE)  # Must always be grayscale.
    sensor.set_framesize(sensor.EVT_2048)  # Must be EVT_1024/2048/.../65536

    clock = time.clock()

    while True:
        clock.tick()

        # Reads up to 2048 events from the camera.
        # Returns the number of valid events (0-2048) or a negative error code.
        # Note that old events in the buffer are not cleared to save CPU time.
        event_count = sensor.ioctl(sensor.IOCTL_GENX320_READ_EVENTS, events)

        # Draws the events on the image. If clear=True then the image is cleared
        # to the default value of "brightness". Event histogram bins have "contrast"
        # added to them for PIX_ON_EVENT events and subtracted from them for
        # PIX_OFF_EVENT events clampped between 0 and 255. Pass clear=False to keep
        # accumulating events in the histogram image.
        img.draw_event_histogram(events[:event_count], clear=True, brightness=128, contrast=64)

        # Push the image to the jpeg buffer for the IDE to pull and display.
        # The IDE pulls frames off the camera at a much lower rate than the
        # onboard camera frame rate printed below.
        img.flush()

        print(event_count, clock.fps())

Example of LED tracking based on histogram data

This example shows how to stream histograms, apply a preset of sensor biases tuned for seeing only high-frequency LEDs and track LEDs.
  1. import sensor
import time

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE) # Must always be grayscale.
sensor.set_framesize(sensor.B320X320) # Must always be 320x320.
sensor.set_framerate(200)
sensor.ioctl(sensor.IOCTL_GENX320_SET_BIASES, sensor.GENX320_BIASES_ACTIVE_MARKER) # Applies sensor biases for seeing LEDs only

clock = time.clock()

while True:
    clock.tick()
    img = sensor.snapshot()

    blobs = img.find_blobs(
        [(120, 140)], invert=True, pixels_threshold=2, area_threshold=4, merge=True
    )

    for blob in blobs:
        img.draw_rectangle(blob.rect(), color=(255, 255, 255))
        img.draw_cross(blob.cx(), blob.cy(), color=(0, 0, 0))

    print(clock.fps())


Example of biases tuning 

This example shows how to stream histograms and tune individual sensor biases one by one.
  1. import sensor
import time

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE) # Must always be grayscale.
sensor.set_framesize(sensor.B320X320) # Must always be 320x320.
sensor.set_framerate(400)

sensor.ioctl(sensor.IOCTL_GENX320_SET_BIAS, sensor.GENX320_BIAS_DIFF_OFF, 28)
sensor.ioctl(sensor.IOCTL_GENX320_SET_BIAS, sensor.GENX320_BIAS_DIFF_ON, 25)
sensor.ioctl(sensor.IOCTL_GENX320_SET_BIAS, sensor.GENX320_BIAS_FO, 34)
sensor.ioctl(sensor.IOCTL_GENX320_SET_BIAS, sensor.GENX320_BIAS_HPF, 40)
sensor.ioctl(sensor.IOCTL_GENX320_SET_BIAS, sensor.GENX320_BIAS_REFR, 10)

clock = time.clock()

while True:
    clock.tick()
    img = sensor.snapshot()
    print(clock.fps())

Example of AFK setting 

This example shows how to stream histograms and set AFK filter to filter out a given range of frequencies.
  1. import sensor
    import time

    sensor.reset()
    sensor.set_pixformat(sensor.GRAYSCALE)  # Must always be grayscale.
    sensor.set_framesize(sensor.B320X320)  # Must always be 320x320.
    sensor.set_framerate(100)

    # Enables AFK filter to remove periodic data in the frequency range from 130Hz to 160Hz
    sensor.ioctl(sensor.IOCTL_GENX320_SET_AFK, 1, 130, 160)
    # Disables AFK filter
    # sensor.ioctl(sensor.IOCTL_GENX320_SET_AFK, 0)

    clock = time.clock()

    while True:
        clock.tick()
        img = sensor.snapshot()
        print(clock.fps())

To buy an OpenMV Cam with GENX320 Event Camera Module, contact OpenMV.

    As a Prophesee customer, get access to your personal ticketing tool, application notes, product manuals, SDK Download Center and more resources.