Getting GPU device error when running the detection pipeline

Getting GPU device error when running the detection pipeline

Hi team,
I am trying to run the detection pipeline on my system.
The command i am entering is 
  1. python3 sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py --object_detector_dir ~/Documents/pre-trained_models/metavision_sdk_3.x/red_event_cube_05_2020/ --record_file /media/e/dataset/data/raw/day/my_recording.raw --display --device gpu
The error i get is:
  1. Namespace(deletion_time=100000, detection_merge_weight=0.7, detector_NMS_IOU_threshold=None, detector_confidence_threshold=None, device='gpu', display=True, end_ts=9223372036854775807, iou_to_match_a_detection=0.2, max_duration=None, max_iou_for_one_det_to_many_tracks=0.5, max_iou_inter_track=0.5, network_input_height=None, network_input_width=None, noise_filtering_threshold=10000, noise_filtering_type='trail', number_of_consecutive_detections_to_create_a_new_track=1, object_detector_dir='/home/exx/Documents/pre-trained_models/metavision_sdk_3.x/red_event_cube_05_2020/', output_detections_filename=None, output_tracks_filename=None, output_video_filename=None, pipeline_delta_t=10000, record_file='/media/e/dataset/data/raw/day/my_recording.raw', start_ts=0, timesurface_delta_t=200000, update_tracklets_between_detections=True, use_descriptor=False)
  2. /home/exx/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:1501: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  3.   return forward_call(*args, **kwargs)
  4. Traceback (most recent call last):
  5.   File "sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py", line 298, in <module>
  6.     main()
  7.   File "sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py", line 294, in main
  8.     run(args)
  9.   File "sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py", line 243, in run
  10.     detections = object_detector.process(ts, frame_buffer)
  11.   File "/usr/lib/python3/dist-packages/metavision_ml/detection_tracking/object_detector.py", line 124, in process
  12.     tensorBoxList = self.model(frame_buffer_tensor, self.detection_threshold)
  13.   File "/home/exx/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  14.     return forward_call(*args, **kwargs)
  15. RuntimeError: The following operation failed in the TorchScript interpreter.
  16. Traceback of TorchScript, serialized code (most recent call last):
  17.   File "code/__torch__/metavision_ml/detection/jitting.py", line 16, in forward
  18.     cls0 = (self.rpn).get_scores(cls, )
  19.     anchors = (self.anchor_generator).forward(xs, x, )
  20.     targets = (self.box_decoder).forward(anchors, loc, cls0, 0.10000000000000001, 0.20000000000000001, score_thresh, 0.5, torch.size(x, 1), )
  21.                ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  22.     return targets
  23.   def forward_network_without_box_decoding(self: __torch__.metavision_ml.detection.jitting.SSD,
  24.   File "code/__torch__/metavision_ml/detection/jitting.py", line 49, in forward
  25.     _7 = torch.add(xy, torch.div(wh, 2), alpha=1)
  26.     box_preds = torch.cat([_6, _7], torch.sub(torch.dim(xy), 1))
  27.     _8 = (self)._box_filtering(box_preds, cls_preds, score_thresh, nms_thresh, )
  28.           ~~~~~~~~~~~~~~~~~~~~ <--- HERE
  29.     box_tensor, batch_index, = _8
  30.     num_tbins = torch.floordiv(torch.len(cls_preds), batch_size)
  31.   File "code/__torch__/metavision_ml/detection/jitting.py", line 80, in _box_filtering
  32.     _19 = torch.expand(torch.unsqueeze(boxes, 2), [-1, num_anchors, num_classes, 4], implicit=False)
  33.     boxes0 = torch.contiguous(_19, memory_format=0)
  34.     _20 = (self)._score_filtering(scores0, score_thresh, )
  35.            ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  36.     scores1, score_indices, = _20
  37.     _21 = torch.view(boxes0, [-1, 4])
  38.   File "code/__torch__/metavision_ml/detection/jitting.py", line 117, in _score_filtering
  39.     mask = torch.ge(scores2, score_threshold)
  40.     _33 = annotate(List[Optional[Tensor]], [mask])
  41.     score_indices0 = torch.contiguous(torch.index(score_indices, _33), memory_format=0)
  42.                                       ~~~~~~~~~~~ <--- HERE
  43.     _34 = annotate(List[Optional[Tensor]], [score_indices0])
  44.     scores3 = torch.index(scores2, _34)

  46. Traceback of TorchScript, original code (most recent call last):
  47.   File "/home/fmenasri/sources/metavision/sdk/modules/ml/python/metavision_ml/detection/jitting.py", line 60, in forward
  48.         box_preds = torch.cat([xy - wh / 2, xy + wh / 2], xy.dim()-1)

  50.         box_tensor, batch_index = self._box_filtering(box_preds, cls_preds, score_thresh, nms_thresh)
  51.                                   ~~~~~~~~~~~~~~~~~~~ <--- HERE

  53.         num_tbins = len(cls_preds) // batch_size
  54.   File "/home/fmenasri/sources/metavision/sdk/modules/ml/python/metavision_ml/detection/jitting.py", line 83, in _box_filtering
  55.         # filtering boxes by score
  56.         boxes = boxes.unsqueeze(2).expand(-1, num_anchors, num_classes, 4).contiguous()
  57.         scores, score_indices = self._score_filtering(scores, score_thresh)
  58.                                 ~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  59.         boxes = boxes.view(-1, 4)[score_indices].contiguous()
  60.         idxs = idxs.view(-1)[score_indices].contiguous()
  61.   File "/home/fmenasri/sources/metavision/sdk/modules/ml/python/metavision_ml/detection/jitting.py", line 112, in _score_filtering
  62.         score_indices = torch.arange(scores.shape[0])
  63.         mask = scores >= score_threshold
  64.         score_indices = score_indices[mask].contiguous()
  65.                         ~~~~~~~~~~~~~~~~~~~ <--- HERE
  66.         scores = scores[score_indices]
  67.         return scores, score_indices
  68. RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)
The pipeline works alright if I use cpu as the device, but not for gpu. Could you please help me with this.
Thanks.
    As a Prophesee customer, join the community conversation. 
    Request your free access today.

      Information

      In this Community Forum, you can discuss products, technology and applications. Only registred users can post but everyone can read. To ask a private question, enter a support ticket in My Area https://support.prophesee.ai/portal/en/newticket