Getting GPU device error when running the detection pipeline

Getting GPU device error when running the detection pipeline

Hi team,
I am trying to run the detection pipeline on my system.
The command i am entering is 
  1. python3 sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py --object_detector_dir ~/Documents/pre-trained_models/metavision_sdk_3.x/red_event_cube_05_2020/ --record_file /media/e/dataset/data/raw/day/my_recording.raw --display --device gpu
The error i get is:
  1. Namespace(deletion_time=100000, detection_merge_weight=0.7, detector_NMS_IOU_threshold=None, detector_confidence_threshold=None, device='gpu', display=True, end_ts=9223372036854775807, iou_to_match_a_detection=0.2, max_duration=None, max_iou_for_one_det_to_many_tracks=0.5, max_iou_inter_track=0.5, network_input_height=None, network_input_width=None, noise_filtering_threshold=10000, noise_filtering_type='trail', number_of_consecutive_detections_to_create_a_new_track=1, object_detector_dir='/home/exx/Documents/pre-trained_models/metavision_sdk_3.x/red_event_cube_05_2020/', output_detections_filename=None, output_tracks_filename=None, output_video_filename=None, pipeline_delta_t=10000, record_file='/media/e/dataset/data/raw/day/my_recording.raw', start_ts=0, timesurface_delta_t=200000, update_tracklets_between_detections=True, use_descriptor=False)
  2. /home/exx/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:1501: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  3.   return forward_call(*args, **kwargs)
  4. Traceback (most recent call last):
  5.   File "sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py", line 298, in <module>
  6.     main()
  7.   File "sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py", line 294, in main
  8.     run(args)
  9.   File "sdk/ml/python_samples/detection_and_tracking_pipeline/detection_and_tracking_pipeline.py", line 243, in run
  10.     detections = object_detector.process(ts, frame_buffer)
  11.   File "/usr/lib/python3/dist-packages/metavision_ml/detection_tracking/object_detector.py", line 124, in process
  12.     tensorBoxList = self.model(frame_buffer_tensor, self.detection_threshold)
  13.   File "/home/exx/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
  14.     return forward_call(*args, **kwargs)
  15. RuntimeError: The following operation failed in the TorchScript interpreter.
  16. Traceback of TorchScript, serialized code (most recent call last):
  17.   File "code/__torch__/metavision_ml/detection/jitting.py", line 16, in forward
  18.     cls0 = (self.rpn).get_scores(cls, )
  19.     anchors = (self.anchor_generator).forward(xs, x, )
  20.     targets = (self.box_decoder).forward(anchors, loc, cls0, 0.10000000000000001, 0.20000000000000001, score_thresh, 0.5, torch.size(x, 1), )
  21.                ~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  22.     return targets
  23.   def forward_network_without_box_decoding(self: __torch__.metavision_ml.detection.jitting.SSD,
  24.   File "code/__torch__/metavision_ml/detection/jitting.py", line 49, in forward
  25.     _7 = torch.add(xy, torch.div(wh, 2), alpha=1)
  26.     box_preds = torch.cat([_6, _7], torch.sub(torch.dim(xy), 1))
  27.     _8 = (self)._box_filtering(box_preds, cls_preds, score_thresh, nms_thresh, )
  28.           ~~~~~~~~~~~~~~~~~~~~ <--- HERE
  29.     box_tensor, batch_index, = _8
  30.     num_tbins = torch.floordiv(torch.len(cls_preds), batch_size)
  31.   File "code/__torch__/metavision_ml/detection/jitting.py", line 80, in _box_filtering
  32.     _19 = torch.expand(torch.unsqueeze(boxes, 2), [-1, num_anchors, num_classes, 4], implicit=False)
  33.     boxes0 = torch.contiguous(_19, memory_format=0)
  34.     _20 = (self)._score_filtering(scores0, score_thresh, )
  35.            ~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  36.     scores1, score_indices, = _20
  37.     _21 = torch.view(boxes0, [-1, 4])
  38.   File "code/__torch__/metavision_ml/detection/jitting.py", line 117, in _score_filtering
  39.     mask = torch.ge(scores2, score_threshold)
  40.     _33 = annotate(List[Optional[Tensor]], [mask])
  41.     score_indices0 = torch.contiguous(torch.index(score_indices, _33), memory_format=0)
  42.                                       ~~~~~~~~~~~ <--- HERE
  43.     _34 = annotate(List[Optional[Tensor]], [score_indices0])
  44.     scores3 = torch.index(scores2, _34)

  46. Traceback of TorchScript, original code (most recent call last):
  47.   File "/home/fmenasri/sources/metavision/sdk/modules/ml/python/metavision_ml/detection/jitting.py", line 60, in forward
  48.         box_preds = torch.cat([xy - wh / 2, xy + wh / 2], xy.dim()-1)

  50.         box_tensor, batch_index = self._box_filtering(box_preds, cls_preds, score_thresh, nms_thresh)
  51.                                   ~~~~~~~~~~~~~~~~~~~ <--- HERE

  53.         num_tbins = len(cls_preds) // batch_size
  54.   File "/home/fmenasri/sources/metavision/sdk/modules/ml/python/metavision_ml/detection/jitting.py", line 83, in _box_filtering
  55.         # filtering boxes by score
  56.         boxes = boxes.unsqueeze(2).expand(-1, num_anchors, num_classes, 4).contiguous()
  57.         scores, score_indices = self._score_filtering(scores, score_thresh)
  58.                                 ~~~~~~~~~~~~~~~~~~~~~ <--- HERE
  59.         boxes = boxes.view(-1, 4)[score_indices].contiguous()
  60.         idxs = idxs.view(-1)[score_indices].contiguous()
  61.   File "/home/fmenasri/sources/metavision/sdk/modules/ml/python/metavision_ml/detection/jitting.py", line 112, in _score_filtering
  62.         score_indices = torch.arange(scores.shape[0])
  63.         mask = scores >= score_threshold
  64.         score_indices = score_indices[mask].contiguous()
  65.                         ~~~~~~~~~~~~~~~~~~~ <--- HERE
  66.         scores = scores[score_indices]
  67.         return scores, score_indices
  68. RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)
The pipeline works alright if I use cpu as the device, but not for gpu. Could you please help me with this.
Thanks.
    As a Prophesee customer, join the community conversation. 
    Request your free access today.