How to improve the writing speed of CSV files?

How to improve the writing speed of CSV files?

# Define a thread function to write to the CSV
def write_to_csv(event_queue, output_file):
with open(output_file, 'w', newline='') as csv_file:
csv_writer = csv.writer(csv_file)
while True:
try:
# Get events from the queue and write them to the CSV
events = event_queue.get(block=True)
for event in events:
csv_writer.writerow(event)
except queue.Empty:
# If the queue is empty and there are no more events to process, the loop exits
break
# Create and start two threads: one for writing to the CSV and one for processing and displaying events
csv_thread = threading.Thread(target=write_to_csv, args=(event_queue, output_file))
# The main thread is responsible for iterating events from the device and queueing them
try:
mv_iterator = EventsIterator.from_device(
device=device)
for events in mv_iterator:
# Place the event in a queue for other threads to process
event_queue.put(events)
except Exception as e:
print(f"An error occurred while iterating events: {e}")
finally:
# When the iteration is complete, a signal is sent to the queue to notify the thread to stop
event_queue.put(None)
This is part of the code I used to write the CSV file. I made the event presentation (opening the camera window) and writing the event to the CSV file into two threads. This allows the two functions to run without interference. However, with the camera on for more than 10 seconds, I could only see about a second of video later using the csv_viewer routine. How can I modify the part of the code that writes to the CSV file so that the two threads are almost in sync? I look forward to and appreciate your advice.

    As a Prophesee customer, join the community conversation. 
    Request your free access today.

      Information

      In this Community Forum, you can discuss products, technology and applications. Only registred users can post but everyone can read. To ask a private question, enter a support ticket in My Area https://support.prophesee.ai/portal/en/newticket