submaster: improve avg frequency calculation for efficiency

[deanlee]

The average frequency calculation is currently the most time-consuming function in SubMaster. The previous implementation kept 10 * freq items in a deque and summed all items each time to check the average frequency. For example, with a 100 Hz service frequency, the queue length would be 1000 items, requiring a sum of all 1000 items for each check. Additionally, checking recent dts involved creating and summing a new list from recv_dts, further increasing inefficiency.

For example, in selfdrived, the average frequency calculation within sm.update() accounted for nearly 30% of the update_event() function’s execution time:

ncalls tottime percall cumtime percall filename:lineno(function)
5000 1.010 0.000 2.179 0.000 selfdrived.py:129(update_events)
5000 0.155 0.000 2.132 0.000 init.py:186(update)
5000 0.653 0.000 1.498 0.000 init.py:196(update_msgs)
31869 0.147 0.000 0.705 0.000 init.py:25(log_from_bytes)
110000 0.188 0.000 0.660 0.000 init.py:120(valid) <- FrequencyTracker.valid()
107605 0.476 0.000 0.476 0.000 {built-in method builtins.sum} <-sum

This PR introduces a separate deque for recent dts values to avoid redundant list copying. It also improves the efficiency of dts sum calculation in record_recv_time(), eliminating the need for costly sum() operations.

As a result, the average frequency calculation time in sm.update() decreased from 0.660s to 0.151s, reducing its impact on selfdrived

ncalls tottime percall cumtime percall filename:lineno(function)
5000 1.117 0.000 2.441 0.000 selfdrived.py:129(update_events)
5000 0.164 0.000 1.917 0.000 init.py:201(update)
5000 0.719 0.000 1.225 0.000 init.py:211(update_msgs)
32031 0.162 0.000 0.781 0.000 init.py:25(log_from_bytes)
27030 0.080 0.000 0.513 0.000 init.py:80(recv_one_or_none)
5000 0.176 0.000 0.428 0.000 selfdrived.py:419(publish_selfdriveState)
4399 0.372 0.000 0.372 0.000 selfdrived.py:235()
5000 0.037 0.000 0.303 0.000 events.py:68(clear)
32044 0.046 0.000 0.294 0.000 contextlib.py:132(enter)
64088 0.283 0.000 0.283 0.000 {built-in method builtins.next}
5000 0.262 0.000 0.262 0.000 events.py:69()
4399 0.069 0.000 0.246 0.000 selfdrived.py:335(ddk)
32044 0.079 0.000 0.217 0.000 contextlib.py:287(helper)
27030 0.177 0.000 0.208 0.000 init.py:117(record_recv_time)
5092 0.131 0.000 0.164 0.000 init.py:30(new_message)
63511 0.156 0.000 0.156 0.000 {built-in method builtins.getattr}
110000 0.135 0.000 0.151 0.000 init.py:136(valid) <-FrequencyTracker.valid()

[sentry-io]

🔍 Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

📄 File: cereal/messaging/init.py

Function	Unhandled Issue
__init__	KjException: capnp/schema.c++:511: failed: struct has no such member; name = peripheralState capnp.lib.capnp i... Event Count: 4
__init__	KeyError: 'pandaStates' cereal.messaging in <list... Event Count: 2
__init__	KjException: capnp/schema.c++:511: failed: struct has no such member; name = PeripheralState capnp.lib.capnp i... Event Count: 2

---

_{Did you find this useful? React with a 👍 or 👎}

[adeebshihadeh]

trigger-jenkins

[adeebshihadeh]

-6% improvement in selfdrived

[github-actions]

This PR has had no activity for 9 days. It will be automatically closed in 2 days if there is no activity.