#47 ✓hold
Christopher Stawarz

mw::SchedulerTestFixture::testPeriod10HzNoPayloadChaffX4 fails more often than not

Reported by Christopher Stawarz | May 11th, 2010 @ 06:03 PM

The unit test mw::SchedulerTestFixture::testPeriod10HzNoPayloadChaffX4 usually fails for me. The timing errors are on the order of 10's to 100's of milliseconds (assuming I'm interpreting the time units correctly).

Comments and changes to this ticket

  • David Cox

    David Cox May 12th, 2010 @ 10:36 AM

    • Assigned user set to “David Cox”

    The stats are all reported in microseconds, and the test is currently set to fail if any event happens more than 1 millisecond late, which might be too aggressive on most machines.

    However, it does look like something is amiss. In January, on this same laptop, I clocked the following scheduler benchmarks:

    testPeriod10HzNoPayload ----------> 99.25% of runs < 100us latency; 100% < 500us
    testPeriod100HzNoPayload: --------> 99.75% of runs < 100us latency; 100% < 500us
    testPeriod10HzNoPayloadChaffX4: --> 99.50% of runs < 100us latency; 2 faults > 1ms
    testPeriod10HzSmallPayload: ------> 97.24% of runs < 100us latency; 100% < 500us
    testOneOffFiringSmallPayload:   ----> 100% of runs < 500us latency


    testPeriod10HzNoPayload: ---------> 99.75% < 100us; 100% < 500us
    testPeriod100HzNoPayload: --------> 100% < 100us
    testPeriod10HzNoPayloadChaffX4:--> 94.21% < 1000us; 97.23% < 15000us 
    testPeriod10HzSmallPayload:-------> 100% < 500us
    testOneOffFiringSmallPayload: ----> 100% < 500us

    So, the chaff test has always been the hardest, but something going very wrong relative to before, though the rest of the tests look basically normal. Those January benchmarks were collected under 10.6, but maybe something changed in the scheduler in an update? It would be helpful to run this test across a variety of different OS / hardware combinations to see if a pattern emerges.

    I also wonder if the problem has more to do with the chaff threads than the scheduled payload -- on my machine the chaff threads are doing a very good job of saturating the CPU cores (occasionally, the Activity Monitor reads numbers > 200% for the test runner, which is weird).

  • David Cox

    David Cox May 12th, 2010 @ 11:19 AM

    Interestingly, demoting the scheduler priority of the chaff threads even a little bit (e.g. -5) appears to drastically alter the latency stats:

    Latency Statistics...
       < 5us:  92.98
       < 10us:  96.74
       < 50us:  97.99
       < 100us:  98.25
       < 500us:  99.75
       < 1000us:  99.75
       < 5000us:  100
       < 10000us:  100
       < 15000us:  100
       > 15000us:  0

    Still would fail the test with 1ms threshold, but the latency distribution is better in line with what you'd want in real life. On closer examination, I'm not sure the chaffx4 test is reasonable in its current form: basically it's asking the system to spin as fast as possible on 4 threads, and schedule a 5th, and there is no indication of the relative priority of these activities.

  • Christopher Stawarz

    Christopher Stawarz July 6th, 2010 @ 01:28 PM

    • Milestone changed from 0.4.4 to 0.4.5
    • Milestone order changed from “0” to “0”
  • Christopher Stawarz

    Christopher Stawarz April 8th, 2011 @ 10:45 AM

    • State changed from “new” to “open”
    • Milestone cleared.
    • Milestone order changed from “5” to “0”
  • Christopher Stawarz

    Christopher Stawarz July 12th, 2016 @ 04:20 PM

    • State changed from “open” to “hold”

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile ยป

The core framework and supporting libraries for the MWorks Suite

People watching this ticket