Lighthouse has a new layout. Prefer the old one? Return to the old layout, and switch back any time from the link at the top of each page.

mw::SchedulerTestFixture::testPeriod10HzNoPayloadChaffX4 fails more often than not

#47

The unit test mw::SchedulerTestFixture::testPeriod10HzNoPayloadChaffX4 usually fails for me. The timing errors are on the order of 10's to 100's of milliseconds (assuming I'm interpreting the time units correctly).

Reported by Christopher Stawarz · May 11th, 2010 @ 06:03 PM

State: hold
Milestone: none
Assigned to: David Cox David Cox

Activity

  1. David Cox
    David Cox
    • Assigned user set to David Cox

    The stats are all reported in microseconds, and the test is currently set to fail if any event happens more than 1 millisecond late, which might be too aggressive on most machines.

    However, it does look like something is amiss. In January, on this same laptop, I clocked the following scheduler benchmarks:

    testPeriod10HzNoPayload ----------> 99.25% of runs < 100us latency; 100% < 500us
    testPeriod100HzNoPayload: --------> 99.75% of runs < 100us latency; 100% < 500us
    testPeriod10HzNoPayloadChaffX4: --> 99.50% of runs < 100us latency; 2 faults > 1ms
    testPeriod10HzSmallPayload: ------> 97.24% of runs < 100us latency; 100% < 500us
    testOneOffFiringSmallPayload:   ----> 100% of runs < 500us latency
    

    Today:

    testPeriod10HzNoPayload: ---------> 99.75% < 100us; 100% < 500us
    testPeriod100HzNoPayload: --------> 100% < 100us
    testPeriod10HzNoPayloadChaffX4:--> 94.21% < 1000us; 97.23% < 15000us 
    testPeriod10HzSmallPayload:-------> 100% < 500us
    testOneOffFiringSmallPayload: ----> 100% < 500us
    

    So, the chaff test has always been the hardest, but something going very wrong relative to before, though the rest of the tests look basically normal. Those January benchmarks were collected under 10.6, but maybe something changed in the scheduler in an update? It would be helpful to run this test across a variety of different OS / hardware combinations to see if a pattern emerges.

    I also wonder if the problem has more to do with the chaff threads than the scheduled payload -- on my machine the chaff threads are doing a very good job of saturating the CPU cores (occasionally, the Activity Monitor reads numbers > 200% for the test runner, which is weird).

    May 12th, 2010 @ 10:36 AM

  2. David Cox
    David Cox

    Interestingly, demoting the scheduler priority of the chaff threads even a little bit (e.g. -5) appears to drastically alter the latency stats:

    Latency Statistics...
       < 5us:  92.98
       < 10us:  96.74
       < 50us:  97.99
       < 100us:  98.25
       < 500us:  99.75
       < 1000us:  99.75
       < 5000us:  100
       < 10000us:  100
       < 15000us:  100
       > 15000us:  0
    

    Still would fail the test with 1ms threshold, but the latency distribution is better in line with what you'd want in real life. On closer examination, I'm not sure the chaffx4 test is reasonable in its current form: basically it's asking the system to spin as fast as possible on 4 threads, and schedule a 5th, and there is no indication of the relative priority of these activities.

    May 12th, 2010 @ 11:19 AM

  3. Christopher Stawarz
    Christopher Stawarz
    • Milestone changed from 0.4.4 to 0.4.5
    • Milestone order changed from 0 to 0

    July 6th, 2010 @ 01:28 PM

  4. Christopher Stawarz
    Christopher Stawarz
    • State changed from new to open
    • Milestone cleared.
    • Milestone order changed from 5 to 0

    April 8th, 2011 @ 10:45 AM

  5. Christopher Stawarz
    Christopher Stawarz
    • State changed from open to hold

    July 12th, 2016 @ 04:20 PM

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.