Monitoring Execution Timing

It is often desirable to determine whether realtime performance criteria are met or how long a certain time domain takes to run. For this purpose each time domain as well as the safety system comprises of a dedicated timer.

The timer is implemented in eeros::PeriodicFunction and is included in every periodic object. It registers the system time each time the executor executes this particular task (tick()). When the task is done, the system time is registered again (tock()). The timer keeps track of the previous measurement and calculates the run time in between consecutive runs as well as period and jitter.

• run time: tock() - tick()
• period: tick()[t=n] - tick()[t=n-1]
• jitter: |actual period - set period|

Of these three measurements the minimal value, maximal value, mean value and the variance are calculated. All times are given in ns.

...
loop {
    counter.tick();
    //code under test (list.run();)
    counter.tock();
}
...

The timer measurements can be used to monitor execution timing and notably realtime performance. After the creation of a periodic task, a monitor function can be added before the task is added to the Executor.
Add the monitor function before adding the periodic task to the executor.

eeros::task::Periodic periodic("control system", dt, timedomain);  // create a periodic with a given period 
// the periodic will run a given time domain
 
periodic.monitors.push_back([&](PeriodicCounter &c, Logger &log) {
  log.info() << "period max: " << c.period.max << ", run mean: " << c.run.mean;
});
 
eeros::Executor::instance().add(periodic);

Each time the periodic runs it will log the maximum period and the mean run time. This is the time the periodic needs to be executed.

If the periodic has a small cycle time (less than 1 second) it is not advisable to update the log output each time. In the following example the output is only updated after 1000 cycles. Depending on the cycle time, this number can be adjusted.

periodic.monitors.push_back([](eeros::PeriodicCounter &c, Logger &log) {
  static int ticks = 0;
  if (++ticks < 1000) return;
  ticks = 0;
  log.info() << "period max: " << c.period.max << ", period min: " << c.period.min << ", period mean: " << c.period.mean;
  c.reset();
});

Every periodic includes a periodic counter object. In case you simply define a time domain and add it to the executor its corresponding periodic object is created internally and you cannot access its periodic counter object. In such cases you manually create a periodic object and pass its time domain as a parameter.
Similarly, if you a define a safety system and add to to the executor as the main task, you cannot access its periodic counter either. Here, you can access its associated periodic as shown below.

eeros::control::TimeDomain td1("td1",0.01, true);  // periodic counter not accessible
eeros::Executor::instance().add(td1);
 
eeros::control::TimeDomain td2("td1",0.01, true);  
eeros::task::Periodic per2("per2",0.01, td2);  // now you can add monitor functions to its periodic counter
eeros::Executor::instance().add(per2);
 
MySafetyProperties ssProperties(0.01);
eeros::safety::SafetySystem safetySys(ssProperties, 0.01);
eeros::Executor::instance().setMainTask(safetySys);
eeros::task::Periodic per3 = eeros::Executor::instance().getMainTask();  // now you can add monitor functions to its periodic counter

You can add a default monitor to a periodic. Such a default monitor will log a message (on level WARN) as soon as the maximum period exceeds a the envisaged period more than a certain limit. The default value for this limit is 5%.

eeros::task::Periodic per1("per1",0.001, td1);
per1.addDefaultMonitor();  // warn if the period exceeds 1.05ms
eeros::task::Periodic per2("per2",0.001, td2);
per2.addDefaultMonitor(0.02);  // warn if the period exceeds 1.02ms

For a certain type of safety critical robots, it is desirable to change into a safe level in case of the jitter in periodicity of a time domain or the safety system itself exceeds a certain level. Add a monitor function, which periodically checks for the critical maximum value and triggers a specific safety event.

periodic.monitors.push_back([=](PeriodicCounter &c, Logger<LogWriter> &log) {
  if (c.period.max >= dt) safetySystem.triggerEvent(myEvent);
});

This measurement provides only statistical data over 1000 (or 100) cycles at a time and not for each iteration. To measure jitter for each iteration use the Logging with a Trace Block.

It is possible to measure a 15% CPU load and over 300us run time per 1ms period. This indicates, that the run time measurement is not accurate.

Because of memory restriction or because writing a file would affect the real time behavior of the system it is maybe not possible to save the output of the terminal directly on the target system. The output can be saved on a different computer with following command:

ssh <username>@<hostname> "<path to eeros project>/eerosproject" > <path to logfile>/<name of logfile>

With this command it is possible to measure for multiple hours or even multiple days.

Especially when implementing your own blocks one wishes to measure the time it takes to run it. This can be very useful for blocks with complex algorithms or in cases your timedomain with many blocks takes to much time to run and you want to pinpoint the culprit. For this purpose add a PeriodicCounter to the block and add tick() and tock() to its run method.

#include <eeros/core/PeriodicCounter.hpp>   // add include file
 
eeros::PeriodicCounter pc;                  // add periodic counter object to your block
 
virtual void run() {
  pc.tick();                                // start ...
  ... // algorithm
  pc.tock();                                // ... and stop the timer

You can then periodically print the mean and maximum run time.