Pipeline configuration files#

The Hello Pipeline! example showed the basic functionality of the Pipeline class. In those examples, the Task, Step and InOutput instances were added to the Pipeline using code. That is, a statement like

1pipeline.AddInOutput<in_output::HelloWorld>("HelloWorld One", true);

was used to add an InOutput of type HelloWorld, and all configuration options were hardcoded (in this case, the boolean specifying whether HelloWorld should output messages). Also, custom code was written to call the Tick() and MainTick() functions at the right time and in the right order. The pipeline namespace contains a lot of code that can handle such tasks (setting configuration options and using the code from the control and predict namespaces as intended) for you, saving you time.

In this example we show

  1. how configuration files can be used to set up the Pipeline with the desired set of InOutput and Step instances, and
  2. how you can use functionality provided in the pipeline namespace to trigger the Tick() calls periodically with high timing accuracy (soft-realtime).

Program main#

First, let's look at the body of the main function (program entry). 

 1int main(int argc, char** argv)
 2{
 3    try {
 4        mpmca::pipeline::MainActor main_actor(argc, argv);
 5        return main_actor.Run();
 6    }
 7    catch (const std::exception& e) {
 8        std::cout << "An exception was caught:\n" << e.what() << std::endl;
 9        return 1;
10    }
11    catch (...) {
12        std::cout << "A general exception was caught." << std::endl;
13        return 1;
14    }
15}

All program functionality is provided through the mpmca::pipeline::MainActor class. The constructor of this class takes the same arguments as the program entry main function (i.e., int argc, char** argv) and reads some command line options from those. The MainActor::Run() function then performs the actual work, i.e., reading the configuration file, setting up the pipeline, and then running the endless update loop in which Tick() and MainTick() are called. The program will exit once one of these conditions occur:

  1. an exception is thrown somewhere in the code before the Pipeline enters the periodic update cycles,
  2. the StateMachine is sent to the TERMINATE (return code 0) or FAIL_TERMINATE state (return code 1), or
  3. the user presses ctrl+c in the console window were the Pipeline is running.

Note that the MainActor class can and will throw exceptions under certain conditions. In the example main.cpp above, two catch statements are present: the first will catch all exceptions, the second will catch any exception not derived from std::exception. All exceptions thrown from MPMCA code are derived from std::exception, but dependencies of the MPMCA might throw other exceptions.

To run the program, a configuration file needs to be specified using the -c command line option. For example, to run the program from the root directory of the MPMCA repository:

1./build/bin/demo_pipeline_config -c applications/demo_pipeline_config/config.json

Required include statements#

Before explaining the structure of the configuration files, it is important to discuss the required #include statements in main.cpp first. That is, all InOutput and Step types that you want to use in your program need to be explicitly included in main.cpp, to ensure that these are properly linked (statically) into your program.

The include statements for demo_pipeline_config are as follows:

1#include "mpmca/pipeline/in_output/delayed_state_follower.hpp"
2#include "mpmca/pipeline/in_output/empty_in_output.hpp"
3#include "mpmca/pipeline/in_output/go_to_run.hpp"
4#include "mpmca/pipeline/in_output/hello_world.hpp"
5#include "mpmca/pipeline/main_actor.hpp"
6#include "mpmca/pipeline/pipeline.hpp"
7#include "mpmca/pipeline/step/hello_world.hpp"

Thus, demo_pipeline_config can add the following InOutput types:

  • DelayedStateFollower,
  • EmptyInOutput,
  • GoToRun, and
  • HelloWorld.

It can also add the HelloWorld step. Other types of InOutput and Step cannot be added.

Configuration files#

The configuration file config.json located inside the applications/demo_pipeline_config folder specifies the same configuration as in Example3() of the Hello Pipeline! example. That is, it adds three InOutput and two Step instances.

 1{
 2    "Pipeline": {
 3        "BaseStepMs": 1,
 4        "Guarded": true,
 5        "MainTickBufferSize": 5,
 6        "PrintStateChanges": true,
 7        "PrintClientStateChanges": true,
 8        "MarkNondeterministicLogOutput": false,
 9        "Logger": {"LoggerType": "SpdLog", "LoggerLevel": "Trace", "ConsoleSinkLevel": "Trace", "FileSinkLevel": "Trace", "Filename": "mpmca.log", "Truncate": false},
10        "InOutputs": {
11            "Hello World InOutput One": {"Type": "HelloWorld", "PrintMessages": true, "ExecutionOrder": 0},
12            "Hello World InOutput Two": {"Type": "HelloWorld", "PrintMessages": true, "ExecutionOrder": 1},
13            "GoToRun": {"Type": "GoToRun", "ExecutionOrder": 2}
14        },
15        "Task": {
16            "TimeStepMs": 3,
17            "Horizon": {"PredictionMaximumTimeStepMs": 30, "Steps": [{"Count": 50, "StepMs": 3}]},
18            "Steps": {
19                "Hello World Step One": {"Type": "HelloWorld", "PrintMessages": true, "ExecutionOrder": 0},
20                "Hello World Step Two": {"Type": "HelloWorld", "PrintMessages": true, "ExecutionOrder": 100}
21            }
22        }
23    }
24}

The following list documents the meaning of the options specified in the example configuration file.

  • Pipeline.BaseStepMs: the fundamental update rate, in milliseconds, of the Pipeline. All InOutput instances will be triggered at this interval.
  • Pipeline.Guarded: controls whether the Pipeline will enter safe mode whenever an exception is caught, see this page.
  • Pipeline.MainTickBufferSize: controls the number of MainTick samples the pipeline may lag behind "realtime", see this page.
  • Pipeline.PrintStateChanges: controls whether state changes of the global state machine should be printed or not.
  • Pipeline.PrintClientStateChanges: controls whether state changes of each client state machine should be printed or not.
  • Pipeline.Logger: controls what type of logger should be used. Current options are Console and SpdLog, whereas the SpdLog option is recommended, as it provides a much richer output and more options to control output.
  • Pipeline.InOutputs: should contain a json object array, specifying the InOutput instances to be added to the Pipeline, see below.
  • Pipeline.Task: specifies the options of the Task.
  • Pipeline.Task.TimeStepMs: specifies the time step between MainTick() calls, and should be an integer multiple of Pipeline.BaseStepMs. If the task time step is less than the base step, MainTick() will be called every Tick() and thus run at the (higher) base time step! In this example, MainTick() will be called every third base sample.
  • Pipeline.Horizon: specifies the options of the Horizon, see this page.
  • Pipeline.Steps: should contain a json object array, specifying the Step instances to be added to the Pipeline, see below.

InOutput and Step configuration#

The specification of InOutput and Step instances is very similar and follow the following template:

1{
2  "Unique name of InOutput or Step": {
3    "Type": "Type name of the InOutput or Step instance",
4    "ExecutionOrder": 0,
5    "MaximumComputationTimeUs": 2000,
6    "Disabled": false,
7    "Other options": "Different for each InOutput and Step"
8  }
9}

The "Type" and "ExecutionOrder" fields are required, the "MaximumComputationTimeUs" and "Disabled" fields are optional. If not specified, "MaximumComputationTimeUs" defaults to 200 microseconds, and "Disabled" defaults to false. If "Disabled" is true, the InOutput or Step will not be constructed nor added to the Pipeline.

The "Type" field should be a string and equal to the type name by which the InOutput or Step was added to the register. For example, the GoToRun InOutput was registered as follows:

1static InOutputRegistrar<GoToRun> GoToRunRegistrar("GoToRun");

and thus it requires "Type": "GoToRun".

The "ExecutionOrder" field controls the order in which the InOutput and Step instances are ticked, and is interpreted as a signed integer. The order of ticking is ascending, i.e., InOutput and Step instances with a smaller value for ExecutionOrder are ticked first. The values of different InOutput and Step instances do not need to be consecutive (e.g., 30, 31, 32, 33, ...). Thus, a configuration file that defines three InOutput instances with -50, 0, and 120 as values for ExecutionOrder, respectively, is valid. The ExecutionOrder option is necessary, because some json implementations will sort object arrays by name upon saving a json file.

Configuration file inheritance#

The Pipeline provides a rudimentary way for configuration files to inherit from other configuration files. For example, see the configuration file config_extended.json inside the applications/demo_pipeline_config folder:

1{
2    "ParentConfigFile": "config.json",
3    "Pipeline": {"InOutputs": {"Hello World InOutput One": {"PrintMessages": false, "ExecutionOrder": 10}}, "Task": {"Steps": {"Hello World Step Two": {"Type": "HelloWorld", "ExecutionOrder": 500}}}}
4}

This configuration file inherits from the config.json file (discussed above) through the "ParentConfigFile" option, and modifies a few settings of one InOutput and one Step.

The path to the "ParentConfigFile" is relative from the path to the config file itself.

The parent configuration file is read first, and its contents are updated using the values of the child configuration file. Fields that are not specified in the child configuration file are left as they were defined in the parent file. Thus, in this example, the "PrintMessages" option of "Hello World InOutput One" is modified from true to false, and the "ExecutionOrder" from 0 to 10, such that it now runs after "Hello World InOutput Two". In this example, the "ExecutionOrder" of "Hello World Step One" is modified from 0 to 500, such that it now runs after "Hello World Step Two". The "PrintMessages" option is not present in the child configuration file, and is thus still true (as it was defined in the parent configuration file).