Software:Process control daemon

Process Control Daemon (PCD) is an open source, light-weight system level process manager/controller for Embedded Linux based projects (such as consumer electronics and network devices).

This tool provides a complementary service for any Embedded Linux driven product and is specialized for the needs of embedded products. PCD was created because Embedded Linux distributions did not provide such a tool. Other similar tools which are available are not suited for embedded products in terms of compatibility, performance and limited hardware resources.

Background

The PCD project was designed and implemented by Hai Shalom, senior software engineer at Texas Instruments Israel at the time, as part of his M.Sc. degree in Computer Science. The project was initiated due to a true need in one of Texas Instruments’ broadband solutions (ARM1176 Running MV-Linux Pro-5.00). The system was originally started with shell scripts, where the init process started the rcS script, and it started some other scripts. Due to the rapid development process of the system's software, more processes and tasks were added, and the scripts became too long, unreadable and hard to maintain. There was no efficient way to control the startup process of the system. Synchronizing inter-process dependency was difficult and determining the order of initialization required a lot of effort.

For example, a daemon which listens on a socket must precede any clients that send data on this socket. With shell scripts, there is no efficient way to verify this condition. A service with a complex state machine that needs to be in a specific state before it is ready has no standard means to synchronize with its clients.

Monitoring the system's health was another issue. There was no entity that monitored and controlled the processes in the system. Therefore, in case of a crash, the system became unstable or unusable until the crashed process was manually detected.

At that time, the runtime debugging capabilities were very limited. In case of a process crash, there was no debug information available, unless compiling a special version that supports low optimizations, debug symbols and a GDB server.

Once PCD was integrated into the product, the final results showed reduced startup time and great improvements in the system's robustness, reliability and availability. The Debugging and remote debugging capabilities were also improved due to the extended debug information which was available in case of a crash (Segmentation fault, bus error, or similar).

The project is no longer active but has been extended to support more embedded architectures and new features.

Features in high-level

• Start up the system in a synchronized and deterministic manner, according to a textual configuration file.
• Control and monitor all the processes and services in the system.
• Recover the system in case of errors or crashes, make it more reliable.
• Provide useful and detailed debug information in case of a crash: Process details, registers dump and memory map.
• Create a graphical representation of the system's processes.

Configuration

The PCD needs to be configured with all the processes, daemons and tasks it controls. The configuration is done in runtime, using a simple, human readable configuration file (AKA PCD Script). Each process has its own rule block which describes it, defines its dependency in other rules, defines its details and what action to perform in case it fails. The rule blocks define the inter-process dependencies in the most deterministic manner, and as a result, the PCD builds a dependency tree out of this information. Once all rules are written, and a dependency tree is established, the PCD can startup the system in the most reliable way, and the boot time will be faster, comparing to script startup. The PCD scripts can also be used in the host machine to generate a graphical representation of the system startup dependencies for better understanding it, and for documentation purposes, as can be seen in the following example:

Crash recovery

One of the big advantages of the PCD, is its debug capabilities and crash recovery abilities. The PCD also implements a crash daemon that constantly monitors the system health, and performs a recovery action in case of a crash. This feature enhances the system's stability, robustness and availability.

The PCD also provides a lot of useful debug information that can help the developers and the test engineers to understand what happened. The debug information is displayed on the system's console, and can be stored in a non-volatile storage for later analysis. The detailed debug information includes all the process details, all signal details, a complete register dump (on supported architectures) and the process's address map.

This debug information is available anytime and everywhere (even in the field) and it does not require a special debug build.

Licensing

The PCD Project is an Open-Source project, and it is licensed under the GNU Lesser General Public License version 2.1, as published by the Free Software Foundation:

• There are no purchase costs or royalty fees.
• The source code is fully available.
• High quality code due to extensive exposure.
• The LGPL allows linking and mixing proprietary code with PCD safely. It is allowed to redistribute the PCD with any other software package, provided that the license terms are kept. Any changes or enhancements which are done on the PCD code must be returned to the community.
• Continuous development: New features and other platforms are now supported.
• Community support and bug fixes.
• In case you need a new feature, either request it in the project tracker system, or join the PCD community and develop it, so others could also enjoy your productivity.

Reviews

The PCD Project received good reviews from software engineers and project managers who used it in their project; "PCD is a huge success in a project I am responsible for. It provides 4 major advantages: It simplifies the initialization code, it improves boot time, it reduces debugging effort (providing great crash logs), and improves the traditional WD implementation (monitoring all processes and act accordingly). All this translate to a better product and efficient development flow".

The PCD Project was presented in the ARM Technology Conference 2010, and received good feedback. "Things are getting a lot easier for software developers who want to use open source Linux in their real time embedded designs, particularly on the ARM architecture".

External links

• PCD Project page at SourceForge.net - Download latest version, browse the code, join the community or get help and support.
• PCD support forum
• PCD Presentation at the ARM Technology Conference 2010
• GNU Lesser General Public License version 2.1