Automate Manufacturing Flows and Systems
Faster, More Reliable Factory Automation with Connext DDS
RTI Connext DDS is a peer-to-peer real-time middleware solution that allows you to represent and monitor the state of real-world objects. This makes it easy to build systems that monitor and update the state of an object as it goes through any workflow. In discrete manufacturing process flows, this allows you to create a system that controls a batch or lot as it is processed by one or more control applications, updating the lot states as they are processed. An example of a discrete processing implementation is demonstrated here for illustration purposes by developing a simple set of use cases that support a Manufacturing Execution System design that can manage the simple dispatching of lots, monitoring of current lot states while detecting and reporting errors throughout the system.
This set of use cases applies if you have any type of system with a workflow:
- Discrete oriented factory automation systems
- Chemical batch plant processing flows
- Pharmaceutical manufacturing flows
- Medical sample processing and analysis
- Any manufacturing system that uses a distributed control system (DCS) to process items in a production line, or a manufacturing execution system (MES) to dispatch and monitor state information.
DDS as a Manufacturing Service Bus (MSB)
To remain competitive, manufacturing systems need to be agile and easily reconfigurable reducing time-to-market and allowing production to be adjusted in reaction to demand, competition, and regulation. Similarly, the need for integrating both supplier components and IT systems is driving the need for more flexible and expandable manufacturing architectures.
Systems built using a Service Oriented Architecture (SOA) have been used successfully to address similar needs within enterprise systems. In the enterprise, SOA systems are built using technologies such as Microsoft WFC, J2EE, and Enterprise Service Buses (ESBs). However these technologies are generally considered not well suited to the more distributed, high-transaction, and near-real-time deterministic needs of manufacturing systems. For example, ESBs are centralized brokers built using web technologies like HTTP and XML which impact performance, robustness, and determinism. Instead, manufacturing systems require a Manufacturing Service Bus (MSB).
The Data-Centric publish-subscribe technologies as specified by the OMG Data-Distribution Service (DDS) standard are an emerging platform for implementing the Manufacturing Service Bus. RTI Connext DDS is a high-performance, highly-scalable implementation of the OMG DDS standard.
The DDS technology provides technical layer for monitoring and integrating the manufacturing control systems that is independent of the programming language, hardware, and operating system used by each component. It provides automatic service discovery, data-normalization, health monitoring, mediation, and many other SOA services directly in the infrastructure. In addition, RTI Connext DDS features the scalability, performance, determinism and real-time QoS support needed to operate in the manufacturing loop. Moreover the serverless peer-to-peer architecture makes deployment in the factory easy – there are no extra services or computers to manage.
This example illustrates the use of DDS to implement an MSB using a chocolate manufacturing process as an example.
A Manufacturing Execution System built as a set of DDS applications will be used to configure the recipes for the different kinds of chocolate, define the batches (lots) to be produced, control the routing of the batch to each workcell, and monitor the state of the manufacturing process. The example could be extrapolated to other discrete manufacturing processes that operate similarly by dispatching lots to different workcells and configure and monitor the operating parameter of each workcell.
This example shows three applications. You can run them on the same machine, or on separate machines in the same network. This example is modeled after a simple subsection of a fictitious chocolate factory, but the concepts of data modeling and quality of service (QoS) tuning are applicable for a range of systems that follow a workflow.
The three applications are:
Recipe generator (RecipeGenerator):
- Provides recipes for creating chocolate
Manufacturing Execution System (ManufacturingExecutionSystem):
- Dispatches a lot by providing an initial state update for a chocolate lot including:
- The first controller that should process the lot
- The name of the recipe it should use
- Monitors the chocolate lots as they progress through the system
Station Controller (StationController):
- Receives state updates about chocolate lots
- Receives all recipes
- Filters the chocolate lot state updates to processes only chocolate lots that are assigned to it or in the completed state
- "Processes" the chocolate lot according to the recipe
- Updates the state of a chocolate lot to assign it to the next station controller in the recipe
- When the MES system receives notification that a lot is in the completed state, the lot is unregistered, allowing it to clean up memory and purge its state information for the specific lot being processed.
The flow of data in this example starts when the Manufacturing Execution System determines which recipe needs to be run next. It sends an update to the ChocolateLotState topic that includes information on the recipe to use, and assigns the lot to the first Station Controller that should process it. All Station Controllers are listening to the chocolate lot state, and using content-filtering to ensure that they only see updates when they should be processing a lot.
Download the Example
To view and download the example source code without pre-built executables, visit the RTI Community DDS Use Cases repository in GitHub.
This download includes:
- Pre-built applications you can run that simulate the distributed applications of a chocolate factory
- Example source code and project files for Windows and Linux systems
Download RTI Connext DDS Professional
If you do not already have RTI Connext DDS Professional installed, download and install it now. You can use a 30-day trial license to try out the product. Your download will include the libraries that are required to run the example, and tools you can use to visualize and debug your distributed system.
Run the Example
On Windows systems, navigate to the EXAMPLE_HOME\ExampleCode\scripts directory. In this directory, there are four separate batch files to start the applications:
- StationController.bat (to start a single Station Controller)
On Linux systems, navigate to the EXAMPLE_HOME/ExampleCode/scripts directory. In this directory, there are four separate script files to start the applications:
- StationController.sh (to start a single Station Controller)
You can run these script or batch files on the same machine, or you can copy this example and run on multiple machines. If you run them on the same machine, they will communicate over the shared memory transport. If you run them on multiple machines, they will communicate over UDP.
If you have access to multiple machines on the same network, start running these applications on separate machines.
If your network doesn't support multicast, you can run this example using only unicast data. The two steps you must take to run with only unicast are:
- Run all three applications with the parameter --no-multicast. This causes the applications to load the .xml files that do not depend on multicast in the network.
- Edit the base_profile_no_multicast.xml file to add the address of the machines that you want to contact. These addresses can be valid UDPv4 or UDPv6 addresses.
<discovery> <initial_peers> <!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! --> <!-- Insert addresses here of machines you want --> <!-- to contact --> <!-- !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! --> <element>127.0.0.1</element> <!-- <element>192.168.1.2</element> --> </initial_peers> </discovery>
EXAMPLE_HOME |-- Docs `-- ExampleCode |-- make |-- win32 |-- scripts `-- src |-- CommonInfrastructure |-- Config |-- Generated |-- Idl `-- . . .
The source code is divided into:
- make - Linux makefiles
- win32 - Windows project and solution files
- scripts - Scripts to run applications
- src - Source code
- CommonInfrastructure - The code that all applications use to start using RTI Connext DDS to send or receive data
- Config - XML QoS configuration files
- Generated - Source files generated from Idl
- Idl - Describes the data types that are sent over the network
- Other directories - Source code for specific applications. RTI Connext DDS publishing and subscribing code is in FooInterface.h and FooInterface.cxx
Build the Example
On all platforms, the first thing you must do is set an environment variable called NDDSHOME. This environment variable must point to the ndds.5.x.x directory inside your RTI Connext DDS installation. For more information on how to set an environment variable, please see the RTI Core Libraries and Utilities Getting Started Guide.
On a Windows system, start by opening the EXAMPLE_HOME\ExampleCode\win32\ChocolateFactory-<compilerver>.sln file. This code is made up of a combination of libraries, source, and IDL files that represent the interface to the application. The Visual Studio solution files are set up to automatically generate the necessary code and link against the required libraries.
To build the applications on a Linux system, change directories to the ExampleCode directory and use the command:
gmake -f make/Makefile.<platform>
The platform you choose will be the combination of your processor, OS, and compiler version. Currently, this example only supports one platform: i86Linux2.6gcc4.5.5
Data Model Considerations
When modeling data in DDS, one of the biggest considerations is "what represents a single element or real-world object within my data streams?" In a system where you are updating the state of a lot as it is processed, it is clear that the lot is a distinct object in the system. In this example, we are modeling our data types in the file ChocolateFactory.idl. RTI Connext DDS uses IDL – the Interface Definition Language defined by the OMG – to define language-independent data types. More information about IDL can be found in the RTI Core Libraries and Utilities Users' Manual.
In DDS, these unique real-world objects are modeled as instances. Instances are described by a set of unique identifiers called keys, which are denoted by the //@key symbol. There are many benefits to having unique instances in a system, including the middleware maintaining a separate logical queue for each instance.
In our example, we use lotID as a key field:
// ID of the lot whose status is being updated long lotID; //@key
There is one wrinkle in our assumption that each chocolate lot should be represented as a unique object: the lot is being acted on by multiple Station Controllers. This means we have a choice. We can key this data by just the lot ID, which means updates for the same lot from different stations are stored in the same logical queue and may overwrite each other if we do not configure the queue properly. Alternatively, we can key this data by both the lot ID and the Station Controller ID. With the second option, we can set our queue size to 1 and still know that updates coming from different station controllers about the same lot will not overwrite each other.
// ID of the lot whose status is being updated long lotID; //@key // ID of the Station Controller producing the status StationControllerKind controller; //@key
Data Model Considerations - QoS
We model all the data in this example as Occasionally Changing State Data, which has the following characteristics:
- It is updated only when the state of some object changes – in this case, a state change occurs when a lot changes state
- That object's state is not constantly changing
- Other applications want to know the current state of each object – even if it was published before they started up
This data must be sent reliably because it is not being sent all the time. The Reliability QoS is configured in the recipe profiles XML file. Note that these settings must be enabled on both the DataWriter and DataReader to ensure reliable delivery.
<reliability> <kind>RELIABLE_RELIABILITY_QOS</kind> </reliability>
One of the benefits of using RTI Connext DDS for sending state data is the ability to send data as it changes, and to also ensure that any interested late-joiner will receive the current state data as soon as it starts up. This means the chocolate lot state can be sent as soon as it changes; if an interested application has not been started yet, it will receive the current lot state as soon as it starts. To enable delivery to late-joiners, data must be sent with a transient-local or higher level of durability.
<reliability> <kind>TRANSIENT_LOCAL_DURABILITY_QOS</kind> </reliability>
In order to deliver only the most recent update to the lot state from each Station Controller, this XML configures the history cache on the DataWriter to maintain a history of size one for each lot state instance.
<history> <kind>KEEP_LAST_HISTORY_QOS</kind> <depth>1</depth> </history>
This data does not need to be tuned for extreme throughput, but we do tune it for fast reliability. Details are described in the FactoryStateData XML profile.
The Station Controller is responsible for:
- Listening for lots that it should process
- Updating the state of a lot to say it is processing that lot
- Sending an update on a lot to say that the next station controller in the recipe should process the lot
- If it is the last Station Controller in the recipe, sending an update on a lot to say that it is completed
When you run the Station Controller, you can specify the type of StationController this application represents.
The valid numbers are — 1: Sugar Controller, 2: Cocoa Butter Controller, 3: Cocoa Liquor Controller, 4: Vanilla Controller, 5: Milk Controller.
The code to create the application's DDS interface is in the StationControllerInterface class. This class is composed of four objects:
The DDSCommunicator object creates the necessary DDS Entities that are used to create the ChocolateRecipeReader, ChocolateLotStateWriter and ChocolateLotStateReader Entities.
The ChocolateLotStateReader class is a wrapper around a DDS DataReader that receives all updates of the chocolate lot state data. It also contains a DDS WaitSet object that is used to block a thread until an update becomes available. This class creates a content-based filter that specifies which chocolate lot state data the Station Controller should receive. This filter is useful because the Station Controller, in this example, is only interested in chocolate lot information if:
- It needs to process it, or
- The chocolate lot is completed (so this Station Controller can purge its knowledge of the chocolate lot).
To wait for a chocolate lot update to arrive, call:
void WaitForChocolateLotUpdates( std::vector< DdsAutoType<com::rti::chocolatefactory::generated::ChocolateLotState> > *lotState);
The ChocolateLotStateWriter class is a wrapper around a DDS DataWriter. The Station Controller publishes updates to the lot state as it is processing the lot and when it is done with the lot. Updates to the chocolate lot are written using the PublishChocolateLotState() method. In addition, this class provides an UnregisterChocolateLotState() method that the Station Controller uses when it receives a notification that a lot is completed. The UnregisterChocolateLotState() method calls the DataWriter's unregister_instance() call, which does two things:
- Notifies all listening applications that this DataWriter is no longer writing a particular instance
- Allows the DataWriter to clean up internal memory resources associated with that instance.
The methods for publishing and unregistering a chocolate lot state are:
void PublishChocolateLotState( DdsAutoType<com::rti::chocolatefactory::generated::ChocolateLotState> &lotState); void UnregisterChocolateLotState( DdsAutoType<com::rti::chocolatefactory::generated::ChocolateLotState> &lotState);
The ChocolateRecipeReader class is a wrapper around a DDS DataReader that allows the application to easily query each recipe by name. When the Station Controller is done processing a chocolate lot, it uses this class to query the next step in the chocolate recipe. Then it updates the chocolate lot state to send it to the next Station Controller in the recipe.
void GetRecipe( std::string recipeName, DdsAutoType<com::rti::chocolatefactory::generated::ChocolateRecipe> *recipe);
While MES Systems often embody much more complex logic, this example is kept very simple to support the illustration of DDS concepts. In this example, the Manufacturing Execution System (MES) sends an initial state update for each lot. This tells the first Station Controller that it will soon be processing a lot. You can specify the number of lots and how quickly that you want to be started as follows:
Number of lots the MES should start:
How quickly to start lots:
The Manufacturing Execution System also subscribes to the chocolate lot state, so it receives updates from itself and from all Station Controllers as they update the state of a lot.
The code to create the application's DDS interface is in the class MESInterface. The class is composed of three objects:
The DDSCommunicator object creates the necessary DDS Entities that are used to create the ChocolateLotStateWriter and ChocolateLotStateReader Entities.
The ChocolateLotStateWriter class is a wrapper around a DDS DataWriter that sends the initial chocolate lot state updates. The ChocolateLotStateReader class is a wrapper around a DDS DataReader that receives all updates of the chocolate lot state data.
The Recipe Generator application sends out three preconfigured chocolate recipes. This is the simplest of the three applications.
The code to create the application's DDS interface is in the RecipePublisherInterface class. This class is composed of two objects:
The DDSCommunicator object creates the necessary DDS Entities that are used to create the ChocolateRecipeDataWriter Entity.
The ChocolateRecipeDataWriter class is a wrapper around a DDS DataWriter that sends the recipes. These recipes are modeled as state data. Each recipe is a separate instance, with the recipe name acting as the key field:
string<MAX_STRING_LENGTH> recipeName; //@key