Experiment Management#
Estimated time to read: 18 minutes
With the Experiment Management module you can overcome the challenges of combining experiment lots with production lots by ensuring that experiments run in a smooth and transparent way for users and the applications that support manufacturing operations.
The Experiment Management module is fully integrated with the MES to support the design of the experiment (DoE) and the execution of the experiment. Experiment lots are tracked and processed in the same way as any other lot and the system automatically enforces the set variations.
Using the Experiment Management module of Critical Manufacturing MES leads a faster speed of learning, increased visibility, tracking and monitoring of experiments, increased operational efficiency, and the reduction of errors.
Info
Experiment Management is a separately licensed module.
This document will guide you through the setup and usage of Experiment Management functionalities.
Overview#
The Experiment Management module allows you to define and carry out controlled experiments on the shop-floor. An Experiment consists of well-defined variations, such as a different Recipe or different BOM, for which you want to test the effect of that variation, typically against a control group, typically assigned to the POR (Process of Record). When using sub-materials (an Experiment Definition mode designated as Sub-Materials), it is possible to assign sub-materials to Material Groups, and then define specific variations at different process Steps for those Material Groups. Because multiple Material Groups are supported, it is possible to carry out several Experiments in a single Material. When no sub-materials are available all variations will be applied to the complete material - this Experiment Definition mode is called Full Material. The execution of the Experiment is enforced by the system and it is transparent for the operator.
Concepts#
The main concepts related with the Experiment Management module are described in the table below:
| Term / Concept | Description |
|---|---|
| Experiment Definition | A structured definition of the specific variations to be applied to a Material or to specific Material Groups during an Experiment. |
| Experiment | The running Experiment based on an Experiment Definition. An Experiment always require a Material. |
| Objective | A goal to be tested by the Experiment Definition, such as cost, yield or performance. |
| Material Group | A group of Sub-Materials that will be subject to some variations together. |
| Step Material Groups | The Steps where Material Groups will be subject to for specific process deviations. |
| Action | A deviation to be applied for a Material or a Material Group at an Experiment Definition Step. |
| Event | The trigger for an Action. |
Table: Experiment Management main concepts
The Experiment Management object model is shown below:
graph TB
A1[Material] --- L1[Experiment]
L1 --- L2[Experiment Material]
L2 --- A2[Material]
L1 === Main[Experiment Definition]
L3[Experiment Definition Material Group] === Main
Main === L4[Experiment Definition Step]
Main === L7[Experiment Definition Objective]
L4 --- L5[Experiment Definition Step Material Group]
L5 --- L6[Experiment Definition Step Material Group Action]
Main --- A3[Flow]
Main -.- A4[Product]
Main -.- A5[Product Group]
classDef mermaid_title color:#000, fill:#fafafa, stroke:#fafafa, stroke-width:0x, font-size:100%, font-weight:200;
classDef mermaid_start color:#000, fill:#fafafa, stroke:#fafafa, color:#fafafa, stroke-width:0x, font-size:100%, visibility: hidden;
classDef mermaid_businessdata color:#000, fill:#65CDE8, stroke:#65CDE8, stroke-width:0px, font-size:100%;
classDef mermaid_nonbusinessdata color:#000, fill:#B7DEE8, stroke:#B7DEE8, stroke-width:0px, font-size:100%;
classDef mermaid_entity color:#000, fill:#FB9F53, stroke:#FB9F53, stroke-width:0px, font-size:100%;
classDef mermaid_entitylinked color:#000, fill:#FCD5B5, stroke:#FCD5B5, stroke-width:0px, font-size:100%;
classDef mermaid_context color:#000, fill:#B9CDE5, stroke:#B9CDE5, stroke-width:0px, font-size:100%;
classDef mermaid_optional color:#000, fill:#B7DEE8, stroke:#65CDE8, stroke-width:1px, font-size:100%, stroke-dasharray: 5 5;
class Main mermaid_entity
class A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12 mermaid_businessdata
class L1,L2,L3,L4,L5,L6,L7,L8,L9 mermaid_entitylinked
class C1,C2,C3,C4,C5,C6 mermaid_context
class N1,N2,N3,N4,N5,N6 mermaid_nonbusinessdata
click Main "../../userguide/business-data/experiment-definition"
click L1 "../../userguide/business-data/experiment"
click A1 "../../userguide/business-data/material"
click A2 "../../userguide/business-data/material"
click A3 "../../userguide/business-data/flow"
click A4 "../../userguide/business-data/product"
click A5 "../../userguide/business-data/product-group" Experiment Management Lifecycle#
There are essentially two core objects that make up the Experiment Management Module as shown in the image below. The Experiment Definition (shown in blue) is a template from which Experiments can be created. The Experiment (shown in green) is the actual running Experiment and that must have a Material associated with it. The Experiment Definition is a versioned object that follows the normal lifecycle of a versioned object.
The Experiment object that represents the running Experiment Definition instance and that always requires a Material follows the lifecycle as shown in the image below and described in the table immediately after:
graph TD
Start -->|"<strong>Assign Material</strong><br>(Material is assigned)"| A1[Created]
A1 -->|"<strong>Move Next</strong><br>(Material reaches the first Step of the Experiment Definition)"| A2[In Progress]
A2 -->|"<strong>Move Next</strong><br>(Material reaches the last Step of the Experiment Definition)"| A3[Completed]
A3 -->|"<strong>Close Experiment</strong><br>(Experiment is closed manually or automatically)"| A4[Closed]
classDef mermaid_title color:#000, fill:#fafafa, stroke:#fafafa, stroke-width:0x, font-size:100%, font-weight:200;
classDef mermaid_start color:#000, fill:#fafafa, stroke:#fafafa, color:#fafafa, stroke-width:0x, font-size:100%, visibility: hidden;
classDef mermaid_businessdata color:#000, fill:#65CDE8, stroke:#65CDE8, stroke-width:0px, font-size:100%;
classDef mermaid_nonbusinessdata color:#000, fill:#B7DEE8, stroke:#B7DEE8, stroke-width:0px, font-size:100%;
classDef mermaid_entity color:#000, fill:#FB9F53, stroke:#FB9F53, stroke-width:0px, font-size:100%;
classDef mermaid_entitylinked color:#000, fill:#FCD5B5, stroke:#FCD5B5, stroke-width:0px, font-size:100%;
classDef mermaid_context color:#000, fill:#B9CDE5, stroke:#B9CDE5, stroke-width:0px, font-size:100%;
classDef mermaid_optional color:#000, fill:#B7DEE8, stroke:#65CDE8, stroke-width:1px, font-size:100%, stroke-dasharray: 5 5;
class Main mermaid_entity
class Start mermaid_start
class A1,A2,A3,A4,A5,A6,A7,A8,A9,A10,A11,A12 mermaid_businessdata
class L1,L2,L3,L4,L5,L6 mermaid_entitylinked
class C1,C2,C3,C4,C5,C6 mermaid_context
class N1,N2,N3,N4,N5,N6 mermaid_nonbusinessdata | State | Description |
|---|---|
| Created | This is the default state when an Experiment is created by assigning a Material to it. There are several restrictions regarding which Materials can be assigned to a particular Experiment Definition such as the Product, the Flow, the Type, the Form as well as the number and form of the Sub-Materials. |
| InProgress | When the Material reaches the first Step defined in the Experiment Definition it becomes InProgress. |
| Completed | When the Material is moved out of the last Experiment Definition Step, it becomes Completed. |
| Closed | When the user marks the Experiment as closed, the Experiment is closed. Typically, the Experiment results are recorded before the Experiment is closed. |
Table: Experiment object lifecycle
Info
An Experiment Definition can be configured to automatically close the Experiment automatically after the last Experiment Step.
During execution there are four important events that the Experiment Management module considers to carry out actions. These are described in the table below:
| Event | Description |
|---|---|
| Queued | Triggered when the Material or Material Group arrives at the Step. |
| Track-In | Triggered when the Material is being Tracked-In. |
| Track-Out | Triggered when the Material is being Tracked-Out. |
| Processed | Triggered after the Material has been Tracked-Out (but not yet moved to the next Step). |
Table: Experiment management events
Any Experiment Definition Action must always be associated with one of the four events. The execution of the Experiment by the system is shown in the image below:
Some actions require that a split takes place before the action is performed. All splits are defined per Experiment Definition Step Material Group and will be evaluated and performed when the Material arrives at the Experiment Definition Step.
Note
At every Experiment Definition Step, there must be one Material Group that is not Split, that is, its Split Type must be None.
| Split Type | Description |
|---|---|
| None | No split is performed. |
| Logical | The Material Group is logically split in the system automatically, even though physically it will remain together. There are three types of physical splits: - For Current Step - a temporary split where the Material Group is merged back at the end of the current Step (in state Processed). - For Multiple Steps - a temporary split where the Material Group is merged back at another Step further ahead in the Flow. In this case, it is necessary to provide a merge point, that is, the merge Flow Path. And also the merge state. - Permanent - in this case, the Material Group is split permanently and will not be merged back again. |
| Physical | The Material Group is split logically and physically. There are three types of physical splits: - For Current Step - a temporary split where the Material Group is merged back at the end of the current Step (in state Processed). - For Multiple Steps - a temporary split where the Material Group is merged back at another Step further ahead in the Flow. In this case, it is necessary to provide a merge point, that is, the merge Flow Path. And also the merge state. - Permanent - in this case, the Material Group is split permanently and will not be merged back again. |
Table: Experiment Material Group split types
Setting Up an Experiment Definition#
Create the Experiment Definition#
An Experiment Definition can be created as any versioned object in the system. The next sub-sections will cover the Experiment Definition creation steps in more detail.
General Data#
The table that follows describes properties that need to be taken into account when creating an Experiment Definition as part of the first step of the Experiment Definition creation wizard as shown in the image below.
| Field | Description |
|---|---|
| Owner Role | If defined, it restricts the modifications of this Experiment Definition to this role. |
| Applicable To | To restrict the material to be selected on an Experiment, there are two options available: Product or Product Group. Depending on the selection, it will only be possible to select a material which fits the option. Neither of the two are mandatory and only the Flow can be defined |
| Flow | The Flow in which the Experiment will be executed. If Product or Product Group is not defined any material running in this flow (And that matches the other conditions below) can be selected for the Experiment. If the selected product has a default Flow, it will be automatically filled out. |
| Maximum Number of Materials | The maximum number of materials which can be used in an Experiment. |
| Mode | The Mode selected determines if the Experiment uses Sub-materials or Full Material. If the selected mode is Full Material, only Materials without Sub-Materials can be used; if the selected mode is Sub-Materials, the field Required Sub-Materials Count must be greater than zero. If zero is indicated, the experiment will be carried out in the entire material. Required Sub-Materials Form can also be indicated. |
| Ignore Sampling Plans | This option determines if during the execution of the Experiment, Sampling Plans are ignored. |
| Ignore SPC Charts | This option determines if during the execution of the Experiment, the DataCollection Parameters values from the Experiment action "SetDataCollection" are ignored and not send to the SPC Chart. |
| Close Experiment Automatically | This option determines if, when completed the last step of the experiment, it will be closed automatically. After completed, it will still be shown in the experiment's list. |
Table: General Data tab steps
Objectives#
The Objectives wizard step, is used to capture the objectives of the Experiment as shown in the table and image below:
| Field | Description |
|---|---|
| Name | Each objective's name must be unique. However, it can be re-used across different Experiment Definitions. The available objectives must be listed in the Lookup table ExperimentDefinitionObjective. |
| Description | An optional field, only used for informative purposes. |
| Target | An optional field, only used for informative purposes. |
Table: Objectives tab steps
Material Groups#
The definition of the Material Groups is necessary if the selected Mode in the General Data tab is Sub-Materials. These groups are used to assign variations (actions) during the course of the Experiment.
Note
These are the primary (main) groups and it is possible to define other groups for a particular Experiment Definition Step.
The first image below displays the Material Groups definition page. Besides this view, a matrix view is also available, as shown in the image below. The matrix view is accessed by selecting the corresponding icon on the top right-hand corner of the screen. The matrix view provides a general overview of the assignment and enables the association and disassociation of a material number to a group, by selecting the corresponding area.
- Each Material Group can define a Material Type. If set, all Sub-Materials assigned to this group will be changed to this material type automatically when the Experiment is created.
- For information purposes, each Material Group can be marked as shippable or not to indicate whether a certain Material Group can be shipped. Remember that this option is for information only and doesn't have any functional effect, meaning that the material can still be shipped.
- There are as many Sub-Materials available to assign to material groups as the value defined in Required Sub-Materials Count.
- Each Sub-Material number must be associated with one and only one Material Group.
- The mapping between the sub-material slot ID and sub-material is performed when it is first assigned to the Experiment.
- Select the mandatory Sub-Materials and optional Sub-Materials. If any Sub-Material is left unselected for any Material Group, a visual indication will be displayed below the Material Groups panel.
When viewing the Material Groups as a matrix, you can choose a specific Sub-Material as mandatory or optional by selecting the proper option on the top right of the matrix and picking the right cell on the matrix.
Define Experiment Definition Steps and Actions#
After creating the basic Experiment Definition object, you then need to define the actions to be performed at each Step for each Material Group. This can be defined in the Experiment Definition Matrix view, by selecting the option Add Step, as shown in the image below:
General Data#
In this tab you must select the Flow Path for which it is intended to perform an action on a Material Group. And if MaterialGroup Mode = Sub-Material, define how different Material Groups are defined for entering and leaving the Step. The available options are:
- Enter - Material Groups are defined when entering the Step.
- Enter and Exit - Material Groups can be defined on entering and leaving the Step.
As shown in the image below:
Info
If option Enter and Exit is selected:
- At Enter - the Split Type must be permanent in order to create new a Material Group at Exit.
- At Exit - only Hold, Change Flow And Step, Temporary Off Flow, Terminate And Send Mail can be selected because only these actions can be set once the Material is Processed.
The Flow Paths selected can be different from the one defined in the Experiment Definition, as there can be operations in other steps that trigger actions on the experiment.
It is possible to mark the Experiment Definition Step to ignore any Sampling Plans so that it does not affect the normal production sampling strategy. It is also possible to mark the Experiment Definition Step to ignore any SPC Charts so that it does not affect the normal production SPC strategy.
Material Groups#
In the case of an Experiment with Sub-Materials, you need to specify the Sub-Materials Groups. It is possible to use the predefined Experiment Definition Groups, or to specify new ones, as shown in the image below. A matrix view is also available by selecting the respective icon on the top right-hand corner of the screen.
For each step's material group, you need to define the Split Type, i.e. the way the material will be split across the experiment, which can be: None, Physical or Logical, as shown in the image below. For the Split Type option None, no further configuration needs to be performed. For the Split Type Physical or Logical, you need to define the Split Scope, and when applicable the Merge Flow Path.
Info
The Split Type is only defined if the selected Experiment Definition Mode is Sub-Materials.
Actions#
In the Actions tab, you can define which actions will be performed for each Material Group, as shown in the image below. Each action must define the event when the action will be triggered: Queued, Track-In, Track-Out or Processed. The available events depend on the selected action, as can be checked in the table below.
Info
It's not necessary to define actions for every Material Group as by default, all Material Groups will follow the standard process unless there is an explicit action defined.
For a detailed a list of possible combinations between Experiment Actions and Events, see Experiment Actions Restrictions.
Depending on the action, additional information may need to be provided. For more information, see Required Information for Experiment Actions.
Note
There is the special security feature ExperimentDefinition.AllActions that provides access to all the available Actions. You can read more about Feature Level Security on the Security page of the User Guide.
Using Experiment Management#
Create Experiment#
To validate the hypothesis of an Experiment Definition, the association with a Material needs to be performed. There are two ways to create an Experiment:
- Select an Experiment Definition and then a Material.
- Select a Material and then an Experiment Definition.
A Material to be selected for an Experiment definition must have the defined:
- Product or Product Group
- Flow
- Material Type
- Material Form (If defined in the Experiment Definition)
- The sub-material form and number of required sub-materials, if the selected Mode is Sub-Materials
After the selection of a Material and an Experiment Definition, if the Mode is Sub-Materials, you need to defined the sub-materials to be assigned to each Sub-Material Numbers of the Experiment Definition, as shown on the image below. The system allows an automatic assignment of the sub-materials to each Sub-Material Numbers, as shown in the image below, with the options:
- Sorted by Sub-Material Name - the first experiment number is assigned with the first sub-material, sorted by name, with an ascending order.
- Sorted by Container and Position - the first experiment number is assigned with the first sub-material, sorted by Container, Position, with an ascending order.
- Random - positions are randomly assigned.
If the Maximum Number of Materials is defined, then the number of experiments records must not exceed this number.
Info
A Material can only be assigned to a single Experiment at one time.
Close Experiment#
When the Material/Sub-materials reach the last Step of the Experiment and all the Actions have been performed, the Experiment transits to the Completed state.
If the Experiment has the Close Experiment Automatically set to true, then it will be Closed automatically, having the option to be edited before being closed. Otherwise, the Close button will be enabled on the Experiment page. Optionally, a Result and a Conclusion can be associated with the Experiment, as shown in the image below:
Experiment Page#
The Experiment page allows you to check its details, actions to be performed and associated sub-materials, for the Experiment Definition Mode of sub-materials. There are two available views: Details and Matrix.
In the Details view you see the configurations of the Experiment, the associated Material and Sub-materials, as shown in the image below:
A Matrix view is also available, and it displays the defined actions for each sub-material per Step and Material Group, as shown in the image below. By selecting a sub-material and expanding the lateral Details tab, further information can be consulted, related to the Experiment, Step, Action and Sub-material.













