Application Layer - Container Stack#
The Critical Manufacturing MES application tier is deployed in a containerized environment, providing solid gains in terms of configuration and usability by taking advantage of a mature containerization architecture such as the one supplied by the Kubernetes or Docker engines.
The application tier takes advantage of a container orchestrator in order to ensure almost boundless horizontal scalability. Additional worker nodes can be added to the cluster and additional instances of specific components can be launched to accommodate extra load on a particular installation.
Software Requirements#
Containerized deployments rely on a container orchestrators. Critical Manufacturing MES supports the following container Orchestrators:
| Orchestrator | Version | On-Premises | Cloud |
|---|---|---|---|
| Vanilla Kubernetes | v1.33 or later | ||
| Canonical Kubernetes | v1.33 or later | ||
| Red Hat OpenShift | v4.15 or later | ||
| Azure Kubernetes Services | |||
| Amazon Elastic Kubernetes Service |
When using these container orchestrators, keep in mind the following restrictions:
-
Operating System
- Any Linux distribution compatible with the selected container orchestrator.
- Recommended: Ubuntu Server 20.04 LTS or Red Hat Enterprise Linux 9.
- Only x64 architecture is supported. (Container images for x86 or ARM architectures are not available).
-
Container Engine
- CRI-O (Kubernetes only)
-
Additional Required Software
- Powershell 7.1 (On-Premises installations only)
Workload characterization#
Due to the flexibility offered by container orchestrators, Critical Manufacturing does not recommend specific hardware configurations, but provides an adequate description of the expected workload generated by the MES system. With this information, system administrators can provision and adequately size a cluster to run the MES application tier, considering high-availability and scalability requirements.
This section offers a description of the MES workload in different example configurations.
Similarly to what happens in the more traditional approach, the exact hardware requirements depend ultimately on the specific customer environment. Please consult with Critical Manufacturing for more specific recommendations adapted to specific deployments.
Hardware requirements for computational resources#
The following table describes the approximate requirements for some sample configurations, depending on their purpose. All examples assume no workloads related with equipment integration.
- Development: Development sandbox with no significant system load.
- Training / Staging: System with a similar configuration to a productive deployment, with higher resource demands.
- Production (MES only): Productive MES deployment with low to medium volume, without significant usage of Data Platform or Machine Learning capabilities. This configuration includes adequate overhead to support a new MES deployment during an upgrade process.
- Production (MES with Data Platform and Machine Learning): Productive MES deployment with medium to high volume, considering usage of Data Platform and Machine Learning features. This configuration includes adequate overhead to support a new MES deployment during an upgrade process.
| Workload | vCPU | Clock speed | Memory |
|---|---|---|---|
| Development | 10 | 2+ GHz | 18 GB |
| Training / Staging | 20 | 2+ GHz | 32 GB |
| Production (MES Only) | 30 | 2+ GHz | 64 GB |
| Production (MES + Data Platform + Machine Learning) | 50 | 2+ GHz | 128 GB |
Table: Hardware requirements for computational resources
Note
The workloads defined above may be combined in the same cluster by simply adding the characteristics of the intended environments to host on the same cluster.
For example, if you want to host two Development Systems (one Staging and one MES-only production system) in the same cluster, the resource requirements will add up, as shown below:
| Workload | vCPU | Memory |
|---|---|---|
| Development 1 | 10 | 18 GB |
| Development 2 | 10 | 18 GB |
| Staging | 20 | 32 GB |
| Production (MES only) | 30 | 64 GB |
| Total | 70 | 132 GB |
Table: Hardware requirements example for cluster
If several environments are deployed to the same cluster, Critical Manufacturing recommends resource quotas to be defined for each environment in order to prevent resource starvation on other environments or applications hosted on the same cluster. Usually, this is only possible on Kubernetes clusters.
Persistent Storage#
The application layer requires access to persistent storage volumes to hold object attachments, documents, installation packages, and other application files. The storage requirements depend heavily on the expected usage of the system.
Persistent volumes can be provisioned in different ways, depending on the deployment target platform. At the time of release of this version, Critical Manufacturing MES supports the following volumes types:
| Volume type | Usage |
|---|---|
| Local | Refers to local path on the node file system. |
| SMB/CIFS | Refers to a shared folder accessible through the SMB protocol. |
| NFS | Refers to a shared folder accessible through the NFS protocol. |
| Azure File | Refers to an Azure File Share available in an Azure Storage Account (for AKS and OpenShift deployments only). |
| Storage Class | Refers to a Kubernetes API for dynamic PV provisioning based on defined storage profiles (provisioners). |
| Persistent Volume | Refers to a cluster-level, provisioned storage resource with an independent lifecycle, accessed via PVCs; can be static or dynamically provisioned. |
Table: Support for persistent volume types
Note
Support for additional volume types may be added in the future.
The table below lists all the required volumes along with the component that requires it. Please note that, especially for Kubernetes deployments, there are different requirements for access modes. Some volumes require Read-Write-Many (RWX) access mode.
Info
Note that due to the specific requirements of some technologies, high-performance block storage is required (identified in the table above). The volumes can be statically provisioned using existing Persistent Volumes (previously created by the customer) or dynamically provisioned using Storage Classes in Kubernetes deployments. For high-performance block storage, we recommend a storage solution with at least 10000 IOPS (Input/Output Operations Per Second)
| Volume | Component | Access Mode | Storage Type | Minimum Size |
|---|---|---|---|---|
| connect-iot-repo | connectiot-manager, envmanager, host | RWX | Any | 2 GB |
| grafana-share | grafana | RWX | Any | 1 GB |
installation-data | aggregation-engine, cube, data-manager, discoveryservices, envmanager, epf-alarm-mng-at, epf-alarm-mng-erh, epf-alarm-mng-mes-eh, grafana, help, host, housekeeper, messagebus, rasa-actions, securityportal, traefik-fwdauth, ui | RWX | Any | 5 GB |
| mes-host-documents | host | RWX | Any | 10 GB |
| ml-agent-folder | mlplatform-agent | RWX | Any | 1 GB |
| ml-agent-folder | mlplatform-training | RWX | Any | 1 GB |
| ml-export-folder | data-manager | RWX | Any | 1 GB |
| ml-export-folder | mlplatform-training | RWX | Any | 1 GB |
| ml-training-folder | mlplatform-training | RWX | Any | 1 GB |
| redis-data | redis | RWO | Block Storage | 1 GB |
| cube | cube | RWO | Any | 5 GB |
Table: Volumes required for persistent storage.
Warning
The grafana-share volume needs to POSIX compliant. It is recommended to avoid using Azure Files as they can cause issues related with this requirement.
Warning
The installation-data volume is used by the system to store installation artifacts consumed by other components. In this volume, a directory named <environmentname>/backups will be created in order to extract the database backups for initial database deployment. This directory must be accessible by the SQL Server engine and the path must be provided during system installation, in order to ensure the initial database backup can be restored. For this reason, dynamic provisioning of this volume is not recommended since it may not generate a deterministic folder path that can be provided to an external SQL Server.
Optional Volume configuration#
| Volume | Component | Access Mode | Storage Type | Minimum Size |
|---|---|---|---|---|
| dagster | dagster | RWO | Any | 1 GB |
| kafka-data | kafka | RWO | Block Storage | 100 GB (per broker) |
| clickhouse-data | clickHouse | RWO | Block Storage | 100 GB |
| clickhouse-log | clickHouse | RWO | Block Storage | 20 GB |
| rabbit-data | rabbit | RWO | Block Storage | 1 GB |
| rabbit-log | rabbit | RWO | Block Storage | 1 GB |
| storage-data | storage | RWO | Block Storage | 25 GB |
| host, discoveryservices, envmanager, help, messagebus, UI | RWX | Any | 20 GB |
Table: Volumes of optional components or optional volumes.
Info
The amount of storage required by the Critical Manufacturing components greatly depend on how the system is modelled and used. The size indications present in this document are a recommendation for low/medium volume sites, based on our experience. During the configuration of the system these values should be adjusted according to the defined parameterization.
Warning
While the mes_logs_share volume does not require Block Storage, it is crucial to consider that the overall system performance will be significantly impacted by the write speed to this volume. For example, a high-latency network share must be avoided in production environments to maintain optimal performance. When installing MES in containerized environments on a Linux-based environment in Ext4 (or a Unix file system family), a lost+found folder will be created (typically through fsck) for the volumes. Kafka containers are deployed together with the application and will not start due to inconsistent naming format. This folder must be either removed before starting the relevant Kafka containers or a different file system must be used that doesn't create this folder.
Scalable Components#
The table below lists all the components along with the images that can be scalable.
| Component | Image |
|---|---|
| core-host | criticalmanufacturing/core-host |
| core-ui | criticalmanufacturing/core-ui |
| data-manager | dataplatform/datamanager |
| edgesquidproxy | criticalmanufacturing/edgesquidproxy |
| epf-alarm-mng-at | dataplatform/epf-alarm-management-action-trigger |
| epf-alarm-mng-erh | dataplatform/epf-alarm-management-event-rule-handler |
| epf-alarm-mng-mes-eh | dataplatform/epf-alarm-management-mes-event-handler |
| grafana | criticalmanufacturing/grafana |
| help | criticalmanufacturing/help |
| host | criticalmanufacturing/host |
| housekeeper | dataplatform/housekeeper |
| messagebus | criticalmanufacturing/messagebus |
| mlplatform-agent | dataplatform/mlplatformagent |
| mlplatform-training | dataplatform/mlplatformagent |
| rasa | criticalmanufacturing/rasa |
| rasa-actions | criticalmanufacturing/rasa-actions |
| reference | criticalmanufacturing/reference |
| securityportal | criticalmanufacturing/securityportal |
| traefik | traefik |
| traefik-fwdauth | criticalmanufacturing/traefik-fwdauth |
| ui | criticalmanufacturing/ui |
Table: Components can be scalable.