Just as hardware is being replaced by software that can perform the same functions, some larger process applications and facilities are also giving way to digitalized and IIoT-enabled counterparts—or at least the portions that software can handle.
For instance, Cameco Fuel Manufacturing in Port Hope and Cobourg, Ontario, makes uranium fuel bundles for Canada’s nuclear reactors. These 19-inch-long bundles each contain 37 fuel rods, and 12 bundles are placed in each of about 480 fuel channels at a typical reactor. The company operates 20 different equipment units at each facility, with the Cobourg plant manufacturing metallic components such as seamless tubing and other parts, while the Port Hope plant converts uranium oxide powder into ceramic pellets, and assembles them into the bundles. These manufacturing and assembly lines were also mostly manual historically with limited automation, which they’ve slowly been trying to upgrade.
“Previously, fuel bundle manufacturing relied on standalone islands of equipment with only low-grade automation and manual material handling. However, from 2010 until now, we began to incorporate some higher-level, automotive-style automation with controls, robots, pallet handlers and other devices from ATS Group,” says Jean-Paul Moniz, technical services coordinator at Cameco. “These devices are still networked via EtherNet/IP to ControlLogix PLCs and HMIs from Rockwell Automation, using its longtime Converged Plantwide Ethernet (CPwE) guidelines. This let us implement overall equipment effectiveness (OEE) and downtime tracking. We enhanced these capabilities, also in 2010, by implementing Ignition web-based SCADA/HMI software from Inductive Automation.”
Find the hidden factory
Moniz reports that IIoT gives Cameco more of the data it requires to monitor and manage production and the genealogy of its products. “IIoT solves what’s mainly a scalability issue. As manufacturing systems advanced and get more complex, their need for IT-level tools and IIoT increases,” explains Moniz. “Initial process data was always available. But, as operations continue to evolve, users want to find the “hidden factory within the factory.’ This is making more granular data visible, and revealing more quality defects, finding root causes, identifying more waste streams, or accounting more accurately for costly raw materials. IIoT can provide more detailed and accurate results, and show them on the SCADA system.”
To implement IIoT initiatives and make them successful, Moniz adds that its advocates must convince their management of the value of IIoT and digitalization, and secure their commitment to maintaining them. “We built our first control architecture over 10 years, and added more IT-based technology recently. However, many legacy controls lock users into their systems, so we also started trying to separate those Layer 3 devices with help from the IT side,” says Moniz. “One of the most well-known IT tools is Git, a distributed, open-source control system that tracks and manage software file changes, and is typically used for source code management in software development. Other IT tools include container images, popularized by Docker, which package code and runtime dependencies together, so programs can run in different computing environments.”
The hybrid (software) plant
More recently, Cameco sought to automate even further by adding IT-grade tools and procedures with help from 4IR Solutions, which is a managed services provider that’s enabling Cameco to add software development and operations (DevOps) to its control environment and pass its quality assurance (QA) tests more easily. “4IR is helping us create a hybrid environment for running Ignition software in containers, as well as databases and PLC software,” says Moniz. “These orchestration and management containers use source control systems like Git, which can also automate infrastructure management in an operations technology (OT) environment. This hybrid environment consists of onsite servers and edge devices that crunch some numbers locally, while a private cloud-computing service handles other calculations and analytics.”
To tailor IT tools for OT users, Joseph Dolivo, CTO at 4IR, reports it recommends a hybrid plant strategy that can store and forward data, maintain local fallback control when network connections are broken, and support disaster recovery (DR) of OT systems using secure, cloud-based backups. This method also enables MES and other services to send analytics results up to the cloud, while sending instructions back down to on-premise systems. 4IR often runs SCADA software like Ignition in the cloud, using data transmitted via MQTT from onboard, device-level sensors or MQTT-powered edge devices.
“Due to the COVID-19 pandemic, many users started using routers from Best Buy to access previously isolated, proprietary networks, so they could look at plant data from their home offices,” says Dolivo. “However, this created a lot more vulnerabilities and potential cyber-attack vectors, and exposed existing ones that had been physically inaccessible before but could now be exploited. A hybrid environment can backup an OT system in the cloud, and it’s a better solution than taking a tape drive to a bank vault.”
Because a hybrid environment via IIoT allows computing anywhere, Moniz adds its flexibility brings digital transformation much closer to OT and the plant-floor. “Running SCADA in a container lets many OT functions run in a parallel environment, and also gives users improved security because they’re no longer running on a regular operating system with its vulnerabilities. This is especially important in nuclear-related industries that must be secure,” says Moniz. “This hybrid setting also gives us a parallel testing environment that’s similar to a model, simulation or digital twin because it lets users test OT technologies like SCADA and MES features before deploying them. Overall, we believe a hybrid environment will reduce our onsite requirements, improve cybersecurity, and give us IT-grade tools and standardized processes. This will let us concentrate more on our processes, which helps because we only have so many people. If we have to focus on patching software and server maintenance, that means less time to add value to our end products. If we can automate dealing with software code and other IT tasks, we can concentrate on improving value.”
Software (care) packages
Dolivo reports it’s crucial for manufacturers to be aware of the ongoing revolution in IT-level software and quickly emerging solutions that can serve on the plant-floor. “We can do cloud-based backups of OT systems. Modern IIoT platforms are improving their integration with version control systems,” says Dolivo. “Docker containers provide a standard way to package and run applications, but they also give us new ways to think about and use software. In the past, software would run in PLCs, PCs and virtual machines (VM), but it would need to be cared for like a pet. Containers run software in a disposable environment that’s sometimes compared to cattle because, if something goes wrong, users can simply spin up a new container in its place.”
Dolivo adds that users can even employ the same processes as a public cloud, but still run their software with their own hardware in their facility. In fact, even though far less hardware is needed lately, what’s still required can be rented from hardware as a service (HaaS) providers. For example, Microsoft rents, maintains and updates 2U rack servers for $450 per month each, and 4IR has an option for including them as part of its service. “Users can run real-time control on a server like this, and use software containers, backups and monitoring,” adds Dolivo. “What hybrid cloud computing does best is centralize data historizing and governance, while workloads themselves remain onsite. This arrangement also meets cybersecurity requirements for data residency and sovereignty.”
