Things to do and not to do in project execution

Greg McMillan and Stan Weiner bring their wits and more than 80 years of process control experience to bear on your questions, comments and problems. Write to them at [email protected]. Follow McMillan's Control Talk Blog.

See more Control Talk articles.

Stan: We are fortunate to have Denis Backer, a lead engineer at the Emerson Midwest Engineering Center, to share his extensive experience.

Greg: I know Denis from our days together at Monsanto and Solutia. I was always impressed with his thoroughness. Denis, why do really smart people sometimes do really poorly in executing projects and running processes?

Denis: It's because they don't know what they don't know. Things you don't do well will cause so much rework that budget, schedule and morale will suffer. First, let's start out with an overview of practices needed by the lead engineer.

Identify all of the customers. There are, of course, the building owners, the key people interested in the dollars, the process engineers, and the production operators. Operators need to know how to start up and shut down the process. However, they're frequently not as well trained on how to address abnormal situations. Process engineers need to know the functionalities of the system that they can use to solve process problems and increase process efficiency. Often not recognized and not included in setting and meeting project objectives are the maintenance technicians, analyzer specialists, and quality, environmental and safety engineers. Quality engineers want assurance the product will meet customer specifications and that management of change procedures are followed. All too often, validation engineers are brought in too late in the project. The requirements that validation adds are typically reasonable, but the fact that they're usually untimely makes them more stressful than they should be. Safety engineers are particularly interested in SIS and pre-startup safety reviews. Maintenance engineers are interested in the AMS and getting timely training on technology that's new to the plant.

Not enough attention is paid to making sure equipment that's selected and installed can be maintained. You don't want equipment arranged so close together that maintenance is full of head knockers and knuckle breakers. This is especially a problem for skid equipment that may not have good instrumentation or installation practices. For example, one skid needed a lot of disassembly to change a thermocouple.

Greg:  Skids are notorious for achieving low bids by skimping as much as possible on the automation.  I just ran into a skid or more figurative was run over by a skid that did not have an operator interface or a conventional PID. The control algorithm was a long series of individual "if-then-else" statements that were a "one of kind" brain freak leaving the user clueless as to functionality and adjustability. I remember a control valve specialist lamenting how the cheapest valves more designed for a pilot plant than a production facility without even positioners were prolific on skids. The backlash and stiction were horrendous but all the customer knew was the low price.

Stan: What is done at the start of a project to make execution more timely and effective?

Denis: The project kickoff meeting has two key functions. The first is to review the overall project execution. This involves listing the major activities and setting the execution order, including parallel paths. The second key function is to assign responsibilities. What does each person on the team contribute and to whom do they deliver their work product? Project deliverables include correct P&IDs (no kidding), instrument and electrical drawings, instrument specifications, spare parts lists, software specifications for the control and historian systems, commissioning plan and validation plan. Not adequately defining expectations or level of detail required will come back to bite you.

The project manager has a template of checklists that needs to be customized for the project. The checklists should be reviewed at least once per month. Everyone must use their checklists and accountability must be managed.

Having the sequence of project execution steps clearly in mind is essential. Someone has to have enough backbone to proactively "pause" the project when important milestones are not met. Communication is the key. Bad news rarely gets better with age. Everyone has made mistakes that affect other people. It is just part of being on a team. The sooner you let your coworkers know what went wrong, the easier the solution will be.

To avoid pauses, you must manage the ordering of long-lead-time items and learn up front how to implement new technologies. The customer must realize the contractor can't make up for schedule deficiencies. Anyone that is late has to know how it affects the rest of the team.
At the end of each project, document lessons learned. Reviewing previous lessons learned at the start of the next project reminds us of the many ways things can go wrong.

Greg: Operator training systems (OTSs) must be realistic enough to show how the process performs for the new technologies, revealing process, interactions and tuning issues. Online process metrics, such as production rate and process efficiency, should be included to help guide the operators. Sometimes neglected is the opportunity to use OTSs to educate all of the customers and get their input.

Denis: Training on technologies new to the plant or innovative uses of existing technologies must be scheduled for engineers and technicians. Don't shy away from scheduling startup assistance with the vendor. Vendors want you to be successful, and they don't want you blaming their equipment when startup goes poorly.

Stan: How do you judge a project as successful or not?

Denis: A successful project functions well, is on time, is on budget, and the customer wants you back.

Greg: What are some of the things that can go wrong, things that prevent a successful project?

Denis: There are many such things: When the person writing the user requirement specifications does not understand the difference between open-loop control and closed-loop control; not preparing for disturbances; not knowing PID control (especially the implications of the "I" part in terms of windup and overshoot); not knowing the PID modes (auto, manual, cascade, remote output); not knowing control strategies (ratio, ramping, feedforward, cascade and override control); not knowing what "failsafe" means (e.g., what happens on power failure); incorrect alarm design; not knowing how to design safety instrumented functions (SIF) or recognizing safety instrumented levels (SILs).

Sequence descriptions need to be more than a narrative of the time everything went smoothly. They often lack separate tables of parameters and tables of equipment. Even a box of cake mix lists the parameters (eggs, water, oil, oven temperature) and equipment (9x12 inch pan). Alarm design needs more attention than it's usually given. An alarm means "get out of the chair and do something." Rarely does the person setting the alarm values give instructions on what to do after standing up. Also, if alarm delays are not set, millisecond blips can shut down plants. Alarm trip values are often not designed in a way that they can be of use. Instead of being set at a level that gives the opportunity to respond in time, some alarms pretty much tell you, "Kiss this batch goodbye."

Denis: When the person writing the user requirement specifications does not understand the difference between of open loop control and closed loop control, not preparing for disturbances, not knowing PID control especially the implications of the "I" part in terms of windup and overshoot, not knowing the PID modes (auto, manual, cascade, remote output), not knowing control strategies (ratio, ramping, feedforward, cascade and override control), not knowing what "failsafe" means (e.g., what happens on power failure), incorrect alarm design, not knowing how to design safety instrumented functions (SIF) or recognizing safety instrumented levels (SIL).

Sequence descriptions are frequently a narrative of the time everything went smoothly.  They often lack separate tables of parameters, tables of equipment.  Even a box of cake mix lists the parameters (eggs, water, oil, oven temperature) and equipment (9x12 inch pan).  You can easily tell when a sequence designer has not been taught about the functions available in a sequence: set setpoint, set output, ramp functions, timed wait, process wait, cancel wait.  Further they don't always remember sequence states: Run, Hold, Retry, Fail, Abort.  Sometimes they don't differentiate between a sequence "shutting down and "completing".  The sequences for hold and restart are usually less well defined. Alarm design needs more attention than usually given. An alarm means "get out of the chair and do something." Rarely does the person setting the alarm values give instructions on what to do after standing up.  Also, if alarm delays are not set, millisecond blips can shut down plants. Alarm trip values are often not designed in a way that they can be of use. Instead of being set at a level that gives the opportunity to respond in time, some alarms pretty much tell you "kiss this batch goodbye."

There are many poor practices for instrumentation and control design. Some that come to mind are; not prototyping user requirements and specifications to make sure the customer is giving enough information to implement the project, inconsistent tagging of items that perform similar functions, not displaying what a sequence is waiting on, not validating operator input data, not tracking the function of each pair of wires and soft signal, not displaying SIF causes, not preparing data flow diagrams for skid equipment, not using fuses or circuit breakers on hardwired permit-to-run circuits, not having the system implementers do a prototype on a small part of the project up front to resolve misunderstandings, not documenting your code, inadequately testing upgrades,  inadequately testing restart after a interlock, not simulating the process. Having the customer participate in the depths of the code during startup frequently results in a poorer solution than re-grouping on your own.

Greg: A major continual challenge is lack of understanding of open loop and closed loop operation and the downside of integral action in a PID controller. In open loop control, the loop is in either output tracking or the manual or remote output mode and the operator or sequence sets the manipulated variable, usually a flow. Process engineers want to do this because their chemical engineering classes, process design simulations, and Process Flow Diagrams (PFD) emphasized achieving process conditions by setting flows. I have had plant engineers and operators ask me to keep both the manipulated flow and the controlled variable (e.g., level or temperature) constant, which is impossible. Closed loop control, where the PID is in auto, cascade or remote cascade modes, the control loop transfers variability from the controlled variables to the manipulated variables. The PID takes care of the inevitable disturbances, unknowns, and non-idealities in industrial plant operation. Process engineers have difficulty turning the responsibility for setting flows over to a controller. This tendency is most apparent in the use of sequences to set flows for traditional batch operation rather than the use of closed loop control for temperature and composition via fed-batch operation. For pressure and level control, the controlled variable will ramp off to a limit in open loop control. Open loop control of pressure is not done because the pressure can rapidly ramp off to a limit or relief setting. In contrast, the ramp rate for level is quite slow. Operators have sometimes made manual level control their primary task because of poor level controller tuning. In fact most of the oscillations in vessel and column control loops are due to excessive integral action. Often the quick fix is simply to increase the reset time by two or more orders of magnitude as noted in the InTech May-June 2015 article "How to Avoid Common Tuning Mistakes." Integral action has no sense of direction and no anticipation causing overshoot and the wrong split range valve to be open besides triggering the slow oscillations as described in the article from the product of the controller gain and reset time being too small. The online "Appendix B - Basics of PID Controllers" from Tuning and Control Loop Performance - 4th Edition (Momentum Press 2015) details how the integral mode meets the human expectation of control action due to impatience, lack of understanding of dead time and the current operator interfaces that just show numeric values rather than intelligently time scaled trends of a PID’s setpoint, process variable, and manipulated flow.

Stan: How would you sum up what you need to do?

Denis: Take responsibility. Develop a sense of what you don't know. Figure out how to get the information you need. Your next project might not be over budget. Your next project might not be late. However, your poor quality on this project will be there until the system is upgraded or replaced. Poor quality is not self-healing.

Top 10 Signs of a Project Disaster

(10) Only the project manager shows up for a project kickoff meeting.
(9) Project manager says the plant people are too busy to participate in the project.
(8) The plant documentation consists of a 4 foot stack of a printout of the existing configuration with a sign on top "Please migrate and write Process Description"
(7) Project manager says goals are cheapest and fastest project.
(6) Project manager says there is no time or money for simulation.
(5) Transmitters are dumb.
(4) Control valves are on-off valves in disguise.
(3) Most of the loops are in manual.
(2) Process engineer asks how to best operate the plant.
(1) Project manager's check list has just 2 entries: project completion date and total cost.