Tom Edwards is senior technical advisor for Opto 22. Edwards can be reached at [email protected] or 951.695.3006.
Situated at the uppermost origins of the Arkansas River in the heart of the Rocky Mountains, the city of Leadville, Colo., was founded in 1877. At an elevation of 10,152 ft, Leadville is the highest incorporated city in the U.S. It is most famous for its rich deposits of gold, silver, copper and other valuable metals.
Dating back to the 1860s, when gold and silver were first discovered in surrounding areas, Leadville had its origins as a mining camp for local prospectors. Over the next several decades, the adjacent areas became saturated with metal mines that penetrated horizontally and vertically deep into the mountainsides. At the time, there was no easy way to get natural water and rainwater out of the mines, so horizontal channels were built underneath the mining infrastructure so that water could flow down and out to the Arkansas River.
By the 1980s, tremendous amounts of water had accumulated in abandoned and deteriorating mines, and the metals had made this water very acidic. This problem worsened to the point that the river and other nearby waterways had become so contaminated that the water was scalding the feet and legs of animals that waded through. After a U.S. Environmental Protection Agency assessment, multiple areas in and around the Leadville Mining District were declared unsafe for human occupation and designated as Superfund sites.
The Leadville Tunnel
The Leadville Mine Drainage Tunnel (LMDT), completed in 1952, was built by the U.S. Bureau of Mines to drain off water from certain areas of the mining district. The tunnel runs approximately 120,000 ft south/southeast to an area just outside of Leadville. Since 1992, the Bureau of Reclamation (which acquired the tunnel in 1959 and assumed sole responsibility for it after the Bureau of Mines was disbanded by the federal government in 1996) has treated the water flowing out of the tunnel—removing dissolved metals and bringing the water quality into compliance with laws and standards so it can be discharged safely into the Arkansas River.
Eugene Csuti has been principally responsible for the automation and electronics for this water treatment plant since 1996. Among other tasks, he has specified, implemented and maintained a comprehensive process control system for the Bureau of Reclamation to monitor water levels, warn of changing conditions and remove metals. The system also adjusts water pH levels, reduces water turbidity and otherwise treats the water before releasing it cleaner than everyday drinking water.
Csuti began by reexamining the installed Opto 22 mistic control platform, which included multiple G4LC32 controllers, and opting to upgrade to the more advanced SNAP PAC system. It features faster and more powerful controllers, Microsoft Windows-based programming and an easy-to-use HMI development tool. The supplier’s products are designed to be backward compatible, so Csuti was able to upgrade to the new hardware and take advantage of its features and commands without having to alter his still-functioning field wiring and I/O.
Csuti worked with Opto-Solutions, an engineering and design firm specializing in machine automation, building and energy management, and other applications. The firm assisted Csuti in upgrading his entire system to the SNAP PAC system platform. The new system features signature distributed architecture with stand-alone controllers communicating to more robust I/Os that monitor and control thousands of points. Opto 22 I/O is unique in that it includes individual I/O processors capable of time-critical, processing-intensive and repetitive tasks (e.g., high-speed counting, input latching, quadrature counting and PID loop control).
PID control. Part of Leadville’s treatment process consists of adding sulfuric acid and other chemicals to the water, which helps contaminants solidify so they can be pumped out as sludge. Using PID loop control regulates this sulfuric acid injection process and helps keep the water’s pH in the acceptable range (typically from 7.8 to 8.0).
“Our Opto system connects to chemical dosers, and PID control speeds up or slows down the injection process to keep the pH correct,” Csuti said.
Distributed architecture. The PID loop control that this process relies on is not performed by the Opto 22 PAC controller, but instead is executed by the remote processors (also known as brains). Offloading the processing-intensive PID loop control to these brains, which are on the I/O rack units situated throughout the facility, pushes control to the I/O level. This type of distributed architecture offers Csuti many benefits.
OptoSolutions President Anthony Dern, who consults regularly with Csuti on implementation and support issues, explained: “With Opto’s distributed SNAP PAC system, the central controller runs the control programs, or ‘strategies,’ and delegates many functions to the remote brains—from simple I/O reads and writes to more advanced functions like high-speed counting, pulse generation and measurement, and thermocouple linearization. So, by design, the SNAP PAC system reduces the chances of a systemwide failure because if the host PAC should malfunction in any way, you still have independent cells operating and performing their own set of tasks without interruption, indefinitely.”
In Csuti’s case, this means that if his central controller gets knocked offline or out of service, any SNAP PAC brains distributed across the facility will not be affected and will continue executing PID calculations and dosing the water as prescribed.
Another benefit of this distributed architecture relates to wiring, as Csuti’s water treatment operations take place in a massive facility designed with a control room that communicates to six remote I/O panels plus two remote sites that communicate via fiber-optics. These all are wired to I/Os that open and close valves, turn devices on and off, and monitor instrumentation. During the day shift, the system operates in manual mode and the panels are used for local control. After hours, overnight and on holidays, the system is switched into automatic mode and the control room takes precedence. In effect, wiring is needed to two distinct locations. For centralized control of the facility’s 2,500-plus I/O points, many long wiring runs would need to be established and managed.
Parallel wiring. One unique aspect of this water treatment application is that upgrading to the SNAP PAC system has meant that Csuti has had to wire the entire facility in parallel, literally duplicating the architecture that already is in place by installing the new hardware and wiring it to the I/O right alongside the old hardware. Although this approach has been more time-consuming than a typical hardware removal and replacement, the nature of operations at the LMDT—and the considerable impact these operations have on the health and well-being of the surrounding populace and environment—have left no other choice.
“Our plant is a 24/7/365 facility that processes about 2.8 million gal per day, and our water treatment operations are absolutely critical to this community,” Csuti said. “We’re in a position where we just can’t shut things down for any significant period of time. As a result, we have fewer options in terms of how we can perform our system upgrade. Wiring everything in duplicate has been tricky. We’ve got I/O and components with jury-rigged mounting all over the place.”
Ultimately, however, the upgrade will be well worth it. The new system is 100 times faster, has better PID handling capabilities and offers higher-density I/O that will save the Bureau of Reclamation a good deal of space.
With the entire control system configured in parallel as it is, Csuti is able to switch over from his old architecture to his new SNAP PAC architecture, fine-tuning it to his exact operating specifications and preferences. With the new system functioning, Csuti evaluates and makes note of adjustments that need to be made. He then can switch back over to the old system and make these adjustments, while still keeping the facility operational. Csuti plans to continue flip-flopping like this until everything is perfect, at which time he will strip away the old system completely.
Failsafes, alarming and reliability. Regulatory bodies continue to monitor the water in and around Leadville. If too much metal is in the water or it is otherwise unclean, the Bureau of Reclamation faces major fines. To guard against this, Csuti designed control strategies dictating that if any processes are not operating within their defined operational guidelines, the control system will issue commands to divert the water output from the river to a secure holding pond until the problem can be corrected and the system reset.
When the plant is monitoring in automatic mode, if a valve is detected in a wrong position or any analog readings are out of their parameters, the control strategy has the system attempt a restart. If the system cannot restart normally, the entire process shuts down in a safe and orderly fashion. During emergency situations, an autodialer activates and calls one of four staff members who immediately acknowledges the call and hurries on site.
The system also monitors generator sets for power outages, sending instant notifications should any occur. Uninterruptible power supplies keep the controllers and control strategies running at all times.
“Extensive programming has enabled this system to respond to many different scenarios and conditions,” Csuti said. “The sophistication of our strategies—close to 80 of them—and the decision-making that takes place within them make our system something of a living entity.”