The wish was to replace these separate environments with a single new application that would look and feel more like a consumer app than a mission-critical operational system. ICT Group took up the challenge.
Henri van de Kolk, project manager at the Vallei & Veluwe Water Board, remembers the market consultation in 2019 well. A video showed what the future SCADA system could look like. The goal of the new application: to deliver tailor-made information to users, on any device. The integrators present were amazed by the possibilities, but also made critical comments. Because in a small demo, everything is often technically possible, but in practice it is usually a lot more unruly. "We found the tender quite exciting. What we asked had never been done before and that can deter tenderers, resulting in a failed tender," says Van de Kolk.
Tailor-made information
What the water board asked was that the Operations Management Interface (OMI) would put the user first. Depending on its role and location, the application should pull the right data from different underlying data sources and display it in a way that the user immediately sees the information they need. Users are administrators, mechanics and employees of the central control. "They all have different information needs," says Van de Kolk. "Technicians need information about the pumps and valves they are (going to) work on, while employees with a directing function want to see (deviating) information from measurements of purification processes. For each user group, you have to create a design for a user interface (UX design) that meets their wishes and needs. The OMI must also be accessible with any device. You should see the same information on a mobile phone as you do on the screen of a laptop or desktop."
Working well together
The technical challenge was huge. ICT Group bid for and won the tender. Van de Kolk explains what was the deciding factor. "ICT Group impressed with their plan and during the interviews. We were very enthusiastic about their approach to designing and realizing the OMI, in combination with user experience and flexibility during the realization. Moreover, they gave us the feeling that we would be able to work well with them throughout the preliminary phase. And in the end, that is perhaps even more important than the technology. Especially in such a large and complex project as this."
Changing environment
The design of the new environment and the migration of all sixteen treatment plants initially took 3.5 years. One thing was certain in advance: during that period, many things in the internal and external environment are going to change, so we will have to constantly adapt our original plan to the new circumstances. One change that threw quite a spanner in the works was the announcement of the new cybersecurity legislation NIS2, says Van de Kolk. "Due to stricter technical requirements, things that AVEVA showed in the demo could no longer be configured in practice because you would no longer comply with the NIS2. It was precisely the display of OMI on a non-Windows device outside the 'secure' water board network that was affected by this." Any where and any device were and are therefore under heavy pressure.
Separation of IT and OT
ICT Group started the project with a baseline measurement of the current systems at the sixteen locations and with interviews with the users. "There were many opportunities for improvement, but at the same time people were so used to it that they didn't think about what could be improved," says Van de Kolk.
Once the initial situation had been mapped out, two teams were formed: one team dealt with the purely technical side: networks, hardware and all software related to the technology. The other team focused on the user interface (OMI). It was a good decision to separate IT from OT very clearly, says Michel Manders, Operations Manager at ICT Group. "Because they are two different sports." He did notice that most employees at the water board did not see the distinction properly. "That makes sense, they just operate something via a screen and don't think about how different IT and OT are at their core and what consequences that has for choices you make."
Best value approach
In the meantime, that knowledge gap has been made up. Van de Kolk is therefore pleased that he has chosen best value as a project approach. "We really said: ICT Group, you are the expert, we will be guided by your advice." Which, by the way, did not mean that every advice was taken at random. "We have always looked at whether this advice also suits us as an organisation. For example, do certain technical choices also suit our people and our processes? Because those three axes are inextricably linked." In addition, of course, there must be a balance between price and quality. The technically best solution is often also the most expensive solution. Sometimes you just have to make concessions. All these themes regularly led to discussions on the cutting edge. Nevertheless, the relationship remained good. Van de Kolk: "I really give ICT Group a big ten out of ten in terms of collaboration. They always have a long-term view. They don't want to prove themselves right, even though they were sometimes right in hindsight. They just want to achieve the best possible solution, taking into account all the difficult circumstances. That's where we always found each other."
Proud of the result
The relationship with ICT Group has therefore never been under pressure, not even when it turned out that the project was running late, he says. "As I mentioned earlier, the cooperation has been very good throughout. We stood side by side, even – and perhaps especially – when the going got tough. We were in the same boat together and knew where we wanted to go. We didn't always agree on the route and sometimes had fierce discussions, but you need them to come up with the best design. Under challenging circumstances, we have really created something innovative that we can be very proud of."
Would you like to know more about the challenges during the project, such as not replacing the PLCs? Read all about it in the deep dive of the project!
Diluted standards
One of the topics under discussion was the decision not to replace the PLCs, knowing that although these PLCs all use the same three basic standards of the water board, those standards have been diluted to such an extent that five different dialects have emerged. By solving all this within SCADA, you move the problems around. Van de Kolk was all too aware of this. "But replacing all PLCs would become enormously complex, risky and expensive. That is why we have chosen to define the scope of the project at the interface with the PLC," he says.
A choice for which ICT Group had to find a good solution, says Manders. "We were going to build one library, but that turned out not to be technically feasible. It has become five, with five dialects of standards. In the long run, this will have to be reduced to a single standard, but we will take that beyond this project."
No PoC after all
The project started with the requirement to implement the new system as a PoC on two treatment plants: one that previously worked with Wonderware's SCADA and one that worked with the phased-out SCADA system. Van de Kolk: "You can then see for a few months whether it works as expected in practice and make adjustments before you roll it out at the fourteen other locations."
That idea was good, but it was difficult in practice. Because the technical team needed two major AVEVA updates to get new but necessary functionality available and to solve problems with the OMI. "It also turned out to be much more technically challenging on their side than expected," says Van de Kolk. The UX team had to deal with Corona restrictions. It was forbidden to bring the water board's user panel together. This has impacted how we work together on the user interface. As a result, the acceptance of the new interface has been a long time coming. In the end, the project team chose to roll out despite the fact that the new standard had not been tested. "Otherwise, we would have to put the project on hold. That would lead to unacceptable risks and costs."
Change management
At the time of the interview, six of the sixteen locations are live with the new SCADA system. The reactions are mixed, says Van de Kolk. "Users have to get used to what is so beautifully called 'situational awareness' and 'management by exception' in management language. In theory, that sounds great, but it means that employees have to learn to trust the data they see on their phone's screen. In the past, they saw diagrams that were incomprehensible to outsiders: a (very) complex representation of purification processes with a lot of information and a lot of colors. But they knew exactly what to look for. It's a huge switch to only see very concise but targeted management information."
The project team will therefore focus even more attention on change management. "Employees do not yet accept why we are doing this. We need to communicate more about that," says Van de Kolk. At the end of the project, when all sixteen locations have been completed, the points for improvement will also be reviewed. On this basis, the OMI will be even better tailored to needs. "Because it turns out that no matter how well you involve people in advance, the proof of the pudding is in the eating. It's only when people really start working with it that they come up with the valuable feedback."