Best framework or system architecture for telemetry / SCADA system design & programming

I asked a question about a good book for this topic.

It's not getting many replies, so maybe there aren't any good books (although I find that strange; I must go search Amazon for myself).

In the absence of a book, maybe there is a good framework, some good URLs, or just general advice.

As I asked in the other question:

I am looking at several projects, all with roughly the same functionality.

Some instruments collect some data (or control some functionality). They communicate by Internet (Ethernet/wifi/GPRS/satellite) with a database server which stores the measurements and provides a browser based means of querying the data, producing reports, etc (and possibly also allows control of the remote equipment).

Can anyone recommend a good book describing an approach to developing such a software architecture, keeping it generic, which tools, languages. test methods, etc to use?

For "book", please substitute "framework, some good URLs, or just general advice".

This is a very common system pattern. Who can advise?

I have written a complete SCADA system (excluding the custom instrumentation hardware). The system was designed to be generic to allow the creation of new hardware models, instruments, and data collection - it was not written like so many SCADA systems for an individual company/plant but is used internationally for thousands of companies/plants.

I was the sole developer/designer with one member of management overseeing and guiding the project. It took longer that way, but it was doable. We looked at other SCADA specific systems/frameworks already out there and decided that since our units were custom, it would be easier and more flexible to write the system from scratch leveraging existing development frameworks and 3rd party components. Looking back, this worked out really well for us because we had the time and skill, but this is generally not the best solution depending on your business/contract model.

I am no longer with that company, however, they still use my software exclusively and I left on excellent terms. I would be happy to answer any general questions you have and to help point you in the right direction.

System Architecture

Here is a high-level overview of what the system consisted of:

• Custom Cellular devices that had generic inputs to accommodate multiple instruments of varying types (analog, digital, pressure, amperage, floats, etc.)
• UDP/TCP packets of custom format were sent by the units across the cell network (GPRS) to our servers (Windows Server 2003 R2). Information was sent regularly for reporting, and on customizable state changes that could be programmed at the device or online (the configuration sent over the cell network).
• A custom Multi-threaded .NET application using TCP/UDP Listeners that grabbed incoming packets (Several Hundred-thousand a day), deciphered custom headers, and routed the packets without further interpretation to the correct database (Some clients required their own standalone system)
• A Microsoft SQL 2005 database that acted as the brain for the entire system. Packets were interpreted using CLR functions and automatically triggered alarms (as configured), compiled reports, and kept a full history
• A custom .NET application to handle alerts by placing phone calls, sending SMS messages, and sending emails. The phone logic was handled by an Intel Dialogic Card over analog lines using a combination of recorded prompts and Text-To-Speech.
• 3 ASP.NET sites:
  • Customer facing site that allowed them to manage their accounts/sub users, track alerts, configure units and alerts, chart data, map devices, export reports, etc.
  • Sales site that allowed the distribution of material to sales people, tracking of individual devices, device health reporting, etc.
  • Internal management site that allowed the creation of customer accounts, the configuration/build of units, and all other administrative features as required.
• There was also a custom internal monitoring system to verify the health of the system and to alert technicians of problems as necessary since the system needed 24/7 uptime.
• In addition we created an iOS app, a mobile site, and a custom web service/client (API) to allow the retrieval of customer data directly by customers to allow them to integrate our solution with their existing (usually custom) SCADA systems.

Those are the components we used and they worked. Doing it again I would change a couple of things. I would use Windows Server 2008 R2, SQL 2008 R2, and instead of a Dialogic card I would use Microsoft TellMe using VoIP. I would also use Silverlight instead of ASP.NET. I really like ASP.NET, but Silverlight can give a much better presentation and can be used outside of the browser as necessary - a common request from SCADA operators.

The sites all used 3rd party components so that charting, and tables didn't have to be written from scratch. There are some SCADA specific components (largely Java based) out there. However, we found most of them to be crude, ugly, or way too specific to be used in our generic system (also expensive! It was easier and more flexible to customize a gauge/charting package to "make" our own).

As mentioned, the brain of the system was the database. This was done because Microsoft SQL is a pretty awesome, well supported product designed for extreme uptimes with great backup and performance options. We were also really impressed with the .NET CLR integration that was possible allowing our custom .NET code to run as a part of that process. The units we were supporting came in a variety of models and could be configured to use any combination of instruments and so keeping the database flexible was key. We used a lot of normalization!

One thing that really helped was to use Recursive CTEs to fake the existence of data when the values were still the default. We did this to save space in the database, but this also enabled us to introduce a layer of abstraction in the database that allowed the queries to be flexible as well.

We had messed with OPC in the past but found it too inflexible, difficult, and irritating for our needs. That was a few years ago though, and I haven't looked at it since.

That is a long and very general answer to your question. I can't give you specific code or go into extreme detail since that information is property of that company, but I can answer some design questions and point you towards frameworks/tools we found helpful. My main advice would be to break everything down into separate components and employ the black box model on each so that individual components can be swapped out/improved as necessary. The scope of the project can seem overwhelming otherwise. Let me know if you have further questions or would like more information, good luck!