Data aggregation using historians for informed decision making

Historian systems have become increasingly popular throughout various industries for their ability to provide operations managers with the data needed to operate and manage their assets efficiently. In the first of two historian articles, we look at what a data historian is, the different types of historians available and some of the hardware requirements.

What is a data historian?

A data historian is a high performance software program that is designed to collect, store, compress (optional) and retrieve production and process data at extremely high speeds. The data is recorded continuously over time from various sources and is saved in a database as time-stamped information. All of the data stored in the archives represents a specific time period to allow historical data to be accessed as required, hence the term historian.

Data historians are used in many industries to capture different information, for industrial systems this can include process plants, PLC and SCADA systems, instrumentation analysis, utilities transmission and distribution data, data centre environmental information, water and wastewater monitoring, plant and heavy equipment analysis, and general environmental monitoring.

The data collected comes from a variety of sources such as Programmable Logic Controllers (PLCs), Distributed Control Systems (DCS), proprietary instrument interfaces, lab instruments, field devices and weather monitoring systems to name a few.

A major benefit of data historians is their ability to collate data over time from one or more areas of plant to provide visibility of the specific system and to allow for prompt decision making based on trends over time. Historians are used to provide transparency of processes, improve productivity, quality and efficiency at all levels of an organisation.

Historians vs Relational Databases

A historian is very different to a relational database such as SQL for example. One major difference between a historian and a relational database is the storage method. Where a relational database grows and requires management, a historian uses a series of ‘flat files’ so that each piece of the puzzle is indexed and stored in fixed size files so that it can be retrieved very quickly, regardless of how old the data is or how much data is stored. A data historian is optimised for time-series data retrieval whereas a relational database is designed for storing contextual or genealogical information.

Types of historians

There are generally two types of historians: operational and enterprise.

Operational historians

Operational historians are used at the operational level for high performance process database applications, with validation, compression and aggregation capabilities. The historian is often embedded or used to provide enhanced data capture from PLC control systems, Telemetry, DCS’s, instrumentation, field devices and SCADA.

Operational historians manage data which relate to processes and production, with its main role consisting of efficient storing of process data for supervisory purposes to analyse plant or production performances and as a reference to real-time operations. If there is an issue with the process or plant, or an operator wishes to compare current and previous data, the historian provides information of what was occurring at an exact point of time. Operational historians are typically used by engineers and plant operators.

Enterprise historians

Compared to operational historians, which are typically found in production facilities, enterprise historians are deployed to support business functions and can be found in a central office, control centre or for larger systems, a data centre. Enterprise historians provide data to a much wider user base than an operational historian typically would. Subsequently, enterprise historians performance is significantly more demanding than local historians used for operations only.

In a multi-tier configuration, an enterprise historian will collect most, if not all of the same data that an operation historian collects. Along with operational data, enterprise historians will collect and aggregate data from additional business sources also.

Enterprise historians are generally used as a gateway to provide service bus access to Enterprise Resource Planning (ERP), Enterprise Asset Management (EAM) and Supply Chain Management systems. From a cyber security standpoint, it is an ideal gateway or interface because by design it isolates the IT and OT systems while still providing essential information to users who need it to operate, design or manage the business more effectively.

Enterprise historians offer the same performance capabilities as operational historians, but their advantage is that the data it produces is generally more holistic and tailored to how the business operates its assets.

Enterprise historians are used at the business level by planners, designers, managers and other stakeholders.

Data access

There is no difference to how data is accessed by either operational or enterprise historians, as it is the technology used that determines the levels of security and permissions.

At an operational level, integrity is maintained by restricting access to data to a select group of users, allowing the business to keep operating without interference regardless of what happens.

Processing power and hard drive space: how much is needed?

The amount of server processing power and hard drive space that will be required for a data historian is dependent on a number of factors, which needs careful consideration for each individual application. To determine hardware requirements requires information such as the number of tags from which data is collected, the rate of alarm and event collection, how often data will be collected, how much data will be stored online for how long and the chosen vendor, of which there are many. It is beyond the scope of this article to provide calculations however Automation IT would be happy to help on an individual basis.

If a historian system and associated infrastructure is too small, it may not capture the necessary data correctly and is liable to reach it’s capacity much quicker than anticipated, this can cause instability and negate any benefits from having an historian in the first place. A larger than necessary historian system will be much more costly to purchase and implement the required hardware and software.

As with many engineered systems, there is typically no one ‘commercial off-the-shelf’ solution suitable for all applications, so it is imperative to employ the services of companies experienced with historian systems, especially where integration into a plant process system such as PLCs are involved. This will ensure the system/s meet the minimum requirements to operate effectively and efficiently, while providing transparent data in the right format at the right time and most importantly, available all of the time.

In the second part of this article, we discuss different historian vendors and what considerations are required for the design, supply, configuration and integration of historian systems.