Data reduction algorithm for correlated data in the smart grid – an open access paper for the IET Smart Grid Journal
About the Lead Author
Dr Zoya Pourmirza, is a research associate at Newcastle University within the School of Electrical and Electronic Engineering. She was awarded her PhD in Information and Communication Technology (ICT) Architecture for Smart Grids from University of Manchester in 2015. Her research expertise includes Smart Grids ICT networks, cyber-security, communication energy efficiency, and data compression.
Zoya carries out a wide range of research for CESI in the area of cyber-security on energy and transport systems.
Contact:- Zoya.Pourmirza@newcastle.ac.uk
About the Co-authors
Dr Sara Walker, School of Engineering, Newcastle University, Newcastle upon Tyne, UK
John Brooke, Freelance Consultant, Manchester, UK
About the Paper
Smart grids are intelligent electrical networks that incorporate information and communication technology (ICT) to provide data services for the grid. In this work, we investigated an ICT architecture at the level of the electrical network where monitoring and control have not previously been deployed. Energy constraints are one of the major limitations of the ICT in the Smart Grid, especially where wireless networking is proposed. The main contribution of this paper is that we proposed a data reduction algorithm suitable for Smart Grid applications which significantly improves the energy efficiency of the communication network by minimizing the communication energy cost while maintaining the integrity and quality of data.
One approach to providing energy efficiency in the communication system is to use a data reduction algorithm to reduce the volume of data prior to transmission. Our survey of data compression algorithms showed that there is no single method that is superior for all forms of data streams. Therefore, we designed and developed a practical data reduction algorithm called DRACO (Data Reduction Algorithm for COrrelated data), on the basis of readings from monitoring devices that are typical of electricity network data patterns. In applications where the metering devices collect data with a high acquisition rate and transmit them to a control unit, a great degree of data correlation occurs. Taking this fact into consideration, we developed a data reduction algorithm that discards the redundant parts between each two consecutive measured values and transmits the changing parts only: these parts are a small portion of the binary representation. This algorithm can improve the energy efficiency of the communication network by transmitting a smaller volume of data while keeping data integrity.
DRACO is envisaged to be implemented on resource-constrained sensors, therefore simplicity in the design of the algorithm is a key issue. It also provides a low level of security for communication between devices since we are transmitting a modified or cipher data instead of raw data.
Validation
In this paper we examined the efficiency of DRACO on both simulated data and real data collected from the substation level of the Grid, which were produced at a very high sampling rate. We demonstrated DRACO can achieve compression ratios of 70%–99% depending on the data characteristics. Figure below shows compression efficiency over 70% for simulated data.
Experimentation
In this paper, we conducted several evaluations and comparisons. For example, we designed an experiment to assess the effect of various sampling rates on the efficiency of DRACO. We examined the data being logged with different frequencies. Figure 2 below shows that, as the frequency of the data acquisition rate increases, the original size of the data will increase. However, as we start to sample more frequently, the correlation between every two consecutive values is higher and DRACO performs best on data with stronger correlations. So, the difference between the original data size and the DRACO reduced data size also grows. Thus, with a higher sampling rate, we could transmit more data about the network, and with the use of the DRACOs we could send this data more efficiently in terms of data volume.
The team also designed another experiment to examine the effect of DRACO on the bit rate. This experiment was carried out to determine the link between significant events in the actual data profile and the maximum/minimum bit rate. As shown in the figure below, the correlation between the two graphs indicates the dependency of the data transfer rate on the rate of change of the quantity being measured (e.g. total active power).
Finally, to assess the efficiency of the DRACO we compared its performance with other data reduction algorithms and showed it performs reasonably good in these comparisons.
Conclusion
In this work, we focused on proving the communication energy awareness and concluded that DRACO is suitable for smart grid applications since it optimizes the network resource consumption and reduces the communication energy cost while maintaining the integrity and quality of data. In near future, the growth in the number of monitoring devices in the smart grid will lead to an explosion in data volume, which will cause storage and network congestion problems. DRACO could also be an initial point for addressing these problems.
The full paper is available to view online.