Politecnico di Milano — Publications

Deep <i>Q</i>-Network with Reinforcement Learning for Fault Detection in Cyber-Physical Systems

Article

Cyber-Physical System (CPS) is an integration of physical components like actuators, sensors and various types of equipment with the Internet possessing computational ability for efficient communication. A Heterogeneous Independent Network (HINT) is a realistic model that is used for the analysis of inter-dependability between the power grid and communications network. In the traditional Deep [Formula: see text]-Learning method, action needs to be stored in the [Formula: see text] table for the prediction. In real case studies, many state and action values affect the performance of the model. Existing Deep [Formula: see text]-Network (DQN) model generates all possible actions for the [Formula: see text]-values and this involves the generation of excessive information that causes the model to overfit. In this research, the Neural Network is applied to estimate the state–action in the DQN and to store the particular state–action value instead of storing all the state–action values as followed in the traditional method. The HINT model provides realistic failure propagation in the network and its state–action value overfits the existing DQN method due to the presence of more information. The proposed DQN with reinforcement learning stores selected state–action values in the [Formula: see text] tables and eliminates irrelevant information that helps to increase the accuracy and reduce the computational time. The DQN with reinforcement learning is applied to adaptively learn the system to select the optimal action in a continuous interaction with a stochastic environment. The proposed DQN model involves the application of reward function to store state–action value with higher probability based on prediction and eliminates other state–action values. Features such as intra-degree, inter-betweenness, substation-betweenness, relay-betweenness and feature vector are extracted and given as input to the DQN to characterize the critical nodes. The proposed DQN method is evaluated on the HINT network and synthetic network to analyze its efficiency in fault detection. The result shows that the HINT network has a lower prediction error compared to the existing Deep Neural Network (DNN) method. The proposed DQN and LSTM models have accuracies of 98% and 93% in fault prediction, respectively.

J. Stanly Jayaprakash, M. Jasmine Pemeena Priyadarsini, B D Parameshachari et al. 2022

Publications

-Infinity robust controller design for the synchronization of master-slave chaotic systems with disturbance input

Robust Redundant Input Reliable Tracking Control for Omnidirectional Rehabilitative Training Walker

Fault diagnosis of induction motors broken rotor bars by pattern recognition based on noise cancelation

A robust aerial image registration method using Gaussian mixture models

Optimal control of Markovian jump systems via a neural network-based ADP iterative algorithm

Deep <i>Q</i>-Network with Reinforcement Learning for Fault Detection in Cyber-Physical Systems

Editorial special issue on recent developments in deep neural networks for intelligent mechatronics

Observer-Based Adaptive Sliding Mode Control for Nonlinear Stochastic Markov Jump Systems via T–S Fuzzy Modeling: Applications to Robot Arm Model

IEEE Transactions on Systems, Man, and Cybernetics publication information

Toward farm-level health management of wind turbine systems: status and scope for improvements

Building a Nationally Representative Sample of Teachers’ Online and Offline: The Public Instructional Network of School Resources

Determinants of moral norms and ascription of responsibility towards water conservation: the case of Mahabad and Miandoab counties.

Haar Wavelet-Based Optimal Control of Time-Varying State-Delayed Systems: A Computational Method

Non-fragile fault-tolerant control for nonlinear Markovian jump systems with intermittent actuator fault

Repetitive control design for switched neutral systems with input time‐delay

Improved max-log map turbo decoding using maximum mutual information combining

A Multi-source Sensors Framework for Mechanical Fault Diagnosis Under Strong Noise

Stability and l1-gain analysis for positive 2D T–S fuzzy state-delayed systems in the second FM model

Manifold of a planar array and its effects on the accuracy of direction-finding systems

Robust H-Infinity Filtering for Networked Control Systems with Markovian Jumps and Packet Dropouts