Optimized Energy Dispatch for Microgrids With Distributed Reinforcement Learning

The increasing integration of renewable energy resources (RES) introduces uncertainties in modern power systems and makes the dynamic energy dispatch (DED) problem challenging. Uncertainties lead to dynamic grid control, which needs to be addressed for the optimized DED. Moreover, since energy usage and power generation are distributed, multiple parties can be involved in the DED problem. Thus, DED should be optimized in a distributed way for efficiency and privacy. With the development of the Internet of Things (IoT) and machine learning technology, various data can be gathered and analyzed to achieve intelligent energy management, and the dynamics of power grids should be considered for optimality. For this purpose, we investigate how reinforcement learning can be used to solve the DED problem for a dynamic microgrid (MG) environment. The objective is to determine the optimal power generation for each generator using fossil fuels at each time slot, to minimize the cumulative cost of power generation in a given time period. To achieve this goal, we first model the MG with the practical impact of batteries, photovoltaic (PV) panels, and load banks (external grids). Then we formulate the optimization problem of minimizing the total generation from fossil fuels. To solve this problem, we propose a distributed reinforcement learning algorithm to reduce communication costs and improve data privacy. In the proposed scheme, each generator is considered as an agent, which shares a global state and only obtains its own local loss. Then, different agents work jointly to minimize the global cost. Theoretical analysis is provided to prove the convergence of the proposed algorithms, which are also tested with real-world datasets. Results show that the policy learned from the proposed algorithms can balance the production and consumption in the MG for both fully and partially observable MG environments while simultaneously reducing the total generation cost from fossil fuels.