Deep reinforcement learning for scheduling of green hydrogen production process

LIU Jingxin; ZHAO Liang

doi:10.11949/0438-1157.20260420

您当前的位置：

首页 >

文章列表页 >

Deep reinforcement learning for scheduling of green hydrogen production process

更新时间：2026-05-09

- Deep reinforcement learning for scheduling of green hydrogen production process
- CIESC Journal (2026)
- 作者机构：
  
  华东理工大学信息科学与工程学院，上海 200237
- 作者简介：
- 基金信息：
- DOI：10.11949/0438-1157.20260420
  CLC： TP 273.1
- Received：28 March 2026，
  
  Revised：2026-05-09，
  
  Accepted：09 May 2026，
- 稿件说明：
移动端阅览
LIU Jingxin, ZHAO Liang. Deep reinforcement learning for scheduling of green hydrogen production process[J/OL]. CIESC Journal, 2026.
DOI：

LIU Jingxin, ZHAO Liang. Deep reinforcement learning for scheduling of green hydrogen production process[J/OL]. CIESC Journal, 2026. DOI： 10.11949/0438-1157.20260420.

摘要

针对风光发电波动导致电解槽负载频繁变化、弃电与购电并存等问题，本文建立了光伏—风电—储能—电解槽耦合制氢过程模型，并将在线调度表述为马尔可夫决策过程；在多目标奖励中考虑产氢收益、购电成本、弃电抑制、负载平滑及储能荷电状态（SOC）安全约束，采用软演员-评论家（SAC）算法获得最优调度策略。基于NASA气象数据与电价序列的算例表明，所提方法在多场景下比专家规则与模型预测控制取得更高产氢量与可再生能源利用率，并显著降低弃电，且单步推理开销可满足5 min调度周期的实时应用需求。

Abstract

To address the issues of frequent load fluctuations of electrolyzers caused by volatile wind and solar power

as well as the coexistence of renewable energy curtailment and electricity purchase costs

this paper establishes a coupled photovoltaic-wind-energy storage-electrolyzer hydrogen production system model

and formulates the online scheduling problem as a Markov decision process. The multi-objective reward function comprehensively considers hydrogen production benefits

electricity purchase costs

renewable energy curtailment mitigation

load smoothing

and state-of-charge (SOC) security constraints

and the Soft Actor-Critic (SAC) algorithm is adopted to obtain the optimal scheduling strategy. Case studies based on NASA meteorological data and electricity price sequences demonstrate that the proposed method achieves higher hydrogen production and renewable energy utilization efficiency than expert rules and model predictive control under various scenarios

significantly reduces renewable energy curtailment

and the single-step inference cost can meet the real-time application requirements of a 5-minute scheduling cycle.

关键词

Keywords

references

国家发展和改革委员会，国家能源局 . 氢能产业发展中长期规划（ 2021 - 2035 年）[EB/OL ] . 2022 . https://zfxxgk.nea.gov.cn/2022-03/23/c_1310525630.htm https://zfxxgk.nea.gov.cn/2022-03/23/c_1310525630.htm .

IEA . Global Hydrogen Review 2024 [R ] . Paris : International Energy Agency , 2024 . https://www.iea.org/reports/global-hydrogen-review-2024.

Hydrogen Council . Hydrogen Scaling Up: A Sustainable Pathway for the Global Energy Transition [R ] . Brussels : Hydrogen Council , 2017 . https://hydrogencouncil.com/en/newsroom/.

Aminaho E N , Aminaho N S , Aminaho F . Techno-economic assessments of electrolyzers for hydrogen production [J ] . Applied Energy , 2025 , 399 : 126515 .

Sayed-Ahmed H , Toldy Á I , Santasalo-Aarnio A . Dynamic operation of proton exchange membrane electrolyzers: Critical review [J ] . Renewable and Sustainable Energy Reviews , 2024 , 189 : 113883 .

Nguyen E , Olivier P , Péra M C , et al . Impact of short-term intermittent operation on experimental industrial PEM and alkaline electrolyzers [J ] . International Journal of Hydrogen Energy , 2025 , 126 : 516 - 530 .

Minnah P , Pope K . Degradation modelling and the impact of intermittent operation on proton exchange membrane electrolyzers [J ] . International Journal of Hydrogen Energy , 2026 , 202 : 152962 .

Johnsen A G , Mitridati L , Zarrilli D , et al . The value of ancillary services for electrolyzers [J ] . Computers & Chemical Engineering , 2026 , 204 : 109360 .

Zeng A D , Wang Z R , Wang J W , et al . Optimal scheduling of electricity–hydrogen–heat integrated energy system considering electrolyzer dynamic temperature and pipeline network heat storage [J ] . International Transactions on Electrical Energy Systems , 2025 , 2025 : 9478361 .

Zhang L Q , Dai W Z , Zhao B , et al . Multi-time-scale economic scheduling method for electro-hydrogen integrated energy system based on day-ahead long-time-scale and intra-day MPC hierarchical rolling optimization [J ] . Frontiers in Energy Research , 2023 , 11 : 1132005 .

Amores E , Rodríguez J , Carreras C . Influence of operation parameters in the modeling of alkaline water electrolyzers for hydrogen production [J ] . International Journal of Hydrogen Energy , 2014 , 39 ( 25 ): 13063 - 13078 .

Masampally V S , Mitra K , Runkana V . Modeling and simulation of hydrogen production in a PEM water electrolyzer at high current densities [J ] . Energy Conversion and Management , 2026 , 352 : 121110 .

Zhang Y T , Zhang T C , Liu S W , et al . Multi-energy complementary optimal scheduling based on hydrogen gas turbine considering the flexibility of electrolyser [J ] . IET Renewable Power Generation , 2024 , 18 ( 12 ): 1972 - 1985 .

Cardona P , Costa-Castelló R , Roda V , et al . Model predictive control of an on-site green hydrogen production and refuelling station [J ] . International Journal of Hydrogen Energy , 2023 , 48 ( 47 ): 17995 - 18010 .

Ulleberg Ø . Modeling of advanced alkaline electrolyzers: a system simulation approach [J ] . International Journal of Hydrogen Energy , 2003 , 28 ( 1 ): 21 - 33 .

Abdin Z , Webb C J , Gray E M . Modelling and simulation of an alkaline electrolyser cell [J ] . Energy , 2017 , 138 : 316 - 331 .

Ashraf H , Kawai E , Mae M , et al . Advancing electrochemical modelling of PEM electrolyzers through robust parameter estimation with the weighted mean of vectors algorithm [J ] . Multiscale and Multidisciplinary Modeling, Experiments and Design , 2025 , 8 ( 9 ): 397 .

Mößle P , Herrmannsdörfer T , Welzl M , et al . Economic optimization for the dynamic operation of a grid connected and battery-supported electrolyzer [J ] . International Journal of Hydrogen Energy , 2025 , 100 : 749 - 759 .

Zheng Y , You S , Bindner H W , et al . Optimal day-ahead dispatch of an alkaline electrolyser system concerning thermal–electric properties and state-transitional dynamics [J ] . Applied Energy , 2022 , 307 : 118091 .

Voulkopoulos X , Dimitriadis C N , Georgiadis M C . Optimal scheduling of a RES–Electrolyzer aggregator in electricity, hydrogen and green certificates markets [J ] . International Journal of Hydrogen Energy , 2024 , 78 : 1099 - 1107 .

Wang J B , Wen J F , Wang J R , et al . Coordinated scheduling of wind-solar-hydrogen-battery storage system for techno-economic-environmental optimization of hydrogen production [J ] . Energy Conversion and Management , 2024 , 314 : 118695 .

Haarnoja T , Zhou A , Abbeel P , et al . Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor [EB/OL ] . 2018 : arXiv : 1801 . 01290 . https://arxiv.org/abs/1801.01290 https://arxiv.org/abs/1801.01290 .

Haarnoja T , Zhou A , Hartikainen K , et al . Soft actor-critic algorithms and applications [EB/OL ] . 2018 : arXiv : 1812 . 05905 . https://arxiv.org/abs/1812.05905 https://arxiv.org/abs/1812.05905 .

Bäumler A , Benterki A , Meng J W , et al . Energy management strategies based on soft actor critic reinforcement learning with a proper reward function design based on battery state of charge constraints [J ] . Journal of Energy Storage , 2024 , 90 : 111797 .

Zhao Y J , Lin F , Yang Z P . Soft actor-critic-based energy management strategy for hybrid energy storage system in urban rail transit DC traction power supply system [J ] . Electric Power Systems Research , 2026 , 252 : 112372 .

Lu W C , Gao Y , Sun Z , et al . An Improved soft actor–critic framework for cooperative energy management in the building cluster [J ] . Applied Sciences , 2025 , 15 ( 16 ): 8966 .

Van Quang T , Doan D T . Online transfer learning (OTL) for accelerating deep reinforcement learning (DRL) for building energy management [J ] . Journal of Building Performance Simulation , 2025 : 1 - 20 .

RMI . 2024 China Power Market Outlook: 10 Key Trends for Market Players [R ] . RMI , 2024 . https://rmi.org/wp-content/uploads/dlm_uploads/2025/01/final-2024-China-Power-Market-Outlook-10-Key-Trends-for-Market-Players-2501117.pdf.

Li D F , Li Z H , Li X J , et al . Optimization method of time-of-use electricity price for the cost savings of power grid investment [J ] . Frontiers in Energy Research , 2024 , 12 : 1361116 .

Dong L , Huang Y , Xu X , et al . Research on priority scheduling strategy for smoothing power fluctuations of microgrid Tie-lines based on PER-DDPG algorithm [J ] . IET Generation , Transmission & Distribution, 2024 , 18 ( 20 ): 3221 - 3233 .

Tang D G , Ge P X , Yuan C Q , et al . Optimal management of coupled hydrogen-electricity energy systems at ports by multi-time scale scheduling [J ] . Applied Energy , 2025 , 391 : 125885 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

ANN-GA integrated framework for intelligent screening of ORC mixture working fluids and performance optimization

Deep reinforcement learning-based intelligent production scheduling for a heterogeneous production line plants

Performance analysis of combined heat and power system equipped with electric boiler and thermal storage device

Optimization analysis of 3D modelling of SOEC stacks taking into account inhomogeneities

Research on performance of PEM electrolyzer driven by multi-objective optimization

Related Author

LI Ling

ZHANG Yu

LIU Linlin

WANG Chao

DU Jian

WANG Dantong

GAO Xiaoyong

ZHANG Li

Related Institution

Institute of Chemical Process Systems Engineering, School of Chemical Engineering, Dalian University of Technology

School of Control Sciences and Engineering, Dalian University of Technology

Department of Automation, China University of Petroleum

Hainan Institute of China University of Petroleum

Huaneng Jinan Huangtai Power Plant Co., Ltd.

⁰