AC Transmission System Expansion Planning using a Q-learning Algorithm

Mateo David Llivisaca Mejía; Santiago Patricio Torres Contreras; Darwin Fabian Astudillo Salinas

doi:10.1109/PowerTech59965.2025.11180229

AC Transmission System Expansion Planning using a Q-learning Algorithm

Mateo David Llivisaca Mejía (First Author)
, Santiago Patricio Torres Contreras
, Darwin Fabian Astudillo Salinas (Last Author)

Departamento de Ingeniería Eléctrica, Electrónica y Telecomunicaciones

Universidad de Cuenca
Departamento de Ingeniería Eléctrica, Electrónica y Telecomunicaciones Universidad de Cuenca

Research output: Contribution to conference › Paper › peer-review

Abstract

The Transmission Network Expansion Planning (TNEP) is a complex optimization problem. Metaheuristic techniques and mathematical methods are currently used to obtain reliable results for linearized or simplified TNEP formulations. However, solving the TNEP problem using the full AC equations remains an issue. The use of machine learning in the context of power systems optimization has increased in recent years. Therefore, this research explores the application of reinforcement learning to solve the AC TNEP problem, framing it as an episodic decision-making process where each episode represents a distinct planning scenario. The results obtained for the Garver 6-bus system show that it is a viable method.

Original language	English
Pages	1-6
Number of pages	6
DOIs	https://doi.org/10.1109/PowerTech59965.2025.11180229
State	Published - 6 Oct 2025
Event	2025 IEEE Kiel PowerTech, PowerTech 2025 - Duration: 29 Jun 2025 → 3 Jul 2025

Conference

Conference	2025 IEEE Kiel PowerTech, PowerTech 2025
Period	29/06/25 → 3/07/25

Keywords

Episodic problem
Q-Learning
Reinforcement Learning
Transmission Expansion Planning

Access to Document

10.1109/PowerTech59965.2025.11180229

Cite this

@conference{3e54366181894b5ba1688b376e20828b,

title = "AC Transmission System Expansion Planning using a Q-learning Algorithm",

abstract = "The Transmission Network Expansion Planning (TNEP) is a complex optimization problem. Metaheuristic techniques and mathematical methods are currently used to obtain reliable results for linearized or simplified TNEP formulations. However, solving the TNEP problem using the full AC equations remains an issue. The use of machine learning in the context of power systems optimization has increased in recent years. Therefore, this research explores the application of reinforcement learning to solve the AC TNEP problem, framing it as an episodic decision-making process where each episode represents a distinct planning scenario. The results obtained for the Garver 6-bus system show that it is a viable method. ",

keywords = "Episodic problem, Q-Learning, Reinforcement Learning, Transmission Expansion Planning, Episodic problem, Q-Learning, Reinforcement Learning, Transmission, Expansion Planning",

author = "\{Llivisaca Mej{\'i}a\}, \{Mateo David\} and \{Torres Contreras\}, \{Santiago Patricio\} and \{Astudillo Salinas\}, \{Darwin Fabian\}",

note = "Publisher Copyright: {\textcopyright} 2025 IEEE. Publisher Copyright: {\textcopyright} 2025 IEEE.; 2025 IEEE Kiel PowerTech, PowerTech 2025 ; Conference date: 29-06-2025 Through 03-07-2025",

year = "2025",

month = oct,

day = "6",

doi = "10.1109/PowerTech59965.2025.11180229",

language = "Ingl{\'e}s",

pages = "1--6",

}

TY - CONF

T1 - AC Transmission System Expansion Planning using a Q-learning Algorithm

AU - Llivisaca Mejía, Mateo David

AU - Torres Contreras, Santiago Patricio

AU - Astudillo Salinas, Darwin Fabian

PY - 2025/10/6

Y1 - 2025/10/6

N2 - The Transmission Network Expansion Planning (TNEP) is a complex optimization problem. Metaheuristic techniques and mathematical methods are currently used to obtain reliable results for linearized or simplified TNEP formulations. However, solving the TNEP problem using the full AC equations remains an issue. The use of machine learning in the context of power systems optimization has increased in recent years. Therefore, this research explores the application of reinforcement learning to solve the AC TNEP problem, framing it as an episodic decision-making process where each episode represents a distinct planning scenario. The results obtained for the Garver 6-bus system show that it is a viable method.

AB - The Transmission Network Expansion Planning (TNEP) is a complex optimization problem. Metaheuristic techniques and mathematical methods are currently used to obtain reliable results for linearized or simplified TNEP formulations. However, solving the TNEP problem using the full AC equations remains an issue. The use of machine learning in the context of power systems optimization has increased in recent years. Therefore, this research explores the application of reinforcement learning to solve the AC TNEP problem, framing it as an episodic decision-making process where each episode represents a distinct planning scenario. The results obtained for the Garver 6-bus system show that it is a viable method.

KW - Episodic problem

KW - Q-Learning

KW - Reinforcement Learning

KW - Transmission Expansion Planning

KW - Episodic problem

KW - Q-Learning

KW - Reinforcement Learning

KW - Transmission

KW - Expansion Planning

UR - https://www.scopus.com/pages/publications/105019303118

UR - https://ieeexplore.ieee.org/document/11180229

U2 - 10.1109/PowerTech59965.2025.11180229

DO - 10.1109/PowerTech59965.2025.11180229

M3 - Artículo

SP - 1

EP - 6

T2 - 2025 IEEE Kiel PowerTech, PowerTech 2025

Y2 - 29 June 2025 through 3 July 2025

ER -

AC Transmission System Expansion Planning using a Q-learning Algorithm

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this