A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows

Biao Yin, Monica Menendez

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In the paper, we propose a model-based reinforcement learning algorithm, i.e., approximate dynamic programming for signal control at isolated intersections with mixed traffic: vehicles and pedestrians. The integrated optimization problem is formulated by the discrete-time dynamic process. The system state is represented by the combination of weighted vehicle queue lengths and the number of waiting pedestrians. The system action is generated in each decision step by the proposed algorithm. To solve the computation issue in conventional dynamic programming, the proposed algorithm adopts a linear approximation function that helps to quickly obtain a near-optimal solution. In simulation, we extract traffic information from the traffic simulator SUMO. The on-line traffic information is provided for the algorithm to make a signal decision. After testing various scenarios, results show that the proposed algorithm has potential control performance. We also reveal the delay changes with different weights assigned to the vehicle and pedestrian components.

Original languageEnglish (US)
Title of host publicationCICTP 2019
Subtitle of host publicationTransportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals
EditorsLei Zhang, Guangjun Zhang, Pan Liu, Jianming Ma
PublisherAmerican Society of Civil Engineers (ASCE)
Pages3123-3135
Number of pages13
ISBN (Electronic)9780784482292
DOIs
StatePublished - Jan 1 2019
Event19th COTA International Conference of Transportation Professionals: Transportation in China - Connecting the World, CICTP 2019 - Nanjing, China
Duration: Jul 6 2019Jul 8 2019

Publication series

NameCICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals

Conference

Conference19th COTA International Conference of Transportation Professionals: Transportation in China - Connecting the World, CICTP 2019
CountryChina
CityNanjing
Period7/6/197/8/19

Fingerprint

Traffic signals
traffic control
Reinforcement learning
learning method
pedestrian
reinforcement
traffic
Dynamic programming
programming
Learning algorithms
Simulators
scenario
Testing
simulation
learning
performance

Keywords

  • Approximate dynamic programming
  • Intersection
  • Pedestrians
  • Signal control

ASJC Scopus subject areas

  • Transportation

Cite this

Yin, B., & Menendez, M. (2019). A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows. In L. Zhang, G. Zhang, P. Liu, & J. Ma (Eds.), CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals (pp. 3123-3135). (CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals). American Society of Civil Engineers (ASCE). https://doi.org/10.1061/9780784482292.270

A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows. / Yin, Biao; Menendez, Monica.

CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals. ed. / Lei Zhang; Guangjun Zhang; Pan Liu; Jianming Ma. American Society of Civil Engineers (ASCE), 2019. p. 3123-3135 (CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yin, B & Menendez, M 2019, A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows. in L Zhang, G Zhang, P Liu & J Ma (eds), CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals. CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals, American Society of Civil Engineers (ASCE), pp. 3123-3135, 19th COTA International Conference of Transportation Professionals: Transportation in China - Connecting the World, CICTP 2019, Nanjing, China, 7/6/19. https://doi.org/10.1061/9780784482292.270
Yin B, Menendez M. A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows. In Zhang L, Zhang G, Liu P, Ma J, editors, CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals. American Society of Civil Engineers (ASCE). 2019. p. 3123-3135. (CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals). https://doi.org/10.1061/9780784482292.270
Yin, Biao ; Menendez, Monica. / A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows. CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals. editor / Lei Zhang ; Guangjun Zhang ; Pan Liu ; Jianming Ma. American Society of Civil Engineers (ASCE), 2019. pp. 3123-3135 (CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals).
@inproceedings{8d8fa3b027f0463f8b974785e96fac3a,
title = "A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows",
abstract = "In the paper, we propose a model-based reinforcement learning algorithm, i.e., approximate dynamic programming for signal control at isolated intersections with mixed traffic: vehicles and pedestrians. The integrated optimization problem is formulated by the discrete-time dynamic process. The system state is represented by the combination of weighted vehicle queue lengths and the number of waiting pedestrians. The system action is generated in each decision step by the proposed algorithm. To solve the computation issue in conventional dynamic programming, the proposed algorithm adopts a linear approximation function that helps to quickly obtain a near-optimal solution. In simulation, we extract traffic information from the traffic simulator SUMO. The on-line traffic information is provided for the algorithm to make a signal decision. After testing various scenarios, results show that the proposed algorithm has potential control performance. We also reveal the delay changes with different weights assigned to the vehicle and pedestrian components.",
keywords = "Approximate dynamic programming, Intersection, Pedestrians, Signal control",
author = "Biao Yin and Monica Menendez",
year = "2019",
month = "1",
day = "1",
doi = "10.1061/9780784482292.270",
language = "English (US)",
series = "CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals",
publisher = "American Society of Civil Engineers (ASCE)",
pages = "3123--3135",
editor = "Lei Zhang and Guangjun Zhang and Pan Liu and Jianming Ma",
booktitle = "CICTP 2019",

}

TY - GEN

T1 - A Reinforcement learning method for traffic signal control at an isolated intersection with pedestrian flows

AU - Yin, Biao

AU - Menendez, Monica

PY - 2019/1/1

Y1 - 2019/1/1

N2 - In the paper, we propose a model-based reinforcement learning algorithm, i.e., approximate dynamic programming for signal control at isolated intersections with mixed traffic: vehicles and pedestrians. The integrated optimization problem is formulated by the discrete-time dynamic process. The system state is represented by the combination of weighted vehicle queue lengths and the number of waiting pedestrians. The system action is generated in each decision step by the proposed algorithm. To solve the computation issue in conventional dynamic programming, the proposed algorithm adopts a linear approximation function that helps to quickly obtain a near-optimal solution. In simulation, we extract traffic information from the traffic simulator SUMO. The on-line traffic information is provided for the algorithm to make a signal decision. After testing various scenarios, results show that the proposed algorithm has potential control performance. We also reveal the delay changes with different weights assigned to the vehicle and pedestrian components.

AB - In the paper, we propose a model-based reinforcement learning algorithm, i.e., approximate dynamic programming for signal control at isolated intersections with mixed traffic: vehicles and pedestrians. The integrated optimization problem is formulated by the discrete-time dynamic process. The system state is represented by the combination of weighted vehicle queue lengths and the number of waiting pedestrians. The system action is generated in each decision step by the proposed algorithm. To solve the computation issue in conventional dynamic programming, the proposed algorithm adopts a linear approximation function that helps to quickly obtain a near-optimal solution. In simulation, we extract traffic information from the traffic simulator SUMO. The on-line traffic information is provided for the algorithm to make a signal decision. After testing various scenarios, results show that the proposed algorithm has potential control performance. We also reveal the delay changes with different weights assigned to the vehicle and pedestrian components.

KW - Approximate dynamic programming

KW - Intersection

KW - Pedestrians

KW - Signal control

UR - http://www.scopus.com/inward/record.url?scp=85070287954&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85070287954&partnerID=8YFLogxK

U2 - 10.1061/9780784482292.270

DO - 10.1061/9780784482292.270

M3 - Conference contribution

AN - SCOPUS:85070287954

T3 - CICTP 2019: Transportation in China - Connecting the World - Proceedings of the 19th COTA International Conference of Transportation Professionals

SP - 3123

EP - 3135

BT - CICTP 2019

A2 - Zhang, Lei

A2 - Zhang, Guangjun

A2 - Liu, Pan

A2 - Ma, Jianming

PB - American Society of Civil Engineers (ASCE)

ER -