Reference Policies for Non-myopic Sequential Network Design and Timing Problems

Joseph Ying Jun Chow, Hamid R. Sayarshad

Research output: Contribution to journalArticle

Abstract

Despite a growing number of studies in stochastic dynamic network optimization, the field remains less well defined and unified than other areas of network optimization. Due to the need for approximation methods like approximate dynamic programming, one of the most significant problems yet to be solved is the lack of adequate benchmarks. The values of the perfect information policy and static policy are not sensitive to information propagation while the myopic policy does not distinguish network effects in the value of flexibility. We propose a scalable reference policy value defined from theoretically consistent real option values based on sampled sequences, and estimate it using extreme value distributions. The reference policy is evaluated on an existing network instance with known sequences (Sioux Falls network from Chow and Regan 2011a): the Weibull distribution demonstrates good fit and sampling consistency with more than 200 samples. The reference policy is further applied in computational experiments with two other types of adaptive network design: a facility location and timing problem on the Simchi-Levi and Berman (1988) network, and Hyytiä et al.’s (2012) dynamic dial-a-ride problem. The former experiment represents an application of a new problem class and use of the reference policy as an upper bound for evaluating sampled policies, which can reach 3 % gap with 350 samples. The latter experiment demonstrates that sensitivity to parameters may be greater than expected, particularly when benchmarked against the proposed reference policy.

Original languageEnglish (US)
Pages (from-to)1-27
Number of pages27
JournalNetworks and Spatial Economics
DOIs
StateAccepted/In press - Dec 14 2015

Fingerprint

Weibull distribution
Experiments
Dynamic programming
Sampling

Keywords

  • Adapted stochastic process
  • Approximate dynamic programming
  • Dynamic dial-a-ride problem
  • Facility location problem
  • Markov decision process
  • Sequential network design problems

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Artificial Intelligence
  • Software

Cite this

Reference Policies for Non-myopic Sequential Network Design and Timing Problems. / Chow, Joseph Ying Jun; Sayarshad, Hamid R.

In: Networks and Spatial Economics, 14.12.2015, p. 1-27.

Research output: Contribution to journalArticle

@article{f0e0f9d442fc4889b0b96bcca99cfbdf,
title = "Reference Policies for Non-myopic Sequential Network Design and Timing Problems",
abstract = "Despite a growing number of studies in stochastic dynamic network optimization, the field remains less well defined and unified than other areas of network optimization. Due to the need for approximation methods like approximate dynamic programming, one of the most significant problems yet to be solved is the lack of adequate benchmarks. The values of the perfect information policy and static policy are not sensitive to information propagation while the myopic policy does not distinguish network effects in the value of flexibility. We propose a scalable reference policy value defined from theoretically consistent real option values based on sampled sequences, and estimate it using extreme value distributions. The reference policy is evaluated on an existing network instance with known sequences (Sioux Falls network from Chow and Regan 2011a): the Weibull distribution demonstrates good fit and sampling consistency with more than 200 samples. The reference policy is further applied in computational experiments with two other types of adaptive network design: a facility location and timing problem on the Simchi-Levi and Berman (1988) network, and Hyyti{\"a} et al.’s (2012) dynamic dial-a-ride problem. The former experiment represents an application of a new problem class and use of the reference policy as an upper bound for evaluating sampled policies, which can reach 3 {\%} gap with 350 samples. The latter experiment demonstrates that sensitivity to parameters may be greater than expected, particularly when benchmarked against the proposed reference policy.",
keywords = "Adapted stochastic process, Approximate dynamic programming, Dynamic dial-a-ride problem, Facility location problem, Markov decision process, Sequential network design problems",
author = "Chow, {Joseph Ying Jun} and Sayarshad, {Hamid R.}",
year = "2015",
month = "12",
day = "14",
doi = "10.1007/s11067-015-9315-5",
language = "English (US)",
pages = "1--27",
journal = "Networks and Spatial Economics",
issn = "1566-113X",
publisher = "Kluwer Academic Publishers",

}

TY - JOUR

T1 - Reference Policies for Non-myopic Sequential Network Design and Timing Problems

AU - Chow, Joseph Ying Jun

AU - Sayarshad, Hamid R.

PY - 2015/12/14

Y1 - 2015/12/14

N2 - Despite a growing number of studies in stochastic dynamic network optimization, the field remains less well defined and unified than other areas of network optimization. Due to the need for approximation methods like approximate dynamic programming, one of the most significant problems yet to be solved is the lack of adequate benchmarks. The values of the perfect information policy and static policy are not sensitive to information propagation while the myopic policy does not distinguish network effects in the value of flexibility. We propose a scalable reference policy value defined from theoretically consistent real option values based on sampled sequences, and estimate it using extreme value distributions. The reference policy is evaluated on an existing network instance with known sequences (Sioux Falls network from Chow and Regan 2011a): the Weibull distribution demonstrates good fit and sampling consistency with more than 200 samples. The reference policy is further applied in computational experiments with two other types of adaptive network design: a facility location and timing problem on the Simchi-Levi and Berman (1988) network, and Hyytiä et al.’s (2012) dynamic dial-a-ride problem. The former experiment represents an application of a new problem class and use of the reference policy as an upper bound for evaluating sampled policies, which can reach 3 % gap with 350 samples. The latter experiment demonstrates that sensitivity to parameters may be greater than expected, particularly when benchmarked against the proposed reference policy.

AB - Despite a growing number of studies in stochastic dynamic network optimization, the field remains less well defined and unified than other areas of network optimization. Due to the need for approximation methods like approximate dynamic programming, one of the most significant problems yet to be solved is the lack of adequate benchmarks. The values of the perfect information policy and static policy are not sensitive to information propagation while the myopic policy does not distinguish network effects in the value of flexibility. We propose a scalable reference policy value defined from theoretically consistent real option values based on sampled sequences, and estimate it using extreme value distributions. The reference policy is evaluated on an existing network instance with known sequences (Sioux Falls network from Chow and Regan 2011a): the Weibull distribution demonstrates good fit and sampling consistency with more than 200 samples. The reference policy is further applied in computational experiments with two other types of adaptive network design: a facility location and timing problem on the Simchi-Levi and Berman (1988) network, and Hyytiä et al.’s (2012) dynamic dial-a-ride problem. The former experiment represents an application of a new problem class and use of the reference policy as an upper bound for evaluating sampled policies, which can reach 3 % gap with 350 samples. The latter experiment demonstrates that sensitivity to parameters may be greater than expected, particularly when benchmarked against the proposed reference policy.

KW - Adapted stochastic process

KW - Approximate dynamic programming

KW - Dynamic dial-a-ride problem

KW - Facility location problem

KW - Markov decision process

KW - Sequential network design problems

UR - http://www.scopus.com/inward/record.url?scp=84949795442&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84949795442&partnerID=8YFLogxK

U2 - 10.1007/s11067-015-9315-5

DO - 10.1007/s11067-015-9315-5

M3 - Article

SP - 1

EP - 27

JO - Networks and Spatial Economics

JF - Networks and Spatial Economics

SN - 1566-113X

ER -