Highly efficient asynchronous execution of large-grained parallel programs

Y. Aumann, Zvi Kedem, K. V. Palem, M. O. Rabin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

An n-thread parallel program P is large-grained if in every parallel step the computations on each of the threads are complex procedures requiring numerous processor instructions. This practically relevant style of programs differs from PRAM programs in its large granularity and the possibility that within a parallel step the computations on different threads may considerably vary in size. Let M be an n-processor asynchronous parallel system, with no restriction on the degree of asynchrony and without any specialized synchronization mechanisms. It is a challenging theoretical as well as practically important problem to ensure correct execution of P on such a parallel machine. Let P be a large-grained program requiring total work W for its execution on a synchronous n-processor parallel system. We present a transformation (compilation) of P into a program C(P) which correctly and efficiently effects the computation of P on the asynchronous machine M. Under moderate assumptions on the granularity of threads and the size of the program variables, execution of C(P) requires just O(W log* n) expected total work, and the memory space overhead is a small multiplicative constant. This result is the first of its kind. The solution involves a number of new concepts and methods. These include methods for storing program and control variables, employing a combination of error correction codes with phase-dependent hashing into memory. We feel that these methods for storing data will have significant practical applications to storage of data on Disk Arrays (RAIDS), as well as additional theoretical implications. The significance of the present work to parallel data-processing programs and large scale parallel numerical computations is obvious.

Original languageEnglish (US)
Title of host publicationAnnual Symposium on Foundatons of Computer Science (Proceedings)
Editors Anon
PublisherPubl by IEEE
Pages271-280
Number of pages10
ISBN (Print)0818643706
StatePublished - 1993
EventProceedings of the 34th Annual Symposium on Foundations of Computer Science - Palo Alto, CA, USA
Duration: Nov 3 1993Nov 5 1993

Other

OtherProceedings of the 34th Annual Symposium on Foundations of Computer Science
CityPalo Alto, CA, USA
Period11/3/9311/5/93

Fingerprint

Data storage equipment
Error correction
Synchronization

ASJC Scopus subject areas

  • Hardware and Architecture

Cite this

Aumann, Y., Kedem, Z., Palem, K. V., & Rabin, M. O. (1993). Highly efficient asynchronous execution of large-grained parallel programs. In Anon (Ed.), Annual Symposium on Foundatons of Computer Science (Proceedings) (pp. 271-280). Publ by IEEE.

Highly efficient asynchronous execution of large-grained parallel programs. / Aumann, Y.; Kedem, Zvi; Palem, K. V.; Rabin, M. O.

Annual Symposium on Foundatons of Computer Science (Proceedings). ed. / Anon. Publ by IEEE, 1993. p. 271-280.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Aumann, Y, Kedem, Z, Palem, KV & Rabin, MO 1993, Highly efficient asynchronous execution of large-grained parallel programs. in Anon (ed.), Annual Symposium on Foundatons of Computer Science (Proceedings). Publ by IEEE, pp. 271-280, Proceedings of the 34th Annual Symposium on Foundations of Computer Science, Palo Alto, CA, USA, 11/3/93.
Aumann Y, Kedem Z, Palem KV, Rabin MO. Highly efficient asynchronous execution of large-grained parallel programs. In Anon, editor, Annual Symposium on Foundatons of Computer Science (Proceedings). Publ by IEEE. 1993. p. 271-280
Aumann, Y. ; Kedem, Zvi ; Palem, K. V. ; Rabin, M. O. / Highly efficient asynchronous execution of large-grained parallel programs. Annual Symposium on Foundatons of Computer Science (Proceedings). editor / Anon. Publ by IEEE, 1993. pp. 271-280
@inproceedings{6442e340c3c84f49aea2ccf7f52afd77,
title = "Highly efficient asynchronous execution of large-grained parallel programs",
abstract = "An n-thread parallel program P is large-grained if in every parallel step the computations on each of the threads are complex procedures requiring numerous processor instructions. This practically relevant style of programs differs from PRAM programs in its large granularity and the possibility that within a parallel step the computations on different threads may considerably vary in size. Let M be an n-processor asynchronous parallel system, with no restriction on the degree of asynchrony and without any specialized synchronization mechanisms. It is a challenging theoretical as well as practically important problem to ensure correct execution of P on such a parallel machine. Let P be a large-grained program requiring total work W for its execution on a synchronous n-processor parallel system. We present a transformation (compilation) of P into a program C(P) which correctly and efficiently effects the computation of P on the asynchronous machine M. Under moderate assumptions on the granularity of threads and the size of the program variables, execution of C(P) requires just O(W log* n) expected total work, and the memory space overhead is a small multiplicative constant. This result is the first of its kind. The solution involves a number of new concepts and methods. These include methods for storing program and control variables, employing a combination of error correction codes with phase-dependent hashing into memory. We feel that these methods for storing data will have significant practical applications to storage of data on Disk Arrays (RAIDS), as well as additional theoretical implications. The significance of the present work to parallel data-processing programs and large scale parallel numerical computations is obvious.",
author = "Y. Aumann and Zvi Kedem and Palem, {K. V.} and Rabin, {M. O.}",
year = "1993",
language = "English (US)",
isbn = "0818643706",
pages = "271--280",
editor = "Anon",
booktitle = "Annual Symposium on Foundatons of Computer Science (Proceedings)",
publisher = "Publ by IEEE",

}

TY - GEN

T1 - Highly efficient asynchronous execution of large-grained parallel programs

AU - Aumann, Y.

AU - Kedem, Zvi

AU - Palem, K. V.

AU - Rabin, M. O.

PY - 1993

Y1 - 1993

N2 - An n-thread parallel program P is large-grained if in every parallel step the computations on each of the threads are complex procedures requiring numerous processor instructions. This practically relevant style of programs differs from PRAM programs in its large granularity and the possibility that within a parallel step the computations on different threads may considerably vary in size. Let M be an n-processor asynchronous parallel system, with no restriction on the degree of asynchrony and without any specialized synchronization mechanisms. It is a challenging theoretical as well as practically important problem to ensure correct execution of P on such a parallel machine. Let P be a large-grained program requiring total work W for its execution on a synchronous n-processor parallel system. We present a transformation (compilation) of P into a program C(P) which correctly and efficiently effects the computation of P on the asynchronous machine M. Under moderate assumptions on the granularity of threads and the size of the program variables, execution of C(P) requires just O(W log* n) expected total work, and the memory space overhead is a small multiplicative constant. This result is the first of its kind. The solution involves a number of new concepts and methods. These include methods for storing program and control variables, employing a combination of error correction codes with phase-dependent hashing into memory. We feel that these methods for storing data will have significant practical applications to storage of data on Disk Arrays (RAIDS), as well as additional theoretical implications. The significance of the present work to parallel data-processing programs and large scale parallel numerical computations is obvious.

AB - An n-thread parallel program P is large-grained if in every parallel step the computations on each of the threads are complex procedures requiring numerous processor instructions. This practically relevant style of programs differs from PRAM programs in its large granularity and the possibility that within a parallel step the computations on different threads may considerably vary in size. Let M be an n-processor asynchronous parallel system, with no restriction on the degree of asynchrony and without any specialized synchronization mechanisms. It is a challenging theoretical as well as practically important problem to ensure correct execution of P on such a parallel machine. Let P be a large-grained program requiring total work W for its execution on a synchronous n-processor parallel system. We present a transformation (compilation) of P into a program C(P) which correctly and efficiently effects the computation of P on the asynchronous machine M. Under moderate assumptions on the granularity of threads and the size of the program variables, execution of C(P) requires just O(W log* n) expected total work, and the memory space overhead is a small multiplicative constant. This result is the first of its kind. The solution involves a number of new concepts and methods. These include methods for storing program and control variables, employing a combination of error correction codes with phase-dependent hashing into memory. We feel that these methods for storing data will have significant practical applications to storage of data on Disk Arrays (RAIDS), as well as additional theoretical implications. The significance of the present work to parallel data-processing programs and large scale parallel numerical computations is obvious.

UR - http://www.scopus.com/inward/record.url?scp=0027803207&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0027803207&partnerID=8YFLogxK

M3 - Conference contribution

SN - 0818643706

SP - 271

EP - 280

BT - Annual Symposium on Foundatons of Computer Science (Proceedings)

A2 - Anon, null

PB - Publ by IEEE

ER -