Improved single-round protocols for remote file synchronization

Utku Irmak, Svilen Mihaylov, Torsten Suel

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    Given two versions of a file, a current version located on one machine and an outdated version known only to another machine, the remote file synchronization problem is how to update the outdated version over a network with a minimal amount of communication. In particular, when the versions are very similar, the total data transmitted should be significantly smaller than the file size. File synchronization problems arise in many application scenarios such as web site mirroring, file system backup and replication, and web access over slow links. An open source tool for this problem, called rsync and included in many Linux distributions, is widely used in such scenarios, rsync uses a single round of messages between the two machines. While recent research has shown that significant additional savings in bandwidth consumption are possible through the use of optimized multi-round protocols, there are many scenarios where multiple rounds are undesirable. In this paper, we study single-round protocols for file synchronization that offer significant improvements over rsync. Our main contribution is a new approach to file synchronization based on the use of erasure codes. Using this approach, we design a single-round protocol that is provably efficient with respect to common measures of file distance, and another optimized practical protocol that shows promising improvements over rsync on our data sets. In addition, we show how to obtain moderate improvements by engineering the rsync approach.

    Original languageEnglish (US)
    Title of host publicationProceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies
    EditorsK. Makki, E. Knightly
    Pages1665-1676
    Number of pages12
    Volume3
    DOIs
    StatePublished - 2005
    EventIEEE INFOCOM 2005 - Miami, FL, United States
    Duration: Mar 13 2005Mar 17 2005

    Other

    OtherIEEE INFOCOM 2005
    CountryUnited States
    CityMiami, FL
    Period3/13/053/17/05

    Fingerprint

    Synchronization
    Network protocols
    Websites
    Bandwidth
    Communication

    ASJC Scopus subject areas

    • Electrical and Electronic Engineering
    • Hardware and Architecture

    Cite this

    Irmak, U., Mihaylov, S., & Suel, T. (2005). Improved single-round protocols for remote file synchronization. In K. Makki, & E. Knightly (Eds.), Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies (Vol. 3, pp. 1665-1676) https://doi.org/10.1109/INFCOM.2005.1498448

    Improved single-round protocols for remote file synchronization. / Irmak, Utku; Mihaylov, Svilen; Suel, Torsten.

    Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies. ed. / K. Makki; E. Knightly. Vol. 3 2005. p. 1665-1676.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Irmak, U, Mihaylov, S & Suel, T 2005, Improved single-round protocols for remote file synchronization. in K Makki & E Knightly (eds), Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies. vol. 3, pp. 1665-1676, IEEE INFOCOM 2005, Miami, FL, United States, 3/13/05. https://doi.org/10.1109/INFCOM.2005.1498448
    Irmak U, Mihaylov S, Suel T. Improved single-round protocols for remote file synchronization. In Makki K, Knightly E, editors, Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies. Vol. 3. 2005. p. 1665-1676 https://doi.org/10.1109/INFCOM.2005.1498448
    Irmak, Utku ; Mihaylov, Svilen ; Suel, Torsten. / Improved single-round protocols for remote file synchronization. Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies. editor / K. Makki ; E. Knightly. Vol. 3 2005. pp. 1665-1676
    @inproceedings{db3f5937af684f15bea7df48f852ad29,
    title = "Improved single-round protocols for remote file synchronization",
    abstract = "Given two versions of a file, a current version located on one machine and an outdated version known only to another machine, the remote file synchronization problem is how to update the outdated version over a network with a minimal amount of communication. In particular, when the versions are very similar, the total data transmitted should be significantly smaller than the file size. File synchronization problems arise in many application scenarios such as web site mirroring, file system backup and replication, and web access over slow links. An open source tool for this problem, called rsync and included in many Linux distributions, is widely used in such scenarios, rsync uses a single round of messages between the two machines. While recent research has shown that significant additional savings in bandwidth consumption are possible through the use of optimized multi-round protocols, there are many scenarios where multiple rounds are undesirable. In this paper, we study single-round protocols for file synchronization that offer significant improvements over rsync. Our main contribution is a new approach to file synchronization based on the use of erasure codes. Using this approach, we design a single-round protocol that is provably efficient with respect to common measures of file distance, and another optimized practical protocol that shows promising improvements over rsync on our data sets. In addition, we show how to obtain moderate improvements by engineering the rsync approach.",
    author = "Utku Irmak and Svilen Mihaylov and Torsten Suel",
    year = "2005",
    doi = "10.1109/INFCOM.2005.1498448",
    language = "English (US)",
    isbn = "0780389689",
    volume = "3",
    pages = "1665--1676",
    editor = "K. Makki and E. Knightly",
    booktitle = "Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies",

    }

    TY - GEN

    T1 - Improved single-round protocols for remote file synchronization

    AU - Irmak, Utku

    AU - Mihaylov, Svilen

    AU - Suel, Torsten

    PY - 2005

    Y1 - 2005

    N2 - Given two versions of a file, a current version located on one machine and an outdated version known only to another machine, the remote file synchronization problem is how to update the outdated version over a network with a minimal amount of communication. In particular, when the versions are very similar, the total data transmitted should be significantly smaller than the file size. File synchronization problems arise in many application scenarios such as web site mirroring, file system backup and replication, and web access over slow links. An open source tool for this problem, called rsync and included in many Linux distributions, is widely used in such scenarios, rsync uses a single round of messages between the two machines. While recent research has shown that significant additional savings in bandwidth consumption are possible through the use of optimized multi-round protocols, there are many scenarios where multiple rounds are undesirable. In this paper, we study single-round protocols for file synchronization that offer significant improvements over rsync. Our main contribution is a new approach to file synchronization based on the use of erasure codes. Using this approach, we design a single-round protocol that is provably efficient with respect to common measures of file distance, and another optimized practical protocol that shows promising improvements over rsync on our data sets. In addition, we show how to obtain moderate improvements by engineering the rsync approach.

    AB - Given two versions of a file, a current version located on one machine and an outdated version known only to another machine, the remote file synchronization problem is how to update the outdated version over a network with a minimal amount of communication. In particular, when the versions are very similar, the total data transmitted should be significantly smaller than the file size. File synchronization problems arise in many application scenarios such as web site mirroring, file system backup and replication, and web access over slow links. An open source tool for this problem, called rsync and included in many Linux distributions, is widely used in such scenarios, rsync uses a single round of messages between the two machines. While recent research has shown that significant additional savings in bandwidth consumption are possible through the use of optimized multi-round protocols, there are many scenarios where multiple rounds are undesirable. In this paper, we study single-round protocols for file synchronization that offer significant improvements over rsync. Our main contribution is a new approach to file synchronization based on the use of erasure codes. Using this approach, we design a single-round protocol that is provably efficient with respect to common measures of file distance, and another optimized practical protocol that shows promising improvements over rsync on our data sets. In addition, we show how to obtain moderate improvements by engineering the rsync approach.

    UR - http://www.scopus.com/inward/record.url?scp=25844459542&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=25844459542&partnerID=8YFLogxK

    U2 - 10.1109/INFCOM.2005.1498448

    DO - 10.1109/INFCOM.2005.1498448

    M3 - Conference contribution

    SN - 0780389689

    VL - 3

    SP - 1665

    EP - 1676

    BT - Proceedings - IEEE INFOCOM 2005. The Conference on Computer Communications - 24th Annual Joint Conference of the IEEE Computer and Communications Societies

    A2 - Makki, K.

    A2 - Knightly, E.

    ER -