Every Shred Helps

Assembling Evidence from Orphaned JPEG Fragments

Emre Durmus, Pawel Korus, Nasir Memon

    Research output: Contribution to journalArticle

    Abstract

    In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

    Original languageEnglish (US)
    Article number8636549
    Pages (from-to)2372-2386
    Number of pages15
    JournalIEEE Transactions on Information Forensics and Security
    Volume14
    Issue number9
    DOIs
    StatePublished - Sep 1 2019

    Fingerprint

    Law enforcement
    Metadata
    Luminance
    Computational complexity
    Imaging techniques
    Sensors

    Keywords

    • Carving
    • data recovery
    • jigsaw puzzle
    • JPEG coding
    • orphaned fragment carving
    • photo carving
    • photo reconstruction

    ASJC Scopus subject areas

    • Safety, Risk, Reliability and Quality
    • Computer Networks and Communications

    Cite this

    Every Shred Helps : Assembling Evidence from Orphaned JPEG Fragments. / Durmus, Emre; Korus, Pawel; Memon, Nasir.

    In: IEEE Transactions on Information Forensics and Security, Vol. 14, No. 9, 8636549, 01.09.2019, p. 2372-2386.

    Research output: Contribution to journalArticle

    Durmus, Emre ; Korus, Pawel ; Memon, Nasir. / Every Shred Helps : Assembling Evidence from Orphaned JPEG Fragments. In: IEEE Transactions on Information Forensics and Security. 2019 ; Vol. 14, No. 9. pp. 2372-2386.
    @article{5d661aac420e425285dd9cebdcc30875,
    title = "Every Shred Helps: Assembling Evidence from Orphaned JPEG Fragments",
    abstract = "In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.",
    keywords = "Carving, data recovery, jigsaw puzzle, JPEG coding, orphaned fragment carving, photo carving, photo reconstruction",
    author = "Emre Durmus and Pawel Korus and Nasir Memon",
    year = "2019",
    month = "9",
    day = "1",
    doi = "10.1109/TIFS.2019.2897912",
    language = "English (US)",
    volume = "14",
    pages = "2372--2386",
    journal = "IEEE Transactions on Information Forensics and Security",
    issn = "1556-6013",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",
    number = "9",

    }

    TY - JOUR

    T1 - Every Shred Helps

    T2 - Assembling Evidence from Orphaned JPEG Fragments

    AU - Durmus, Emre

    AU - Korus, Pawel

    AU - Memon, Nasir

    PY - 2019/9/1

    Y1 - 2019/9/1

    N2 - In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

    AB - In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

    KW - Carving

    KW - data recovery

    KW - jigsaw puzzle

    KW - JPEG coding

    KW - orphaned fragment carving

    KW - photo carving

    KW - photo reconstruction

    UR - http://www.scopus.com/inward/record.url?scp=85066619988&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85066619988&partnerID=8YFLogxK

    U2 - 10.1109/TIFS.2019.2897912

    DO - 10.1109/TIFS.2019.2897912

    M3 - Article

    VL - 14

    SP - 2372

    EP - 2386

    JO - IEEE Transactions on Information Forensics and Security

    JF - IEEE Transactions on Information Forensics and Security

    SN - 1556-6013

    IS - 9

    M1 - 8636549

    ER -