Every Shred Helps: Assembling Evidence from Orphaned JPEG Fragments

Emre Durmus, Pawel Korus, Nasir Memon

Research output: Contribution to journalArticle

Abstract

In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

Original languageEnglish (US)
Article number8636549
Pages (from-to)2372-2386
Number of pages15
JournalIEEE Transactions on Information Forensics and Security
Volume14
Issue number9
DOIs
StatePublished - Sep 1 2019

Fingerprint

Law enforcement
Metadata
Luminance
Computational complexity
Imaging techniques
Sensors

Keywords

  • Carving
  • data recovery
  • jigsaw puzzle
  • JPEG coding
  • orphaned fragment carving
  • photo carving
  • photo reconstruction

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

Cite this

Every Shred Helps : Assembling Evidence from Orphaned JPEG Fragments. / Durmus, Emre; Korus, Pawel; Memon, Nasir.

In: IEEE Transactions on Information Forensics and Security, Vol. 14, No. 9, 8636549, 01.09.2019, p. 2372-2386.

Research output: Contribution to journalArticle

@article{a6d365de22434bbc881a1abcc2cf485d,
title = "Every Shred Helps: Assembling Evidence from Orphaned JPEG Fragments",
abstract = "In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.",
keywords = "Carving, data recovery, jigsaw puzzle, JPEG coding, orphaned fragment carving, photo carving, photo reconstruction",
author = "Emre Durmus and Pawel Korus and Nasir Memon",
year = "2019",
month = "9",
day = "1",
doi = "10.1109/TIFS.2019.2897912",
language = "English (US)",
volume = "14",
pages = "2372--2386",
journal = "IEEE Transactions on Information Forensics and Security",
issn = "1556-6013",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "9",

}

TY - JOUR

T1 - Every Shred Helps

T2 - Assembling Evidence from Orphaned JPEG Fragments

AU - Durmus, Emre

AU - Korus, Pawel

AU - Memon, Nasir

PY - 2019/9/1

Y1 - 2019/9/1

N2 - In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

AB - In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

KW - Carving

KW - data recovery

KW - jigsaw puzzle

KW - JPEG coding

KW - orphaned fragment carving

KW - photo carving

KW - photo reconstruction

UR - http://www.scopus.com/inward/record.url?scp=85066619988&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066619988&partnerID=8YFLogxK

U2 - 10.1109/TIFS.2019.2897912

DO - 10.1109/TIFS.2019.2897912

M3 - Article

AN - SCOPUS:85066619988

VL - 14

SP - 2372

EP - 2386

JO - IEEE Transactions on Information Forensics and Security

JF - IEEE Transactions on Information Forensics and Security

SN - 1556-6013

IS - 9

M1 - 8636549

ER -