Every Shred Helps: Assembling Evidence from Orphaned JPEG Fragments

Emre Durmus, Pawel Korus, Nasir Memon

Research output: Contribution to journalArticle

Abstract

In this paper, we address the problem of forensic photo carving, which serves as one of the key sources of digital evidence in modern law enforcement. We propose efficient algorithms for assembling meaningful photographs from orphaned photofragments, carved without access to file headers, meta-data, or compression settings. The addressed problem raises a novel variant of a jigsaw puzzle with an unknown number of mixed images, missing pieces, and severe brightness and colorization artifacts. We construct an efficient compatibility metric for matching puzzle pieces and a corresponding image stitching procedure which allows us to mitigate these artifacts. To facilitate photo assembly, we perform a forensic analysis of the fragments to provide clues about their location within the frame of the imaging sensor. The proposed algorithm formulates the assembly problem as finding non-overlapping sets in an interval graph spanned over the input fragments. The algorithm exhibits lower computational complexity compared with a popular puzzle-solving approach based on minimal spanning trees.

Original languageEnglish (US)
Article number8636549
Pages (from-to)2372-2386
Number of pages15
JournalIEEE Transactions on Information Forensics and Security
Volume14
Issue number9
DOIs
StatePublished - Sep 1 2019

    Fingerprint

Keywords

  • Carving
  • data recovery
  • jigsaw puzzle
  • JPEG coding
  • orphaned fragment carving
  • photo carving
  • photo reconstruction

ASJC Scopus subject areas

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications

Cite this