Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing

Clément Farabet, Rafael Paz, Jose Pérez-Carrasco, Carlos Zamarreño-Ramos, Alejandro Linares-Barranco, Yann LeCun, Eugenio Culurciello, Teresa Serrano-Gotarredona, Bernabe Linares-Barranco

Research output: Contribution to journalArticle

Abstract

Most scene segmentation and categorization architectures for the extraction of features in images and patches make exhaustive use of 2D convolution operations for template matching, template search, and denoising. Convolutional Neural Networks (ConvNets) are one example of such architectures that can implement general-purpose bio-inspired vision systems. In standard digital computers 2D convolutions are usually expensive in terms of resource consumption and impose severe limitations for efficient real-time applications. Nevertheless, neuro-cortex inspired solutions, like dedicated Frame-Based or Frame-Free Spiking ConvNet Convolution Processors, are advancing real-time visual processing. These two approaches share the neural inspiration, but each of them solves the problem in different ways. Frame-Based ConvNets process frame by frame video information in a very robust and fast way that requires to use and share the available hardware resources (such as: multipliers, adders). Hardware resources are fixed- and time-multiplexed by fetching data in and out. Thus memory bandwidth and size is important for good performance. On the other hand, spike-based convolution processors are a frame-free alternative that is able to perform convolution of a spike-based source of visual information with very low latency, which makes ideal for very high-speed applications. However, hardware resources need to be available all the time and cannot be time-multiplexed. Thus, hardware should be modular, reconfigurable, and expansible. Hardware implementations in both VLSI custom integrated circuits (digital and analog) and FPGA have been already used to demonstrate the performance of these systems. In this paper we present a comparison study of these two neuro-inspired solutions. A brief description of both systems is presented and also discussions about their differences, pros and cons.

Original languageEnglish (US)
Article numberArticle 32
JournalFrontiers in Neuroscience
Issue numberAPR
DOIs
StatePublished - 2012

Keywords

  • Address-event-representation
  • Convolutional neural network
  • Fpga
  • Frame-free vision
  • Image convolutions
  • Spike-based convolutions
  • Vhdl

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Farabet, C., Paz, R., Pérez-Carrasco, J., Zamarreño-Ramos, C., Linares-Barranco, A., LeCun, Y., ... Linares-Barranco, B. (2012). Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing. Frontiers in Neuroscience, (APR), [Article 32]. https://doi.org/10.3389/fnins.2012.00032

Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing. / Farabet, Clément; Paz, Rafael; Pérez-Carrasco, Jose; Zamarreño-Ramos, Carlos; Linares-Barranco, Alejandro; LeCun, Yann; Culurciello, Eugenio; Serrano-Gotarredona, Teresa; Linares-Barranco, Bernabe.

In: Frontiers in Neuroscience, No. APR, Article 32, 2012.

Research output: Contribution to journalArticle

Farabet, C, Paz, R, Pérez-Carrasco, J, Zamarreño-Ramos, C, Linares-Barranco, A, LeCun, Y, Culurciello, E, Serrano-Gotarredona, T & Linares-Barranco, B 2012, 'Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing', Frontiers in Neuroscience, no. APR, Article 32. https://doi.org/10.3389/fnins.2012.00032
Farabet, Clément ; Paz, Rafael ; Pérez-Carrasco, Jose ; Zamarreño-Ramos, Carlos ; Linares-Barranco, Alejandro ; LeCun, Yann ; Culurciello, Eugenio ; Serrano-Gotarredona, Teresa ; Linares-Barranco, Bernabe. / Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing. In: Frontiers in Neuroscience. 2012 ; No. APR.
@article{c347a13228a840bfba3663c41a52eb01,
title = "Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing",
abstract = "Most scene segmentation and categorization architectures for the extraction of features in images and patches make exhaustive use of 2D convolution operations for template matching, template search, and denoising. Convolutional Neural Networks (ConvNets) are one example of such architectures that can implement general-purpose bio-inspired vision systems. In standard digital computers 2D convolutions are usually expensive in terms of resource consumption and impose severe limitations for efficient real-time applications. Nevertheless, neuro-cortex inspired solutions, like dedicated Frame-Based or Frame-Free Spiking ConvNet Convolution Processors, are advancing real-time visual processing. These two approaches share the neural inspiration, but each of them solves the problem in different ways. Frame-Based ConvNets process frame by frame video information in a very robust and fast way that requires to use and share the available hardware resources (such as: multipliers, adders). Hardware resources are fixed- and time-multiplexed by fetching data in and out. Thus memory bandwidth and size is important for good performance. On the other hand, spike-based convolution processors are a frame-free alternative that is able to perform convolution of a spike-based source of visual information with very low latency, which makes ideal for very high-speed applications. However, hardware resources need to be available all the time and cannot be time-multiplexed. Thus, hardware should be modular, reconfigurable, and expansible. Hardware implementations in both VLSI custom integrated circuits (digital and analog) and FPGA have been already used to demonstrate the performance of these systems. In this paper we present a comparison study of these two neuro-inspired solutions. A brief description of both systems is presented and also discussions about their differences, pros and cons.",
keywords = "Address-event-representation, Convolutional neural network, Fpga, Frame-free vision, Image convolutions, Spike-based convolutions, Vhdl",
author = "Cl{\'e}ment Farabet and Rafael Paz and Jose P{\'e}rez-Carrasco and Carlos Zamarre{\~n}o-Ramos and Alejandro Linares-Barranco and Yann LeCun and Eugenio Culurciello and Teresa Serrano-Gotarredona and Bernabe Linares-Barranco",
year = "2012",
doi = "10.3389/fnins.2012.00032",
language = "English (US)",
journal = "Frontiers in Neuroscience",
issn = "1662-4548",
publisher = "Frontiers Research Foundation",
number = "APR",

}

TY - JOUR

T1 - Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing

AU - Farabet, Clément

AU - Paz, Rafael

AU - Pérez-Carrasco, Jose

AU - Zamarreño-Ramos, Carlos

AU - Linares-Barranco, Alejandro

AU - LeCun, Yann

AU - Culurciello, Eugenio

AU - Serrano-Gotarredona, Teresa

AU - Linares-Barranco, Bernabe

PY - 2012

Y1 - 2012

N2 - Most scene segmentation and categorization architectures for the extraction of features in images and patches make exhaustive use of 2D convolution operations for template matching, template search, and denoising. Convolutional Neural Networks (ConvNets) are one example of such architectures that can implement general-purpose bio-inspired vision systems. In standard digital computers 2D convolutions are usually expensive in terms of resource consumption and impose severe limitations for efficient real-time applications. Nevertheless, neuro-cortex inspired solutions, like dedicated Frame-Based or Frame-Free Spiking ConvNet Convolution Processors, are advancing real-time visual processing. These two approaches share the neural inspiration, but each of them solves the problem in different ways. Frame-Based ConvNets process frame by frame video information in a very robust and fast way that requires to use and share the available hardware resources (such as: multipliers, adders). Hardware resources are fixed- and time-multiplexed by fetching data in and out. Thus memory bandwidth and size is important for good performance. On the other hand, spike-based convolution processors are a frame-free alternative that is able to perform convolution of a spike-based source of visual information with very low latency, which makes ideal for very high-speed applications. However, hardware resources need to be available all the time and cannot be time-multiplexed. Thus, hardware should be modular, reconfigurable, and expansible. Hardware implementations in both VLSI custom integrated circuits (digital and analog) and FPGA have been already used to demonstrate the performance of these systems. In this paper we present a comparison study of these two neuro-inspired solutions. A brief description of both systems is presented and also discussions about their differences, pros and cons.

AB - Most scene segmentation and categorization architectures for the extraction of features in images and patches make exhaustive use of 2D convolution operations for template matching, template search, and denoising. Convolutional Neural Networks (ConvNets) are one example of such architectures that can implement general-purpose bio-inspired vision systems. In standard digital computers 2D convolutions are usually expensive in terms of resource consumption and impose severe limitations for efficient real-time applications. Nevertheless, neuro-cortex inspired solutions, like dedicated Frame-Based or Frame-Free Spiking ConvNet Convolution Processors, are advancing real-time visual processing. These two approaches share the neural inspiration, but each of them solves the problem in different ways. Frame-Based ConvNets process frame by frame video information in a very robust and fast way that requires to use and share the available hardware resources (such as: multipliers, adders). Hardware resources are fixed- and time-multiplexed by fetching data in and out. Thus memory bandwidth and size is important for good performance. On the other hand, spike-based convolution processors are a frame-free alternative that is able to perform convolution of a spike-based source of visual information with very low latency, which makes ideal for very high-speed applications. However, hardware resources need to be available all the time and cannot be time-multiplexed. Thus, hardware should be modular, reconfigurable, and expansible. Hardware implementations in both VLSI custom integrated circuits (digital and analog) and FPGA have been already used to demonstrate the performance of these systems. In this paper we present a comparison study of these two neuro-inspired solutions. A brief description of both systems is presented and also discussions about their differences, pros and cons.

KW - Address-event-representation

KW - Convolutional neural network

KW - Fpga

KW - Frame-free vision

KW - Image convolutions

KW - Spike-based convolutions

KW - Vhdl

UR - http://www.scopus.com/inward/record.url?scp=84862236095&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84862236095&partnerID=8YFLogxK

U2 - 10.3389/fnins.2012.00032

DO - 10.3389/fnins.2012.00032

M3 - Article

JO - Frontiers in Neuroscience

JF - Frontiers in Neuroscience

SN - 1662-4548

IS - APR

M1 - Article 32

ER -