Learning spatiotemporal features with 3D convolutional networks

Du Tran, Lubomir Bourdev, Robert Fergus, Lorenzo Torresani, Manohar Paluri

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets, 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets, and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8% accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.

Original languageEnglish (US)
Title of host publicationProceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages4489-4497
Number of pages9
Volume11-18-December-2015
ISBN (Electronic)9781467383912
DOIs
StatePublished - Feb 17 2016
Event15th IEEE International Conference on Computer Vision, ICCV 2015 - Santiago, Chile
Duration: Dec 11 2015Dec 18 2015

Other

Other15th IEEE International Conference on Computer Vision, ICCV 2015
CountryChile
CitySantiago
Period12/11/1512/18/15

Fingerprint

Convolution
Classifiers
Deep learning

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2016). Learning spatiotemporal features with 3D convolutional networks. In Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015 (Vol. 11-18-December-2015, pp. 4489-4497). [7410867] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2015.510

Learning spatiotemporal features with 3D convolutional networks. / Tran, Du; Bourdev, Lubomir; Fergus, Robert; Torresani, Lorenzo; Paluri, Manohar.

Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. Vol. 11-18-December-2015 Institute of Electrical and Electronics Engineers Inc., 2016. p. 4489-4497 7410867.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Tran, D, Bourdev, L, Fergus, R, Torresani, L & Paluri, M 2016, Learning spatiotemporal features with 3D convolutional networks. in Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. vol. 11-18-December-2015, 7410867, Institute of Electrical and Electronics Engineers Inc., pp. 4489-4497, 15th IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, 12/11/15. https://doi.org/10.1109/ICCV.2015.510
Tran D, Bourdev L, Fergus R, Torresani L, Paluri M. Learning spatiotemporal features with 3D convolutional networks. In Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. Vol. 11-18-December-2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 4489-4497. 7410867 https://doi.org/10.1109/ICCV.2015.510
Tran, Du ; Bourdev, Lubomir ; Fergus, Robert ; Torresani, Lorenzo ; Paluri, Manohar. / Learning spatiotemporal features with 3D convolutional networks. Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015. Vol. 11-18-December-2015 Institute of Electrical and Electronics Engineers Inc., 2016. pp. 4489-4497
@inproceedings{dd6ac3cf18454e6d97f480f126701ab7,
title = "Learning spatiotemporal features with 3D convolutional networks",
abstract = "We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets, 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets, and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8{\%} accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.",
author = "Du Tran and Lubomir Bourdev and Robert Fergus and Lorenzo Torresani and Manohar Paluri",
year = "2016",
month = "2",
day = "17",
doi = "10.1109/ICCV.2015.510",
language = "English (US)",
volume = "11-18-December-2015",
pages = "4489--4497",
booktitle = "Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

TY - GEN

T1 - Learning spatiotemporal features with 3D convolutional networks

AU - Tran, Du

AU - Bourdev, Lubomir

AU - Fergus, Robert

AU - Torresani, Lorenzo

AU - Paluri, Manohar

PY - 2016/2/17

Y1 - 2016/2/17

N2 - We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets, 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets, and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8% accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.

AB - We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets, 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets, and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8% accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.

UR - http://www.scopus.com/inward/record.url?scp=84973865953&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973865953&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2015.510

DO - 10.1109/ICCV.2015.510

M3 - Conference contribution

AN - SCOPUS:84973865953

VL - 11-18-December-2015

SP - 4489

EP - 4497

BT - Proceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -