Towards the automatic classification of avian flight calls for bioacoustic monitoring

Justin Salamon, Juan Pablo Bello, Andrew Farnsworth, Matt Robbins, Sara Keen, Holger Klinck, Steve Kelling

Research output: Contribution to journalArticle

Abstract

Automatic classification of animal vocalizations has great potential to enhance the monitoring of species movements and behaviors. This is particularly true for monitoring nocturnal bird migration, where automated classification of migrants' flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we investigate the automatic classification of bird species from flight calls, and in particular the relationship between two different problem formulations commonly found in the literature: classifying a short clip containing one of a fixed set of known species (N-class problem) and the continuous monitoring problem, the latter of which is relevant to migration monitoring. We implemented a state-of-the-art audio classification model based on unsupervised feature learning and evaluated it on three novel datasets, one for studying the N-class problem including over 5000 flight calls from 43 different species, and two realistic datasets for studying the monitoring scenario comprising hundreds of thousands of audio clips that were compiled by means of remote acoustic sensors deployed in the field during two migration seasons. We show that the model achieves high accuracy when classifying a clip to one of N known species, even for a large number of species. In contrast, the model does not perform as well in the continuous monitoring case. Through a detailed error analysis (that included full expert review of false positives and negatives) we show the model is confounded by varying background noise conditions and previously unseen vocalizations. We also show that the model needs to be parameterized and benchmarked differently for the continuous monitoring scenario. Finally, we show that despite the reduced performance, given the right conditions the model can still characterize the migration pattern of a specific species. The paper concludes with directions for future research.

Original languageEnglish (US)
Article numbere0166866
JournalPLoS One
Volume11
Issue number11
DOIs
StatePublished - Nov 1 2016

Fingerprint

Bioacoustics
bioacoustics
flight
Surgical Instruments
Birds
taxonomy
Monitoring
monitoring
Animal Vocalization
vocalization
Acoustics
birds
Noise
Research Design
Learning
Error analysis
acoustics
Conservation
Animals
learning

ASJC Scopus subject areas

  • Medicine(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Agricultural and Biological Sciences(all)

Cite this

Salamon, J., Bello, J. P., Farnsworth, A., Robbins, M., Keen, S., Klinck, H., & Kelling, S. (2016). Towards the automatic classification of avian flight calls for bioacoustic monitoring. PLoS One, 11(11), [e0166866]. https://doi.org/10.1371/journal.pone.0166866

Towards the automatic classification of avian flight calls for bioacoustic monitoring. / Salamon, Justin; Bello, Juan Pablo; Farnsworth, Andrew; Robbins, Matt; Keen, Sara; Klinck, Holger; Kelling, Steve.

In: PLoS One, Vol. 11, No. 11, e0166866, 01.11.2016.

Research output: Contribution to journalArticle

Salamon, J, Bello, JP, Farnsworth, A, Robbins, M, Keen, S, Klinck, H & Kelling, S 2016, 'Towards the automatic classification of avian flight calls for bioacoustic monitoring', PLoS One, vol. 11, no. 11, e0166866. https://doi.org/10.1371/journal.pone.0166866
Salamon, Justin ; Bello, Juan Pablo ; Farnsworth, Andrew ; Robbins, Matt ; Keen, Sara ; Klinck, Holger ; Kelling, Steve. / Towards the automatic classification of avian flight calls for bioacoustic monitoring. In: PLoS One. 2016 ; Vol. 11, No. 11.
@article{fd81e9e2181c424d8e573e859e2b0016,
title = "Towards the automatic classification of avian flight calls for bioacoustic monitoring",
abstract = "Automatic classification of animal vocalizations has great potential to enhance the monitoring of species movements and behaviors. This is particularly true for monitoring nocturnal bird migration, where automated classification of migrants' flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we investigate the automatic classification of bird species from flight calls, and in particular the relationship between two different problem formulations commonly found in the literature: classifying a short clip containing one of a fixed set of known species (N-class problem) and the continuous monitoring problem, the latter of which is relevant to migration monitoring. We implemented a state-of-the-art audio classification model based on unsupervised feature learning and evaluated it on three novel datasets, one for studying the N-class problem including over 5000 flight calls from 43 different species, and two realistic datasets for studying the monitoring scenario comprising hundreds of thousands of audio clips that were compiled by means of remote acoustic sensors deployed in the field during two migration seasons. We show that the model achieves high accuracy when classifying a clip to one of N known species, even for a large number of species. In contrast, the model does not perform as well in the continuous monitoring case. Through a detailed error analysis (that included full expert review of false positives and negatives) we show the model is confounded by varying background noise conditions and previously unseen vocalizations. We also show that the model needs to be parameterized and benchmarked differently for the continuous monitoring scenario. Finally, we show that despite the reduced performance, given the right conditions the model can still characterize the migration pattern of a specific species. The paper concludes with directions for future research.",
author = "Justin Salamon and Bello, {Juan Pablo} and Andrew Farnsworth and Matt Robbins and Sara Keen and Holger Klinck and Steve Kelling",
year = "2016",
month = "11",
day = "1",
doi = "10.1371/journal.pone.0166866",
language = "English (US)",
volume = "11",
journal = "PLoS One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "11",

}

TY - JOUR

T1 - Towards the automatic classification of avian flight calls for bioacoustic monitoring

AU - Salamon, Justin

AU - Bello, Juan Pablo

AU - Farnsworth, Andrew

AU - Robbins, Matt

AU - Keen, Sara

AU - Klinck, Holger

AU - Kelling, Steve

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Automatic classification of animal vocalizations has great potential to enhance the monitoring of species movements and behaviors. This is particularly true for monitoring nocturnal bird migration, where automated classification of migrants' flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we investigate the automatic classification of bird species from flight calls, and in particular the relationship between two different problem formulations commonly found in the literature: classifying a short clip containing one of a fixed set of known species (N-class problem) and the continuous monitoring problem, the latter of which is relevant to migration monitoring. We implemented a state-of-the-art audio classification model based on unsupervised feature learning and evaluated it on three novel datasets, one for studying the N-class problem including over 5000 flight calls from 43 different species, and two realistic datasets for studying the monitoring scenario comprising hundreds of thousands of audio clips that were compiled by means of remote acoustic sensors deployed in the field during two migration seasons. We show that the model achieves high accuracy when classifying a clip to one of N known species, even for a large number of species. In contrast, the model does not perform as well in the continuous monitoring case. Through a detailed error analysis (that included full expert review of false positives and negatives) we show the model is confounded by varying background noise conditions and previously unseen vocalizations. We also show that the model needs to be parameterized and benchmarked differently for the continuous monitoring scenario. Finally, we show that despite the reduced performance, given the right conditions the model can still characterize the migration pattern of a specific species. The paper concludes with directions for future research.

AB - Automatic classification of animal vocalizations has great potential to enhance the monitoring of species movements and behaviors. This is particularly true for monitoring nocturnal bird migration, where automated classification of migrants' flight calls could yield new biological insights and conservation applications for birds that vocalize during migration. In this paper we investigate the automatic classification of bird species from flight calls, and in particular the relationship between two different problem formulations commonly found in the literature: classifying a short clip containing one of a fixed set of known species (N-class problem) and the continuous monitoring problem, the latter of which is relevant to migration monitoring. We implemented a state-of-the-art audio classification model based on unsupervised feature learning and evaluated it on three novel datasets, one for studying the N-class problem including over 5000 flight calls from 43 different species, and two realistic datasets for studying the monitoring scenario comprising hundreds of thousands of audio clips that were compiled by means of remote acoustic sensors deployed in the field during two migration seasons. We show that the model achieves high accuracy when classifying a clip to one of N known species, even for a large number of species. In contrast, the model does not perform as well in the continuous monitoring case. Through a detailed error analysis (that included full expert review of false positives and negatives) we show the model is confounded by varying background noise conditions and previously unseen vocalizations. We also show that the model needs to be parameterized and benchmarked differently for the continuous monitoring scenario. Finally, we show that despite the reduced performance, given the right conditions the model can still characterize the migration pattern of a specific species. The paper concludes with directions for future research.

UR - http://www.scopus.com/inward/record.url?scp=84996572317&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84996572317&partnerID=8YFLogxK

U2 - 10.1371/journal.pone.0166866

DO - 10.1371/journal.pone.0166866

M3 - Article

C2 - 27880836

AN - SCOPUS:84996572317

VL - 11

JO - PLoS One

JF - PLoS One

SN - 1932-6203

IS - 11

M1 - e0166866

ER -