Approximate search of audio queries by using DTW with phone time boundary and data augmentation

Haikua Xu; Jingyong Hou; Xiong Xiao; Van Tung Pham; Cheung Chi Leung; Lei Wang; Van Hai Do; Hang Lv; Lei Xie; Bin Ma; Eng Siong Chng; Haizhou Li

doi:10.1109/ICASSP.2016.7472835

Approximate search of audio queries by using DTW with phone time boundary and data augmentation

Haikua Xu, Jingyong Hou, Xiong Xiao, Van Tung Pham, Cheung Chi Leung, Lei Wang, Van Hai Do, Hang Lv, Lei Xie, Bin Ma, Eng Siong Chng, Haizhou Li

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

15 Scopus citations

Abstract

Dynamic Time Warping (DTW) is widely used in language independent query-by-example (QbE) spoken term detection (STD) tasks due to its high performance. However, there are two limitations of DTW based template matching, 1) it is not straightforward to perform approximate match of audio queries; 2) DTW is sensitive to the mismatch of signal conditions between the query and the speech search data. To allow approximate search, we propose a partial template matching strategy using phone time boundary information generated by a phone recognizer. To have more invariant representation of audio signals, we use bottleneck features (BNF) as the input of DTW. The BNF network is trained from augmented data, which is generated by adding reverberation and additive noises to the clean training data. Experimental results on QUESST 2015 task shows the effectiveness of the proposed methods for QbE-STD when the queries and search data are both distorted by reverberation and noises.

Original language	English
Title of host publication	2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	6030-6034
Number of pages	5
ISBN (Electronic)	9781479999880
DOIs	https://doi.org/10.1109/ICASSP.2016.7472835
State	Published - 18 May 2016
Event	41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Shanghai, China Duration: 20 Mar 2016 → 25 Mar 2016

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2016-May
ISSN (Print)	1520-6149

Conference

Conference	41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016
Country/Territory	China
City	Shanghai
Period	20/03/16 → 25/03/16

Keywords

data augmentation
DTW
partial matching
Query-by-example
spoken term detection

Access to Document

10.1109/ICASSP.2016.7472835

Cite this

Xu, H., Hou, J., Xiao, X., Pham, V. T., Leung, C. C., Wang, L., Do, V. H., Lv, H., Xie, L., Ma, B., Chng, E. S., & Li, H. (2016). Approximate search of audio queries by using DTW with phone time boundary and data augmentation. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings (pp. 6030-6034). Article 7472835 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2016-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2016.7472835

Xu, Haikua ; Hou, Jingyong ; Xiao, Xiong et al. / Approximate search of audio queries by using DTW with phone time boundary and data augmentation. 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 6030-6034 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

@inproceedings{fe2e22c82dec4b31b41320611a7e659a,

title = "Approximate search of audio queries by using DTW with phone time boundary and data augmentation",

abstract = "Dynamic Time Warping (DTW) is widely used in language independent query-by-example (QbE) spoken term detection (STD) tasks due to its high performance. However, there are two limitations of DTW based template matching, 1) it is not straightforward to perform approximate match of audio queries; 2) DTW is sensitive to the mismatch of signal conditions between the query and the speech search data. To allow approximate search, we propose a partial template matching strategy using phone time boundary information generated by a phone recognizer. To have more invariant representation of audio signals, we use bottleneck features (BNF) as the input of DTW. The BNF network is trained from augmented data, which is generated by adding reverberation and additive noises to the clean training data. Experimental results on QUESST 2015 task shows the effectiveness of the proposed methods for QbE-STD when the queries and search data are both distorted by reverberation and noises.",

keywords = "data augmentation, DTW, partial matching, Query-by-example, spoken term detection",

author = "Haikua Xu and Jingyong Hou and Xiong Xiao and Pham, {Van Tung} and Leung, {Cheung Chi} and Lei Wang and Do, {Van Hai} and Hang Lv and Lei Xie and Bin Ma and Chng, {Eng Siong} and Haizhou Li",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 ; Conference date: 20-03-2016 Through 25-03-2016",

year = "2016",

month = may,

day = "18",

doi = "10.1109/ICASSP.2016.7472835",

language = "英语",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "6030--6034",

booktitle = "2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings",

}

Xu, H, Hou, J, Xiao, X, Pham, VT, Leung, CC, Wang, L, Do, VH, Lv, H, Xie, L, Ma, B, Chng, ES & Li, H 2016, Approximate search of audio queries by using DTW with phone time boundary and data augmentation. in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings., 7472835, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2016-May, Institute of Electrical and Electronics Engineers Inc., pp. 6030-6034, 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016, Shanghai, China, 20/03/16. https://doi.org/10.1109/ICASSP.2016.7472835

Approximate search of audio queries by using DTW with phone time boundary and data augmentation. / Xu, Haikua; Hou, Jingyong; Xiao, Xiong et al.
2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2016. p. 6030-6034 7472835 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2016-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Approximate search of audio queries by using DTW with phone time boundary and data augmentation

AU - Xu, Haikua

AU - Hou, Jingyong

AU - Xiao, Xiong

AU - Pham, Van Tung

AU - Leung, Cheung Chi

AU - Wang, Lei

AU - Do, Van Hai

AU - Lv, Hang

AU - Xie, Lei

AU - Ma, Bin

AU - Chng, Eng Siong

AU - Li, Haizhou

PY - 2016/5/18

Y1 - 2016/5/18

N2 - Dynamic Time Warping (DTW) is widely used in language independent query-by-example (QbE) spoken term detection (STD) tasks due to its high performance. However, there are two limitations of DTW based template matching, 1) it is not straightforward to perform approximate match of audio queries; 2) DTW is sensitive to the mismatch of signal conditions between the query and the speech search data. To allow approximate search, we propose a partial template matching strategy using phone time boundary information generated by a phone recognizer. To have more invariant representation of audio signals, we use bottleneck features (BNF) as the input of DTW. The BNF network is trained from augmented data, which is generated by adding reverberation and additive noises to the clean training data. Experimental results on QUESST 2015 task shows the effectiveness of the proposed methods for QbE-STD when the queries and search data are both distorted by reverberation and noises.

AB - Dynamic Time Warping (DTW) is widely used in language independent query-by-example (QbE) spoken term detection (STD) tasks due to its high performance. However, there are two limitations of DTW based template matching, 1) it is not straightforward to perform approximate match of audio queries; 2) DTW is sensitive to the mismatch of signal conditions between the query and the speech search data. To allow approximate search, we propose a partial template matching strategy using phone time boundary information generated by a phone recognizer. To have more invariant representation of audio signals, we use bottleneck features (BNF) as the input of DTW. The BNF network is trained from augmented data, which is generated by adding reverberation and additive noises to the clean training data. Experimental results on QUESST 2015 task shows the effectiveness of the proposed methods for QbE-STD when the queries and search data are both distorted by reverberation and noises.

KW - data augmentation

KW - DTW

KW - partial matching

KW - Query-by-example

KW - spoken term detection

UR - http://www.scopus.com/inward/record.url?scp=84973334065&partnerID=8YFLogxK

U2 - 10.1109/ICASSP.2016.7472835

DO - 10.1109/ICASSP.2016.7472835

M3 - 会议稿件

AN - SCOPUS:84973334065

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 6030

EP - 6034

BT - 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016

Y2 - 20 March 2016 through 25 March 2016

ER -

Xu H, Hou J, Xiao X, Pham VT, Leung CC, Wang L et al. Approximate search of audio queries by using DTW with phone time boundary and data augmentation. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2016. p. 6030-6034. 7472835. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP.2016.7472835

Approximate search of audio queries by using DTW with phone time boundary and data augmentation

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this