SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network
Brain-inspired Spiking Neural Network (SNN) has demonstrated its effectiveness and efficiency in vision, natural language, and speech understanding tasks, indicating their capacity to "see", "listen", and "read". In this paper, we design \textbf{SpikeVoice}, which performs high-quality Text-To-Speec...
Збережено в:
| Автори: | , , , , , , |
|---|---|
| Формат: | Стаття |
| Опубліковано: |
2024
|
| Предмети: | |
| Онлайн доступ: | https://arxiv.org/abs/2408.00788 http://arxiv.org/abs/2408.00788 |
| Теги: |
Немає тегів, Будьте першим, хто поставить тег для цього запису!
|
| _version_ | 1879175960031395841 |
|---|---|
| author | Wang, Kexin Zhang, Jiahong Ren, Yong Yao, Man Shang, Di Xu, Bo Li, Guoqi |
| author_browse | Li, Guoqi Ren, Yong Shang, Di Wang, Kexin Xu, Bo Yao, Man Zhang, Jiahong |
| author_facet | Wang, Kexin Zhang, Jiahong Ren, Yong Yao, Man Shang, Di Xu, Bo Li, Guoqi |
| author_sort | Wang, Kexin |
| building | arXiv |
| collection | Acceso abierto |
| description | Brain-inspired Spiking Neural Network (SNN) has demonstrated its effectiveness and efficiency in vision, natural language, and speech understanding tasks, indicating their capacity to "see", "listen", and "read". In this paper, we design \textbf{SpikeVoice}, which performs high-quality Text-To-Speech (TTS) via SNN, to explore the potential of SNN to "speak". A major obstacle to using SNN for such generative tasks lies in the demand for models to grasp long-term dependencies. The serial nature of spiking neurons, however, leads to the invisibility of information at future spiking time steps, limiting SNN models to capture sequence dependencies solely within the same time step. We term this phenomenon "partial-time dependency". To address this issue, we introduce Spiking Temporal-Sequential Attention STSA in the SpikeVoice. To the best of our knowledge, SpikeVoice is the first TTS work in the SNN field. We perform experiments using four well-established datasets that cover both Chinese and English languages, encompassing scenarios with both single-speaker and multi-speaker configurations. The results demonstrate that SpikeVoice can achieve results comparable to Artificial Neural Networks (ANN) with only 10.5 energy consumption of ANN. |
| first_indexed | 2026-10-16T00:37:21Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-2408.00788</identifier><datestamp>2024-08-05</datestamp>
<dc:title>SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network</dc:title>
<dc:creator>Wang, Kexin</dc:creator>
<dc:creator>Zhang, Jiahong</dc:creator>
<dc:creator>Ren, Yong</dc:creator>
<dc:creator>Yao, Man</dc:creator>
<dc:creator>Shang, Di</dc:creator>
<dc:creator>Xu, Bo</dc:creator>
<dc:creator>Li, Guoqi</dc:creator>
<dc:subject>Neural and Evolutionary Computing</dc:subject>
<dc:subject>Machine Learning</dc:subject>
<dc:description>Brain-inspired Spiking Neural Network (SNN) has demonstrated its effectiveness and efficiency in vision, natural language, and speech understanding tasks, indicating their capacity to "see", "listen", and "read". In this paper, we design \textbf{SpikeVoice}, which performs high-quality Text-To-Speech (TTS) via SNN, to explore the potential of SNN to "speak". A major obstacle to using SNN for such generative tasks lies in the demand for models to grasp long-term dependencies. The serial nature of spiking neurons, however, leads to the invisibility of information at future spiking time steps, limiting SNN models to capture sequence dependencies solely within the same time step. We term this phenomenon "partial-time dependency". To address this issue, we introduce Spiking Temporal-Sequential Attention STSA in the SpikeVoice. To the best of our knowledge, SpikeVoice is the first TTS work in the SNN field. We perform experiments using four well-established datasets that cover both Chinese and English languages, encompassing scenarios with both single-speaker and multi-speaker configurations. The results demonstrate that SpikeVoice can achieve results comparable to Artificial Neural Networks (ANN) with only 10.5 energy consumption of ANN. </dc:description>
<dc:description>9 pages</dc:description>
<dc:date>2024-07-17</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/2408.00788</dc:identifier>
<dc:identifier>http://arxiv.org/abs/2408.00788</dc:identifier>
</oai_dc:dc> |
| id | arxiv-2408.00788 |
| institution | arXiv |
| last_indexed | 2026-10-16T00:37:21Z |
| publishDate | 2024 |
| publishDateRange | 2024 |
| publishDateSort | 2024 |
| record_format | oai_dc |
| spelling | arxiv-2408.007882024-08-05 SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network Wang, Kexin Zhang, Jiahong Ren, Yong Yao, Man Shang, Di Xu, Bo Li, Guoqi Neural and Evolutionary Computing Machine Learning Brain-inspired Spiking Neural Network (SNN) has demonstrated its effectiveness and efficiency in vision, natural language, and speech understanding tasks, indicating their capacity to "see", "listen", and "read". In this paper, we design \textbf{SpikeVoice}, which performs high-quality Text-To-Speech (TTS) via SNN, to explore the potential of SNN to "speak". A major obstacle to using SNN for such generative tasks lies in the demand for models to grasp long-term dependencies. The serial nature of spiking neurons, however, leads to the invisibility of information at future spiking time steps, limiting SNN models to capture sequence dependencies solely within the same time step. We term this phenomenon "partial-time dependency". To address this issue, we introduce Spiking Temporal-Sequential Attention STSA in the SpikeVoice. To the best of our knowledge, SpikeVoice is the first TTS work in the SNN field. We perform experiments using four well-established datasets that cover both Chinese and English languages, encompassing scenarios with both single-speaker and multi-speaker configurations. The results demonstrate that SpikeVoice can achieve results comparable to Artificial Neural Networks (ANN) with only 10.5 energy consumption of ANN. 9 pages 2024-07-17 text https://arxiv.org/abs/2408.00788 http://arxiv.org/abs/2408.00788 |
| spellingShingle | Neural and Evolutionary Computing Machine Learning Wang, Kexin Zhang, Jiahong Ren, Yong Yao, Man Shang, Di Xu, Bo Li, Guoqi SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network |
| title | SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network |
| title_full | SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network |
| title_fullStr | SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network |
| title_full_unstemmed | SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network |
| title_short | SpikeVoice: High-Quality Text-to-Speech Via Efficient Spiking Neural Network |
| title_sort | spikevoice high quality text to speech via efficient spiking neural network |
| topic | Neural and Evolutionary Computing Machine Learning |
| url | https://arxiv.org/abs/2408.00788 http://arxiv.org/abs/2408.00788 |
| work_keys_str_mv | AT wangkexin spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork AT zhangjiahong spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork AT renyong spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork AT yaoman spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork AT shangdi spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork AT xubo spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork AT liguoqi spikevoicehighqualitytexttospeechviaefficientspikingneuralnetwork |