Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search
Text-based person search aims at retrieving target person in an image gallery using a descriptive sentence of that person. It is very challenging since modal gap makes effectively extracting discriminative features more difficult. Moreover, the inter-class variance of both pedestrian images and desc...
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Format: | Artículo |
| Udgivet: |
2021
|
| Fag: | |
| Online adgang: | https://arxiv.org/abs/2101.03036 http://arxiv.org/abs/2101.03036 |
| Tags: |
Ingen Tags, Vær først til at tagge denne postø!
|
| _version_ | 1879172724152074240 |
|---|---|
| author | Gao, Chenyang Cai, Guanyu Jiang, Xinyang Zheng, Feng Zhang, Jun Gong, Yifei Peng, Pai Guo, Xiaowei Sun, Xing |
| author_browse | Cai, Guanyu Gao, Chenyang Gong, Yifei Guo, Xiaowei Jiang, Xinyang Peng, Pai Sun, Xing Zhang, Jun Zheng, Feng |
| author_facet | Gao, Chenyang Cai, Guanyu Jiang, Xinyang Zheng, Feng Zhang, Jun Gong, Yifei Peng, Pai Guo, Xiaowei Sun, Xing |
| author_sort | Gao, Chenyang |
| building | arXiv |
| collection | Acceso abierto |
| description | Text-based person search aims at retrieving target person in an image gallery using a descriptive sentence of that person. It is very challenging since modal gap makes effectively extracting discriminative features more difficult. Moreover, the inter-class variance of both pedestrian images and descriptions is small. So comprehensive information is needed to align visual and textual clues across all scales. Most existing methods merely consider the local alignment between images and texts within a single scale (e.g. only global scale or only partial scale) then simply construct alignment at each scale separately. To address this problem, we propose a method that is able to adaptively align image and textual features across all scales, called NAFS (i.e.Non-local Alignment over Full-Scale representations). Firstly, a novel staircase network structure is proposed to extract full-scale image features with better locality. Secondly, a BERT with locality-constrained attention is proposed to obtain representations of descriptions at different scales. Then, instead of separately aligning features at each scale, a novel contextual non-local attention mechanism is applied to simultaneously discover latent alignments across all scales. The experimental results show that our method outperforms the state-of-the-art methods by 5.53% in terms of top-1 and 5.35% in terms of top-5 on text-based person search dataset. The code is available at https://github.com/TencentYoutuResearch/PersonReID-NAFS |
| first_indexed | 2026-10-15T23:45:55Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-2101.03036</identifier><datestamp>2021-01-11</datestamp>
<dc:title>Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search</dc:title>
<dc:creator>Gao, Chenyang</dc:creator>
<dc:creator>Cai, Guanyu</dc:creator>
<dc:creator>Jiang, Xinyang</dc:creator>
<dc:creator>Zheng, Feng</dc:creator>
<dc:creator>Zhang, Jun</dc:creator>
<dc:creator>Gong, Yifei</dc:creator>
<dc:creator>Peng, Pai</dc:creator>
<dc:creator>Guo, Xiaowei</dc:creator>
<dc:creator>Sun, Xing</dc:creator>
<dc:subject>Computer Vision and Pattern Recognition</dc:subject>
<dc:description>Text-based person search aims at retrieving target person in an image gallery using a descriptive sentence of that person. It is very challenging since modal gap makes effectively extracting discriminative features more difficult. Moreover, the inter-class variance of both pedestrian images and descriptions is small. So comprehensive information is needed to align visual and textual clues across all scales. Most existing methods merely consider the local alignment between images and texts within a single scale (e.g. only global scale or only partial scale) then simply construct alignment at each scale separately. To address this problem, we propose a method that is able to adaptively align image and textual features across all scales, called NAFS (i.e.Non-local Alignment over Full-Scale representations). Firstly, a novel staircase network structure is proposed to extract full-scale image features with better locality. Secondly, a BERT with locality-constrained attention is proposed to obtain representations of descriptions at different scales. Then, instead of separately aligning features at each scale, a novel contextual non-local attention mechanism is applied to simultaneously discover latent alignments across all scales. The experimental results show that our method outperforms the state-of-the-art methods by 5.53% in terms of top-1 and 5.35% in terms of top-5 on text-based person search dataset. The code is available at https://github.com/TencentYoutuResearch/PersonReID-NAFS </dc:description>
<dc:date>2021-01-08</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/2101.03036</dc:identifier>
<dc:identifier>http://arxiv.org/abs/2101.03036</dc:identifier>
</oai_dc:dc> |
| id | arxiv-2101.03036 |
| institution | arXiv |
| last_indexed | 2026-10-15T23:45:55Z |
| publishDate | 2021 |
| publishDateRange | 2021 |
| publishDateSort | 2021 |
| record_format | oai_dc |
| spelling | arxiv-2101.030362021-01-11 Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search Gao, Chenyang Cai, Guanyu Jiang, Xinyang Zheng, Feng Zhang, Jun Gong, Yifei Peng, Pai Guo, Xiaowei Sun, Xing Computer Vision and Pattern Recognition Text-based person search aims at retrieving target person in an image gallery using a descriptive sentence of that person. It is very challenging since modal gap makes effectively extracting discriminative features more difficult. Moreover, the inter-class variance of both pedestrian images and descriptions is small. So comprehensive information is needed to align visual and textual clues across all scales. Most existing methods merely consider the local alignment between images and texts within a single scale (e.g. only global scale or only partial scale) then simply construct alignment at each scale separately. To address this problem, we propose a method that is able to adaptively align image and textual features across all scales, called NAFS (i.e.Non-local Alignment over Full-Scale representations). Firstly, a novel staircase network structure is proposed to extract full-scale image features with better locality. Secondly, a BERT with locality-constrained attention is proposed to obtain representations of descriptions at different scales. Then, instead of separately aligning features at each scale, a novel contextual non-local attention mechanism is applied to simultaneously discover latent alignments across all scales. The experimental results show that our method outperforms the state-of-the-art methods by 5.53% in terms of top-1 and 5.35% in terms of top-5 on text-based person search dataset. The code is available at https://github.com/TencentYoutuResearch/PersonReID-NAFS 2021-01-08 text https://arxiv.org/abs/2101.03036 http://arxiv.org/abs/2101.03036 |
| spellingShingle | Computer Vision and Pattern Recognition Gao, Chenyang Cai, Guanyu Jiang, Xinyang Zheng, Feng Zhang, Jun Gong, Yifei Peng, Pai Guo, Xiaowei Sun, Xing Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search |
| title | Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search |
| title_full | Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search |
| title_fullStr | Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search |
| title_full_unstemmed | Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search |
| title_short | Contextual Non-Local Alignment over Full-Scale Representation for Text-Based Person Search |
| title_sort | contextual non local alignment over full scale representation for text based person search |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2101.03036 http://arxiv.org/abs/2101.03036 |
| work_keys_str_mv | AT gaochenyang contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT caiguanyu contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT jiangxinyang contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT zhengfeng contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT zhangjun contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT gongyifei contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT pengpai contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT guoxiaowei contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch AT sunxing contextualnonlocalalignmentoverfullscalerepresentationfortextbasedpersonsearch |