All in One: Multi-task Prompting for Graph Neural Networks
Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, mak...
Tallennettuna:
| Päätekijät: | , , , , |
|---|---|
| Aineistotyyppi: | Artikkeli |
| Julkaistu: |
2023
|
| Aiheet: | |
| Linkit: | https://arxiv.org/abs/2307.01504 http://arxiv.org/abs/2307.01504 |
| Tagit: |
Ei tageja, Lisää ensimmäinen tagi!
|
| _version_ | 1879175208197160960 |
|---|---|
| author | Sun, Xiangguo Cheng, Hong Li, Jia Liu, Bo Guan, Jihong |
| author_browse | Cheng, Hong Guan, Jihong Li, Jia Liu, Bo Sun, Xiangguo |
| author_facet | Sun, Xiangguo Cheng, Hong Li, Jia Liu, Bo Guan, Jihong |
| author_sort | Sun, Xiangguo |
| building | arXiv |
| collection | Acceso abierto |
| description | Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. In this way, the prompting idea from NLP can be seamlessly introduced to the graph area. Then, to further narrow the gap between various graph tasks and state-of-the-art pre-training strategies, we further study the task space of various graph applications and reformulate downstream problems to the graph-level task. Afterward, we introduce meta-learning to efficiently learn a better initialization for the multi-task prompt of graphs so that our prompting framework can be more reliable and general for different tasks. We conduct extensive experiments, results from which demonstrate the superiority of our method. |
| first_indexed | 2026-10-16T00:25:26Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-2307.01504</identifier><datestamp>2023-12-19</datestamp>
<dc:title>All in One: Multi-task Prompting for Graph Neural Networks</dc:title>
<dc:creator>Sun, Xiangguo</dc:creator>
<dc:creator>Cheng, Hong</dc:creator>
<dc:creator>Li, Jia</dc:creator>
<dc:creator>Liu, Bo</dc:creator>
<dc:creator>Guan, Jihong</dc:creator>
<dc:subject>Social and Information Networks</dc:subject>
<dc:subject>Artificial Intelligence</dc:subject>
<dc:subject>Machine Learning</dc:subject>
<dc:description>Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. In this way, the prompting idea from NLP can be seamlessly introduced to the graph area. Then, to further narrow the gap between various graph tasks and state-of-the-art pre-training strategies, we further study the task space of various graph applications and reformulate downstream problems to the graph-level task. Afterward, we introduce meta-learning to efficiently learn a better initialization for the multi-task prompt of graphs so that our prompting framework can be more reliable and general for different tasks. We conduct extensive experiments, results from which demonstrate the superiority of our method. </dc:description>
<dc:description>KDD 23 Best Research Paper Award, which is the first for Hong Kong and Mainland China. A Python Library is released as ProG: https://github.com/sheldonresearch/ProG Submitted to SIGKDD'23 in 03 Feb 2023; Receive Acceptance in 17 May 2023 (Rating 3 4 4 4); Submit to arXiv 1st time in 4 Jul 2023</dc:description>
<dc:date>2023-07-04</dc:date>
<dc:date>2023-12-17</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/2307.01504</dc:identifier>
<dc:identifier>http://arxiv.org/abs/2307.01504</dc:identifier>
</oai_dc:dc> |
| id | arxiv-2307.01504 |
| institution | arXiv |
| last_indexed | 2026-10-16T00:25:26Z |
| publishDate | 2023 |
| publishDateRange | 2023 |
| publishDateSort | 2023 |
| record_format | oai_dc |
| spelling | arxiv-2307.015042023-12-19 All in One: Multi-task Prompting for Graph Neural Networks Sun, Xiangguo Cheng, Hong Li, Jia Liu, Bo Guan, Jihong Social and Information Networks Artificial Intelligence Machine Learning Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of graph prompts and language prompts with the prompt token, token structure, and inserting pattern. In this way, the prompting idea from NLP can be seamlessly introduced to the graph area. Then, to further narrow the gap between various graph tasks and state-of-the-art pre-training strategies, we further study the task space of various graph applications and reformulate downstream problems to the graph-level task. Afterward, we introduce meta-learning to efficiently learn a better initialization for the multi-task prompt of graphs so that our prompting framework can be more reliable and general for different tasks. We conduct extensive experiments, results from which demonstrate the superiority of our method. KDD 23 Best Research Paper Award, which is the first for Hong Kong and Mainland China. A Python Library is released as ProG: https://github.com/sheldonresearch/ProG Submitted to SIGKDD'23 in 03 Feb 2023; Receive Acceptance in 17 May 2023 (Rating 3 4 4 4); Submit to arXiv 1st time in 4 Jul 2023 2023-07-04 2023-12-17 text https://arxiv.org/abs/2307.01504 http://arxiv.org/abs/2307.01504 |
| spellingShingle | Social and Information Networks Artificial Intelligence Machine Learning Sun, Xiangguo Cheng, Hong Li, Jia Liu, Bo Guan, Jihong All in One: Multi-task Prompting for Graph Neural Networks |
| title | All in One: Multi-task Prompting for Graph Neural Networks |
| title_full | All in One: Multi-task Prompting for Graph Neural Networks |
| title_fullStr | All in One: Multi-task Prompting for Graph Neural Networks |
| title_full_unstemmed | All in One: Multi-task Prompting for Graph Neural Networks |
| title_short | All in One: Multi-task Prompting for Graph Neural Networks |
| title_sort | all in one multi task prompting for graph neural networks |
| topic | Social and Information Networks Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2307.01504 http://arxiv.org/abs/2307.01504 |
| work_keys_str_mv | AT sunxiangguo allinonemultitaskpromptingforgraphneuralnetworks AT chenghong allinonemultitaskpromptingforgraphneuralnetworks AT lijia allinonemultitaskpromptingforgraphneuralnetworks AT liubo allinonemultitaskpromptingforgraphneuralnetworks AT guanjihong allinonemultitaskpromptingforgraphneuralnetworks |