Publications

Visual Question Answering (VQA) can facilitate social convenience, which needs to study complex joint reasoning in the visual and language over external knowledge. Recently, Knowledge-Based VQA has attracted the attention of researchers. There are many sources of external knowledge, including visual, textual, and commonsense knowledge, which can effectively improve the reasoning ability of the VQA model. However, introducing different knowledge sources increases the probability of retrieving irrelevant facts and generating noise and further impacts the model's performance. Existing approaches use contrast and prompt learning, visual matrices, density retrieval, etc., to address the noise but bring complex processes. Furthermore, the knowledge representation in these approaches is limited to specific knowledge forms, such as the triple of the knowledge graphs. To address the challenges, we propose a multi-modal joint-guided (MMJG) external knowledge introduction method. The method is to select more relevant external knowledge to the current question through the attention of multi-modal information. Unlike any existing method, our approach learns an adaptive selection module to select external knowledge that is more relevant to the question. Our approach is not specific to a particular knowledge form. The comparison and ablation experiments on the benchmark dataset show that our method achieves better results and demonstrates that our method is more effective.

Semantic search promises to provide more accurate result than present-day keyword search. However, progress with semantic search has been delayed due to the complexity of its query languages. In this paper, we explore a novel approach of adapting keywords to querying...

OBJECTIVE To investigate the inhibitory effects of single and compound phenolic acids on mixed algae. METHODS Salicylic acid, cinnamic acid and pyrogallic acid were chosen individually or in pairs to act on mixed algae of Microcystis aeruginosa and Chlorella pyrenoidosa. RESULTS Three phenolic acids that singled or paired showed certain inhibitory effects on the mixed algae, of which were pyrogallic acid > salicylic acid > cinnamic acid in single phenolic acid, the EC50s of the three phenolic acids on the mixture of Microcystis aeruginosa and Chlorella pyrenoidosa were 7.87, 64.90 and 89.34 mg/L at the day 6; and inhibition of the paired phenolic acid group was pyrogallic acid & salicylic acid group > pyrogallic acid & cinnamic acid group > salicylic acid & cinnamic acid, the first 2 groups had synergistic algal inhibitory effects and the last paired group showed additive action. CONCLUSION Some specifically combined phenolic acids can lead to synergistic inhibiting effects on algae, which has better environmental safety for application due to the use of single phenolic acid was massively decreased.

To quickly obtain new labeled data, we can choose crowdsourcing as an alternative way at lower cost in a short time. But as an exchange, crowd annotations from non-experts may be of lower quality than those from experts. In this paper, we propose an approach to performing crowd annotation learning for Chinese Named Entity Recognition (NER) to make full use of the noisy sequence labels from multiple annotators. Inspired by adversarial learning, our approach uses a common Bi-LSTM and a private Bi-LSTM for representing annotator-generic and -specific information. The annotator-generic information is the common knowledge for entities easily mastered by the crowd. Finally, we build our Chinese NE tagger based on the LSTM-CRF model. In our experiments, we create two data sets for Chinese NER tasks from two domains. The experimental results show that our system achieves better scores than strong baseline systems.

In this paper, we propose the task of personal knowledge extraction to get structured knowledge from personal data in daily life. The existing information extraction methods struggle to handle this task due to the personal data’s multi-source, fine-grained, dynamic, and personalized nature. They fail to select necessary extraction tasks adaptively, cope with diverse scenarios in daily life, and overlook the assistance of historical personal data for the extraction task. Thus, we propose a novel Memory-Enhanced Task-Adaptive Personal Knowledge Extraction method called Meta-PKE. We introduce a task selection module to select the necessary extraction tasks without manual specification according to input personal data. When executing the selected extraction tasks, we record the historical data as the memory and design a memory-enhanced progressive extraction module. Structured personal knowledge is extracted in a coarse-to-fine manner aided by the optimal historical data from a carefully designed memory selection strategy. In addition, we propose a knowledge re-identification module to ensure the completeness of the extracted personal knowledge while avoiding the hallucinations engendered by the large language models. Extensive experiments reflect that, only utilizing the model with a small number of parameters (7B v.s. > 100B), Meta-PKE outperforms the state-of-the-art methods by near 15%, 20%, and 10% on 3 datasets, which cover not only daily but also non-daily scenarios more efficiently.

Commit messages are natural language descriptions of code changes, which are important for software evolution such as code understanding and maintenance. However, previous methods are trained on the entire dataset without considering the fact that a portion of commit messages adhere to good practice (i.e., good-practice commits), while the rest do not. On the basis of our empirical study, we discover that training on good-practice commits significantly contributes to the commit message generation. Motivated by this finding, we propose a novel knowledge-aware denoising learning method called KADEL. Considering that good-practice commits constitute only a small proportion of the dataset, we align the remaining training samples with these good-practice commits. To achieve this, we propose a model that learns the commit knowledge by training on good-practice commits. This knowledge model enables supplementing more information for training samples that do not conform to good practice. However, since the supplementary information may contain noise or prediction errors, we propose a dynamic denoising training method. This method composes a distribution-aware confidence function and a dynamic distribution list, which enhances the effectiveness of the training process. Experimental results on the whole MCMD dataset demonstrate that our method overall achieves state-of-the-art performance compared with previous methods.

Based on Semantic Web technologies, knowledge graphs help users to discover information of interest by using live SPARQL services. Answer-seekers often examine intermediate results iteratively and modify SPARQL queries repeatedly in a search session. In this context,...