Advances in Knowledge Discovery and Data Mining

Citation plays a pivotal role in determining the associations among research articles. It portrays essential information in indicative, supportive, or contrastive studies. The task of inline citation classification aids in extrapolating these relationships; However, existing studies are still immature and demand further scrutiny. Current datasets and methods used for inline citation classification only use citation-marked sentences constraining the model to turn a blind eye to domain knowledge and neighboring contextual sentences. In this paper, we propose a new dataset, named 3Cext, which along with the cited sentences, provides discourse information using the vicinal sentences to analyze the contrasting and entailing relationships as well as domain information. We propose PeriCite, a Transformer-based deep neural network that fuses peripheral sentences and domain knowledge. Our model achieves the state-of-the-art on the 3Cext dataset by \(+0.09\) F1 against the best baseline. We conduct extensive ablations to analyze the efficacy of the proposed dataset and model fusion methods.

AR shopping has attracted significant attention with the emergence of popular AR applications, e.g., IKEA Place and Nike AR Fit. However, most AR stores merely provide auxiliary information for each user, instead of group recommendations that leverage new AR features. In this paper, we make the first attempt to explore AR shopping to simultaneously: i) leverage flexible display and item replacement to maximize preference, ii) construct an immersive environment with virtual item haptic feedback, and iii) stimulate social interactions with common user interests. We formulate the Tangible AR Group Shopping (TARGS) problem to partition users and recommend virtual substitutes. We then develop the Social-aware Tangible AR Replacement and Recommendation (STAR3) system, including a connected graph neural network for modeling graph features from different domains, a virtual-physical item mapping model to enable haptic experience by leveraging on-shelf items’ passive haptic feedback, and a multi-goal recommender to dynamically split user groups and recommend substitutes for satisfaction maximization. Experimental results manifest that STAR3 outperforms baselines by \(23\%\) to \(63\%\) in three real-world datasets and a user study.

The spread of rumors in online networks threatens public safety and results in economic losses. To overcome this problem, a lot of work studies the problem of rumor control which aims at limiting the spread of rumors. However, all previous work ignores the relationship between the influence block effect and counts of impressions on the user. In this paper, we study the problem of minimizing the spread of rumors when impression counts. Given a graph G(V, E), a rumor set \(R \in V\), and a budget k, it aims to find a protector set \(P \in V \backslash R\) to minimize the spread of the rumor set R under the budget k. Due to the impression counts, two following challenges of our problem need to be overcome: (1) our problem is NP-hard; (2) the influence block is non-submodular, which means a straightforward greedy approach is not applicable. Hence, we devise a branch-and-bound framework for this problem with a (\(1-1/e-\epsilon \)) approximation ratio. To further improve the efficiency, we speed up our framework with a progressive upper bound estimation method, which achieves a (\(1-1/e-\epsilon - \rho \)) approximation ratio. We conduct experiments on real-world datasets to verify the efficiency, effectiveness, and scalability of our methods.

Data missing is inevitable in Intelligent Transportation Systems (ITSs). Although many methods have been proposed for traffic data imputation, it is still very challenging because of two reasons. First, the ground truth of missing data is actually inaccessible, which makes most imputation methods hard to be trained. Second, incomplete data would easily mislead the model to learn unreliable spatial-temporal dependencies, which finally hurts the imputation performance. In this paper, we proposes a novel \(\underline{{\boldsymbol{G}}}\)enerative-\(\underline{{\boldsymbol{C}}}\)ontrastive-\(\underline{{\boldsymbol{A}}}\)ttentive \(\underline{{\boldsymbol{S}}}\)patial-\(\underline{{\boldsymbol{T}}}\)emporal \(\underline{{\boldsymbol{N}}}\)etwork (GCASTN) for traffic data imputation. It combines the ideas of generative and contrastive self-supervised learning together to develop a new training paradigm for imputation without relying on the ground truth of missing data. In addition, it introduces nearest missing interval to describe missing data and a novel \(\underline{{\boldsymbol{M}}}\)issing-\(\underline{{\boldsymbol{A}}}\)ware \(\underline{{\boldsymbol{A}}}\)ttention (MAA) mechanism is designed to utilize nearest missing interval to guide the model to adaptively learn the reliable spatial-temporal dependencies of incomplete traffic data. Extensive experiments covering three types of missing scenarios on two real-world traffic flow datasets demonstrate that GCASTN outperforms the state-of-the-art baselines.

Spatio-temporal traffic patterns reflecting the mobility behavior of road users are essential for learning effective general-purpose road representations. Such patterns are largely neglected in state-of-the-art road representation learning, mainly focusing on modeling road topology and static road features. Incorporating traffic patterns into road network representation learning is particularly challenging due to the complex relationship between road network structure and mobility behavior of road users. In this paper, we present TrajRNE – a novel trajectory-based road embedding model incorporating vehicle trajectory information into road network representation learning. Our experiments on two real-world datasets demonstrate that TrajRNE outperforms state-of-the-art road representation learning baselines on various downstream tasks.

Accurate spatio-temporal prediction is essential for capturing city dynamics and planning mobility services. State-of-the-art deep spatio-temporal predictive models depend on rich and representative training data for target regions and tasks. However, the availability of such data is typically limited. Furthermore, existing predictive models fail to utilize cross-correlations across tasks and cities. In this paper, we propose MetaCitta, a novel deep meta-learning approach that addresses the critical challenges of data scarcity and model generalization. MetaCitta adopts the data from different cities and tasks in a generalizable spatio-temporal deep neural network. We propose a novel meta-learning algorithm that minimizes the discrepancy between spatio-temporal representations across tasks and cities. Our experiments with real-world data demonstrate that the proposed MetaCitta approach outperforms state-of-the-art prediction methods for zero-shot learning and pre-training plus fine-tuning. Furthermore, MetaCitta is computationally more efficient than the existing meta-learning approaches.

Our work focuses on anomaly detection in cyber-physical systems. Prior literature has three limitations: (1) Failing to capture long-delayed patterns in system anomalies; (2) Ignoring dynamic changes in sensor connections; (3) The curse of high-dimensional data samples. These limit the detection performance and usefulness of existing works. To address them, we propose a new approach called deep graph stream support vector data description (SVDD) for anomaly detection. Specifically, we first use a transformer to preserve both short and long temporal patterns of monitoring data in temporal embeddings. Then we cluster these embeddings according to sensor type and utilize them to estimate the change in connectivity between various sensors to construct a new weighted graph. The temporal embeddings are mapped to the new graph as node attributes to form weighted attributed graph. We input the graph into a variational graph auto-encoder model to learn final spatio-temporal representation. Finally, we learn a hypersphere that encompasses normal embeddings and predict the system status by calculating the distances between the hypersphere and data samples. Extensive experiments validate the superiority of our model, which improves F1-score by 35.87%, AUC by 19.32%, while being 32 times faster than the best baseline at training and inference.

Text classification is a widely used task in natural language processing. However, the presence of stereotype bias in text classification can lead to unfair and inaccurate predictions. Stereotype bias is particularly prevalent in words that are unevenly distributed across classes and are associated with specific categories. This bias can be further strengthened in pre-trained models on large natural language datasets. Prior works to remove stereotype bias have mainly focused on specific demographic groups or relied on specific thesauri without measuring the influence of stereotype words on predictions. In this work, we present a causal analysis of how stereotype bias occurs and affects text classification, and propose a framework to mitigate stereotype bias. Our framework detects potential stereotype bias words using SHAP values and alleviates bias in the prediction stage through a counterfactual approach. Unlike existing debiasing methods, our framework does not rely on existing stereotype word sets and can dynamically evaluate the influence of words on stereotype bias. Extensive experiments and ablation studies show that our approach effectively improves classification performance while mitigating stereotype bias.

Emotion-cause pair extraction (ECPE) aims to extract all potential pairs of emotions and corresponding cause(s) from a given document. Current methods have focused on extracting possible emotion-cause pairs by directly analyzing the given documents on the basis of a large training set. However, there are many hard-matching emotion-cause pairs that require commonsense knowledge to understand. Exploiting only the given documents is insufficient to capture the latent semantics behind these hard-matching emotion-cause pairs, which may downgrade the performance of existing ECPE methods. To fill this gap, we propose a Knowledge-Enhanced Hierarchical Transformers framework for the ECPE task. Specifically, we first inject commonsense knowledge into the given documents to construct the knowledge-enhanced clauses. To incorporate the injected knowledge into the clause representations, we then develop a hierarchical Transformers module that leverages two different types of transformer blocks to encode knowledge-enriched clause representations at both global and local stages. Experimental results show that our method achieves state-of-the-art performance.

Generating informative, coherent and fluent responses to user queries is challenging yet critical for a rich user experience and the eventual success of dialogue systems. Knowledge-grounded dialogue systems leverage external knowledge to induce relevant facts in a dialogue. These systems need to understand the semantic relatedness between the dialogue context and the available knowledge, thereby utilising this information for response generation. Although various innovative models have been proposed, they neither utilise the semantic entailment between the dialogue history and the knowledge nor effectively process knowledge from both structured and unstructured sources. In this work, we propose PICKD, a two-stage framework for knowledgeable dialogue. The first stage involves the Knowledge Selector choosing knowledge pertinent to the dialogue context from both structured and unstructured knowledge sources. PICKD leverages novel In-Situ prompt tuning for knowledge selection, wherein prompt tokens are injected into the dialogue-knowledge text tokens during knowledge retrieval. The second stage employs the Response Generator for generating fluent and factual responses by utilising the retrieved knowledge and the dialogue context. Extensive experiments on three domain-specific datasets exhibit the effectiveness of PICKD over other baseline methodologies for knowledge-grounded dialogue. The source is available at https://​github.​com/​rajbsk/​pickd.

Detecting fake news on social media is an increasingly important problem, because of the rapid dissemination and detrimental impact of fake news. Graph-based methods that encode news propagation paths into tree structures have been shown to be effective. Existing studies based on such methods represent the propagation of news through static graphs or coarse-grained graph snapshots. They do not capture the full dynamics of graph evolution and hence the temporal news propagation patterns. To address this issue and model dynamic news propagation at a finer-grained level, we propose a temporal graph-based model. We join this model with a neural Hawkes process model to exploit the distinctive self-exciting patterns of true news and fake news on social media. This creates a highly effective fake news detection model that we named SEAGEN. Experimental results on real datasets show that SEAGEN achieves an accuracy of fake news detection of over 93% with an advantage of over 2.5% compared to other state-of-the-art models.

Important natural language processing tasks such as machine translation and document summarization have made enormous strides in recent years. However, their performance is still partially limited by the standard training objectives, which operate on single tokens rather than on more global features. Moreover, such standard objectives do not explicitly consider the source documents, potentially affecting their alignment with the predictions. For these reasons, in this paper, we propose using an Optimal Transport (OT) training objective to promote a global alignment between the model’s predictions and the source documents. In addition, we present an original implementation of the OT objective based on the Sinkhorn divergence between the final hidden states of the model’s encoder and decoder. Experimental results over machine translation and abstractive summarization tasks show that the proposed approach has been able to achieve statistically significant improvements across all experimental settings compared to our baseline and other alternative objectives. A qualitative analysis of the results also shows that the predictions have been able to better align with the source sentences thanks to the supervision of the proposed objective.

The task of Chinese Spelling Check (CSC) is to detect and correct spelling errors in Chinese sentences. Since the scale of labeled CSC training set is quite small, we propose an unsupervised Chinese spelling correction framework based on detectors. Two kinds of detectors: Dec-Err and Dec-Eva, are proposed to leverage the contextual information to detect misspelled characters and evaluate the corrections respectively. Both detectors are fine-tuned with our proposed hybrid mask strategy. Dec-Eva is a transformer encoder based detector, of which we modify the attention connections to reuse the contextual information and parallel evaluate possible corrections. Compared with supervised and unsupervised state-of-the-art methods, experimental studies show that our method achieves competitive results. Further empirical studies reveal the efficiency and flexibility of our method.

Entity matching (EM) is a fundamental task in data integration, which involves identifying records that refer to the same real-world entity. Unsupervised EM is often preferred in real-world applications, as labeling data is often a labor-intensive process. However, existing unsupervised methods may not always perform well because the assumptions for these methods may not hold for tasks in different domains. In this paper, we propose QA-Matcher, an unsupervised EM model that is domain-agnostic and doesn’t require any particular assumptions. Our idea is to frame EM as question answering (QA) by utilizing a trained QA model. Specifically, we generate a question that asks which record has the characteristics of a particular record and a passage that describes other records. We then use the trained QA model to predict the record pair that corresponds to the question-answer as a match. QA-Matcher leverages the power of a QA model to represent the semantics of various types of entities, allowing it to identify identical entities in a QA-like fashion. In extensive experiments on 16 real-world datasets, we demonstrate that QA-Matcher outperforms unsupervised EM methods and is competitive with supervised methods.

In an e-commerce business, the ability to parse postal addresses into sub-component entities (such as building, locality) is essential to take automated actions at scale for successful delivery of shipments. The entities can be leveraged to build applications for logistics related operations, e.g. geocoding, assessing address completeness. Training an accurate address parser requires a significant number of manually labeled examples which is very expensive to create, especially when trying to build model(s) for multiple countries with unique address structure. To tackle this problem, in this paper, we present a novel Unsupervised Domain Adaptation (UDA) framework to transfer knowledge acquired by training a parser on labeled data from one country (source domain) to another (target domain) with unlabeled data. We specifically propose a multi-task student-teacher model comprising of three components: 1) specialized teachers trained on source data to create a pseudo labeled dataset, 2) consistency regularization, that uses a new data augmentation technique for sequence tagging data, and 3) boundary detection, leveraging signals in addresses like commas and text box boundaries. Multiple experiments on diverse address datasets (In this paper, we do not reveal the name of the e-commerce countries on which we evaluate our models due to business confidentiality. We also mask finer address details with (XX) to preserve customer’s privacy.) demonstrate that our approach outperforms state-of-the-art UDA baselines for Named Entity Recognition (NER) task in terms of F1-score by 2–9%.

Sentiment transfer aims at revising the input text to satisfy a given sentiment polarity while retaining the original semantic content. The nucleus of sentiment transfer lies in precisely separating the sentiment information from the content information. Existing explicit approaches generally identify and mask sentiment tokens simply based on prior linguistic knowledge and manually-defined rules, leading to low generality and undesirable transfer performance. In this paper, we view the positions to be masked as the learnable parameters, and further propose a novel AM-ST model to learn adaptive task-relevant masks based on the attention mechanism. Moreover, a sentiment-aware masked language model is further proposed to fill in the blanks in the masked positions by incorporating both context and sentiment polarity to capture the multi-grained semantics comprehensively. AM-ST is thoroughly evaluated on two popular datasets, and the experimental results demonstrate the superiority of our proposal.

In this paper, we propose an end-to-end system for unsupervised text style transfer (UTST). Prior studies on UTST work on the principle of disentanglement between style and content features, which successfully accomplishes the task of generating style-transferred text. The success of a style transfer system depends on three criteria, viz. Style transfer accuracy, Content preservation of source, and Fluency of the generated text. Generated text by disentanglement-based method achieves better style transfer performance but suffers from the lack of content preservation as the previous works suggest. To develop an all-around solution to all three aspects, we use a reinforcement learning-based training objective that gives rewards to the model for generating fluent style transferred text while preserving the source content. On the modeling aspect, we develop a shared encoder and style-specific decoder architecture which uses the Transformer architecture as a backbone. This modeling choice enables us to frame a differentiable back translation objective aiding better content preservation as shown through a careful ablation study. We conclude this paper with both automatic and human evaluation, showing the superiority of our proposed method on sentiment and formality style transfer tasks. Code is available at https://​github.​com/​newcodevelop/​Unsupervised-TST.

Aspect Sentiment Triplet Extraction (ASTE) is a challenging task in modern natural language processing concerning the automatic extraction of (aspect phrase, opinion phrase, sentiment polarity) triplets from a given text. Current state-of-the-art methods achieve relatively high results by analyzing all possible spans extracted from a text. Due to a high number of analyzed spans, span-level methods usually apply some kind of pruning operators that interrupt the gradient flow. They also do not analyze interrelations between spans while constructing model output, relying on independent, sequential predictions for candidate triplets. This paper presents a new span-level approach that applies a learnable extractor of spans and a differentiable span selector that enables end2end training. The approach relies on a fully connected pairwise CRF model to capture interrelations between spans while constructing the output. Conducted experiments demonstrated that the proposed approach achieves superior results in terms of F1-score in comparison to other, state-of-the-art ASTE methods.

There has been an upward trend of fake news propagation on social media. To solve the fake news propagation problem, it is crucial to understand which media posts (e.g., tweets) cause fake news to disseminate widely, and further what lexicons inside a tweet play essential roles for the propagation. However, only modeling the correlation between social media posts and dissemination will find a spurious relationship between them, provide imprecise dissemination prediction, and incorrect important lexicons identification because it did not eliminate the effect of the confounder variable. Additionally, existing causal inference models cannot handle numerical and textual covariates simultaneously. Thus, we propose a novel causal inference model that combines the textual and numerical covariates through soft-prompt learning, and removes irrelevant information from the covariates by conditional treatment generation toward learning effective confounder representation. Then, the model identifies critical lexicons through a post-hoc explanation method. Our model achieves the best performance against baseline methods on two fake news benchmark datasets in terms of dissemination prediction and important lexicon identification related to the dissemination. The code is available at https://​github.​com/​bigheiniu/​CausalFakeNews.

Due to the success of the pre-trained language model (PLM), existing PLM-based summarization models show their powerful generative capability. However, these models are trained on general-purpose summarization datasets, leading to generated summaries failing to satisfy the needs of different readers. To generate summaries with topics, many efforts have been made on topic-focused summarization. However, these works generate a summary only guided by a prompt comprising topic words. Despite their success, these methods still ignore the disturbance of sentences with non-relevant topics and only conduct cross-interaction between tokens by attention module. To address this issue, we propose a topic-arc recognition objective and topic-selective graph network. First, the topic-arc recognition objective is used to model training, which endows the capability to discriminate topics for the model. Moreover, the topic-selective graph network can conduct topic-guided cross-interaction on sentences based on the results of topic-arc recognition. In the experiments, we conduct extensive evaluations on NEWTS and COVIDET datasets. Results show that our methods achieve state-of-the-art performance.

Traffic accident forecasting is of vital importance to the intelligent transportation and public safety. Spatial-temporal learning is the mainstream approach to exploring complex evolving patterns. However, two intrinsic challenges lie in traffic accident forecasting, preventing the straightforward adoption of spatial-temporal learning. First, the temporal observations of traffic accidents exhibit ultra-rareness due to the inherent properties of accident occurrences (Fig. 1(a)), which leads to the severe scarcity of risk samples in learning accident patterns. Second, the spatial distribution of accidents is severely imbalanced from region to region (Fig. 1(b)), which poses a serious challenge to forecast the spatially diversified risks. To tackle the above challenges, we propose RiskContra, a Contra stive learning approach with multi-kernel networks, to forecast the Risk of traffic accidents. Specifically, to address the first challenge (i.e. temporal rareness), we design a novel contrastive learning approach, which leverages the periodic patterns to derive a tailored mixup strategy for risk sample augmentation. This way, the contrastively learned features can better represent the risk samples, thus capturing higher-quality accident patterns for forecasting. To address the second challenge (i.e. spatial imbalance), we design the multi-kernel networks to capture the hierarchical correlations from multiple spatial granularities. This way, disparate regions can utilize the multi-granularity correlations to enhance the forecasting performance across regions. Extensive experiments corroborate the effectiveness of each devised component in RiskContra.

This study proposes a framework for generating customized trend lines that consider user preferences and input time series shapes. The existing trend estimators fail to capture individual needs and application domain requirements. The proposed framework obtains users’ preferred trends by asking users to draw trend lines on sample datasets. The experiments and case studies demonstrate the effectiveness of the model. Code and dataset are available at https://​github.​com/​Anthony860810/​Generating-Personalized-Trend-Line-Based-on-Few-Labelings-from-One-Individual.

Irregularly sampled multivariate time series classification tasks become prevalent due to widespread application of sensors. However, different collection frequencies or sensor failures presents nontrivial challenges since mainstream methods generally assume aligned measurements across sensors (variables). Besides, most existing studies fail to account for the relationship between misaligned patterns and classification tasks. To this end, we propose a Global view-guided Autoregressive Residual Network (GARNet), which mainly adopts a generation-and-sampling strategy to deal with the partially observed data at each timestamp. Specifically, we first leverage a Structure-augmented Global Information Extractor (SGIE) to capture the global semantic information in the whole conditioning window. Then, a Global view-guided Autoregressive Recurrent Neural Network (GARNN) is developed to capture the local temporal dynamics hidden in latent factors. Finally, a Masked Temporal Information Aggregator (MTIA) is proposed to attentively aggregate the extracted latent factors at each timestamp for the classification task. Experimental results on two real-world datasets show that GARNet outperforms state-of-the-art methods.

Periodicity or repetition detection has a wide varieties of use cases in human activity tracking, music pattern discovery, physiological signal monitoring and more. While there exists a broad range of research, often the most practical approaches are those based on simple quantities that are conserved over periodic repetition, such as auto-correlation or Fourier transform. Unfortunately, these periodicity-based approaches do not generalise well to quasi-periodic (variable period) scenarios. In this research, we exploit the time warping invariance of path signatures to find linearly accumulating quantities with respect to quasi-periodic repetition, and propose a novel repetition detection algorithm Recurrence Point Signed Area Persistence. We show that our approach can effectively deal with repetition detection with period variations, which similar unsupervised methods tend to struggle with.

Time Series Forecasting (TSF) is well established in domains dealing with temporal data to predict future events yielding the basis for strategic decision-making. Previous research indicated that forecasting models are vulnerable to adversarial attacks, that is, maliciously crafted perturbations of the original data with the goal of altering the model’s predictions. However, attackers targeting specific outcomes pose a substantially more severe threat as they could manipulate the model and bend it to their needs. Regardless, there is no systematic approach for targeted adversarial learning in the TSF domain yet. In this paper, we introduce targeted attacks on TSF in a systematic manner. We establish a new experimental design standard regarding attack goals and perturbation control for targeted adversarial learning on TSF. For this purpose, we present a novel indirect sparse black-box evasion attack on TSF, n Vita. Additionally, we adapt the popular white-box attacks Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM). Our experiments confirm not only that all three methods are effective but also that current state-of-the-art TSF models are indeed susceptible to attacks. These results motivate future research in this area to achieve higher reliability of forecasting models.

Dealing with an unbounded data stream involves overcoming the assumption that data is identically distributed and independent. A data stream can, in fact, exhibit temporal dependencies (i.e., be a time series), and data can change distribution over time (concept drift). The two problems are deeply discussed, and existing solutions address them separately: a joint solution is absent. In addition, learning multiple concepts implies remembering the past (a.k.a. avoiding catastrophic forgetting in Neural Networks’ terminology). This work proposes Continuous Progressive Neural Networks (cPNN), a solution that tames concept drifts, handles temporal dependencies, and bypasses catastrophic forgetting. cPNN is a continuous version of Progressive Neural Networks, a methodology for remembering old concepts and transferring past knowledge to fit the new concepts quickly. We base our method on Recurrent Neural Networks and exploit the Stochastic Gradient Descent applied to data streams with temporal dependencies. Results of an ablation study show a quick adaptation of cPNN to new concepts and robustness to drifts.

Multivariate time series usually have complex and time-varying dependencies among variables. In order to spot changes and interpret temporal dynamics, it is essential to understand these dependencies and how they evolve over time. However, the problem of acquiring and monitoring them is extremely challenging due to the dynamic and nonlinear interactions among time series. In this paper, we propose a dynamic dependency learning method, which learns dependency latent space with a two-level attention model. The first level is a bi-sided attention module to learn the short-term dependencies. Once the sequence of short-term dependencies is collected over a certain period of time, a temporal self-attention module is applied to obtain the actual dependencies for the current timestamp. The coordinates in latent space is descriptor of the temporal dynamic. We apply this descriptor to change point detection, and experiments show that our proposed method outperforms popular baselines.