Publications

No abstract is provided for this article.

This work presents a novel method for applying test-time augmentation (TTA) to tabular data. We used TTA along with an ensemble of 42 models to achieve higher performance on the MIT Global Open Source Severity of Illness Score dataset consisting of 131,051 ICU visits and outcomes. This method achieved an AUC of 0.915 on the private test set (19,669 admissions) and won first place at Stanford University's WiDS Datathon 2020 challenge on Kaggle, while the Acute Physiology and Chronic Health Evaluation (APACHE) IV model (commonly used for ICU survival prediction in the literature) achieved an AUC of 0.868. In addition to increasing the AUC score, our method also reduces "unfair" bias.

Large Language Models (LLMs) have shown remarkable capabilities across various natural language processing tasks but often struggle to excel uniformly in diverse or complex domains. We propose a novel ensemble method - Diverse Fingerprint Ensemble (DFPE), which leverages the complementary strengths of multiple LLMs to achieve more robust performance. Our approach involves: (1) clustering models based on response "fingerprints" patterns, (2) applying a quantile-based filtering mechanism to remove underperforming models at a per-subject level, and (3) assigning adaptive weights to remaining models based on their subject-wise validation accuracy. In experiments on the Massive Multitask Language Understanding (MMLU) benchmark, DFPE outperforms the best single model by 3% overall accuracy and 5% in discipline-level accuracy. This method increases the robustness and generalization of LLMs and underscores how model selection, diversity preservation, and performance-driven weighting can effectively address challenging, multi-faceted language understanding tasks.

Basketball is one of the most popular types of sports in the world. Recent technological developments have made it possible to collect large amounts of data on the game, analyze it, and discover new insights. We propose a novel approach for modeling basketball games using deep reinforcement learning. By analyzing multiple aspects of both the players and the game, we are able to model the latent connections among players' movements, actions, and performance, into a single measure - the Q-Ball. Using Q-Ball, we are able to assign scores to the performance of both players and whole teams. Our approach has multiple practical applications, including evaluating and improving players' game decisions and producing tactical recommendations. We train and evaluate our approach on a large dataset of National Basketball Association games, and show that the Q-Ball is capable of accurately assessing the performance of players and teams. Furthermore, we show that Q-Ball is highly effective in recommending alternatives to players' actions.

Drug-drug interactions (DDIs) are preventable causes of medical injuries and often result in doctor and emergency room visits. Previous research demonstrates the effectiveness of using matrix completion approaches based on known drug interactions to predict unknown Drug-drug interactions. However, in the case of a new drug, where there is limited or no knowledge regarding the drug's existing interactions, such an approach is unsuitable, and other drug's preferences can be used to accurately predict new Drug-drug interactions.We propose adjacency biomedical text embedding (ABTE) to address this limitation by using a hybrid approach which combines known drugs' interactions and the drug's biomedical text embeddings to predict the DDIs of both new and well known drugs.Our evaluation demonstrates the superiority of this approach compared to recently published DDI prediction models and matrix factorization-based approaches. Furthermore, we compared the use of different text embedding methods in ABTE, and found that the concept embedding approach, which involves biomedical information in the embedding process, provides the highest performance for this task. Additionally, we demonstrate the effectiveness of leveraging biomedical text embedding for additional drugs' biomedical prediction task by presenting text embedding's contribution to a multi-modal pregnancy drug safety classification.Text and concept embeddings created by analyzing a domain-specific large-scale biomedical corpora can be used for predicting drug-related properties such as Drug-drug interactions and drug safety prediction. Prediction models based on the embeddings resulted in comparable results to hand-crafted features, however text embeddings do not require manual categorization or data collection and rely solely on the published literature.

In this paper, we present a generic, query-efficient black-box attack against API call-based machine learning malware classifiers. We generate adversarial examples by modifying the malware's API call sequences and non-sequential features (printable strings). The adversarial examples will be misclassified by the target malware classifier without affecting the malware's functionality. In contrast to previous studies, our attack minimizes the number of malware classifier queries required. In addition, in our attack, the attacker must only know the class predicted by the malware classifier; the attacker knowledge of the malware classifier's confidence score is optional. We evaluate the attack effectiveness when attacks are performed against a variety of malware classifier architectures, including recurrent neural network (RNN) variants, deep neural networks, support vector machines, and gradient boosted decision trees. Our attack success rate is about 98% when the classifier's confidence score is known and 88% when just the classifier's predicted class is known. We implement four state-of-the-art query-efficient attacks and show that our attack requires fewer queries and less knowledge about the attacked model's architecture than other existing query-efficient attacks, making it practical for attacking cloud-based malware classifiers at a minimal cost.

In recent years, machine learning algorithms, and more specifically deep learning algorithms, have been widely used in many fields, including cyber security. However, machine learning systems are vulnerable to adversarial attacks, and this limits the application of machine learning, especially in non-stationary, adversarial environments, such as the cyber security domain, where actual adversaries (e.g., malware developers) exist. This article comprehensively summarizes the latest research on adversarial attacks against security solutions based on machine learning techniques and illuminates the risks they pose. First, the adversarial attack methods are characterized based on their stage of occurrence, and the attacker’ s goals and capabilities. Then, we categorize the applications of adversarial attack and defense methods in the cyber security domain. Finally, we highlight some characteristics identified in recent research and discuss the impact of recent advancements in other adversarial learning domains on future research directions in the cyber security domain. To the best of our knowledge, this work is the first to discuss the unique challenges of implementing end-to-end adversarial attacks in the cyber security domain, map them in a unified taxonomy, and use the taxonomy to highlight future research directions.

The query-performance prediction task is to estimate retrieval effectiveness with no relevance judgments. Pre-retrieval prediction methods operate prior to retrieval time. Hence, these predictors are often based on analyzing the query and the corpus upon which retrieval is performed. We propose a {\em corpus-independent} approach to pre-retrieval prediction which relies on information extracted from Wikipedia. Specifically, we present Wikipedia-based features that can attest to the effectiveness of retrieval performed in response to a query {\em regardless} of the corpus upon which search is performed. Empirical evaluation demonstrates the merits of our approach. As a case in point, integrating the Wikipedia-based features with state-of-the-art pre-retrieval predictors that analyze the corpus yields prediction quality that is consistently better than that of using the latter alone.

A context-aware recommender system (CARS) applies sensing and analysis of user context to provide personalized services. The contextual information can be driven from sensors in order to improve the accuracy of the recommendations. Yet, generating accurate recommendations is not enough to constitute a useful system from the users' perspective, since certain contextual information may cause different issues, such as draining the user's battery, privacy issues, and more. Adding high-dimensional contextual information may increase both the dimensionality and sparsity of the model. Previous studies suggest reducing the amount of contextual information by selecting the most suitable contextual information using a domain knowledge. Another solution is compressing it into a denser latent space, thus disrupting the ability to explain the recommendation item to the user, and damaging users' trust. In this paper we present an approach for selecting low-dimensional subsets of the contextual information and incorporating them explicitly within CARS. Specifically, we present a novel feature-selection algorithm, based on genetic algorithms (GA), that outperforms SOTA dimensional-reduction CARS algorithms, improves the accuracy and the explainability of the recommendations, and allows for controlling user aspects, such as privacy and battery consumption. Furthermore, we exploit the top subsets that are generated along the evolutionary process, by learning multiple deep context-aware models and applying a stacking technique on them, thus improving the accuracy while remaining at the explicit space. We evaluated our approach on two high-dimensional context-aware datasets driven from smartphones. An empirical analysis of our results validates that our proposed approach outperforms SOTA CARS models while improving transparency and explainability to the user.

Traffic measurements, road safety studies, and surveys are required for efficient road planning and ensuring the safety of transportation. Unfortunately, these methods can be cumbersome and very expensive. In this paper we point out a source of transportation information that is based on collaborative community-based navigation applications, such as Waze. Partial and anonimized information publicly exposed by Waze through their application provides valuable information that can significantly ease the future of transportation studies. Moreover, we show that Waze user reports may expose locations plagued with accidents but in lacking police coverage. This knowledge may help police departments to improve road safety by relocating the police units to these locations. Lastly, the data discussed in this paper connects transportation and road safety research to location based services and social network platforms.

Classifying medical reports written in Hebrew is challenging due to the ambiguity and complexity of the language. This study proposes Text Test Time Augmentation (TTTA), a novel method to improve the classification accuracy of cancer severity levels from PET-CT diagnostic reports in Hebrew. Hebrew, being a morphologically rich language, often leads to each word having multiple ambiguous interpretations. TTTA leverages test-time augmentation to enhance text information retrieval and model robustness. During training and testing phases, this method generates and evaluates sets of augmentations to enhance the semantics extracted from each report. Experiments utilize a large institutional report repository from Ziv hospital, Israel, where physicians manually labeled the reports. The results demonstrate that the proposed TTTA approach achieves superior performance over baseline models without TTA, improving PR-AUC by 15.18% on classifying cancer severity levels. The study highlights the efficacy of TTTA in extracting essential medical concepts from free text reports and accurately classifying the severity of cancer. The approach addresses the limitations of prior methods and contributes towards improved automated analysis of Hebrew medical reports. TTTA has the potential to assist physicians in cancer diagnosis and treatment planning.

In this paper, we present a generic, query-efficient black-box attack against API call-based machine learning malware classifiers. We generate adversarial examples by modifying the malware's API call sequences and non-sequential features (printable strings), and these adversarial examples will be misclassified by the target malware classifier without affecting the malware's functionality. In contrast to previous studies, our attack minimizes the number of malware classifier queries required. In addition, in our attack, the attacker must only know the class predicted by the malware classifier; attacker knowledge of the malware classifier's confidence score is optional. We evaluate the attack effectiveness when attacks are performed against a variety of malware classifier architectures, including recurrent neural network (RNN) variants, deep neural networks, support vector machines, and gradient boosted decision trees. Our attack success rate is around 98% when the classifier's confidence score is known and 64% when just the classifier's predicted class is known. We implement four state-of-the-art query-efficient attacks and show that our attack requires fewer queries and less knowledge about the attacked model's architecture than other existing query-efficient attacks, making it practical for attacking cloud-based malware classifiers at a minimal cost.

ABSTRACT Background One of the limiting toxicities of BTKi is the development of atrial fibrillation (AF), with an incidence of 3%–16%. Aim This study aimed to identify patients with chronic lymphocytic leukemia (CLL) starting both first‐ and second‐generation BTKis who are at high risk of developing AF using a machine learning approach. Methods The CLL cohort is based on data obtained from electronic medical records from Maccabi, the second‐largest healthcare organization in Israel. The optimal scoring model was determined using the Risk‐calibrated Supersparse Linear Integer Model (RiskSLIM) algorithm. Results A total of 3964 patients with a CLL diagnosis were available in the database. Of these, 208 patients started BTKi during the study period (125 on ibrutinib and 83 on acalabrutinib), and 16 patients developed AF during follow‐up. In addition to well‐established factors such as age, sex, and hypertension, the algorithm detected other factors associated with a high risk for AF: type of BTKi used, low eGFR, elevated absolute monocytes (> 1100/μL), elevated CRP, elevated CK, and elevated B2MG (> 2.5 mg/L). Based on the total AF‐free survival (AFS), we identified three main risk groups: low (0–6), intermediate (7–11) and high (≥ 12). The median AF‐free survival (AFS) was 28 and 56 months for the high‐risk and intermediate‐risk groups, respectively, and was not reached for the low‐risk group. The difference between the groups was statistically significant ( p = 0.0013). Conclusion Our novel score has a high concordance index for predicting the development of AF in patients with CLL treated with first‐ and second‐generation BTKis. It combines age, sex, medical history, and laboratory tests associated with inflammatory status and disease burden and can be applied in clinical settings worldwide.

Image understanding relies heavily on accurate multi-label classification. In recent years, deep learning (DL) algorithms have become very successful tools for multi-label classification of image objects, and various implementations of DL algorithms have been released for public use in the form of application programming interfaces (APIs). In this study, we evaluate and compare 10 of the most prominent publicly available APIs in a best-of-breed challenge. The evaluation is performed on the Visual Genome labeling benchmark dataset using 12 well-recognized similarity metrics. In addition, for the first time in this kind of comparison, we use a semantic similarity metric to evaluate the semantic similarity performance of these APIs. In this evaluation, Microsoft's Computer Vision, TensorFlow, Imagga, and IBM's Visual Recognition performed better than the other APIs. Furthermore, the new semantic similarity metric provided deeper insights for comparison.