Engineering and Information Technology Collected Works - Research Publications
Permanent URI for this collection
Search Results
1 - 3 of 3
-
ItemAn Uncertainty-Accuracy-Based Score Function for Wrapper Methods in Feature Selection(Institute of Electrical and Electronics Engineers, 2023)Feature Selection (FS) is an effective preprocessing method to deal with the curse of dimensionality in machine learning. Redundant features in datasets decrease the classification performance and increase the computational complexity. Wrapper methods are an important category of FS methods that evaluate various feature subsets and select the best one using performance measures related to a classifier. In these methods, the accuracy of classifiers is the most common performance measure for FS. Although the performance of classifiers depends on their uncertainty, this important criterion is neglected in these methods. In this paper, we present a new performance measure called Uncertainty-Accuracy-based Performance Measure for Feature Selection (UAPMFS) that uses an ensemble approach to measure both the accuracy and uncertainty of classifiers. UAPMFS uses bagging and uncertainty confusion matrix. This performance measure can be used in all wrapper methods to improve FS performance. We design two experiments to evaluate the performance of UAPMFS in wrapper methods. In experiments, we use the leave-one-variable-out strategy as the common strategy in wrapper methods to evaluate features. We also define a feature score function based on UAPMFS to rank and select features. In the first experiment, we investigate the importance of considering uncertainty in the FS process and show how neglecting uncertainty affects FS performance. In the second experiment, we compare the performance of the UAPMFS-based feature score function with the most common feature score functions for FS. Experimental results show the effectiveness of the proposed performance measure on different datasets.
-
ItemUncertainty in Selective Bagging: A Dynamic Bi-objective Optimization Model(Society for Industrial and Applied Mathematics, 2023-01)Bagging is a common approach in ensemble learning that generates a group of classifiers through bootstrapping for classification tasks. Despite its wide applications, generating redundant classifiers remains a central challenge in bagging. In recent years, many selective bagging models have been presented to deal with this challenge. These models mostly focused on the accuracy of classifiers and the diversity among them. Despite the importance of uncertainty in the performance of ensemble classifiers, this criterion has been neglected in selective bagging models. In this paper, we propose a two-stage selective bagging model. In the first stage, we formalize the selective bagging problem as a bi-objective optimization model considering both the uncertainty and accuracy of classifiers. We propose an adaptive evolutionary Two-Arch2 algorithm, named Diverse-Two-Arch2, to solve the bi-objective model. The output of this stage is a subset of classifiers that are diverse, certain about correct predictions, and uncertain about incorrect predictions. While most selective bagging models focus on the selection of a fixed subset of classifiers for all test samples (static approach), our proposed model has a dynamic approach to the selection process. So, in the second stage of the model, we select only certain classifiers to make an ensemble prediction for each test sample. Experimental results on twenty data sets and comparing with two ensemble models, and five state-of-the-art dynamic selective bagging models show the outperformance of the proposed model. We also compare the performance of the proposed Diverse-Two-Arch2 to alternative evolutionary computation methods.
-
ItemExpert-Machine Collaborative Decision Making: We Need Healthy Competition(IEEE COMPUTER SOC, 2022-09-01)Much has been written and discussed in previous years about human-AI interaction. However, the debate so far has mainly concentrated on "Aaverage" decision makers, neglecting important differences when it is experts who require support. In this article, we are going to talk about expert-machine collaboration for decision-making. We investigate the current approaches for expert decision support and exemplify the inefficiency of this approach for a real clinical decision-making problem. We propose two solutions for expert-machine collaboration to overcome the shortcomings of the current state of the art. We think that the proposed approaches open new horizons for expert-machine collaborative decision-making.