Data cleaning using machine learning techniques /
Ayat Mahmoud Ahmed Mohamed
Data cleaning using machine learning techniques / تنقية البيانات باستخدام تقنيات تعلم الآلة Ayat Mahmoud Ahmed Mohamed ; Supervised Sherif Mazen , Ayman Elkilany , Farid Ali - Cairo : Ayat Mahmoud Ahmed Mohamed , 2021 - 79 Leaves : charts ; 30cm
Thesis (Ph.D.) - Cairo University - Faculty of Computers and Artificial Intelligence - Department of Information Systems
Data quality is one of the most important problems in data management, since corrupt data often leads to inaccurate data analytics results and wrong business decisions. Detecting and repairing dirty data is one of the perennial challenges in data analytics, and failure to do so can result in inaccurate analytics and unreliable decisions. In todays era of internet, the amount of data generation is growing and increasing, some of the data related to medical, e-commerce, social networking are of great importance. But many of these datasets are imbalanced that is some records belonging to same category are very large number and some are very rare. In other words, Imbalanced class distribution is a scenario where the number of observations belonging to one class is significantly lower than those belonging to the other classes. This problem is predominant in scenarios where anomaly detection is crucial like electricity pilferage, fraudulent transactions in banks, identification of rare diseases, etc. Most of the classical methods of machine learning algorithms have demonstrated shortcomings when used with imbalanced data. Conventional machine learning algorithms do not work well for imbalanced data classification because it assumes equal costs for each class.Thus, conventional machine learning algorithms could be biased and inaccurate.This thesis explores the nature of imbalanced data classification problem, introduces a survey on existing machine learning algorithms along with suggested taxonomy for all imbalanced data learning approaches.It also introduces a comparative study between the existing machine learning algorithms with respect to some factors. Then it proposes three solutions to the challenge of imbalanced data classification
Classification Data Cleaning Imbalanced
Data cleaning using machine learning techniques / تنقية البيانات باستخدام تقنيات تعلم الآلة Ayat Mahmoud Ahmed Mohamed ; Supervised Sherif Mazen , Ayman Elkilany , Farid Ali - Cairo : Ayat Mahmoud Ahmed Mohamed , 2021 - 79 Leaves : charts ; 30cm
Thesis (Ph.D.) - Cairo University - Faculty of Computers and Artificial Intelligence - Department of Information Systems
Data quality is one of the most important problems in data management, since corrupt data often leads to inaccurate data analytics results and wrong business decisions. Detecting and repairing dirty data is one of the perennial challenges in data analytics, and failure to do so can result in inaccurate analytics and unreliable decisions. In todays era of internet, the amount of data generation is growing and increasing, some of the data related to medical, e-commerce, social networking are of great importance. But many of these datasets are imbalanced that is some records belonging to same category are very large number and some are very rare. In other words, Imbalanced class distribution is a scenario where the number of observations belonging to one class is significantly lower than those belonging to the other classes. This problem is predominant in scenarios where anomaly detection is crucial like electricity pilferage, fraudulent transactions in banks, identification of rare diseases, etc. Most of the classical methods of machine learning algorithms have demonstrated shortcomings when used with imbalanced data. Conventional machine learning algorithms do not work well for imbalanced data classification because it assumes equal costs for each class.Thus, conventional machine learning algorithms could be biased and inaccurate.This thesis explores the nature of imbalanced data classification problem, introduces a survey on existing machine learning algorithms along with suggested taxonomy for all imbalanced data learning approaches.It also introduces a comparative study between the existing machine learning algorithms with respect to some factors. Then it proposes three solutions to the challenge of imbalanced data classification
Classification Data Cleaning Imbalanced