AI-Powered Feature Engineering in Data Science Pipelines Using Automated Feature Selection and Embedding Techniques
Keywords:
AI-Powered Feature Engineering, Automated Feature Selection, Embedding Techniques, Data Science Pipelines, Machine Learning, Feature ExtractionAbstract
Feature engineering is a crucial component in data science pipelines, enhancing the performance of machine learning models by transforming raw data into meaningful representations. Traditional feature selection methods are often manual and time-intensive, limiting scalability and efficiency. AI-powered feature engineering leverages automated feature selection, deep learning embeddings, and meta-learning frameworks to streamline feature extraction. This paper explores recent advancements in AI-driven feature selection techniques, compares traditional and automated approaches, and evaluates their impact on model performance and computational efficiency.
References
Rachakatla, SK, & Ravichandran, P. "Scalable Machine Learning Workflows in Data Warehousing: Automating Model Training and Deployment with AI." ResearchGate, 2022.
Mustapha, SD., et al. "Review of Advances in AI-Powered Monitoring and Diagnostics for CI/CD Pipelines." ResearchGate, 2024.
Zhang, F., et al. "OAG: Linking Entities Across Large-Scale Heterogeneous Knowledge Graphs." MDPI Applied Sciences, 2023.
Ramachandran, K. K. (2024). The role of artificial intelligence in enhancing financial data security. International Journal of Artificial Intelligence & Applications (IJAIAP), 3(1), 1–11.
Peters, L., & Han, J. "Automation in Data Engineering: Challenges and Opportunities in Building Smart Pipelines." ResearchGate, 2024.
Gao, H., & Liu, S. "AI-Powered Feature Engineering in Automated Machine Learning." IEEE Transactions on AI, 2023.
Chen, Y., & Sun, J. "Exploring Deep Learning-Based Feature Selection in Data Science Pipelines." Elsevier Expert Systems with Applications, 2023.
vinay, S. B. (2024). Identifying research trends using text mining techniques: A systematic review. International Journal of Data Mining and Knowledge Discovery (IJDMKD), 1(1), 1–11.
Huang, R., & Lin, D. "Graph-Based Feature Embedding for Complex Data Structures." ACM Computing Surveys, 2024.
Johnson, K., & Li, X. "Hyperparameter Optimization for Automated Feature Engineering." Springer AI Review, 2023.
Liu, Z., & Wang, C. "Deep Learning-Based Dimensionality Reduction for Large Datasets." ScienceDirect Journal of AI Research, 2022.
Vasudevan, K. (2024). The influence of AI-produced content on improving accessibility in consumer electronics. Indian Journal of Artificial Intelligence and Machine Learning (INDJAIML), 2(1), 1–11.
Jin, Y., & Zhang, L. "Neural Networks for Feature Selection: A Survey." arXiv Preprint, 2023.
