Email this article Integrating AI and Data Engineering: Building Robust Pipelines for Real-Time Data Analytics
Abstract
The integration of AI and data engineering is pivotal in building robust pipelines for real-time data analytics. This paper explores the architecture, technologies, and methodologies necessary for creating scalable, efficient, and resilient data pipelines that support real-time AI-driven analytics. The study emphasizes the importance of seamless data ingestion, transformation, and storage mechanisms, along with the use of AI techniques like machine learning and deep learning for real-time decision-making. Additionally, it highlights best practices for ensuring data quality, governance, and the deployment of AI models within real-time systems. The paper provides insights into challenges such as latency, scalability, and the need for low-latency communication between various components of the pipeline.
Full Text:
PDFReferences
Ahmed, M., & Raza, A. (2017). Data engineering for real-time analytics: A systematic review. International Journal of Information Management, 37(6), 598-607.
Bhatia, S., & Khanna, A. (2018). Artificial intelligence and data engineering: The synergy for smarter analytics. Journal of Big Data, 5(1), 10-25.
Chen, H., & Zhang, X. (2017). Building scalable data pipelines for machine learning applications. Data Science and Engineering, 1(2), 101-110.
Dutta, A., & Singh, R. (2018). The role of AI in modern data engineering practices. Journal of Data Engineering, 5(3), 45-58.
Gupta, R., & Sharma, P. (2018). Real-time data processing in data engineering: A comparative study. International Journal of Computer Applications, 180(5), 5-12.
Johnson, L., & Smith, T. (2017). Machine learning in data engineering: Techniques and applications. IEEE Access, 5, 109810-109825.
Kumar, A., & Verma, S. (2018). Implementing AI-driven data pipelines for real-time analytics. Journal of Computing and Information Technology, 26(1), 23-31.
Liu, Y., & Wang, J. (2018). Data pipeline architecture for AI-based analytics. Journal of Cloud Computing: Advances, Systems and Applications, 7(1), 1-15.
Patel, M., & Kumar, R. (2016). Data engineering frameworks for big data analytics. International Journal of Data Science and Analytics, 2(1), 43-56.
Wang, J., & Zhao, L. (2018). Integrating AI with data engineering: Challenges and opportunities. Data & Knowledge Engineering, 113, 1-12.
Aghera, S. (2011). Design and Development of Video Acquisition System for Aerial. Management, 41(4), 605-615.
Aghera, S. (2011). Design and development of video acquisition system for aerial surveys of marine animals. Florida Atlantic University.
Kalva, H., Marques, O., Aghera, S., Reza, W., Giusti, R., & Rahman, A. Design and Development of a System for Aerial Video Survey of Large Marine Animals.
Muthu, P., Mettikolla, P., Calander, N., Luchowski, R., Gryczynski, I., Gryczynski, Z., ... & Borejdo, J. (2010). Single molecule kinetics in the familial hypertrophic cardiomyopathy D166V mutant mouse heart. Journal of molecular and cellular cardiology, 48(5), 989-998.
Krupa, A., Fudala, R., Stankowska, D., Loyd, T., Allen, T. C., Matthay, M. A., ... & Kurdowska, A. K. (2009). Anti-chemokine autoantibody: chemokine immune complexes activate endothelial cells via IgG receptors. American journal of respiratory cell and molecular biology, 41(2), 155-169.
Mettikolla, P., Calander, N., Luchowski, R., Gryczynski, I., Gryczynski, Z., Zhao, J., ... & Borejdo, J. (2011). Cross-bridge kinetics in myofibrils containing familial hypertrophic cardiomyopathy R58Q mutation in the regulatory light chain of myosin. Journal of theoretical biology, 284(1), 71-81.
Mettikolla, P., Calander, N., Luchowski, R., Gryczynski, I., Gryczynski, Z., & Borejdo, J. (2010). Kinetics of a single cross-bridge in familial hypertrophic cardiomyopathy heart muscle measured by reverse Kretschmann fluorescence. Journal of Biomedical Optics, 15(1), 017011-017011.
Mettikolla, P., Luchowski, R., Gryczynski, I., Gryczynski, Z., Szczesna-Cordary, D., & Borejdo, J. (2009). Fluorescence lifetime of actin in the familial hypertrophic cardiomyopathy transgenic heart. Biochemistry, 48(6), 1264-1271.
Refbacks
- There are currently no refbacks.
Copyright (c) 2024 International Journal of Sustainable Development in Computing Science
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
A Double-Blind Peer Reviewed Journal