Balachandra Keley*

Department of Computer Sciences, United States

*Corresponding Author: Balachandra Keley, Department of Computer Sciences, United States.

Received: January 24, 2024; Published: January 29, 2024

Abstract

The paper delves into the challenges and opportunities presented by large and diverse datasets, discussing the flexibility of Data Lakes as reservoirs capable of storing raw data and serving as a foundation for advanced analytics. It addresses the scalability, schema-on-read architecture, and integration with big data technologies that characterize Data Lakes. The metaphorical concept of "Bridging Horizons" captures the essence of seamlessly connecting vast datasets and extracting actionable intelligence, promising a transformative impact on decision-making processes. The abstract invites readers to embark on a journey through the evolving landscape of data management, where the convergence of Big Data and Data Lakes opens new horizons for innovative insights and strategic advancements.

Keywords: Data Engineering; Big Data; Data Lake; Data Streaming; Kafka; Spark; Data Architecture; Metadata; Data Ingestion; Data Process; Technology

References

1. Miloslavskaya N and Tolstoy A. “Big data, fast data and data lake concepts”. Procedia Computer Science 88 (2016): 300-305.
2. Hai R., et al. “Constance: An intelligent data lake system”. In Proceedings of the 2016 international conference on management of data (2016): 2097-2100.
3. Siddiqa A., et al. “Big data storage technologies: a survey”. Frontiers of Information Technology and Electronic Engineering 18 (2017): 1040-1070.
4. Meehan J., et al. “Data Ingestion for the Connected World”. In CIDR 17 (2017): 8-11.
5. Subramaniam P., et al. “Comprehensive and comprehensible data catalogs: the what, who, where, when, why, and how of metadata management”. arXiv preprint arXiv:2103.07532 (2021).
6. Hendrycks D., et al. “Augmix: A simple data processing method to improve robustness and uncertainty”. arXiv preprint arXiv:1912.02781 (2019).
7. Souibgui M., et al. “Data quality in ETL process: A preliminary study”. Procedia Computer Science 159 (2019): 676-687.
8. Kelleher JD., et al. “Fundamentals of machine learning for predictive data analytics: algorithms, worked examples, and case studies”. MIT Press (2020).

Citation

Citation: Balachandra Keley. “Bridging Horizons: Navigating Big Data and Data Lakes for Unprecedented Insights". Acta Scientific Computer Sciences 6.2 (2024): 02-05.

Copyright

Copyright: © 2024 Balachandra Keley. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.