Siddharth Chandracodekaro.hashnode.net·Dec 2, 2021Big Data Open Source FrameworksBig Data is a term used to define large scale data sets that are too complex to be manipulated with basic DBMS. Handling Big Data requires sophisticated hardware and software technologies. Just as open-source has been the primary reason for the Big D...Discuss·77 likes·369 readsScalabig data
Vikas Solegaonkarthewiz.hashnode.net·Dec 8, 2020Data Analytics on AWS — What, Why & HowThey say, Data is the new Oil! But, unlike the traditional oil, the Data is growing everyday — every second. Big data is bigger than ever, and it will grow a lot more, with the developments in 5G and IoT. The natural choice for storing and processing...Discuss·50 likes·166 reads2Articles1Week
Aman Anandamananandrai.hashnode.net·Dec 4, 2020Most Popular Tools for Data Scientists in 2020As the world entered the era of Big Data it was necessary to store this data and then technologies like Hadoop were used to solve this issue. Big data is a term used to describe a collection of data that is huge in size and yet growing exponentially ...Discuss·40 likes·124 readsMachine Learning
Karl Bolingerkbolinger.hashnode.net·Apr 24, 2023Understanding ETL and ELT Workflows in Data Engineering: An Easy Guide with ExamplesData engineering is a complex field where many different technologies, frameworks, and techniques come into play. Two of the most common data processing workflows data engineers use are ETL and ELT. ETL stands for Extract, Transform, and Load, while ...Discussdata-engineering
Islam O. Elgoharyiogohary.hashnode.net·Apr 24, 20235 Tips on Data Engineering2 years ago, I have developed an interest in data engineering and fortunately, I recently got a chance to work as a data engineer. In this article, I will write the takeaways from my experience and what I learned so far. What Is Data Engineering? Dat...Discuss·99 readsdata-engineering
Karl Bolingerkbolinger.hashnode.net·Apr 18, 2023SQL vs. NoSQL Databases: An Overview of Data Storage OptionsWhen it comes to data storage options, two main types of databases stand out: SQL and NoSQL. Each of these database types has its own strengths and weaknesses, and choosing between them often depends on a specific project's specific needs. In this ar...DiscussData Engineering Basicsdatabasemanagement
Siddhant Jhathetechwhiz.hashnode.net·Apr 10, 2023From Chaos to Insightful DecisionsBig Data and Machine Learning are two of the most important concepts in the world of technology today. They have the power to transform industries and create new opportunities for businesses. However, for many people, these concepts are still shroude...DiscussData Science
Elvis Davidtechml.hashnode.net·Apr 4, 2023Revolutionizing Data Integration with Airbyte: A Comprehensive GuideIn this tutorial, we'll give a comprehensive guide on Airbyte API, the benefits of using it, and a step-by-step guide. Introduction Data integration is an essential aspect of modern data-driven organizations. However, it can be a daunting task to con...Discuss·3 likesairbyte
Karl Bolingerkbolinger.hashnode.net·Mar 31, 2023Essential Skills for Data Engineers: Tools, Languages, and FrameworksTLDR: This article explores the essential skills required for data engineers to succeed, including proficiency in tools, languages, and frameworks. It provides real-world examples of these skills used in the music streaming and financial services ind...DiscussData Engineering Basicsdata-engineering
padmanabha reddypadmanabha.hashnode.net·Mar 31, 2023Apache Spark - CoreApache spark is a General-purpose, in-memory compute engine. It is a plug-and-play compute engine - we can plug spark with any storage system(S3, Local storage, HDFC etc..) and any resource manager(YARN, Kubernetes, Mesos, etc). Spark on top of Hadoo...Discussdata-engineering
Sanket Singhsanketsingh.hashnode.net·Mar 31, 2023Getting Started With Data EngineeringWhat is Data Engineering? It is a type of software engineering that focuses on designing, developing, testing and maintaining architectures such as databases and large-scale processing systems. It is a process of storing, processing and extracting in...Discussdata-engineering
Amul GauravforCling Multi Solutions blogcmsteam.hashnode.net·Mar 31, 2023A Comprehensive Guide to Big Data: Tools, Applications, and InfrastructureIntroduction to Big Data In the past decade, the term "Big Data" has become increasingly popular. It refers to the vast amounts of structured and unstructured data that organizations collect, process, and analyze daily. The volume, variety, and veloc...Discussbig data
Renjitha Krenjithak.hashnode.net·Mar 30, 2023Demystifying Big Data Analytics with Apache Spark : Part-1Posted by Renjitha K in Renjitha K's Blog on Mar 25, 2023 2:27:13 PM As the amount of data generated by individuals and businesses continue to grow exponentially, the need for technologies like Apache Spark that can process and analyze large dataset...Discuss·2 likes·101 readsspark
