Tag: Apache Spark

The Hadoop Module & High-level Architecture

The Hadoop Module & High-level Architecture

The Apache Hadoop Module: Hadoop Common: this includes the common utilities that support the other Hadoop modules HDFS: the Hadoop Distributed File System provides unrestricted, high-speed access to the application data. Hadoop YARN: this technology accomplishes scheduling of job and efficient management of the cluster resource. MapReduce: highly efficient methodology for parallel processing of huge … Continue reading The Hadoop Module & High-level Architecture

Advertisements
Introduction to Apache Spark Dataset

Introduction to Apache Spark Dataset

1. Objective – Apache Spark Dataset In this blog on Apache Spark dataset, you can read all about what is dataset in Spark. Why the Spark DataSet needed, what is the encoder and what is their significance in the dataset? You will get the answer to all these questions in this blog. Moreover, we will … Continue reading Introduction to Apache Spark Dataset

Hadoop vs Spark – Choosing the Right Big Data Software

Hadoop vs Spark – Choosing the Right Big Data Software

Considered competitors or enemies in Big Data space by many, Apache Hadoop and Apache Spark are the most looked-for technologies and platforms for big data analytics. More interestingly, in the present time, companies that have been managing and performing big data analytics using Hadoop have also started implementing Spark in their everyday organizational and business … Continue reading Hadoop vs Spark – Choosing the Right Big Data Software

Real-time Big Data Pipeline with Hadoop, Spark & Kafka

Real-time Big Data Pipeline with Hadoop, Spark & Kafka

Defined by 3Vs that are velocity, volume, and variety of the data, big data sits in the separate row from the regular data. Though big data was the buzzword since last few years for data analysis, the new fuss about big data analytics is to build up real-time big data pipeline. In a single sentence, … Continue reading Real-time Big Data Pipeline with Hadoop, Spark & Kafka