Tag: apache hadoop

Apache Flink – Flink vs Spark vs Hadoop

Apache Flink – Flink vs Spark vs Hadoop

Here is a comprehensive table, which shows the comparison between three most popular big data frameworks: Apache Flink, Apache Spark and Apache Hadoop. Apache Hadoop Apache Spark Apache Flink Year of Origin 2005 2009 2009 Place of Origin MapReduce (Google) Hadoop (Yahoo) University of California, Berkeley Technical University of Berlin Data Processing Engine Batch Batch … Continue reading Apache Flink – Flink vs Spark vs Hadoop

The Hadoop Module & High-level Architecture

The Hadoop Module & High-level Architecture

The Apache Hadoop Module: Hadoop Common: this includes the common utilities that support the other Hadoop modules HDFS: the Hadoop Distributed File System provides unrestricted, high-speed access to the application data. Hadoop YARN: this technology accomplishes scheduling of job and efficient management of the cluster resource. MapReduce: highly efficient methodology for parallel processing of huge … Continue reading The Hadoop Module & High-level Architecture

Hadoop vs Spark – Choosing the Right Big Data Software

Hadoop vs Spark – Choosing the Right Big Data Software

Considered competitors or enemies in Big Data space by many, Apache Hadoop and Apache Spark are the most looked-for technologies and platforms for big data analytics. More interestingly, in the present time, companies that have been managing and performing big data analytics using Hadoop have also started implementing Spark in their everyday organizational and business … Continue reading Hadoop vs Spark – Choosing the Right Big Data Software

Real-time Big Data Pipeline with Hadoop, Spark & Kafka

Real-time Big Data Pipeline with Hadoop, Spark & Kafka

Defined by 3Vs that are velocity, volume, and variety of the data, big data sits in the separate row from the regular data. Though big data was the buzzword since last few years for data analysis, the new fuss about big data analytics is to build up real-time big data pipeline. In a single sentence, … Continue reading Real-time Big Data Pipeline with Hadoop, Spark & Kafka

Taking the hard work out of Apache Hadoop

Taking the hard work out of Apache Hadoop

Why did IBM decide to create its own Hadoop and Spark distribution, and why does it need a reference architecture? The ability to collect, manage and analyze big data is one of the key tenets of the IBM cognitive business strategy, as well as being central to the Internet of Things. We see a lot … Continue reading Taking the hard work out of Apache Hadoop