Category: HDFS

HDFS Features and Goals

HDFS Features and Goals

The Hadoop Distributed File System (HDFS) is a distributed file system. It is a core part of Hadoop which is used for data storage. It is designed to run on commodity hardware. Unlike other distributed file system, HDFS is highly fault-tolerant and can be deployed on low-cost hardware. It can easily handle the application that … Continue reading HDFS Features and Goals

Getting Started with Big Data Integration using HDFS and DMX-h

Getting Started with Big Data Integration using HDFS and DMX-h

Introduction The data researchers no longer depend only on interviews, surveys, observational studies to collect data. Instead, they have switched to the faster ways of data collection which includes leveraging internet, cameras, smartphones, drones, bots and many more. Later, the collected data is used by organization / governments to make business decisions. But, before that, … Continue reading Getting Started with Big Data Integration using HDFS and DMX-h

Hadoop Architecture – YARN, HDFS and MapReduce

Hadoop Architecture – YARN, HDFS and MapReduce

Hadoop Architecture In this post, we are going to discuss about Apache Hadoop 2.x Architecture and How it’s components work in detail. Hadoop 2.x Architecture Apache Hadoop 2.x or later versions are using the following Hadoop Architecture. It is a Hadoop 2.x High-level Architecture. We will discuss in-detailed Low-level Architecture in coming sections. Hadoop Common … Continue reading Hadoop Architecture – YARN, HDFS and MapReduce

The Hadoop Module & High-level Architecture

The Hadoop Module & High-level Architecture

The Apache Hadoop Module: Hadoop Common: this includes the common utilities that support the other Hadoop modules HDFS: the Hadoop Distributed File System provides unrestricted, high-speed access to the application data. Hadoop YARN: this technology accomplishes scheduling of job and efficient management of the cluster resource. MapReduce: highly efficient methodology for parallel processing of huge … Continue reading The Hadoop Module & High-level Architecture