Skip to main content

Big Data Analytics

 Big Data Analytics


CO1: Understanding BigData Defining Data, Types of Data, Structured Data, Semi-Structured Data, Unstructured Data, How data being Generated, Different sources of Data Generation, Rate at which data is being generated, Different V’s, Volume, Variety, Velocity, Veracity, Value, How a single person is contributing towards BigData, Significance for BigData, Reason for BigData, Understanding RDBMS and why it is failing to store BigData. Future of BigData, BigData use cases for major IT Industries.

PPT | Material | Sample Questions

CO2: Introduction to Hadoop What is Hadoop, Apache Community, Cluster, Node, Commodity Hardware, Rack Awareness, History of Hadoop, Need for Hadoop, How is Hadoop Important, Apache Hadoop Ecosystem, Different Hadoop offering, Hadoop 1.x Architecture, Apache Hadoop Framework, Master-Slave Architecture, Advantages of Hadoop. 

 PPT| Material | Sample Questions

CO3: Storage Unit Hadoop Distributed File System, Design of HDFS, HDFS Concept, How files are stored in HDFS, Hadoop File system, Replication factor, Name Node, Secondary Name Node, Job Tracker, Task Tracker, Data Node, FS Image, Edit-logs, Check-pointing Concept, HDFS Federation, HDFS High availability. Architectural description for Hadoop Cluster, When to use or not to use HDFS, Block Allocation in Hadoop Cluster, Read operation in HDFS, Write operation in HDFS, Hadoop Archives, Data Integrity in HDFS, Compression & Input Splits. 

Processing Unit What is MapReduce, History of MapReduce, How does MapReduce work, Input files, Input Format types Output Format Types, Text Input Format, Key-Value Input Format, Sequence File Input Format, Input split, Record Reader, MapReduce overview, Mapper Phase, Reducer Phase, Sort and Shuffle Phase, Importance of MapReduce, Data Flow, Counters, Combiner Function, Partition Function, Joins, Map Side Join, Reduce Side Join, MapReduce Web UI, Job Scheduling, Task Scheduling, Fault Tolerance, Writing MapReduce Application, Driver Class, Mapper Class, Reducer Class, Serialization, File-Based Data Structure, Writing a simple MapReduce program to Count Number of words, MapReduce WorkFlows.

PPT| Material | Sample Questions

CO4: YARN &Hadoop Cluster YARN, YARN Architecture, YARN Components, Resource Manager, Node Manager, Application Master, Concept of Container, Difference between Hadoop 1.x and 2.x Architecture, Execution of Job in Yarn Cluster, Comparing and Contrasting Hadoop with Relational Databases. Cluster Specification, Cluster Setup and Installation Creating Hadoop user Installing Hadoop, SSH Configuration, Hadoop Configuration, Hadoop daemon properties, Different modes of Hadoop, Standalone Mode, Pseudo Distributed Mode, Fully Distributed Modes.

PPT | Material | Sample Questions


Important Question cum Long Answer Questions

Comments

Post a Comment

Popular posts from this blog

Big Data Analytics Programs

  List of Programs for Big Data Analytics   CLICK ON ME 1.  Implement the following Data structures in Java       a)  Linked Lists            b)   Stacks       c)  Queues     d)   Set            e)   Map 2.  Perform setting up and Installing Hadoop in its three operating modes:      Standalone,     Pseudo distributed,     Fully distributed. 3.  Implement the following file management tasks in Hadoop:    a) Adding files and directories    b) Retrieving files    c) Deleting files 4. Run a basic Word Count Map Reduce program to understand Map Reduce Paradigm. 5. Write a Map Reduce program that mines weather data.     Weather sensors collecting data every hour at many locations across the globe gather a large volume of log data, which is a good candidate for analysis with MapReduce since it is semi-structured and record-oriented. 6. Implement Matrix Multiplication with Hadoop Map Reduce 7. Write a MapReduce program to count the occurrence of similar words in a file. Use partitioner to part
Post a Comment
Read more

How to Install Parrot Operating System in Virtual Box using OVA

Step by Step Process of Parrot OS Installation What is Parrot OS Parrot is a free and open-source Linux system based on Debian that is popular among security researchers, security experts, developers, and privacy-conscious users. It comes with cyber security and digital forensics arsenal that is totally portable. It also includes everything you'll need to make your own apps and protect your online privacy. Parrot is offered in Home and Security Editions, as well as a virtual machine and a Docker image, featuring the KDE and Mate desktop environments. Features of Parrot OS The following are some of the features of Parrot OS that set it apart from other Debian distributions: Tor, Tor chat, I2P, Anonsurf, and Zulu Crypt, which are popular among developers, security researchers, and privacy-conscious individuals, are included as pre-installed development, forensics, and anonymity applications. It has a separate "Forensics Mode" that does not mount any of the system's hard
Post a Comment
Read more

Word Count Map Reduce program

  Aim: Run a basic Word Count Map Reduce program to understand Map Reduce Paradigm   Program: Source Code import java.io.IOException; import java.util.StringTokenizer; import org.apache.hadoop.conf.Configuration;// provides access to configuration parameters import org.apache.hadoop.fs.Path;// Path class names a file or directory in a HDFS import org.apache.hadoop.io.IntWritable;// primtive Writable Wrapper class for integers. import org.apache.hadoop.io.Text;// This class stores text and provides methods to serialize, deserialize, and compare texts at byte level import org.apache.hadoop.mapreduce.Job;//Job class allows the user to configure the job, submit it, control its execution, and query the state //The Hadoop Map-Reduce framework spawns one map task for each InputSplit generated by the InputFormat for the job import org.apache.hadoop.mapreduce.Mapper;//Maps input key/value pairs to a set of intermediate key/value pairs. import org.apache.hadoop.mapred
Post a Comment
Read more