Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

hardware requirement for hadoop installation on Laptop

I am planning to gain some experience on Hadoop , MapReduce and other big data things. Initially I want to install it on laptop. I was wondering what is best Laptop hardware recommended for installing Hadoop on windows. I assume I have to install a single node installation, if not please guide me the correct configuration. regards Nadeem

like image 499
Nhqazi Avatar asked Jun 17 '14 11:06

Nhqazi


People also ask

Can I run Hadoop on my laptop?

You can install Hadoop in your system as well which would be a feasible way to learn Hadoop. We will be installing single node pseudo-distributed hadoop cluster on windows 10. Prerequisite: To install Hadoop, you should have Java version 1.8 in your system. Download the file according to your operating system.

What kind of hardware is best for Hadoop?

What kind of hardware scales best for Apache Hadoop? Hadoop scales best with dual core machines or processors having 4 to 8 GB of RAM that use ECC memory based on the requirements of the workflow.


1 Answers

I know you mentioned wanting to install on Windows, but Cloudera is offering single-node Hadoop Linux VM images that can get you up and running in no time. You even have examples and scripts included, it's a very good place to start.

https://www.cloudera.com/downloads.html

(Don't forget to look at the Getting Started section.)

In my opinion, if you want to learn about Big Data and Hadoop, you should also invest some time in familiarising yourself with Linux, as most of the real environments out there are Linux-based.

System Requirements: Per Cloudera page, the VM takes 4GB RAM and 3GB of disk space. This means your laptop should have more than that (I'd recommend 8GB+). Storage-wise, as long as you have enough to test with small and medium-sized data sets (10s of GB), you'll be fine. As for the CPU, if your machine has that amount of RAM you'll most likely be fine. I'm using a single-node crappy Pentium G3210 with 4GB of ram for testing my small jobs and it works just fine.

Later if you outgrow this environment you can simply move to your own pseudo-distributed setup.

Of course, if what I just said makes no sense to you, then you have some reading ahead...

like image 70
Matt Fortier Avatar answered Oct 12 '22 09:10

Matt Fortier