Set up a Hadoop Cluster – Part 4

apache-ambari-projectThis is part 4 of my series about setting up an inexpensive Hadoop cluster using refurbished desktop PCs as Linux servers. (see here for parts 1, 2 and 3).

I installed HortonWorks HDP 2.0. As part of the installation, the main server also installs Ambari – an open source server application that is used to monitor and manage Hadoop clusters (hence my reason for adding a fourth server with more memory than the other three) and Nagios – a server monitoring application. Continue reading “Set up a Hadoop Cluster – Part 4”

Set Up a Hadoop Cluster – Part 2

StaticIPIn part 1 of this series, I talked about starting a project to set up a cluster of inexpensive refurbished PCs (cheap) to act as a test Hadoop cluster, and some of the issues I encountered in getting the operating system installed. With this part, I’ll talk a little bit about configuring these workstations to get them setup for Hadoop. (And I did want to add, I introduced a fourth PC into the mix, to act as the main Hadoop server – a Zotac Zbox PC with 4GB of memory.)

Static IPs

On all of the PCs, I added a static IP, using the DHCP assigned address. Before you start changing your connection info, I recommend opening a terminal window and entering the “ifconfig” command. This will show you the existing address for your PC, the gateway, the net mask and a lot of other info you probably won’t care about. Continue reading “Set Up a Hadoop Cluster – Part 2”

Set up a Hadoop Cluster – Part 1

cluster1I have posted a few times about various aspects of my experimenting with Hadoop and other Big Data platforms. Between Thanksgiving and Christmas, I even maxed out the memory of my desktop PC (a Mac Mini) so I could run multiple virtual machines to act as a Hadoop cluster. It worked, but performance was poor.

Since then, I’ve been looking into getting a small number of cheap PCs to setup as a physical cluster.  Continue reading “Set up a Hadoop Cluster – Part 1”