This is part 4 of a series about setting up a single-node Hadoop Yarn system for sandbox use. Part 1 was here, part 2 here, and part 3 here. I have another series for using MapReduceV1, which is here. I’m hoping to keep this series in a similar order as the original set of articles, and will deviate only when necessary. All the content here is based on the Cloudera documentation, but I’ve modified it to be easier to follow for setting up a pseudo cluster and added additional content where necessary.
Please be careful when copying lines from these articles to paste into Hadoop config files or a terminal window. I have found that the double hyphen characters used in the comment lines may copy over as a long hyphen instead. This is likely to cause issues when attempting to run the various components.
Before starting, make sure that Python 2.6 or 2.7 is installed on the server. This is easy to accomplish, by opening a terminal window, and from the command line, enter: python
If Python is installed, it will load up and display the version of the software. On my test PC, it responded with Python 2.6.6. Return to the command line by entering at the Python prompt: quit()
Switch to superuser mode, and start the installation of hue with this command:
yum install hue
Any needed dependencies will be located and installed along with HUE, so the time it will take will vary according to that and on your Internet connection, but on my system it took less than ten minutes to download and install 17 packages. (For MRv1 there is an additional package that is not needed with MRv2.)
Once it completes installation, and provided you had no errors, enter at a command prompt:
service hue start
Reopen a web browser and point it to your system name and port 8888 to access the HUE web interface. So for example, if your system was called HadoopTest, you would enter http://HadoopTest:8888 in your browser. You’ll be prompted for a user ID and password and you’ll be warned that this will become your administrator account, so be careful to remember what you enter.
HUE will open up with the Quick Start window, and probably will be displaying a number of problems with your configuration. That’s all normal, so don’t worry about it. You’ll need to configure HUE and several other applications to get it working properly.
Because we are setting up a pseudo-clustered machine, I’m assuming that you have direct access to the system and are not accessing the Hadoop machine from outside your local firewall. With that consideration, we don’t need to setup HttpFS on this machine.
Once again using the terminal window as root, edit the file:
/etc/hadoop/conf/hdfs-site.xml and add this property before the final </configuration> tag:
Save the file, exit the editor and restart your HDFS system with this command:
for x in `cd /etc/init.d ; ls hadoop-hdfs-*` ; do sudo service $x restart ; done
Now we need to configure the system to allow Hue to submit requests for any user or group with access to the system. As root from a terminal prompt, edit the file /etc/hadoop/conf/core-site.xml and add these two properties before the final </configuration> tag:
<!– Hue WebHDFS proxy user setting –>
Now edit the file: /etc/hue/conf/hue.ini. Near the top is a section called [desktop] with the first parameter there being secret_key. Again, because this system is not meant to be a production server, you can leave it blank or add a random string of characters after the secret_key=. I recommend adding some characters, because not having any will cause HUE to generate a warning later.
Now search for [[hdfs_clusters]] and look for a comment section below it that starts with “#Use WebHDFS/HttpFs as the communication mechanism“. At the end of that section, add this:
webhdfs_url=http://<full server name with domain>:50070/webhdfs/v2/
substituting the fully qualified domain name for your server where indicated. Save the file and exit back to a terminal prompt.
Restart the HUE server service hue start and at this point the HUE web interface is available, and you can access HBase through it. However, you will see several error messages when you login. We need to work through these next time.