Provisioning a Cluster with Chef Server

{section}
{column:width=75%}
These instructions outline the steps for installing LucidWorks Big Data on a local cluster using Chef Server. This is the preferred method of installation, and is the simplest approach. However, if you cannot use Chef Server because your nodes do not have access to the internet, please see the instructions on [bigdata:Provisioning a Cluster with Chef Solo].

{note}
These instructions are designed for a three-node cluster, but can be easily extended for larger clusters.
{note}

Before starting the installation, please review the section [bigdata:Planning Your Cluster]. It is important to have a plan for how the cluster will be structured, and which services will be co-located, before starting this process.

The installation process is simpler with Chef Server than with Chef Solo, but still requires a number of steps. After reviewing the system requirements and setting up each node of the cluster, you will install Chef Server, configure Knife, and then finally, set up LucidWorks Big Data with a series of {{knife}} commands.
{column}
{column:width=25%}
{panel}
Topics in this section:
{toc:maxLevel=2}
{panel}
{column}
{section}

h2. System Requirements

h3. Ubuntu
* Ubuntu 12.04 installed on all virtual machines (VMs) and hosting boxes, using {{ubuntu-12.04-server-amd64.iso}}. During installation:
** Install SSH Server
** Create user 'ubuntu' with password of 'lucid'
** Set UTC set as time zone
* [Oracle Java JDK v6.x|http://www.oracle.com/technetwork/java/javase/overview/index.html]
* Internet access is required on all nodes for the installation of system packages

h3. CentOS

* CentOS 5.8 installed on all virtual machines and hosting boxes. During installation:
** Install {{xfsprogs}}
** Install {{zip}} and {{unzip}}
** Set UTC set as time zone
* [Oracle Java JDK v6.x|http://www.oracle.com/technetwork/java/javase/overview/index.html]
* Internet access is required on all nodes for the installation of system packages

h3. Installing Oracle JDK 6.x

# Download the Java JDK {{.bin}} file from [http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloads-javase6-419409.html].
# Copy {{.bin}} file to the server.
# Install it as {{./filename.bin}}. Accept the license and proceed with the installation.

{note}Due to compatibility requirements, OpenJDK is not supported.{note}

h2. Pre-Requisites

h3. Virtualization Software

The cluster installation is done with a prepared virtual machine with Chef Server and the cookbooks to install LucidWorks Big Data. To use the VM, you should have [Oracle VM VirtualBox|https://www.virtualbox.org/], [VMWare Workstation|http://www.vmware.com/products/workstation/overview.html] or [VMWare ESXi|http://www.vmware.com/products/vsphere-hypervisor/overview.html] installed on the hosting box.

h3. Create Appliances

Create two appliances, a small and a large. The small should have 4Gb of memory and 20Gb of disk, while the large should have 12Gb of memory and 100Gb of disk.

h3. VNC

Install [x11vnc|http://www.karlrunge.com/x11vnc/] on the hosting box with the following steps:

# Install VNC packages with {{sudo apt-get install x11vnc vnc-java}}
# Set up a client password with {{x11vnc \-storepasswd}}
# Open ports 5800 and 5900 through your firewall
# Run {{sudo x11vnc \-auth guess \-forever \-usepw \-httpdir /usr/share/vnc-java/ \-httpport 5800 &}}

h2. Downloads and Credentials

After you have signed up for LucidWorks Big Data, you will be given instructions separately on how to access the virtual machine disk images.

Chef Server VM is distributed along with Lucidworks Big Data package to help provision a cluster on premise.

h2. Requirements

h3. Hardware Requirements

Following are the hardware requirements for the VM only. This is in addition to the hardware required to run the operating system and existing applications.
* CPU: Dual Core or higher
* Memory: 2 GB RAM or higher
* Disk Space: 20 GB Disk Space available for the VM

The VM is configured with a bridged network mode, which will automatically take an IP address from the local network. Accessing the web-based user interfaces or the APIs will require using either the IP or a hostname you specify for the VM.

h2. Set Up VM with Chef Server

We have included general instructions that can be used with any virtualization software package, and specific instructions using VirtualBox and VMWare.

h3. General Instructions

The general steps to install the virtual machine package are:

# Import the VM according to the instructions for your virtualization software.
# Login to the new machine with username 'ubuntu' and password 'lucid123'.
# Validate Chef Server with the steps in [#Validate Chef Server Installation].
# Copy the LucidWorks Big Data installation package to the other nodes of the cluster.
# Prepare Chef Server with the steps in [#Prepare Chef Server].

h3. Specific Instructions for Some Virtualization Tools

h4. Importing VM with VirtualBox

If you are using [VirtualBox|https://www.virtualbox.org/], follow these specific instructions to launch the VM.

# From the File menu, select "Import Appliance". This will open a Appliance Import Wizard. Click on *Choose* and browse to the directory which you downloaded the package, select the {{.ovf}} file.\\
\\ !vbox-01-importApplianceWizard.png|border=1!\\
\\
# The next screen will show "Appliance Import Settings" where you could customize the VM configurations. Its recommended to assign two or more CPUs and 2GB or more memory for this VM. Choose *Import* after reviewing the configurations. It will take a few minutes for the complete VM to be imported. Once done, it will show up in your VMs list in Oracle VM VirtualBox Manager.
\\
\\ !vbox-02-applianceImportSettings.png|border=1!\\
\\
# Start the VM and wait for it to boot. Once booted, you can login with username 'ubuntu' and password 'lucid123'.

h4. Importing VM with VMWare Workstation

If you are using [VMWare Workstation|http://www.vmware.com/products/workstation/overview.html], follow these instructions to launch the VM.

# From the main screen, select "Open Virtual Machine".\\
\\ !vmware-001openVirtualMachine.png|border=1!\\
\\
# Browse to the location on disk where you have extracted the VM package, select the {{.ovf}} file, and then select Import.\\
\\
It will take a few minutes for the VM to be imported into VMWare Workstation. Once complete, it will appear in the list of the VMs on the left side of the screen.\\
\\
# Power on the VM.\\
\\ !vmware-005powerOnVM.png|border=1!\\
\\
# While VM is being started, you may see a warning screen. Press Yes and continue.\\
\\ !vmware-006clickYesOnError.png|border=1!\\
\\
You should get a login prompt shortly. Your VM is imported and ready.

h4. Importing VM with VMWare ESXi

If you are using [VMWare ESXi|http://www.vmware.com/products/vsphere-hypervisor/overview.html], follow these instructions to launch the VM.

# Download and extract VM bundle.\\
\\
# Use {{ovftool}} to import the VM onto esxi with following syntax:\\
\\
{{ovftool \--lax <FILE_NAME>.ovf vi://username:pass@my_esx_host}}

h3. Validate Chef Server Installation

Once logged in, check the client list by running the {{knife}} command {{knife client list}}. You should get a list of 3 clients, like this:

{code:borderStyle=solid|borderColor=#666666}
chef-validator
chef-webui
root
{code}

h3. Prepare Chef Server

# Copy and extract LWBD installation package\\
\\
# Change into top level directory of LWBD installation package and run the following script to upload chef code onto chef server:\\
{code:borderStyle=solid|borderColor=#666666}
./chef_upload.sh
{code}
# Validate by listing roles and cookbooks
## {{knife role list}}
### Expected output: hadoop, sda, lwe roles
## {{knife cookbooks list}}
### Expected output: hadoop, zookeeper cookbooks

h2. Installation

# Set password-less sudo:
## {{sudo visudo}}
## Add {{ubuntu ALL=(ALL) NOPASSWD: ALL}} as the last line of the file. If there is another line with {{NOPASSWD}} above this one, remove it.
# Synchronize the date using {{ntpdate \-s \-b \-p 8 \-u 129.132.2.21}}
# Run {{sudo apt-get update}}
# Run {{sudo apt-get curl}}

h3. Prepare Client Nodes Before First Boot

On each client box, before the first time they are booted, perform the following steps:

# Select Bridged Adapter under "Network \-> Attached to"
# Select Paravirtualized Network under "Network \-> Adapter Type"
# Reinitialize MAC address under "Network \-> Mac Address"

h3. Configure Network Access

# Set Static IP. For example, assuming 172.16.10.199 as the 'master' node, {{/etc/network/interfaces}} might look like:
{code:borderStyle=solid|borderColor=#666666}
iface eth0 inet static
address 172.16.10.199
netmask 255.255.255.0
network 172.16.10.0
broadcast 172.16.10.255
gateway 172.16.10.1
{code}\\
# Make sure the DNS can be resolved by updating {{/etc/resolv.conf}}:
{code:borderStyle=solid|borderColor=#666666}
nameserver 4.2.2.2
nameserver 66.7.224.17
{code}\\
# Assign a unique hostname to each host in {{/etc/hostname}}
\\
\\
# Modify {{/etc/hosts}} so that each node can look up the other nodes and the Chef server:
{code:borderStyle=solid|borderColor=#666666}
172.16.10.197 sda
172.16.10.198 slave
172.16.10.199 master
172.16.10.189 chef
{code}

h3. System Usage Monitor

By default, the LucidWorks System Usage Monitor, which sends anonymous usage statistics to LucidWorks, will be enabled during provisioning. For more information about this feature, see the section [bigdata:System Usage Monitor]. If you would prefer to disable the monitor, you can edit the Chef cookbook on the master node that enables it during installation. In {{cookbooks/hadoop/attributes/default.rb}}, find the line:

{code:html|borderStyle=solid|borderColor=#666666}
default["heartbeat"]["enabled"] = true
{code}

Change {{true}} to {{false}}. Save the file, and then proceed with the rest of the installation.

h3. Launch Nodes

h4. Ubuntu

Run {{knife}} commands from chef server VM to install the LucidWorks Big Data services. Run the following commands in the order shown below, replacing the {{<IP_ADDRESS>}} in each command with the actual IP of each node. If multiple services will be installed on the same node ("kafka" and "sda", for example), you should still run the commands separately as below.

{code:borderStyle=solid|borderColor=#666666}
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[zabbix_server]" -N env01-zabbix-server
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[zookeeper]" -N env01-zk
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[hadoop_namenode],role[hadoop_jobtracker],role[hbase_master]" \
-N env01-NameNode-JobTracker-HbaseMaster
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[hadoop_slave]" -N env01-Hadoop_Slave
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[kafka]" -N env01-kafka
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[lwe-core],role[lwe-ui],role[lwe-connectors]" -N env01-lwe
knife bootstrap <IP_ADDRESS> -x ubuntu --sudo -E env01 \
-r "role[sda]" -N env01-sda
{code}

{tip}
If expanding beyond three nodes, you would add the additional IPs to {{/etc/hosts}}, then run the {{knife}} command to install the services for the additional node.
{tip}

h4. CentOS

Run {{knife}} commands from chef server VM to install the LucidWorks Big Data services. Run the following commands in the order shown below, replacing the {{<IP_ADDRESS>}} in each command with the actual IP of each node. If multiple services will be installed on the same node ("kafka" and "sda", for example), you should still run the commands separately as below.

{code:borderStyle=solid|borderColor=#666666}
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[zabbix_server]" -N env01-zabbix-server
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[zookeeper]" -N env01-zk
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[hadoop_namenode],role[hadoop_jobtracker],role[hbase_master]" -N env01-NameNode-JobTracker-HbaseMaster
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[hadoop_slave]" -N env01-Hadoop_Slave
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[kafka]" -N env01-kafka
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[lwe-core],role[lwe-ui],role[lwe-connectors]" -N env01-lwe
knife bootstrap <IP_ADDRESS> -x root -E env01 \
-r "role[sda]" -N env01-sda
{code}

{tip}
If expanding beyond three nodes, you would add the additional IPs to {{/etc/hosts}}, then run the {{knife}} command to install the services for the additional node.
{tip}

h3. Attach 'sdaevents'

The 'sdaevents' role (and associated service) should run on each node that hosts the HBase services ({{hbasemaster}} and {{regionserver}}). However, it can only be launched after all the other services. Once the other nodes have been launched, run this command:
{code:borderStyle=solid|borderColor=#666666}
knife node run_list add env01-sda "role[sdaevents]"
{code}

h2. Validating the Installation

Once the {{knife}} commands have been completed, you can validate the installation with any of the approaches described in [bigdata:Validating Installations].

h2. Related Topics

* [bigdata:Planning Your Cluster]
* [bigdata:Adding or Removing Nodes]
* [bigdata:Starting and Stopping Services]
* [bigdata:Logs and System Directories]

{navbar}