Ambari LDAP setup and Kerberos Setup

 Ambari steps to configure LDAP

Note: These steps performed on Ambari version 2.2.2 with HDP 2.3.2 hortonworks hadoop version.
  1. Configure /etc/ambari-server/conf/ambari.properties

Used ambari-ldap-ad.sh to update ambari.properties file

cat <<-‘EOF’ | sudo tee -a /etc/ambari-server/conf/ambari.properties
authentication.ldap.baseDn=dc=example,dc=com
authentication.ldap.managerDn=cn=ldap-connect,ou=users,ou=hdp,dc=example,dc=com
authentication.ldap.primaryUrl=activedirectory.example.com:389
authentication.ldap.bindAnonymously=false
authentication.ldap.dnAttribute=distinguishedName
authentication.ldap.groupMembershipAttr=member
authentication.ldap.groupNamingAttr=cn
authentication.ldap.groupObjectClass=group
authentication.ldap.useSSL=false
authentication.ldap.userObjectClass=user
authentication.ldap.usernameAttribute=sAMAccountName
EOF

Now run the command:

#ambari-server setup-ldap

It will read all required properties from the ambari.properties file which got setup above. Some important properties are:

Primary URL* (host:port): (activedirectory.example.com:389)

Base DN* (dc=example,dc=com)

Manager DN* (cn=ldap-connect,ou=users,ou=hdp,dc=example,dc=com)

Enter Manager Passwrod*: ******

Re-Enter passwrod: ******

ambari-server start
 Ambari LDAP SYNC

create users.csv or groups.csv with required users and groups to be sync with Ambari.

echo “user1,user2,user3” > users.txt

echo “group1,group2,group3” > groups.txt

ambari-server sync-ldap --user users.txt

ambari-server sync-ldap --group groups.txt

Enter Ambari Admin login: admin

Enter Ambari Admin password: *******

AMBARI KERBEROS SETUP

Pre requisite: Get the Service Principal (Ad service account if AD is configured for Kerberos)

Steps to create new service principal, setpassword and create keytab  (AD with centrify configuration)
  1.  Create Ambari Service Principal (Service account in Active directory, typically we take help of AD admin team to create this AD service account)

ambari-adm@example.com

adkeytab --new --upn ambari-adm@example.com --keytab ambari-adm.keytab --container "OU=Hadoop,OU=Application,DC=example,DC=com" -V ambari-adm --user adadmin@example.com --ignore

3. Set passwod for the new principal (Ad service account)

adpasswd  -a adadmin@example.com ambari-adm@example.com

4.Generate Keytab file for this user account (Again AD admin will help)

adkeytab -A --ignore -u adadmin@example.com -K ambari-adm.keytab -e arcfour-hmac-md5 --fource --newpassword P@$$w0rd -S ambari-adm  ambari-adm -V
Now setup ambari with kerberos
 ambari-server setup-security

Select option: 3

Setup Ambari Kerberos JAAS configuration.

Enter Ambari Server’s kerberos Principal Name: amabri-adm@example.com

Enter keytab path: /root/ambari-adm@example.com

Note: keep 600 permissions the keytab file

Once setup is done, need to configure kerberos principal

Hive View configuration:

Hive Authentication=auth=KERBEROS;principal=hive/<hive host fqdn>@EXAMPLE.COM;hive.server2.proxy.user=$(username)

WebHDFS Authentication=auth=KERBEROS;proxyuser=ambari-adm@EXAMPLE.COM

It requires proxy user configuration (personification) in HADOOP configuration: setup_HDFS_proxy_user

 

hadoop.proxyuser.ambari-adm.hosts=*
hadoop.proxyuser.ambari-adm.groups=*
hadoop.proxyuser.ambari-adm.users=*

 

Advertisements

Set Up LDAP Authentication – Ambari

https://ambari.apache.org/1.2.1/installing-hadoop-using-ambari/content/ambari-chap2-4.html

vi /etc/ambari-server/conf/ambari.properties

client.security=ldap

ambari-server setup-ldap
Using python  /usr/bin/python
Setting up LDAP properties...
Primary URL* {host:port} (adserver.abc.com:3268):
Secondary URL {host:port} :
Use SSL* [true/false] (false):
User object class* (user):
User name attribute* (sAMAccountName):
Group object class* (group):
Group name attribute* (cn):
Group member attribute* (member):
Distinguished name attribute* (distinguishedName):
Base DN* (dc=abc,dc=com):
Referral method [follow/ignore] :
Bind anonymously* [true/false] (false):
Manager DN* (cn=<AD service account>,OU=Hadoop,OU=Applications,DC=abc,DC=com):
Enter Manager Password* :
Re-enter password:
====================
Review Settings
====================
authentication.ldap.managerDn: cn=<AD service account>,OU=Hadoop,OU=Applications,DC=abc,DC=com
authentication.ldap.managerPassword: *****
Save settings [y/n] (y)? y
Saving...done
Ambari Server 'setup-ldap' completed successfully.

To Sync the groups.
vi groups.csv
<add all the ad groups which need to be sync with ambari>

ambari-server sync-ldap --groups groups.csv

To Sync the users. create users.csv file with list of ad user accounts separated by comma.
ambari-server sync-ldap --user users.csv

hadoop: How to take ambari backup

AMBARI backups can be taken using posgresql backup command pg_dump.

AMBARI BACKUP

ambari-server stop
  • Stop the original Ambari Server.
    ambari-server stop
  • Create a directory to hold the database backups.
    cd /tmp
    mkdir dbdumps
    cd dbdumps/
  • Create the database backups.
    pg_dump -U $AMBARI-SERVER USERNAME ambari > ambari.sql Password: $AMBARI-SERVER PASSWORD
    pg_dump -U $MAPRED USERNAME ambarirca > ambarirca.sql Password: $MAPRED PASSWORD

Postgresql backup commands from its man page:

EXAMPLES
To dump a database called mydb into a SQL-script file:

$ pg_dump mydb > db.sql

To reload such a script into a (freshly created) database named newdb:

$ psql -d newdb -f db.sql

To dump a database into a custom-format archive file:

$ pg_dump -Fc mydb > db.dump

To reload an archive file into a (freshly created) database named newdb:

$ pg_restore -d newdb db.dump

To dump a single table named mytab:

$ pg_dump -t mytab mydb > db.sql

To  dump  all  tables whose names start with emp in the detroit schema, except for the table
named employee_log:

$ pg_dump -t ’detroit.emp*’ -T detroit.employee_log mydb > db.sql

To dump all schemas whose names start with east or  west  and  end  in  gsm,  excluding  any
schemas whose names contain the word test:

$ pg_dump -n ’east*gsm’ -n ’west*gsm’ -N ’*test*’ mydb > db.sql

The same, using regular expression notation to consolidate the switches:

$ pg_dump -n ’(east|west)*gsm’ -N ’*test*’ mydb > db.sql

To dump all database objects except for tables whose names begin with ts_:

$ pg_dump -T ’ts_*’ mydb > db.sql

To  specify an upper-case or mixed-case name in -t and related switches, you need to double-
quote the name; else it will be folded to lower case (see Patterns  [psql(1)]).  But  double
quotes are special to the shell, so in turn they must be quoted.  Thus, to dump a single ta-
ble with a mixed-case name, you need something like

$ pg_dump -t ’”MixedCaseName”’ mydb > mytab.sql

 

 

Want to Try it out AMBARI (Management layer on top of Hadoop to deploy and manage Hadoop Stack based on puppet)

Organizations can benefit from a management layer on top of Hadoop. Ambari helps to build management layer quickly.Here is the 2 quick start guides both are using Oracle Virtual box and vagrant for quick setup.

Quick Start Guide – Installing a cluster with Ambari (with local VMs)

This document shows how to quickly set up a cluster using Ambari on your local machine using virtual machines.
This utilizes VirtualBox and Vagrant so you will need to install both.

https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide

Hortonworks quick guide

http://hortonworks.com/hadoop-tutorial/introducing-apache-ambari-deploying-managing-apache-hadoop/