This is the documentation for CDH 4.6.0.
Documentation for other versions is available at Cloudera Documentation.

HttpFS Security Configuration

This section describes how to configure HttpFS CDH4 with Kerberos security on a Hadoop cluster:

For more information about HttpFS, see http://archive.cloudera.com/cdh4/cdh/4/hadoop/hadoop-hdfs-httpfs/index.html.

  Important:

To enable HttpFS to work with Kerberos security on your Hadoop cluster, make sure you perform the installation and configuration steps in Configuring Hadoop Security in CDH4.

  Important:

If the NameNode, Secondary NameNode, DataNode, JobTracker, TaskTrackers, ResourceManager, NodeManagers, HttpFS, or Oozie services are configured to use Kerberos HTTP SPNEGO authentication, and two or more of these services are running on the same host, then all of the running services must use the same HTTP principal and keytab file used for their HTTP endpoints.

Configuring the HttpFS Server to Support Kerberos Security

  1. Create an HttpFS service user principal that is used to authenticate with the Hadoop cluster. The syntax of the principal is: httpfs/<fully.qualified.domain.name>@<YOUR-REALM> where: fully.qualified.domain.name is the host where the HttpFS server is running YOUR-REALM is the name of your Kerberos realm
    kadmin: addprinc -randkey httpfs/fully.qualified.domain.name@YOUR-REALM.COM
  2. Create a HTTP service user principal that is used to authenticate user requests coming to the HttpFS HTTP web-services. The syntax of the principal is: HTTP/<fully.qualified.domain.name>@<YOUR-REALM> where: 'fully.qualified.domain.name' is the host where the HttpFS server is running YOUR-REALM is the name of your Kerberos realm
    kadmin: addprinc -randkey HTTP/fully.qualified.domain.name@YOUR-REALM.COM
      Important:

    The HTTP/ component of the HTTP service user principal must be upper case as shown in the syntax and example above.

  3. Create keytab files with both principals.
    $ kadmin
    kadmin: xst -k httpfs.keytab httpfs/fully.qualified.domain.name
    kadmin: xst -k http.keytab HTTP/fully.qualified.domain.name
  4. Merge the two keytab files into a single keytab file:
    $ ktutil
    ktutil: rkt httpfs.keytab
    ktutil: rkt http.keytab
    ktutil: wkt httpfs-http.keytab
  5. Test that credentials in the merged keytab file work. For example:
    $ klist -e -k -t httpfs-http.keytab
  6. Copy the httpfs-http.keytab file to the HttpFS configuration directory. The owner of the httpfs-http.keytab file should be the httpfs user and the file should have owner-only read permissions.
  7. Edit the HttpFS server httpfs-site.xml configuration file in the HttpFS configuration directory by setting the following properties:

    Property

    Value

    httpfs.authentication.type

    kerberos

    httpfs.hadoop.authentication.type

    kerberos

    httpfs.authentication.kerberos.principal

    HTTP/<HTTPFS-HOSTNAME>@<YOUR-REALM.COM>

    httpfs.authentication.kerberos.keytab

    /etc/hadoop-httpfs/conf/httpfs-http.keytab

    httpfs.hadoop.authentication.kerberos.principal

    httpfs/<HTTPFS-HOSTNAME>@<YOUR-REALM.COM>

    httpfs.hadoop.authentication.kerberos.keytab

    /etc/hadoop-httpfs/conf/httpfs-http.keytab

    httpfs.authentication.kerberos.name.rules

    Use the value configured for 'hadoop.security.auth_to_local' in 'core-site.xml'

      Important:

    You must restart the HttpFS server to have the configuration changes take effect.

Using curl to access an URL Protected by Kerberos HTTP SPNEGO

  Important:

Your version of curl must support GSS and be capable of running curl -V.

To configure curl to access an URL protected by Kerberos HTTP SPNEGO:

  1. Run curl -V:
    $ curl -V
    curl 7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l
    zlib/1.2.3
    Protocols: tftp ftp telnet dict ldap http file https ftps
    Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz
  2. Login to the KDC using kinit.
    $ kinit
    Please enter the password for tucu@LOCALHOST:
  3. Use curl to fetch the protected URL:
    $ curl --negotiate -u : -b ~/cookiejar.txt -c ~/cookiejar.txt http://localhost:14000/webhdfs/v1/?op=liststatus

    where: The --negotiate option enables SPNEGO in curl. The -u : option is required but the user name is ignored (the principal that has been specified for kinit is used). The -b and -c options are used to store and send HTTP cookies.