7.6.4. Kerberos authentication with syslog-ng hdfs() destination

Version 3.10 and later supports Kerberos authentication to authenticate the connection to your Hadoop cluster. syslog-ng OSE assumes that you already have a Hadoop and Kerberos infrastructure.

Prerequisites: 

  • You have configured your Hadoop infrastructure to use Kerberos authentication.

  • You have a keytab file and a principal for the host running syslog-ng OSE. For details, see the Kerberos documentation.

  • You have installed and configured the Kerberos client packages on the host running syslog-ng OSE. (That is, Kerberos authentication works for the host, for example, from the command line using the kinit user@REALM -k -t <keytab_file> command.)

destination d_hdfs {
    hdfs(client-lib-dir("/hdfs-libs/lib")
    hdfs-uri("hdfs://hdp-kerberos.syslog-ng.balabit:8020")
    kerberos-keytab-file("/opt/syslog-ng/etc/hdfs.headless.keytab")
    kerberos-principal("hdfs-hdpkerberos@MYREALM")
    hdfs-file("/var/hdfs/test.log"));
};