12.01.2021

Hortonworks Winutils

39

Mmi

  1. Hortonworks Hadoop Winutils
  2. Hortonworks Winutils.exe
  3. Hortonworks Winutils

We would like to show you a description here but the site won’t allow us.

Hortonworks Winutils
  • Java 8 : We are going to use the Java 8 Function interface and the hot Lambda expressions.
  • Maven 3 : Just to automate collecting the project dependencies.
  • Eclipse : My usual IDE for Java/JavaEE developments.

Hortonworks Hadoop Winutils

  1. Download the executable winutils from the Hortonworks repository.
  2. Create a dummy directory where you place the downloaded executable winutils.exe. For example : C:SparkDevbin.
  3. Add the environment variable HADOOP_HOME which points to C:SparkDev. You have 2 choices :
    • Windows > System Setting
    • Eclipse > Your Class which can be run as a Java Application (containing the static main method) > Right Click > Run as > Run Configurations > Evironment Tab :

We would like to show you a description here but the site won’t allow us. Winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows - cdarlint/winutils.

Konica

Create a Maven Project. Configure pom.xml as follows :

Hortonworks Winutils.exe

Hortonworks Winutils

The benefit of creating a local Spark context is the possibility to run everything locally without being in need of deploying Spark Server separately as a master. This is very interesting while development phase. So here it is the basic configuration :

Hortonworks Winutils

Now that we have an operational environment, let’s move to punchy examples of RDD Tranformations and Actions tutorial.