Michael Miklavcic's Blog

home

Getting PigUnit Running With Hadoop 2 On Windows

06 Apr 2014

I’ll keep this short and sweet. Currently, if you try to run PigUnit on a Windows machine, you’ll get an exception as follows:

java.io.IOException: Could not locate executable null\bin\winutils.exe 
in the Hadoop binaries.

I’ve put together a set of Maven project templates that accomplish two goals:

  1. Provide poms for adding test dependencies with Maven
  2. Provide a way to get around the Windows IOException

I’ve created a freely-available GitHub project to solve both problems. Check it out here - maven project template. First build the “testing” project (mvn clean install), then do the same in the pig-deps project. You just need to reference the pig-deps project in your Maven project as follows:

<dependency>
    <groupId>com.michaelmiklavcic</groupId>
    <artifactId>pig-test-deps</artifactId>
    <version>1.0-SNAPSHOT</version>
</dependency>

Then be sure to invoke the PigUnitUtil.runFix() method. See example here - PigUnitUtilTest

Credit and thanks to @cestella (http://www.caseystella.com) and @mwacc for their contributions.