DEV Community

Discussion on: Building Hadoop native libraries on Mac in 2019

Collapse
 
ajndesai profile image
ajndesai • Edited

Thanks for this article. I get this error when compiling to generate package using this below command:

mvn package -Pdist,native -DskipTests -Dtar

branch: branch-3.2

[INFO] Apache Hadoop MapReduce NativeTask ................. FAILURE [ 1.766 s]

[ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.2.2-SNAPSHOT:cmake-compile (cmake-compile) on project hadoop-mapreduce-client-nativetask: make failed with error code 2 -> [Help 1]

Collapse
 
lanshunfang profile image
001027261_Shunfang Lan

It's because hadoop doesn't support macOS native building now.

See github.com/apache/hadoop/blob/trun...

Note that building Hadoop 3.1.1/3.1.2/3.2.0 native code from source is broken
on macOS. For 3.1.1/3.1.2, you need to manually backport YARN-8622. For 3.2.0,
you need to backport both YARN-8622 and YARN-9487 in order to build native code.
Collapse
 
zejnilovic profile image
Saša Zejnilović

Hello, I don't see enough of the message. Do you have java 8? Or maybe more of the error (line with cause by)?

I have tried building this now and it works for me.

Collapse
 
ajndesai profile image
ajndesai • Edited

Yes I have java 8 and that's my java home. Hadoop, hive setup and working though. I read some of the compression codecs works only with native libraries and jobs will fail with java libraries.

I tried this on mac Mojave OS 10.14.6