loading...

re: Installing and Running Hadoop and Spark on Windows VIEW POST

TOP OF THREAD FULL DISCUSSION
re: Hi David, It sounds like you're trying to run this program by double-clicking on it. You should run it in the cmd prompt like: C:\> start-yar...
 

No. I am using the cmd console.
For example if i try to type just hadoop in console, it shows me some options. But if i try to type yarn it says:

'yarn' is not recognized as an internal or external command,
operable program or batch file.

I am attaching the images where it can be seen: thepracticaldev.s3.amazonaws.com/i...

These error messages are giving you hints about what's going wrong. It looks like your %PATH% is set up correctly and hadoop is on it, but you can't run the hadoop command by itself. That's what the error message is telling you. You need to include additional command-line arguments.

Try running hadoop version and see if you get any output.

When i execute this command "hadoop version" i get this:

Hadoop 2.9.1
Subversion github.com/apache/hadoop.git -r e30710aea4e6e55e69372929106cf119af06fd0e
Compiled by root on 2018-04-16T09:33Z
Compiled with protoc 2.5.0
From source with checksum 7d6d2b655115c6cc336d662cc2b919bd
This command was run using /C:/BigData/hadoop-2.9.1/share/hadoop/common/hadoop-common-2.9.1.jar

But if i try execute just "yarn" i get:

'yarn' is not recognized as an internal or external command,
operable program or batch file.

Right, so hadoop is working fine. yarn isn't a command that you run, it's just the resource negotiator that the HDFS (Hadoop Distributed File System) uses behind the scenes to manage everything.

If you successfully ran start-yarn.cmd and start-dfs.cmd, you're good to go! Try uploading a file to HDFS with:

C:\> hadoop fs -put <file name here> /

...and checking that it's been uploaded with

C:\> hadoop fs -ls /

Hi,
Thanks for your answer.
But the problem is exactly that. When a i run the command start-yarn.cmd i get:

This file does not have an app associated with it for performing this action. Please install an app or, if one is already installed, create an association in the defaul apps settings page.

So, i tried to see what the content is for the file start-yarn.cmd and it has a call to yarn command. So i tried to call it in a independent console and i get the same error. That is the reason why i think the problem is yarn, the command as is.

Okay, I think we're getting close. Can you echo %PATH% and share the result?

start-yarn.cmd should be within the Hadoop /sbin directory. If you haven't added it to your path correctly, maybe that's why you can't access it.

Thanks for the answer.

Here it is: echo %path%

Result:
C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;E:\app\dserranoa\product\11.2.0\client_1;E:\app\dserranoa\product\11.2.0\client_1\bin;C:\oraclexe\app\oracle\product\11.2.0\server\bin;;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\TortoiseGit\bin;C:\Program Files\PuTTY\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI\wbin;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\ManagementStudio\;C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files\nodejs\;C:\Program Files\Microsoft SQL Server\110\DTS\Binn\;C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies\;C:\Program Files (x86)\Bitvise SSH Client;C:\Program Files\dotnet\;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files (x86)\Brackets\command;C:\WINDOWS\System32\OpenSSH\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files\Microsoft SQL Server\140\Tools\Binn\;C:\Program Files\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\;C:\Program Files\Java\jdk1.8.0_121\bin;C:\Program Files\MySQL\MySQL Shell 8.0\bin;C:\Users\dserranoa\AppData\Local\Microsoft\WindowsApps;C:\Progra~1\Java\jdk1.8.0_121;C:\BigData\hadoop-2.9.1;C:\BigData\hadoop-2.9.1\bin;C:\BigData\hadoop-2.9.1\sbin

I have attached the image of my environment variables.

Huh. Can you run:

C:\> dir C:\BigData\hadoop-2.9.1\sbin

...and give the result?

Sure,
Here it is:

Volume in drive C has no label.
Volume Serial Number is 8276-D962

Directory of C:\BigData\hadoop-2.9.1\sbin

11/09/2019 09:55 a.m.

.

11/09/2019 09:55 a.m. ..

16/04/2018 06:52 a.m. 2.752 distribute-exclude.sh

11/09/2019 09:55 a.m. FederationStateStore

16/04/2018 06:52 a.m. 6.475 hadoop-daemon.sh

16/04/2018 06:52 a.m. 1.360 hadoop-daemons.sh

16/04/2018 06:52 a.m. 1.640 hdfs-config.cmd

16/04/2018 06:52 a.m. 1.427 hdfs-config.sh

16/04/2018 06:52 a.m. 3.148 httpfs.sh

16/04/2018 06:52 a.m. 3.677 kms.sh

16/04/2018 06:52 a.m. 4.134 mr-jobhistory-daemon.sh

16/04/2018 06:52 a.m. 1.648 refresh-namenodes.sh

16/04/2018 06:52 a.m. 2.145 slaves.sh

16/04/2018 06:52 a.m. 1.779 start-all.cmd

16/04/2018 06:52 a.m. 1.471 start-all.sh

16/04/2018 06:52 a.m. 1.128 start-balancer.sh

16/04/2018 06:52 a.m. 1.401 start-dfs.cmd

16/04/2018 06:52 a.m. 3.734 start-dfs.sh

16/04/2018 06:52 a.m. 1.357 start-secure-dns.sh

16/04/2018 06:52 a.m. 1.571 start-yarn.cmd

16/04/2018 06:52 a.m. 1.347 start-yarn.sh

16/04/2018 06:52 a.m. 1.770 stop-all.cmd

16/04/2018 06:52 a.m. 1.462 stop-all.sh

16/04/2018 06:52 a.m. 1.179 stop-balancer.sh

16/04/2018 06:52 a.m. 1.455 stop-dfs.cmd

16/04/2018 06:52 a.m. 3.206 stop-dfs.sh

16/04/2018 06:52 a.m. 1.340 stop-secure-dns.sh

16/04/2018 06:52 a.m. 1.642 stop-yarn.cmd

16/04/2018 06:52 a.m. 1.340 stop-yarn.sh

16/04/2018 06:52 a.m. 4.295 yarn-daemon.sh

16/04/2018 06:52 a.m. 1.353 yarn-daemons.sh

28 File(s) 61.236 bytes

3 Dir(s) 101.757.034.496 bytes free

So start-dfs.cmd works, but start-yarn.cmd doesn't? Weird. They're both in the same directory. That doesn't make much sense.

I'm not sure how I can help further without being at your terminal. I'd say maybe try starting from scratch? Sometimes, it's easy to miss a small step or two.

mmm well, i tried to do the same process in another machine and it happened again. The same error. The yarn daemons are not running.

I have checked different options but i have not could find any solution yet.

I don't know if yarn needs some additional installation or something like that or if there is another environment variable that i am not setting up.

I am really lost here.
What kind of command would you use in my console?

I would start from scratch, and make sure the correct version (8) of Java is installed, and re-install Hadoop. Then, I would double-check all of the environment variables.

Can you try adding the environment variables as system environment variables, rather than user environment variables? You may need to be an Administrator to do this.

If all of that checks out, and the %PATH% is correct, and all of the .cmd files are on the path, I'm not sure what else I would do. There's no reason why those commands shouldn't work if they're on the %PATH%.

I appreciate your help.

I have already added the variables to the system but the problem is still there.

I would really appreciate that you tell me If you have another ideas to solve this issue.

I think it is also weird but it seems something related to yarn. I will look for more info , more tricks and if i solve it i will post here.

Thanks so much.

Code of Conduct Report abuse