May 29, 2014
The simplified YARN log management can be a life saver. Being able to do
yarn application and
yarn logs is certainly handy if you do a lot of debugging and development. The alternative is that horrible web UI.
Initially I came across this problem retrieving logs from YARN according to the documentation. Let us say you have an application with the id
application_1401296437855_0024. Trying a
yarn logs -applicationId application_1401296437855_0024 left me as a question mark with the error:
14/05/29 03:24:54 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/192.168.109.138:8050 Logs not available at /app-logs/root/logs/application_1401296437855_0024 Log aggregation has not completed or is not enabled.
Note that the logs is obviously stored in HDFS, and a
hdfs dfs -ls /app-logs/ shows that the logs are stored under
/app-logs/hdfs. That meaning the log request must be issued as the
So that’s basically it. You can find more information about YARN log management on the Hortonworks blog.