I have a number of hive queries that my system executes on a regular basis. When you look at the job tracker, they show up as "SELECT field, other_field ..... (Stage-1)" and similar. That's not particularly helpful to me, so I added:
set mapred.job.name = more helpful name;
to the query. Now I can tell them apart better. However, now my queries that get split into multiple stages all show up as the same name. What I'd ideally like is something along the lines of set mapred.job.name = more helpful name (Stage-%d);
where the %d would get replaced by the current stage number.
Is this possible, and does anyone know how?
I'm not sure there is a way to implement exactly what you wish but I can offer something else.
Instead of using set mapred.job.name
you can add a comment in the beginning of the query with a more helpful name
like this :-- this is a more helpful name
SELECT field, other_field ....
Then, in the jobtracker you'll see -- this is a more helpful name ..... (Stage-%d)
"
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With