How to get hidden input value with JSoup?

Question

I have data

<input name="authenticity_token" type="hidden" value="aiUlw1Yh4W47lPQearSEdTkU0rhKpziZOweq5PMTV0Q=" />

I am trying to select it with Jsoup.

 Element input = doc.select("input[name=authenticity_token]").first();

 String auth_token = input.attr("value");

But this not working.

I am recieving a null pointer exception at String auth_token =.

java.lang.NullPointerException at crime.ic.Main.main(Main.java:2)

What am I doing wrong?

Answer 1

After some researching, I think I know what happens here: the content you want to load may come from some JavaScript and is not visible/existent upon page loading. The loading of JS happens afterwards and fills data in place, and this is not Jsoup can do. So it is beyond reach of Jsoup.

What we need is simulate/automated the loading of JS. Such tools exist.

I found this answer and this question helpful.