The first step in this process is to choose a web scraper for your project. We obviously recommend ParseHub. Not only is it free to use, but it also works with all kinds of websites. With ParseHub, web scraping is as simple as clicking on the data you want and downloading it as an excel sheet or JSON file.
Using the Maven artifact org.json:json
I got the following code, which I think is quite short. Not as short as possible, but still usable.
package so4308554;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
import java.nio.charset.Charset;
import org.json.JSONException;
import org.json.JSONObject;
public class JsonReader {
private static String readAll(Reader rd) throws IOException {
StringBuilder sb = new StringBuilder();
int cp;
while ((cp = rd.read()) != -1) {
sb.append((char) cp);
}
return sb.toString();
}
public static JSONObject readJsonFromUrl(String url) throws IOException, JSONException {
InputStream is = new URL(url).openStream();
try {
BufferedReader rd = new BufferedReader(new InputStreamReader(is, Charset.forName("UTF-8")));
String jsonText = readAll(rd);
JSONObject json = new JSONObject(jsonText);
return json;
} finally {
is.close();
}
}
public static void main(String[] args) throws IOException, JSONException {
JSONObject json = readJsonFromUrl("https://graph.facebook.com/19292868552");
System.out.println(json.toString());
System.out.println(json.get("id"));
}
}
Here are couple of alternatives versions with Jackson (since there are more than one ways you might want data as):
ObjectMapper mapper = new ObjectMapper(); // just need one
// Got a Java class that data maps to nicely? If so:
FacebookGraph graph = mapper.readValue(url, FaceBookGraph.class);
// Or: if no class (and don't need one), just map to Map.class:
Map<String,Object> map = mapper.readValue(url, Map.class);
And specifically the usual (IMO) case where you want to deal with Java objects, can be made one liner:
FacebookGraph graph = new ObjectMapper().readValue(url, FaceBookGraph.class);
Other libs like Gson also support one-line methods; why many examples show much longer sections is odd. And even worse is that many examples use obsolete org.json library; it may have been the first thing around, but there are half a dozen better alternatives so there is very little reason to use it.
The easiest way: Use gson, google's own goto json library. https://code.google.com/p/google-gson/
Here is a sample. I'm going to this free geolocator website and parsing the json and displaying my zipcode. (just put this stuff in a main method to test it out)
String sURL = "http://freegeoip.net/json/"; //just a string
// Connect to the URL using java's native library
URL url = new URL(sURL);
URLConnection request = url.openConnection();
request.connect();
// Convert to a JSON object to print data
JsonParser jp = new JsonParser(); //from gson
JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent())); //Convert the input stream to a json element
JsonObject rootobj = root.getAsJsonObject(); //May be an array, may be an object.
String zipcode = rootobj.get("zip_code").getAsString(); //just grab the zipcode
If you don't mind using a couple libraries it can be done in a single line.
Include Apache Commons IOUtils & json.org libraries.
JSONObject json = new JSONObject(IOUtils.toString(new URL("https://graph.facebook.com/me"), Charset.forName("UTF-8")));
Maven dependency:
<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>2.7</version>
</dependency>
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With