is there any link to show all public repositories in GitHub?

Question

Yesterday, I was trying to get a list of all public repositories in GitHub, but I didn't find any link.

And for example in Sourceforge you can list all proyects by categories or in Google code you can search all for all proyects.

Yes, I tried to search with keywords like as "*" or "%" or empty string, but you only see this page https://github.com/search?q=&type=Everything&repo=&langOverride=&start_value=1

Answer 1

You can list all repositories in github using the following request:

https://api.github.com/repositories?since=0

it will return the first "n" repositories of id>0 as a JSON Array. You should process this "n", storing the "id". When you reach the end of the "page", you just hit again with since=lastId: for example:

https://api.github.com/repositories?since=300

This is the ONLY way I discovered to list ALL repositories, because the SEARCH api has a limit of 1000 repos per search. If you intend to process ALL repos, you should be prepared to cope with the rate-limit:

https://developer.github.com/v3/rate_limit/

Authenticated users get better limits, you can use an access_token (look at the documentation). BE CAREFUL, DON´T PUSH TOO MANY REQUESTS. If you need to filter the repos, you will need to perform an extra query (to search API) for each repo. Be prepared to deal with more than ten million repositories. Java Example: (with javax.json.Json)

int id=0;
do {
    URL url = new URL("https://api.github.com/repositories?since="+id+"&access_token="+oauth);
    // implement callApi such as Json.createReader(url.openStream()), but please make it sleep for a minute if the limit got reached        
    try (JsonReader rdr = callApi(url)) {
        JsonArray results = rdr.readArray();
        for (JsonObject result : results.getValuesAs(JsonObject.class)) {
            id = result.getInt("id");
            String name = result.getString("name");
            boolean priv = result.getBoolean("private");
            ... do whatever you want...
        }
    }
} while (some stop condition);

Good Luck, I lost some time to discover this.

Answer 2

Check out this search. I'm not sure if that lists all public repos, but I bet this is a good start. (the search may take a few seconds, so be patient)