Getting streamable link from Google Drive [Code is ready, Access is denied]

Question

I'm trying to reduplicate the process of fetching downloadable links from Google Drive, just like the one used in Internet Download Manager (Windows).

I'm doing the following:

1. Downloading the HTML source code of the link.
2. Using regex to search for the fmt_stream_map.
3. Decoding.
4. Trying to reach the link.
5. Access denied.

The same approach is used in Internet download manager, and it's working well. I'm fetching the link using my Phone, and accessing it from my phone. So basically it's the same IP and same device.

My code first downloads the source code. Search for the map list, then store the quality and its description in an Array. After that i search for the fmt_stream_map for the links, and add them in a final model, to access them easily.

I have 3 classes, two of them are models, and the last one is the main class for this process.

public class ItemStreamList {

    private String quality;
    private String description;

    public ItemStreamList(){

    }

    public ItemStreamList(String quality, String description){
        this.quality = quality;
        this.description = description;
    }

    public String getQuality() {
        return quality;
    }

    public void setQuality(String quality) {
        this.quality = quality;
    }

    public String getDescription() {
        return description;
    }

    public void setDescription(String description) {
        this.description = description;
    }
}





public class ItemLink {

    private String quality;
    private String qualityDesc;
    private String link;

    public ItemLink(){

    }

    public ItemLink(String quality, String qualityDesc, String link){
        this.quality = quality;
        this.qualityDesc = qualityDesc;
        this.link = link;
    }

    public String getQualityDesc() {
        return qualityDesc;
    }

    public void setQualityDesc(String qualityDesc) {
        this.qualityDesc = qualityDesc;
    }

    public String getLink() {
        return link;
    }

    public void setLink(String link) {
        this.link = link;
    }

    public String getQuality() {
        return quality;
    }

    public void setQuality(String quality) {
        this.quality = quality;
    }
}

Now we come to the main class. It's a "bit" documented.

  @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);

        getPageHTML("https://drive.google.com/file/d/0B7--EhvK76QDNmduLWFZMXh1dGs/view");
    }

    private void getPageHTML(final String mURL){
        new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    Document mDoc = Jsoup.connect(mURL).get();
                    String mHTML = mDoc.toString();

                    boolean hasSetStreamMapList = setStreamMapList(mHTML);

                    String mStreamMap = getMatchRegex(mHTML,"\"fmt_stream_map\",\"","\"]");
                    mStreamMap = org.apache.commons.text.StringEscapeUtils.unescapeJava(mStreamMap);
                    String[] mStreamMapQualities = mStreamMap.split(",");

                    if(hasSetStreamMapList){
                        List<ItemLink> mLinks = new ArrayList<>();

                        for (int i = 0; i < mStreamMapQualities.length; i++){
                            String[] mLinksArray = mStreamMapQualities[i].split("\\|");
                            String mLink = mLinksArray[1];
                            mLink = mLink.replaceAll("%2",",");
                            mLinks.add(new ItemLink(mLinksArray[0],getQualityDescription(mLinksArray[0]),mLink));
                        }

                        for (int i = 0; i < mLinks.size(); i++){
                            Log.i("StreamMap","Quality: " + mLinks.get(i).getQuality() + " - " + mLinks.get(i).getQualityDesc() + "\n" + "Link: " + mLinks.get(i).getLink());
                            Log.i("StreamMap","---------------------------");
                        }

                        startActivity(new Intent(Intent.ACTION_VIEW, Uri.parse(mLinks.get(0).getLink())));


                    } else {
                        Log.i("StreamMap","Stream Map is NOT set");
                    }

                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }).start();
    }

    private List<ItemStreamList> mStreamListItems;

    private String getQualityDescription(String mQuality){
        // Loop through the Stream Map's Array
        for (int i = 0; i < mStreamListItems.size(); i++){
            // If the Quality contains the Param, then return it
            if(mStreamListItems.get(i).getQuality().contains(mQuality))
                return mStreamListItems.get(i).getDescription();
        }
        // Didn't find the quality, return null
        return null;
    }

    private boolean setStreamMapList(String mSource){

        // Define the Array
        mStreamListItems = new ArrayList<>();

        // Get the fmt_list from the HTML
        String mStreamMapList = getMatchRegex(mSource,"\"fmt_list\",\"","\"]");
        // Check if isn't null
        if(mStreamMapList != null){
            // Split the qualities by ","
            String[] mStreamMapListArray = mStreamMapList.split(",");
            // Loop through the Array
            for (int i = 0; i < mStreamMapListArray.length; i++){
                /*
                Split the array by "/"
                First index has the quality
                Second index has the quality's description
                */
                String[] mModelArray = mStreamMapListArray[i].split("/");
                mStreamListItems.add(new ItemStreamList(mModelArray[0],mModelArray[1]));
            }

            return true;

        }

        // It's null or not set, return false
        return false;
    }

    private String getMatchRegex(String mSource, String mFirst, String mSecond){
        String regexString = Pattern.quote(mFirst) + "(.*?)" + Pattern.quote(mSecond);
        Pattern mPattern = Pattern.compile(regexString);
        Matcher mMatcher = mPattern.matcher(mSource);
        while(mMatcher.find())
        {
            String resultString = mMatcher.group(1);


            return resultString;
        }
        return null;
    } 

Answer 1

You get a 403 code, because you do not pass any headers in your request on the fmt_stream_map link.

When opening the top link in the browser, the fmt_stream_map link is queried using all these headers :

The required headers are not documented, as you are not supposed to integrate google drive this way

Answer 2

I think the stream-links are restricted to the YouTube-like video viewer in some way. That could cause the “403 Access denied” errors. I don’t think you’ll be able to download the stream-links this way. I don't know what method IDM uses (couldn't find src code), but I'd assume they're integrating with Google API in some way since that would be the more correct and time-durable way to do it.

Here are some alternatives.

1. If you just want the file, try using the Google Drive Android API.

2. If you wanted to play the video, consider using the YouTube Android Player API.

3. A last resort would be to dig into the YouTube player and figure out how to duplicate the way it requests those URLs.