how to get html content form flutterWebViewPlugin ( webview ) in flutter

Question

My question is pretty simple. how can I get the html data form the loaded page in webview. The data is in JSON format. for a reason I cannot use a post request for any get request here so I need to get the data with webview only. how to achive this ?

flutterWebViewPlugin.onUrlChanged.listen((String url) {
      if (url.contains('/recon?')) {
        print('printing url : $url');
        // need to get data ( html content ) on this url
        flutterWebViewPlugin.close();
        Navigator.of(context).pop(url);
      }
    });
});

WebviewScaffold(
                userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0",
                url: myurl,               
              )

Answer 1

You can do this with official webview library:

WebView(
  initialUrl: model.luxtrustUrl,
  onWebViewCreated: (controller) {
    _controller = controller;
  },
  javascriptMode: JavascriptMode.unrestricted,
  gestureNavigationEnabled: true,
  onPageFinished: (_) {
    readJS();
    model.hideProgress();
  },
)

void readJS() async{
    String html = await _controller.evaluateJavascript("window.document.getElementsByTagName('html')[0].outerHTML;");    
    print(html);
}

Answer 2

You can use my plugin flutter_inappwebview, which is a Flutter plugin that allows you to add inline WebViews or open an in-app browser window and has a lot of events, methods, and options to control WebViews.

To get HTML data, you can simply use var html = await controller.evaluateJavascript(source: "window.document.getElementsByTagName('html')[0].outerHTML;"); inside the onLoadStop event.

Here is an example using your userAgent:

import 'dart:async';
import 'dart:developer';
import 'package:flutter/material.dart';
import 'package:flutter_inappwebview/flutter_inappwebview.dart';

Future main() async {
  WidgetsFlutterBinding.ensureInitialized();
  runApp(new MyApp());
}

class MyApp extends StatefulWidget {
  @override
  _MyAppState createState() => new _MyAppState();
}

class _MyAppState extends State<MyApp> {
  InAppWebViewController webView;
  String myurl = "https://github.com/flutter";

  @override
  void initState() {
    super.initState();
  }

  @override
  void dispose() {
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      home: Scaffold(
        appBar: AppBar(
          title: const Text('InAppWebView Example'),
        ),
        body: Container(
            child: Column(children: <Widget>[
              Expanded(
                  child: InAppWebView(
                    initialUrl: myurl,
                    initialHeaders: {},
                    initialOptions: InAppWebViewGroupOptions(
                      crossPlatform: InAppWebViewOptions(
                        debuggingEnabled: true,
                        userAgent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:61.0) Gecko/20100101 Firefox/61.0"
                      ),
                    ),
                    onWebViewCreated: (InAppWebViewController controller) {
                      webView = controller;
                    },
                    onLoadStart: (InAppWebViewController controller, String url) {

                    },
                    onLoadStop: (InAppWebViewController controller, String url) async {
                      if (url.contains('/recon?')) {
                        // if JavaScript is enabled, you can use
                        var html = await controller.evaluateJavascript(
                            source: "window.document.getElementsByTagName('html')[0].outerHTML;");

                        log(html);
                      }
                    },
                  ))
            ])),
      ),
    );
  }
}

Just set the myurl variable to your url.

Through window.document.getElementsByTagName('html')[0].outerHTML; you will get all the HTML string. Instead, if you need only the body text, you can change it with window.document.body.innerText;.