UIWebView getting image url with elementFromPoint()

Question

I'm trying to get the url of an image tapped in a UIWebView using UILongPressGestureRecognizer.

I sort of have this working, but i'm not sure if the coordinates are off, or it's just not finding an img tag in the DOM as it doesn't always work.

I am using this code:

int displayWidth = [[self.webView stringByEvaluatingJavaScriptFromString:@"window.outerWidth"] intValue];
CGFloat scale = self.webView.frame.size.width / displayWidth;
CGPoint pt = [gesture locationInView:self.webView];
pt.x /= scale;
pt.y /= scale;
NSString *imgURL = [NSString stringWithFormat:@"document.elementFromPoint(%f, %f).parentElement.getElementsByTagName(\"img\")[0].src", pt.x, pt.y];
NSString *urlToSave = [self.webView stringByEvaluatingJavaScriptFromString:imgURL];

If I use a website that starts off zoomed right out and has images in a grid, and I zoom in then select an image, I get given the image at the top left corner of the page instead of the one clicked on. Not sure if it's a zoom issue, offset issue, or DOM issue?

EDIT ------------

For starters, the coordinates are definitely off! If I click a point on a webpage in the UIWebView I get these result: ScrollY: 0 Click pointY: 89 Y *= scale += scrollY = 86.8293

Now if I scroll the page up, so the point I clicked is in line with the top (approximately at y=0) I get these results: ScrollY: 144 Click pointY:1 Y *= scale += scrollY = 144.976

Now before the calculations, the scroll seems off. The point was 89, but when scrolled to that point the scroll reads 144. Why would that be?

I'm getting the scroll from window.pageYOffset

Answer 1

Put this 2 javascript function in a file named JSTools.js

function getHTMLElementsAtPoint(x,y) {
var tags = "";
var e;
var offset = 0;
while ((tags.search(",(A|IMG),") < 0) && (offset < 20)) {
    tags = ",";
    e = document.elementFromPoint(x,y+offset);
    while (e) {
        if (e.tagName) {
            tags += e.tagName + ',';
        }
        e = e.parentNode;
    }
    if (tags.search(",(A|IMG),") < 0) {
        e = document.elementFromPoint(x,y-offset);
        while (e) {
            if (e.tagName) {
                tags += e.tagName + ',';
            }
            e = e.parentNode;
        }
    }

    offset++;
}
return tags;}

function getLinkSRCAtPoint(x,y) {
var tags = "";
var e = "";
var offset = 0;
while ((tags.length == 0) && (offset < 20)) {
    e = document.elementFromPoint(x,y+offset);
    while (e) {
        if (e.src) {
            tags += e.src;
            break;
        }
        e = e.parentNode;
    }
    if (tags.length == 0) {
        e = document.elementFromPoint(x,y-offset);
        while (e) {
            if (e.src) {
                tags += e.src;
                break;
            }
            e = e.parentNode;
        }
    }
    offset++;
}
return tags;}

Then in your controller

- (void)longPressRecognized:(UILongPressGestureRecognizer *)gestureRecognizer {
if (gestureRecognizer.state == UIGestureRecognizerStateBegan) {
    CGPoint point = [gestureRecognizer locationInView:webView];
    longpressTouchedPoint = point;
    // convert point from view to HTML coordinate system
    CGSize viewSize = [webView frame].size;
    CGSize windowSize = [webView windowSize];

    CGFloat f = windowSize.width / viewSize.width;
    if ([[[UIDevice currentDevice] systemVersion] doubleValue] >= 5.) {
        point.x = point.x * f;
        point.y = point.y * f;
    } else {
        // On iOS 4 and previous, document.elementFromPoint is not taking
        // offset into account, we have to handle it
        CGPoint offset = [webView scrollOffset];
        point.x = point.x * f + offset.x;
        point.y = point.y * f + offset.y;
    }

    // Load the JavaScript code from the Resources and inject it into the web page
    NSString *path = [[NSBundle mainBundle] pathForResource:@"JSTools" ofType:@"js"];
    NSString *jsCode = [NSString stringWithContentsOfFile:path encoding:NSUTF8StringEncoding error:nil];
    [webView stringByEvaluatingJavaScriptFromString: jsCode];


    // call js functions
    NSString *tags = [webView stringByEvaluatingJavaScriptFromString:
                      [NSString stringWithFormat:@"getHTMLElementsAtPoint(%i,%i);",(NSInteger)point.x,(NSInteger)point.y]];
    NSString *tagsSRC = [webView stringByEvaluatingJavaScriptFromString:
                         [NSString stringWithFormat:@"getLinkSRCAtPoint(%i,%i);",(NSInteger)point.x,(NSInteger)point.y]];

    NSLog(@"src : %@",tags);
    NSLog(@"src : %@",tagsSRC);

    NSString *url = nil;
    if ([tags rangeOfString:@",IMG,"].location != NSNotFound) {
        url = tagsSRC;    // Here is the image url!
    }

Mixed from several source, credit to the web community!

EDIT

You also have to create a category on uiwebview with that two function :

- (CGSize)windowSize {
CGSize size;
size.width = [[self stringByEvaluatingJavaScriptFromString:@"window.innerWidth"] integerValue];
size.height = [[self stringByEvaluatingJavaScriptFromString:@"window.innerHeight"] integerValue];
return size;}

- (CGPoint)scrollOffset {
CGPoint pt;
pt.x = [[self stringByEvaluatingJavaScriptFromString:@"window.pageXOffset"] integerValue];
pt.y = [[self stringByEvaluatingJavaScriptFromString:@"window.pageYOffset"] integerValue];
return pt;}

Answer 2

I've recently achieved a bit similar goal, but I didn't determine "touch" coordinates.

My answer can help if you have permissions to modify html/javascript source.

WHAT I DID:

On each image <img... html DOM element I put onClick javascript handler to change window.location

f.e

javascript function clickedImage(imageURL) {
    window.location = "customScheme://"+imageURL; // attached image url to be able to read it inside `UIWebView`
}

On UIWebView delegate's method ignore links like above, BUT, we are able to get image URL.

- (BOOL)webView:(UIWebView *)webView shouldStartLoadWithRequest:(NSURLRequest *)request navigationType:(UIWebViewNavigationType)navigationType
{

  if ([[request.URL scheme] isEqualToString:@"customScheme"]) {
        //Fetching image URL
        NSLog(@"Clicked on image with URL %@", [request.URL host])
        ...
        // Always return NO not to allow `UIWebView` process such links
        return NO; 
    }
    ....
}

Sorry, if this approach doesn't fully fit your question, but I guess this is an option you can follow and dig deeply.

Small note: to use UILongPressGestureRecognizer you can simulate this behaviour by making javascript instead: Long Press in JavaScript?

Answer 3

You can get element by following code

        <script type="text/javascript">

            (function($) {
             $.fn.longClick = function(callback, timeout) {
             var timer;
             timeout = timeout || 500;
             $(this).mousedown(function(e) {
                               timer = setTimeout(function() { callback(e.offsetX,e.offsetY); }, timeout);
                               return false;
                               });
             $(document).mouseup(function() {
                                 clearTimeout(timer);
                                 return false;
                                 });
             };

             })(jQuery);


            $(window).longClick(function(x,y)
            {
              var element = document.elementFromPoint(x, y);
              alert(nodeToString(element));
            },600);


            function nodeToString ( node ) {
                var tmpNode = document.createElement( "div" );
                tmpNode.appendChild( node.cloneNode( true ) );
                var str = tmpNode.innerHTML;
                tmpNode = node = null; // prevent memory leaks in IE
                return str;
            }
        </script>

This is just js code. mousedown and mouseup will not work for ios . You need to use other alternative event for ios webView.