I am already parsing pages with the HtmlAgilityPack, and getting most img sources. However many websites include img urls in places other than the img src attributes (e.g. inlined javascript, a different attribute, a different element). I would like to cast a slightly wider net and run a regex on the entire html string capture the following in a regex.
I imagine this would be simple to write, however I am not an awesome regexer. I imagine the parts would look like this
Can anyone help me fill the blanks?
Thanks
Answer
(https?:)?//?[^\'"<>]+?\.(jpg|jpeg|gif|png)
There are a number of ad-hoc regular expressions for matching URLs out there, but none that I am aware of claim total reliability. However, this one will attempt to satisfy your conditions.
According to [1], valid URL characters (which are not reserved) are alphanumeric and the symbols $-_.+!*'(),
. However, there are reserved characters as well, which are +/?%#&
which is concisely given by [2] -- I couldn't find a list in the bulk of the RFC. I know there are other characters used for query strings though, namely =;
, so those need inclusion. Then you run into issues that not everyone properly encodes their URL characters, so spaces may be present among other things (which I do not know how to account for as how a browser auto-corrects things can be mystifying).
Therefore, you might just assume that anything can be in a URL, but merely it must start with something particular and end with something particular (which you provided) but this is still unreliable.
@(https?:)?//?[^'"<>]+?\.(jpg|jpeg|gif|png)@
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With