Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Nokogiri: Searching for <div> using XPath

I use Nokogiri (Rubygem) css search to look for certain <div> inside my html. It looks like Nokogiri's css search doesn't like regex. I would like to switch to Nokogiri's xpath search as this seems to support regex in search strings.

How do I implement the (pseudo) css search mentioned below in an xpath search?

require 'rubygems'
require 'nokogiri'

value = Nokogiri::HTML.parse(<<-HTML_END)
  "<html>
    <body>
      <p id='para-1'>A</p>
      <p id='para-22'>B</p>
      <h1>Bla</h1>
      <p id='para-3'>C</p>
      <p id='para-4'>D</p>
      <div class="foo" id="eq-1_bl-1">
        <p id='para-5'>3</p>
      </div>
    </body>
  </html>"
HTML_END

# my_block is given
my_bl = "1"
# my_eq corresponds to this regex
my_eq = "\/[0-9]+\/"

# FIXME The following line should be changed to an xpath search.
if my_div = value.css("div#eq-#{my_eq}_bl-#{my_bl}.foo").first
  # doing some stuff with the <p> inside the div
end
like image 259
Javier Avatar asked Mar 16 '09 11:03

Javier


2 Answers

Mike Dalessio (one half of the Nokogiri core developers) gave me an answer on #nokogiri (irc.freenode.net). Looks like neither Nokogiri CSS nor XPath search do support regex matching. This is his solution on how to search for regular expressions with Nokogiri:

require 'rubygems'
require 'nokogiri'

value = Nokogiri::HTML.parse(<<-HTML_END)
  "<html>
    <body>
      <p id='para-1'>A</p>
      <p id='para-22'>B</p>
      <h1>Bla</h1>
      <p id='para-3'>C</p>
      <p id='para-4'>D</p>
      <div class="foo" id="eq-1_bl-1">
        <p id='para-5'>3</p>
      </div>
      <div class="bar" id="eq-1_bl-1">
        <p id='para-5'>3</p>
      </div>
    </body>
  </html>"
HTML_END

# my_block is given
my_bl = "1"
# my_eq corresponds to this regex
my_eq = "[0-9]+"
# full regex to search for in node ids
full_regex = %r(eq-#{my_eq}_bl-#{my_bl})

filter_by_id = Class.new do
  attr_accessor :matches

  def initialize(regex)
    @regex = regex
    @matches = []
  end

  def filter(node_set)
    @matches += node_set.find_all { |x| x['id'] =~ @regex }
  end
end.new(full_regex)

value.css("div.foo:filter()", filter_by_id)
filter_by_id.matches.each do |node|
  puts node
end
like image 58
Javier Avatar answered Nov 06 '22 22:11

Javier


A simpler approach based on the answer above:

regex = /subject|header/
headers = doc.css("table td:nth-child(1) div").find_all do
  |h| h['class'] =~ regex
end

Thank you for posting this question.

like image 32
ringe Avatar answered Nov 07 '22 00:11

ringe