wkhtmltopdf error when generating pdf with lots of pages with header / footer

Question

I am using pdfkit (which uses wkhtmltopdf under the hood) to generate PDFs in my rails app. Following the guide here i've gotten it mostly working for basic cases of PDFs. I am now running into an issue when attempting to generate a PDF with lots of pages thats also has headers/footers. The error I see from wkhtmltopdf in console when attempting to generate the PDF is:

QEventDispatcherUNIXPrivate(): Unable to create thread pipe: Too many open files
QEventDispatcherUNIXPrivate(): Can not continue without a thread pipe

A minimal example of the html which can be used to recreate the error:

<!-- content of pdf_header_url is the character "h" -->
<meta content="<%= pdf_header_url %>" name="pdfkit-header-html"/>
<!-- content of pdf_footer_url is the character "f" -->
<meta content="<%= pdf_footer_url %>" name="pdfkit-footer_html"/>
<% [*1..3].each do |j|%>
  <h1><%= j %></h1>
  <ul>
    <% [*1..1000].each do |i|%>
      <li><%= i %></li>
    <% end %>
  </ul>
<% end %>

Note that removing the of headers/footers tags allows the pdf to render fine.

The actual ruby code to generate the PDF is:

def view_report
  html = render_to_string(:template => 'pdf/pdf_body.html', :layout => false)
  kit = PDFKit.new(html)
  pdf = kit.to_pdf
  send_data pdf, :type => 'application/pdf', :disposition => 'inline', :filename => 'foo.pdf'
end

Visiting this controllers route will generate the PDF. And then lastly, I also have a controller for the header/footer, since those "partials" need to be fetchable via url:

class PdfController < ApplicationController

  def header
    render :layout => false
  end

  def footer
    render :layout => false
  end

end

The values of pdf_header_url and pdf_footer_url are literally just "h" and "f" for the sake of a minimal reproducible example.

Does anyone familiar with wkhtmltopdf have any recommendations on furthur debug steps to take to get around this issue?

Answer 1

I was getting the same error message today and I solved the problem with a very easy solution. The problem was that my header and footer needed to be complete html documents with the html, head and body tags. Also, see if you can get the generated html output of your header and footer and validate them.

Answer 2

Open Files Limit.

I ensured my header and footer files were complete HTML documents as Tom Hirschfeld suggested, but I was still getting this error of too many open files.

After scouring the internets I discovered that you need to UP your limit of the number of files a single process is allowed to have open. In my case, I was generating PDFs with hundreds to thousands of pages. It worked fine without headers and footers but when incorporating headers and footers it seems to hit this open file ceiling.

There are different ways to adjust this setting based on the system you are running but here is what worked for me on an Ubuntu server:

Add the following to the end of /etc/security/limits.conf:

# Sets the open file maximum here.
# Generating large PDFs hits the default ceiling (1024) quickly. 
*    hard nofile 65535
*    soft nofile 65535
root hard nofile 65535 # Need these two lines because the wildcards
root soft nofile 65535 # (the * above) are not applied to the root user.

A good reference for the ulimit command can be found here.

I hope that puts some people on the right track.

Answer 3

wkhtmltopdf uses 2 file descriptor per page (one each for header and footer) which are required for generating the per-page custom variables. You'll have to edit /etc/security/limits.conf to set nofile (i.e. maximum no of open files) to a suitably high number -- some experimentation may be required to find a value which works for you.