Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Ruby File IO: Can't open url as File object

I have a function in my code that takes a string representing the url of an image and creates a File object from that string, to be attached to a Tweet. This seems to work about 90% of the time, but occasionally fails.

require 'open-uri'
attachment_url = "https://s3.amazonaws.com/FirmPlay/photos/images/000/002/443/medium/applying_too_many_jobs_-_daniel.jpg?1448392757"
image = File.new(open(attachment_url))

If I run the above code it returns TypeError: no implicit conversion of StringIO into String. If I change open(attachment_url) to open(attachment_url).read I get ArgumentError: string contains null byte. I also tried stripping out the null bytes from the file like so, but that also made no difference.

image = File.new(open(attachment_url).read.gsub("\u0000", ''))

Now if I try the original code with a different image, such as the one below, it works fine. It returns a File object as expected:

attachment_url = "https://s3.amazonaws.com/FirmPlay/photos/images/000/002/157/medium/mike_4.jpg"

I thought maybe it had something to do with the params in the original url, so I stripped those out, but it made no difference. If I open the images in Chrome they appear to be fine.

I'm not sure what I'm missing here. How can I resolve this issue?

Thanks!

Update

Here is the working code I have in my app:

filename = self.attachment_url.split(/[\/]/)[-1].split('?')[0]
stream = open(self.attachment_url)
image = File.open(filename, 'w+b') do |file|
    stream.respond_to?(:read) ? IO.copy_stream(stream, file) : file.write(stream)
    open(file)
end

Jordan's answer works except that calling File.new returns an empty File object, whereas File.open returns a File object containing the image data from stream.

like image 621
Daniel Bonnell Avatar asked Dec 08 '15 16:12

Daniel Bonnell


People also ask

How do I read data from a file in Ruby?

Opening a File in Ruby There are two methods which are widely used − the sysread(n) and the read() method. The open method is used to open the file, while the sysread(n) is used to read the first "n" characters from a file, and the read() method is used to read the entire file content.

What are the Ruby file open modes?

Ruby allows the following open modes: "r" Read-only, starts at beginning of file (default mode). "r+" Read-write, starts at beginning of file. "w" Write-only, truncates existing file to zero length or creates a new file for writing.

Does file read close the file Ruby?

Ruby read file into array with File. The File. readlines method reads the whole file into an array of lines. The method automatically closes the file for us. Since the method reads the whole file at once, it is suitable for smaller files.


1 Answers

The reason you're getting TypeError: no implicit conversion of StringIO into String is that open sometimes returns a String object and sometimes returns a StringIO object, which is unfortunate and confusing. Which it does depends on the size of the file. See this answer for more information: open-uri returning ASCII-8BIT from webpage encoded in iso-8859 (Although I don't recommend using the ensure-encoding gem mentioned therein, since it hasn't been updated since 2010 and Ruby has had significant encoding-related changes since then.)

The reason you're getting ArgumentError: string contains null byte is that you're trying to pass the image data as the first argument to File.new:

image = File.new(open(attachment_url))

The first argument of File.new should be a filename, and null bytes aren't allowed in filenames on most systems. Try this instead:

image_data = open(attachment_url)

filename = 'some-filename.jpg'

File.new(filename, 'wb') do |file|
  if image_data.respond_to?(:read)
    IO.copy_stream(image_data, file)
  else
    file.write(image_data)
  end
end

The above opens the file (creating it if it doesn't exist; the b in 'wb' tells Ruby that you're going to write binary data), then writes the data from image_data to it using IO.copy_stream if it's a StreamIO object or File#write otherwise, then closes the file again.

like image 106
Jordan Running Avatar answered Oct 21 '22 13:10

Jordan Running