I have a file with some duplicate entries. It looks like
Los Angeles, 6
Los Angeles, 6
New York, 31
New York, 31
New YOrk, 31
.
.
.
Now I want to get rid of the duplicate data. What I try to do is to use each_line, see if the line equals to the next line, if they are the same then just skip, and write to a new file. The problem is how should I get the next line of that file? Or any other suggestions to do this?
Comparing against the next line is unnecessary, since you can easily see if the current line matches the previous one:
file = File.open("filename", "r")
previous_line = nil
file.each_line { |line|
if line == previous_line
# duplicate line; ignore
else
# different; do whatever you want
end
# remember this line so we can compare against it
previous_line = line
}
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With