Menu
Looking for articles, documentation or straight head knowledge of how different source control systems differentiate (or detect) the type of file (binary vs. text). Of particular interest is how Git does it vs Mercurial.
Do they look at: File extensions? File signatures or content (ie. is this file UTF8)? A mix of things?
431
Text files are organized around lines, each of which ends with a newline character ('\n'). The source code files are themselves text files. A binary file is the one in which data is stored in the file in the same way as it is stored in the main memory for processing.
One of the advantages of binary files is that they are more efficient. In terms of memory, storing values using numeric formats such as IEEE 754, rather than as text characters, tends to use less memory. In addition, binary formats also offer advantages in terms of speed of access.
In the text file, the newline character is converted to carriage-return/linefeed before being written to the disk. In binary file, conversion of newline to carriage-return and linefeed does not take place. Text files are used to store data more user friendly. Binary files are used to store data more compactly.
Binary files also usually have faster read and write times than text files, because a binary image of the record is stored directly from memory to disk (or vice versa). In a text file, everything has to be converted back and forth to text, and this takes time. C supports the file-of-structures concept very cleanly.
SVN:
When you first add or import a file into Subversion, the file is examined to determine if it is a binary file. Currently, Subversion just looks at the first 1024 bytes of the file; if any of the bytes are zero, or if more than 15% are not ASCII printing characters, then Subversion calls the file binary. This heuristic might be improved in the future, however.
http://subversion.apache.org/faq.html#binary-files
Git works in a similar way. Git usually guesses correctly whether a blob contains text or binary data by examining the beginning of the contents - It checks for any occurrence of a zero byte (NUL “character”) in the first 8000 bytes.
http://git-scm.com/docs/gitattributes
And from Git source:
#define FIRST_FEW_BYTES 8000
int buffer_is_binary(const char *ptr, unsigned long size)
{
if (FIRST_FEW_BYTES < size)
size = FIRST_FEW_BYTES;
return !!memchr(ptr, 0, size);
}
http://git.kernel.org/?p=git/git.git;a=blob;f=xdiff-interface.c;h=0e2c169227ad29b5bf546c6c1b97e1a1d8ed7409;hb=HEAD
And @tonfa makes a good point that "Also note that the only place where it cares about a file being text vs. binary is for diplaying diff, and for doing merges. The storage format does not care about it."
151
Mercurial looks for some occurence of the null character (\0) in the content of the file. If there's one, then the file is considered as binary. Otherwise it is considered as textual, unless explicitely mentionned.
I guess git uses the same approach.
33
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
