Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Least used delimiter character in normal text < ASCII 128

People also ask

What is the least used ASCII character?

In addition, while the less common symbols—including the ampersand—are the least common of all characters, a comma is actually seen more often than certain letters of the alphabet, such as 'v' or 'j'.

What are ASCII delimiters?

Delimited ASCII is a data format in which fields and records are separated by selected characters called delimiters. Field and record delimiters are distinct. You can set the delimiters in the source or target properties for this connector. This connector sets field width in bytes.

What delimiter character should you use for regular text input?

+1. Escaping is the only solution where any character can be in the data. I usually prefer | myself (along with \| and `\` as escapes) but tilde is as good as any other.

What delimiter should I use?

Any character may be used to separate the values, but the most common delimiters are the comma, tab, and colon. The vertical bar (also referred to as pipe) and space are also sometimes used.


I would choose "Unit Separator" ASCII code "US": ASCII 31 (0x1F)

In the old, old days, most things were done serially, without random access. This meant that a few control codes were embedded into ASCII.

ASCII 28 (0x1C) File Separator - Used to indicate separation between files on a data input stream.
ASCII 29 (0x1D) Group Separator - Used to indicate separation between tables on a data input stream (called groups back then).
ASCII 30 (0x1E) Record Separator - Used to indicate separation between records within a table (within a group).  These roughly map to a tuple in modern nomenclature.
ASCII 31 (0x1F) Unit Separator - Used to indicate separation between units within a record.  The roughly map to fields in modern nomenclature.

Unit Separator is in ASCII, and there is Unicode support for displaying it (typically a "us" in the same glyph) but many fonts don't display it.

If you must display it, I would recommend displaying it in-application, after it was parsed into fields.


Assuming for some embarrassing reason you can't use CSV I'd say go with the data. Take some sample data, and do a simple character count for each value 0-127. Choose one of the ones which doesn't occur. If there is too much choice get a bigger data set. It won't take much time to write, and you'll get the answer best for you.

The answer will be different for different problem domains, so | (pipe) is common in shell scripts, ^ is common in math formulae, and the same is probably true for most other characters.

I personally think I'd go for | (pipe) if given a choice but going with real data is safest.

And whatever you do, make sure you've worked out an escaping scheme!


Probably | or ^ or ~ you could also combine two characters


When using different languages, this symbol: ¬

proved to be the best. However I'm still testing.


You said "printable", but that can include characters such as a tab (0x09) or form feed (0x0c). I almost always choose tabs rather than commas for delimited files, since commas can sometimes appear in text.

(Interestingly enough the ascii table has characters GS (0x1D), RS (0x1E), and US (0x1F) for group, record, and unit separators, whatever those are/were.)

If by "printable" you mean a character that a user could recognize and easily type in, I would go for the pipe | symbol first, with a few other weird characters (@ or ~ or ^ or \, or backtick which I can't seem to enter here) as a possibility. These characters +=!$%&*()-'":;<>,.?/ seem like they would be more likely to occur in user input. As for underscore _ and hash # and the brackets {}[] I don't know.


How about you use a CSV style format? Characters can be escaped in a standard CSV format, and there's already a lot of parsers already written.