Rune vs byte ranging over string

Question

According to https://blog.golang.org/strings and my testings, it looks like while we range a string, the characters we get are rune type, but if we get it by str[index], they will be byte type, why is it?

Answer 1

To the first level, the why is because that's how the language is defined. The String type tells us that:

A string value is a (possibly empty) sequence of bytes. The number of bytes is called the length of the string and is never negative. Strings are immutable: once created, it is impossible to change the contents of a string.

and:

A string's bytes can be accessed by integer indices 0 through len(s)-1.

Meanwhile, range is a clause you can insert into a for statement, and the specification says:

The expression on the right in the "range" clause is called the range expression, which may be ... [a] string ...

and:

2. For a string value, the "range" clause iterates over the Unicode code points in the string starting at byte index 0. On successive iterations, the index value will be the index of the first byte of successive UTF-8-encoded code points in the string, and the second value, of type rune, will be the value of the corresponding code point. If the iteration encounters an invalid UTF-8 sequence, the second value will be 0xFFFD, the Unicode replacement character, and the next iteration will advance a single byte in the string.

If you want to know why the language is defined that way, you really have to ask the definers themselves. However, note that if for ranged only over the bytes, you'd need to construct your own fancier loops to range over the runes. Given that for ... range does work through the runes, if you want to work through the bytes in string s instead, you can write:

for i := 0; i < len(s); i++ {
    ...
}

and easily access s[i] inside the loop. You can also write:

for i, b := range []byte(s) {
}

and access both index i and byte b inside the loop. (Conversion from string to []byte, or vice versa, can require a copy since []byte can be modified. In this case, though, the range does not modify it and the compiler can optimize away the copy. See icza's comment below or this answer to golang: []byte(string) vs []byte(*string).) So you have not lost any ability, just perhaps a smidgen of concision.

Answer 2

Just a quick and simple answer on why the language is defined this way.

Think about what a rune is. A rune represents a Unicode code point, which can be composed of multiple bytes and also have different representations depending on the encoding.

Now think what doing mystring[i] would mean if that returned a rune and not a byte. Since you cannot know the length of each rune without scanning the string, that operation would require scanning the whole string every single time, thus making array-like access take O(n) instead of O(1).

It would be very counter-intuitive for the users of the language if mystring[i] scanned the whole string every time, and also more complex for the language developers. This is why most programming languages (like Go, Rust, Python) differentiate between Unicode characters and bytes, and sometimes only support indexing on bytes.

Accessing a string one rune at a time is instead much simpler when iterating from the beginning of it, like for example using range. Consecutive bytes can be scanned and grouped together until they form a valid Unicode character that can be returned as a rune, moving on to the next one.