Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to know the number of characters in utf8 string

i want to know is there a simple way to determine the number of characters in UTF8 string. For example, in windows it can be done by:

  1. converting UTF8 string to wchar_t string
  2. use wcslen function and get result

But I need more simpler and crossplatform solution.

Thanks in advance.

like image 892
akmal Avatar asked Dec 22 '22 09:12

akmal


2 Answers

UTF-8 characters are either single bytes where the left-most-bit is a 0 or multiple bytes where the first byte has left-most-bit 1..10... (with the number of 1s on the left 2 or more) followed by successive bytes of the form 10... (i.e. a single 1 on the left). Assuming that your string is well-formed you can loop over all the bytes and increment your "character count" every time you see a byte that is not of the form 10... - i.e. counting only the first bytes in all UTF-8 characters.

like image 88
borrible Avatar answered Dec 31 '22 10:12

borrible


The entire concept of a "number of characters" does not really apply to Unicode, as codes do not map 1:1 to glyphs. The method proposed by @borrible is fine if you want to establish storage requirements in uncompressed form, but that is all that it can tell you.

For example, there are code points like the "zero width space", which do not take up space on the screen when rendered, but occupy a code point, or modifiers for diacritics or vowels. So any statistic would have to be specific to the concrete application.

A proper Unicode renderer will have a function that can tell you how many pixels will be used for rendering a string if that information is what you're after.

like image 25
Simon Richter Avatar answered Dec 31 '22 10:12

Simon Richter