Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is there a strlen() that works with char16_t?

Tags:

c

unicode

c11

As the question says:

typedef __CHAR16_TYPE__ char16_t; 

int main(void)
{
  static char16_t test[] = u"Hello World!\n";

  printf("Length = %d", strlen(test)); // strlen equivalent for char16_t ???

  return 0;
}

I searched and found only C++ solutions.

My compiler is GCC 4.7.

Edit:

To clarify, I was searching for a solution that returns the count of code points, not the count of characters.

These two are quite different for UTF-16 strings containing characters outside the BMP.

like image 394
Jens Mühlenhoff Avatar asked Jan 25 '13 18:01

Jens Mühlenhoff


People also ask

What can I use instead of strlen?

If you're using a char[] , then you can use sizeof(str) - 1 instead.

Can we use strlen with string?

For C++ strings, there's no reason to use strlen . Just use string::length : Clarity: The length() (or size() ) member functions unambiguously give back the length of the string.

Can strlen fail?

Ok, I need to add some explanation. My application is getting a string from a shared memory (which is of some length), therefore it could be represented as an array of characters. If there is a bug in the library writing this string, then the string would not be zero terminated, and the strlen could fail.


3 Answers

Here's your basic strlen:

int strlen16(const char16_t* strarg)
{
   int count = 0;
   if(!strarg)
     return -1; //strarg is NULL pointer
   char16_t* str = strarg;
   while(*str)
   {
      count++;
      str++;
   }
   return count;
}

Here's a more efficient and popular strlen:

int strlen16(const char16_t* strarg)
{
   if(!strarg)
     return -1; //strarg is NULL pointer
   char16_t* str = strarg;
   for(;*str;++str)
     ; // empty body
   return str-strarg;
}

Hope this helps.

Warning: This doesn't work properly when counting the characters (not code points) of a UTF-16 string. This is especially true when __STDC_UTF_16__ is defined to 1.

UTF-16 is variable length (2 bytes per character in the BMP or 4 bytes per character outside the BMP) and that is not covered by these functions.

like image 106
askmish Avatar answered Oct 12 '22 16:10

askmish


std::char_traits has this.

#include <string>

std::char_traits<char16_t>::length(yourchar16pointerhere);
like image 31
Raven Avatar answered Oct 12 '22 16:10

Raven


#include <string.h>
#include <wchar.h>
#include <uchar.h>

#define char8_t char
#define strlen8 strlen
#define strlen16 strlen16
#define strlen32(s) wcslen((const wchar_t*)s)

static inline size_t strlen16(register const char16_t * string) {
    if (!string) return 0;
    register size_t len = 0;
    while(string[len++]);
    return len;
}

You should expect the number of char16_t characters to be returned, as opposed to byte count.

Optimized 32-Bit Intel Atom Assembly View:

gcc -Wpedantic -std=iso9899:2011 -g3 -O2 -MMD -faggressive-loop-optimizations -fkeep-inline-functions -march=atom -mtune=atom -fomit-frame-pointer -mssse3 -mieee-fp -mfpmath=sse -fexcess-precision=fast -mpush-args -mhard-float -fPIC ...

.Ltext0:
    .p2align 4,,15
    .type   strlen16, @function
strlen16:
.LFB20:
    .cfi_startproc
.LVL0:
    mov edx, DWORD PTR 4[esp]
    xor eax, eax
    test    edx, edx
    je  .L4
    .p2align 4,,15
.L3:
.LVL1:
    lea eax, 1[eax]
.LVL2:
    cmp WORD PTR -2[edx+eax*2], 0
    jne .L3
    ret
.LVL3:
    .p2align 4,,7
    .p2align 3
.L4:
    ret
    .cfi_endproc
.LFE20:
    .size   strlen16, .-strlen16

Here an Intel disassembly:

static inline size_t strlen16(register const char16_t * string) {
   0:   8b 54 24 04             mov    edx,DWORD PTR [esp+0x4]
    if (!string) return 0;
   4:   31 c0                   xor    eax,eax
   6:   85 d2                   test   edx,edx
   8:   74 16                   je     20 <strlen16+0x20>
   a:   8d b6 00 00 00 00       lea    esi,[esi+0x0]
    register size_t len = 0;
    while(string[len++]);
  10:   8d 40 01                lea    eax,[eax+0x1]
  13:   66 83 7c 42 fe 00       cmp    WORD PTR [edx+eax*2-0x2],0x0
  19:   75 f5                   jne    10 <strlen16+0x10>
  1b:   c3                      ret    
  1c:   8d 74 26 00             lea    esi,[esi+eiz*1+0x0]
    return len;
}
  20:   c3                      ret    
  21:   eb 0d                   jmp    30 <AnonymousFunction0>
  23:   90                      nop
  24:   90                      nop
  25:   90                      nop
  26:   90                      nop
  27:   90                      nop
  28:   90                      nop
  29:   90                      nop
  2a:   90                      nop
  2b:   90                      nop
  2c:   90                      nop
  2d:   90                      nop
  2e:   90                      nop
  2f:   90                      nop
like image 28
Hypersoft Systems Avatar answered Oct 12 '22 15:10

Hypersoft Systems