Converting Unicode codepoints to UTF-8 in C using iconv

Question

I want to convert a 32-bit value, which represents a Unicode codepoint, into a sequence of chars which is the utf-8 encoded string containing only the character corresponding to the codepoint.

For example, I want to turn the value 955 into the utf-8 encoded string "λ".

I tried to do this using iconv, but I could not get the desired result. Here is the code that I wrote:

#include <stdio.h>
#include <iconv.h>
#include <stdint.h>

int main(void)
{
  uint32_t codepoint = U'λ';
  char *input = (char *) &codepoint;
  size_t in_size = 2; // lower-case lambda is a 16-bit character (0x3BB = 955)

  char output_buffer[10];
  char *output = output_buffer;
  size_t out_size = 10;

  iconv_t cd = iconv_open("UTF-8", "UTF-32");

  iconv(cd, &input, &in_size, &output, &out_size);

  puts(output_buffer);

  return 0;
}

When I run it, only a newline is printed (puts automatically prints a newline,-- the first byte of outout_buffer is '\0').

What is wrong with my understanding or my implementation?

Answer 1

As said by minitech, you must use size = 4 for UTF32 in an uint32_t, and you must preset the buffer to null to have the terminating null after conversion.

This code works on Ubuntu :

#include <stdio.h>
#include <iconv.h>
#include <stdint.h>
#include <memory.h>

int main(void)
{
  uint32_t codepoint = 955;
  char *input = (char *) &codepoint;
  size_t in_size = 4; // lower-case lambda is a 16-bit character (0x3BB = 955)

  char output_buffer[10];
  memset(output_buffer, 0, sizeof(output_buffer));
  char *output = output_buffer;
  size_t out_size = 10;

  iconv_t cd = iconv_open("UTF-8", "UTF-32");

  iconv(cd, &input, &in_size, &output, &out_size);

  puts(output_buffer);

  return 0;
}

Answer 2

Two problems:

1. Since you’re using UTF-32, you need to specify 4 bytes. The “lower-case lambda is a 16-bit character (0x3BB = 955)” comment isn’t true for a 4-byte fixed-width encoding; it’s 0x000003bb. Set size_t in_size = 4;.

2. iconv doesn’t add null terminators for you; it adjusts the pointers it’s given. You’ll want to add your own before calling puts.

   *output = '\0';
   puts(output_buffer);