Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Unicode sample text file for testing for Unicode related problems? [closed]

I am looking for a sample text unicode file (UTF-8) that can be used for testing different problems related with text encoding and decoding including:

  • low ascii character usage, like first 32 codes
  • characters outside BMP
  • NFC related issues
  • XML encoding/decoding issues

Mainly I want to copy the text into clipboard, paste it in an HTML text-area of the application, and be able to retrieve it from a page after.

This would enable to identify different Unicode related problems that could occur at decoding, encoding or even database level.

like image 817
sorin Avatar asked Feb 17 '23 13:02

sorin


1 Answers

This page has been used to test web browsers, with texts in several scripts: https://www.kermitproject.org/utf8.html

The Gothic entry for "I can eat glass" in particular is outside of BMP: 𐌼𐌰𐌲 πŒ²πŒ»πŒ΄πƒ πŒΉΜˆπ„πŒ°πŒ½, 𐌽𐌹 πŒΌπŒΉπƒ π…πŒΏ 𐌽𐌳𐌰𐌽 πŒ±π‚πŒΉπŒ²πŒ²πŒΉπŒΈ.

Normalization forms and XML processing are usually not problematic when moving data around, so there are no common samples that test those two in particular.

like image 112
Joni Avatar answered Apr 27 '23 05:04

Joni