Vim: Encoding (Unicode) in Terminal under Windows

Question

I don't know why, but this topic seems to be badly documented and is covered with controversies as nobody knows the real answer (except for maybe Mr. Moolenaar, who rarely answers anyway).

So basically I've raised a discussion here, and it went dead pretty quickly, probably because there are not too many people using Vim in terminal mode on Windows.

My encoding settings look as follows:

if has('multi_byte')
  if empty(&termencoding)
    let &termencoding = &encoding
  endif
  let &encoding     = 'utf-8'
  let &fileencoding = 'utf-8'
endif

Of course, I have no problems running under GVim: can type any characters, and my patched Consolas for Powerline works just fine. The problems start when I try to run Vim in terminal mode. I use ConEmu, a feature-rich terminal emulator for Windows. It claims to officially support Unicode out of the box. For example, I can run the following test script:

chcp 65001 & (cmd /c type "%~dpn0.cmd") & pause & goto :EOF

English:     texts, web pages and documents
Graves,etc:  à á â ã ä å æ ç è é ê ë ì í î ï
Greek:       ΐ Α Β Γ Δ Ε Ζ Η Θ Ι Κ Λ Μ Ν Ξ Ο
Arabic:      ڠ ڡ ڢ ڣ ڤ ڥ ڦ ڧ ڨ ک ڪ ګ ڬ ڭ ڮ گ
Full width:  ＠ Ａ Ｂ Ｃ Ｄ Ｅ Ｆ Ｇ Ｈ Ｉ Ｊ Ｋ Ｌ Ｍ Ｎ Ｏ
Romanian:    texte, pagini Web şi a documentelor
Vietnamese:  văn bản, các trang web và các tài liệu
Russian:     тексты, веб-страницы и документы
Japanese:    テキスト、Webページや文書
Yiddish:     טעקסץ, וועב זייַטלעך און דאָקומענטן
Hindi:       पाठ, वेब पृष्ठों और दस्तावेज
Thai:        ข้อความ หน้า เว็บ และ เอกสาร
Korean:      텍스트, 웹 페이지 및 문서
Chinese:     文本，網頁和文件

and I can see all the symbols correctly in ConEmu. Yes, the test script turns on the 65001 codepage. I've already discovered that Vim cannot work with the 65001 codepage at all, so this seems not to be an option anyway. The default codepage in the terminal is 437, and I can also type something like Russian in ConEmu with this default codepage, and it is displayed correctly.

Reading through :h termencoding, I see that Windows uses Unicode by default to pass symbols. Then, I don't understand why when I type anything non-ANSI in terminal Vim, I see ? symbols? Airline does not display fancy symbols from patched Consolas as well. How to configure true Unicode for terminal Vim on Windows? By the way, &termencoding reports 437 as well.

Could somebody, once and for all, please, explain to me is Unicode support for terminal Vim on Windows there (and how to configure it) or not?

Answer 1

I've wondered about this myself too and in the past tried ConEmu and gave up after struggling to get console vim with 256 colors and fancy fonts working on it.

So today I tried out for sometime again and surprise, surprise - things seem to be working. Given all the extreme sensitiveness to versions, I'm going to try and list down versions of everything

VIM - Vi IMproved 7.4 (2013 Aug 10, compiled Aug  1 2014 09:38:34)
MS-Windows 32-bit console version
Included patches: 1-389
Compiled by raghuramanr@ADITI

ConEmu

140723 Alpha

Windows: Win 7x64

ConEmu settings in .vimrc:

" ConEmu
if !empty($CONEMUBUILD)
    echom "Running in conemu"
    set termencoding=utf8
    set term=xterm
    set t_Co=256
    let &t_AB="\e[48;5;%dm"
    let &t_AF="\e[38;5;%dm"
    " termcap codes for cursor shape changes on entry and exit to
    " /from insert mode
    " doesn't work
    "let &t_ti="\e[1 q"
    "let &t_SI="\e[5 q"
    "let &t_EI="\e[1 q"
    "let &t_te="\e[0 q"
endif

Steps:

chcp 65001
vim.exe

I still can't get a blinking cursor in vim which is confusing. Still better than before when stuff would be messed up.