How does Vim guess background color on xterm?

Question

Vim has the ability to correctly guess the background color of an Xterm in order to set its internal option bg to either dark or white according to the terminal's one. Vim is able to do that correctly only when TERM is set to either xterm{,-color,-256color} or linux but no others like tmux or screen.

How does Vim to guess that?

I've found that most people forces setting up the background option to either dark or light in their .vimrc file; but I'd like a way to guess the same way Vim does, independently of the terminal being xterm, tmux, screen.

Answer 1

The default setting is defined (hard-coded) in Vim source code (written in the C programming language). There was a fix to not use "dark" in GUI variants of Vim with "linux" TERM since version 6.1.136, and it helps me to find actual code:

http://ftp.twaren.net/vim/patches/6.1.136

Patch 6.1.136
Problem:    When $TERM is "linux" the default for 'background' is "dark", even
        though the GUI uses a light background. (Hugh Allen)
Solution:   Don't mark the option as set when defaulting to "dark" for the
        linux console.  Also reset 'background' to "light" when the GUI
        has a light background.
Files:      src/option.c

The logic is here, default values: https://fossies.org/dox/vim-7.4/option_8c_source.html#l00566

  563     {"background",  "bg",   P_STRING|P_VI_DEF|P_RCLR,
  564                 (char_u *)&p_bg, PV_NONE,
  565                 {
  566 #if (defined(MSDOS) || defined(OS2) || defined(WIN3264)) && !defined(FEAT_GUI)
  567                 (char_u *)"dark",
  568 #else
  569                 (char_u *)"light",
  570 #endif

Detect terminal background: https://fossies.org/dox/vim-7.4/option_8c_source.html#l03754

 3725     /* For DOS console the default is always black. */
 3726 #if !((defined(MSDOS) || defined(OS2) || defined(WIN3264)) && !defined(FEAT_GUI))
 3727     /*
 3728      * If 'background' wasn't set by the user, try guessing the value,
 3729      * depending on the terminal name.  Only need to check for terminals
 3730      * with a dark background, that can handle color.
 3731      */
 3732     idx = findoption((char_u *)"bg");
 3733     if (idx >= 0 && !(options[idx].flags & P_WAS_SET)
 3734                          && *term_bg_default() == 'd')
 3735     {
 3736     set_string_option_direct(NULL, idx, (char_u *)"dark", OPT_FREE, 0);
 3737     /* don't mark it as set, when starting the GUI it may be
 3738      * changed again */
 3739     options[idx].flags &= ~P_WAS_SET;
 3740     }
 3741 #endif
 3754 /*
 3755  * Return "dark" or "light" depending on the kind of terminal.
 3756  * This is just guessing!  Recognized are:
 3757  * "linux"      Linux console
 3758  * "screen.linux"   Linux console with screen
 3759  * "cygwin"     Cygwin shell
 3760  * "putty"      Putty program
 3761  * We also check the COLORFGBG environment variable, which is set by
 3762  * rxvt and derivatives. This variable contains either two or three
 3763  * values separated by semicolons; we want the last value in either
 3764  * case. If this value is 0-6 or 8, our background is dark.
 3765  */
 3766     static char_u *
 3767 term_bg_default()
 3768 {
 3769 #if defined(MSDOS) || defined(OS2) || defined(WIN3264)
 3770     /* DOS console nearly always black */
 3771     return (char_u *)"dark";
 3772 #else
 3773     char_u  *p;
 3774
 3775     if (STRCMP(T_NAME, "linux") == 0
 3776         || STRCMP(T_NAME, "screen.linux") == 0
 3777         || STRCMP(T_NAME, "cygwin") == 0
 3778         || STRCMP(T_NAME, "putty") == 0
 3779         || ((p = mch_getenv((char_u *)"COLORFGBG")) != NULL
 3780         && (p = vim_strrchr(p, ';')) != NULL
 3781         && ((p[1] >= '0' && p[1] <= '6') || p[1] == '8')
 3782         && p[2] == NUL))
 3783     return (char_u *)"dark";
 3784     return (char_u *)"light";
 3785 #endif
 3786 }

GUI fix:

 4044 gui_bg_default()
 4045 {
 4046     if (gui_get_lightness(gui.back_pixel) < 127)
 4047     return (char_u *)"dark";
 4048     return (char_u *)"light";
 4049 }

UPDATE: For vim-7.4.1689 (debian/ubuntu) I rebuilt Vim package with debugging info (-O2 -g default option; before strip and pack into deb) and ran TERM=xterm gdb --args vim-7.4.1689/src/vim-basic/vim -e with watch p_bg. First change of p_bg was by term_bg_default() to light, and second is from ... main: may_req_termresponse() (if FEAT_TERMRESPONSE was defined) -> vpeekc_nomap -> vpeekc -> vgetorpeek -> check_termcode -> set_option_value("bg",..) -> set_string_option.

https://github.com/vim/vim/blob/54c10ccf9274880e83093a99690e7bfa9a2d2fa8/src/term.c

Line 3302 - may_req_bg_color() color, called from main.c just after starttermcap() / log message "start termcap"; I added preprocessed definition of the request:

/*
 * Similar to requesting the version string: Request the terminal background
 * color when it is the right moment.
 */
    void
may_req_bg_color(void)
...

    {(int)KS_RBG,   "\033]11;?\007",

    {(int)KS_RBG,   "[RBG]"},

Line 4286 - handle response from termcap request:

check_termcode( ....

    /* Check for background color response from the terminal:
     *
     *       {lead}11;rgb:{rrrr}/{gggg}/{bbbb}{tail}
     *
     * {lead} can be <Esc>] or OSC
     * {tail} can be '\007', <Esc>\ or STERM.
     *
     * Consume any code that starts with "{lead}11;", it's also
     * possible that "rgba" is following.
     */

        if (i - j >= 21 && STRNCMP(tp + j + 3, "rgb:", 4) == 0
            && tp[j + 11] == '/' && tp[j + 16] == '/'
            && !option_was_set((char_u *)"bg"))
        {/* TODO: don't set option when already the right value */
            LOG_TR("Received RBG");
            rbg_status = RBG_GOT;
            set_option_value((char_u *)"bg", 0L, (char_u *)(
                (3 * '6' < tp[j+7] + tp[j+12] + tp[j+17])
                ? "light" : "dark"), 0);
            reset_option_was_set((char_u *)"bg");

Added by 7.4.757 patch http://ftp.vim.org/vim/patches/7.4/7.4.757

Patch 7.4.757
Problem:    Cannot detect the background color of a terminal.
Solution:   Add T_RBG to request the background color if possible. (Lubomir
            Rintel)
Files:      src/main.c, src/term.c, src/term.h, src/proto/term.pro

#define T_RBG   (term_str(KS_RBG))  /* request background RGB */

Your tmux/screen may not implement '[RBG]' request; check by running in xterm and in tmux and compare outputs:

echo -e '\033]11;?\007'

---

You may define COLORFGBG (and there were bugs about it in neovim: https://github.com/neovim/neovim/issues/2764) There is bug report for gnome-terminal to set it https://bugzilla.gnome.org/show_bug.cgi?id=733423

Various terminals, including urxvt and konsole, set an environment variable "COLORFGBG" to allow applications to detect the foreground and background colors. Various programs, such as Vim, use this to determine whether to use a color scheme that works best on a light or dark background. Please consider setting this variable in gnome-terminal as well.

Egmont has some option in the bug https://bugzilla.gnome.org/show_bug.cgi?id=733423 to set COLORFGBG with .bashrc.

You may change this in your "vimrc" (~/.vimrc), or globally on your OS by changing /usr/share/vim/vimrc or in vim sources by recompiling custom version or in Vim sources by reporting a bug to Vim authors / sending them patches.

The documentation of "bg" describes part of logic and gives solutions for vimrc:

When setting 'background' to the default value with:

  :set background&

Vim will guess the value. In the GUI this should work correctly, in other cases Vim might not be able to guess the right value.

When starting the GUI, the default value for 'background' will be "light". When the value is not set in the .gvimrc, and Vim detects that the background is actually quite dark, 'background' is set to "dark". ....

Normally this option would be set in the .vimrc file. Possibly depending on the terminal name. Example:

      :if &term == "pcterm"
        :  set background=dark
        :endif

When this option is set, the default settings for the highlight groups will change. To use other settings, place ":highlight" commands AFTER the setting of the 'background' option.