Any way to process escape key in canonical mode?

Question

In unix plain C termios programming, if I am using canonical mode to receive a line of input from the user, how can I process the escape key? In general, if the user is entering a line of text and presses escape nothing happens. I would like to cancel the current input if the user presses escape. I know I can process individual characters, but then I lose all the benefits of canonical mode (backspaces, etc).

Answer 1

Original

With all due credits to Jonathan Leffler for his comment that hinted me at the right direction, at the bottom is my annotated first termios program demonstrator (Thanks!).

The key is to use tcgetattr(ttyfd, &attributes) on the current terminal's file descriptor to retrieve its current attributes into in a struct termios, edit the attributes, then apply the changes with tcsetattr(ttyfd, when, &attributes).

One of the attributes is the "kill" character - the character that causes the entire currently-buffered line to be discarded. It is set by indexing into the c_cc member array of struct termios and setting attr.c_cc[VKILL] to whatever one wants (Here, to Esc, which is equal to octal 033).

The kill character should be restored to its previous value on exit.

#include <termios.h>
#include <fcntl.h>
#include <stdio.h>


int main(){
    char   buf[80];
    int    numBytes;
    struct termios original, tattered;
    int    ttyfd;
    
    /* Open the controlling terminal. */
    ttyfd = open("/dev/tty", O_RDWR);
    if(ttyfd < 0){
        printf("Could not open tty!\n");
        return -1;
    }
    
    /**
     * Get current terminal properties, save them (including current KILL char),
     * set the new KILL char, and make this the new setting.
     */
    
    tcgetattr(ttyfd, &original);
    tattered = original;
    tattered.c_cc[VKILL] = 033;/* New killchar, 033 == ESC. */
    tcsetattr(ttyfd, TCSANOW, &tattered);
    
    /**
     * Simple test to see whether it works.
     */
    
    write(1, "Please enter a line: ", 21);
    numBytes = read(0, buf, sizeof buf);
    write(1, buf, numBytes);
    
    /**
     * Restore original settings.
     */
    
    tcsetattr(ttyfd, TCSANOW, &original);
     
    /* Clean up. */
    close(ttyfd);
    
    return 0;
}

This demo appears to work on Mac OS X 10.6.8. I've also tested this on Linux, and apparently Esc to kill the buffer appears to be the default, as if I print out c_cc[VKILL] I obtain 27 == 033 == ESC.

Edit

The below attempts as closely as possible to imitate the behaviour you described in your comment. It sets c_cc[VEOL2] to Esc; EOL2 is the alternate End-of-Line. It also removes Esc as the kill character, since you want to receive the line.

What now happens is that if a normal Ret is pressed, all is normal. However, if Esc is pressed, the last character in the buffer is set to Esc, a condition which may be tested (although only after reading and buffering the whole line first).

Below is a demonstrator according to your clarified specs. It waits for a line of input and echoes it back with

• <CANCELLED> if the line was terminated with Esc and
• <NORMAL > if the line was terminated with Ret.

Enjoy!

#include <termios.h>
#include <fcntl.h>
#include <stdio.h>


int main(){
    char   buf[80];
    int    numBytes;
    struct termios original, tattered;
    int    ttyfd;

    /* Open the controlling terminal. */
    ttyfd = open("/dev/tty", O_RDWR);
    if(ttyfd < 0){
        printf("Could not open tty!\n");
        return -1;
    }

    /**
     * Get current terminal properties, save them (including current KILL char),
     * set the new KILL char, and make this the new setting.
     */

    tcgetattr(ttyfd, &original);
    tattered = original;
    tattered.c_cc[VKILL] = 0;  /* <Nada> */
    tattered.c_cc[VEOL2] = 033;/* Esc */
    tcsetattr(ttyfd, TCSANOW, &tattered);

    /**
     * Simple test to see whether it works.
     */

    fputs("Please enter a line: ", stdout);
    fflush(stdout);
    numBytes = read(0, buf, sizeof buf);
    if(buf[numBytes-1]==033){/* Last character is Esc? */
        buf[numBytes-1] = '\n';/* Substitute with newline */
        fputs("\n<CANCELLED> ", stdout);   /* Print newline to move to next line */
    }else{
        fputs("<NORMAL   > ", stdout);
    }
    fwrite(buf, 1, numBytes, stdout);

    /**
     * Restore original settings.
     */

    tcsetattr(ttyfd, TCSANOW, &original);

    /* Clean up. */
    close(ttyfd);

    return 0;
}

Answer 2

You need to set the EOF character to ESC instead of Enter using the tcsetattr() function. For more detailed info visit http://pubs.opengroup.org/onlinepubs/7908799/xbd/termios.html#tag_008_001_009