Accessible screenreading of JavaFX "console" output

Question

I'm trying to create a simple UI that is accessible for screenreaders. I've been mostly successful, but I can't manage to design the UI in a way that has the screenreader read new text output.

Currently, I have a TextArea displaying the output of an anonymous PrintStream created and set by System.setOut. Sometimes I open up a TextField for string inputs, but I've been working with just the TextArea to test the reading of text (for now it just listens for keystrokes to display more text for testing purposes).

The issue is this: when new text is added via System.out to the TextArea, the screenreader does not read it. I am still able to navigate upward with the arrow keys to read what was added but it is not read when first added. Is there any way to get the screenreader to treat my TextArea more like a standard console (in which it reads all new text automatically)? I'm using NVDA.

Things I have tried:
- Using TextArea.notifyAccessibleAttributeChanged(AccessibleAttribute.TEXT)
- Using TextArea.requestFocus() and TextArea.notifyAccessibleAttributeChanged(AccessibleAttribute.FOCUS_NODE)
- Disabling autoflush on the PrintStream while using TextArea.setAccessibleText(theNewText) during a flush
- Using a hidden Label set to the new text and focusing on it (I'm still fiddling with this one; Screenreaders can't read actual "hidden" text so I'm trying to find a way to draw it but also be "invisible", perhaps behind the TextArea somehow)
- Changing focus to another Node and back, which doesn't work as I like because it reads the other Nodes accessible stuff and then reads the entire body of the TextArea
- Various combinations of these

I just can't seem to get it to work. I feel like I'm missing something simple and obvious here, but the JavaFX Accessibility API is still relatively new and I can't find solutions to specific problems like this one.

Here's the relevant code of my Application, if it helps any:

@Override
public void start(Stage primaryStage) {
    try {
        primaryStage.setTitle("Test");
        root = new BorderPane();
        root.setFocusTraversable(false);
        Scene scene = new Scene(root,800,600);
        scene.getStylesheets().add(getClass().getResource("application.css").toExternalForm());
        primaryStage.setScene(scene);
        //Create middle
        output = new TextArea();
        output.setEditable(false);
        output.setFocusTraversable(false); //I've tried true also, just to test
        output.setAccessibleRole(AccessibleRole.TEXT_AREA);
        root.setCenter(output);
        ...
        //Begin
        primaryStage.show();
        Thread th = new Thread(new AppMain());
        th.start();
    } catch(Exception e) {
        e.printStackTrace();
    }
}

@Override
public void init() {
    //Set output to TextArea
    System.setOut(new PrintStream(new OutputStream() {
        @Override
        public void write(int b) throws IOException {
            appendTextArea(String.valueOf((char) b));
        }
    }, true)); //I've also overriden flush while this is false, see above
}

public void appendTextArea(String str) {
    Platform.runLater(() -> {
        output.appendText(str);
    });
}

I seriously appreciate any help or suggestions you can provide. I've been messing with this small issue for way too long, and I'm still new to JavaFX. Thank you!

Answer 1

Here is a full working example based off of your code.

Disclosure: For the screen reader I'm using the "Voice Over Utility" on my Mac, but hopefully that doesn't differ too much from your environment.

The key was to utilize the Control#executeAccessibleAction method.

Example:

TextArea.executeAccessibleAction(AccessibleAction.SET_TEXT_SELECTION, start, finish);

The Application Class

package application;

import java.io.PrintStream;
import java.io.IOException;
import java.io.OutputStream;

import javafx.application.Application;
import javafx.application.Platform;
import javafx.stage.Stage;
import javafx.scene.AccessibleAction;
import javafx.scene.AccessibleRole;
import javafx.scene.Scene;
import javafx.scene.control.TextArea;
import javafx.scene.layout.BorderPane;

public class Main extends Application
{
    private TextArea    output;
    private BorderPane  root;

    private final StringBuilder STR_BUFFER = new StringBuilder();
    private static final String NEW_LINE   = System.lineSeparator();

    @Override
    public void start(Stage primaryStage)
    {
        try
        {
            primaryStage.setTitle("Test");
            root = new BorderPane();
            root.setFocusTraversable(false);
            Scene scene = new Scene(root, 800, 600);
            scene.getStylesheets().add(getClass().getResource("application.css").toExternalForm());
            primaryStage.setScene(scene);

            // Create middle
            output = new TextArea();
            output.setEditable(false);
            output.setFocusTraversable(true); // I've tried true also, just to test 

            // ----------------------------------------------
            // Tell the Screen Reader what it needs to access
            // ----------------------------------------------
            output.setAccessibleRole(AccessibleRole.TEXT_AREA);

            root.setCenter(output);
            // ...

            // Begin
            primaryStage.show();

            // start the thread
            Thread th = new Thread(new AppMain());
            th.start();
        }
        catch (Exception e)
        {
            e.printStackTrace();
        }
    }

    @Override
    public void init()
    {
        // Set output to TextArea when we have a full string
        System.setOut(new PrintStream(new OutputStream()
        {
            @Override
            public void write(int b) throws IOException
            {
                if (b == '\r')
                {
                    return;
                }
                if (b == '\n')
                {
                    final String text = STR_BUFFER.toString() + NEW_LINE;
                    appendTextArea(text);
                    STR_BUFFER.setLength(0);
                }
                else
                {
                    STR_BUFFER.append((char) b);
                }
            }
        }, true));
    }

    public void appendTextArea(String str)
    {
        Platform.runLater(() ->
        {
            int anchor = output.getText().length();

            output.appendText(str);

            // just to clear it
            output.positionCaret(0);

            // ----------------------------------------------
            // Tell the Screen Reader what it needs to do
            // ----------------------------------------------
            output.executeAccessibleAction(AccessibleAction.SET_TEXT_SELECTION, anchor, anchor + str.length());
        });
    }

    public static void main(String[] args)
    {
        launch(args);
    }
}

The Thread Class

/*
 * Just to simulate a feed to the console (textArea).
 * This will die after 1 minute.
 */
package application;

public class AppMain implements Runnable
{
    @Override
    public void run()
    {
        int i = 0;
        long start = System.currentTimeMillis();

        while (System.currentTimeMillis() - start < 60000)
        {
            try
            {
                Thread.sleep(3000);
            }
            catch (InterruptedException e)
            {}
            System.out.println("This is line number " + ++i);
        }
    }
}