What alternative user input techniques should be adopted in programming?

Question

Programming is particularly different to, for example word processing due to the wealth of special symbols etc that need to be entered.

Of the current crop of new user interface techniques, which are suited to programming and why?

Or is the idea of a language syntax the problem, should we be programming more symbolically, and if so, how would this affect user interface?

Edit: When I specified user interface techniques, i left it open to both using existing hardware (mouse/keyboard) and some other things, like multi-touch, gesture recognition, augmented reality (see HitLabNz for some great examples). I'm interested in how we can apply these to programming.

Answer 1

I was just thinking about this. I was going to write a blog post about it, I may as well get started here. In programming, I think we need more than just a new input method, there needs to be a new metaphor to go with it. It's a three tiered thing. Model-Metaphor-Interface.

I've been thinking more and more lately that language is a poor metaphor for representing a computation. Language is something we use for communication. You could look at a program as a communication to a computer, and simultaneously a communication to other programmers. But there are other ways to communicate other than just the written word. I'm working on a list here, feel free to edit this post to add more stuff to the list.

Methods of Communication

• Speech
• Body Language
• Gestures
• Facial Expressions
• Sign Language
• Painting/Drawing
• dials, buttons, sliders, pointing, dragging (gui)

Another metaphor for programming is building. Here's some possible ways of building functional things, that could form the basis for a programming interface

Methods of building functional things

• Gears/springs and other mechanics
• paper folding/cutting/gluing
• patch cords
• electronic circuits
• hinges, ball bearings, wheels
• fountains valves and pipes
• Archimedes machines: pulleys, levers, screws
• Lego

Yet another way of specifying a computation is by definition.

Methods of Definition - Constraints - Categorization - Set Theory - Properties - Symptoms - Logic tables - Rules - Railroads (as in railroad diagrams)

But keep in mind why we're doing this. There's obviously some weaknesses in the way programming languages work now, (otherwise we wouldn't want to make new languages) so let's keep them in mind while we're designing our new languages

problems with current languages

• The interface is hidden

• the APIs are hidden

• Side effects are a huge cause of bugs- Any part of a program can effect any other part.

• Refactoring- Sometimes you find that you're repeating yourself, so you need an easy way to factor out the repetition into a macro, or a function, or some other metaphor. This is largely done by hand (or semi-automatically in java) by a massive text manipulation effort. Is there a new metaphor that would make such a thing look utterly silly?

• You need an easy way to define your own building blocks, or "words" or idioms, to use to build more complex structures. Your own tools, your own parts of the environment. A lot of languages don't let you do this in a first class way.

• compilers punish the programmer severely for the slightest mistake.

• Variables lack a sense of time- There's no way to query the history of all the values a variable has been set to in the past. In other words, can we have a programming language where we can "rewind" the progress of our program? The fact that a variable can change, frequently to unexpected values is another source of bugs. This is the other half of the side effects problem

• most programming languages have a fairly steep learning curve

• Making reference to library or widget X throughout your code largely marries you to that library- Making it difficult to switch to a similar equivalent library without a lot of refactoring. This is largely to do with the fact that libraries have names, and in order to use a library, we're hardcoding the name of that library and its methods throughout our code. Is there a better way?

• Poor parallelism, multithreading leads to bugs, race conditions, deadlocks. Is there a better approach to parallelism that makes such bugs impossible? This issue alone is causing the creation of many new languages.

• Think beyond the computer screen, people. Maybe the keyboard is the most efficient interface for entering in complex relationships and symbols. Are you sure? There are more alternatives than just a mouse, or a touch screen, or a tablet. Zillions of ways of interacting with a computer- We have just all settled on one or two rather ordinary ways.

Answer 2

Just about every effort to have a non-textual programming language has fallen flat on its face. It's very hard to be both precise and efficient without a textual language.

Where a lot of the UI effort goes is in making better tools. For example, you can just use a simple text editor to do the programming or you can have full-blown IDEs like Visual Studio or Eclipse. Beyond that, there are visualization and design tools like Rational Rose. These tools offer complementary ways of exploring and/or modifying the underlying code.