TL;DR: make complex UIs driven by a domain-specific command-line language, which itself has interactive controls built into the flow of the commands.

Am I a crazy linuxperson who prefers to sit in front of a terminal all day? Nope.

Some fifteen years ago I was lucky to witness Autocad’s interface that’s pretty much all GUI, but that was having this little terminal with its own command language where you could draw things with precision. “Move 1 cm left, draw 2 cm forward”. You type some commands with precise numbers in a comfortably large textfield, and you see the results immediately. Mistyped something? Undo and tweak the command. You get the same WYSIWYG feedback, but without tinkering with mouse over millions of tiny textfields, dropdowns, panels and palettes.

Some seventeen years ago I was lucky to learn Wolfram Mathematica. That was quite different: a primarily text-based interface, but supercharged to be interactive, structured, with rich formatting and interactivity. This is how a command-line within a GUI should feel like.

I used Photoshop years before that day and then years after. And I still wish that maybe someday someone would make a graphics editor like Autocad was doing for ages, but making it even more interactive and responsive, akin to Mathematica’s notebook language.

When to use command line?

Command line may be intimidating for some simple, isolated and possibly rarely performed tasks. But when amount of compound complexity shoots through the roof, the right click menus and millions of tiny windows and modal dialogue panels just don’t scale.

When you need to do X, then Y, then Z and customize each action with 5 parameters out of 55, and then try the same with various tweaks here and there, you really want a streamlined interface. And only a text-like flow is going to deliver that.

Pros of a command-line driven UI:

1. Precision: just type “thickness 0.33” and it’ll be 0.33 in an instance. In Photoshop today you have to either quickly drag a slider somewhere, which is not precise, but quick. Or hunt for a teeny-tiny textbox, then click it, then enter numbers. Awful!
2. Composition: if you need to express “same brush thickness as half the margin between these two things” you can do just that, instead of manually calculating and typing two-three numbers in various spots several times over, as you try different combinations. Just re-run the same command with one number tweaked each time.
3. Collaboration: you can copy-paste actions and edit them like a regular source code, without wasting time navigating through windows and buttons or pressing quite invisible hotkeys after some video tutorial.
4. Editable history: if your whole document is driven by the command line, you can simply store the whole document as a giant list of commands. So you can go back and rewrite/refactor it to your liking. Things can get much more powerful, for free: you don’t have to do advanced tinkering with the commands unless you really want to.

There are, of course, some problems, but we can use computers to solve computer problems:

Discoverability: how do i learn the syntax of all the commands?

First, if you have a language aimed at composability, your command set would probably be a fraction of the number of menus and windows that you already have. E. g. you won’t be having three different ways to apply color adjustment, if you have a line that describes the adjustment, and another line that attaches it to a group of layers.

Second, no one prevents you from finding command in-place as you type and showing placeholder bubbles and documentation, or even interactive controls, in-place. Have you ever typed formulas in Excel? Same thing, and you can make it arbitrarily smart, with fuzzy text matching and contextual interactive UI elements, like they do in programming IDEs such as IntelliJ IDEA or Xcode.

Third, you can still have good-old menus and dialogues to fill in the commands. But have you noticed the Mac’s “search commands” field in every Help menu? That’s a tiny command line that lets you find a command by typing words. Take a hint!

Programming is hard, I’m a visual person!

It is absolutely possible to make this sort of interface progressively accessible. Draw things with your mouse or click some commonly used toolbar buttons: the UI will fill in the generated commands. You can ignore them, or you can go and tweak the numbers.

Example: draw a rectangle on a screen. The UI will show the rectangle and the command “rect x:1, y:2, w:301, h:499”. Wanna round numbers? Just click them and edit, right from keyboard. Wanna two of those? Copy-paste, add 100 to the y-coordinate. And see the result instantly. Easy and natural! Once you try it (provided feedback is instant and autocomplete is smart), you will never go back.

Do I have to type all bezier curve parameters like a robot?

Of course not! Imagine an object “curve from 0 to 1”. It is the same thing that we use for color adjustment or a shadow profile. Just let it be a type. And then you can render it within the command line, like a floating 2D object that you can interact with. And if you really have to, switch it into all-text mode with all nasty curve parameters shown as-is.

How do we move forward?

1. Think of your process as a domain-specific program, with nouns (objects) and verbs (actions).
2. Make a file format out of that language.
3. Drive the UI by the language.
4. Make super-charged autocomplete. See the leading IDEs for example.
5. Embed UI into the language. Color picker for a color, sliders for numerical values, 2D controls for 2D objects etc.
6. Programming text editors is no fun, especially if the text has custom floating interactive controls all over it. So we really have to create a reusable toolkit for working with text-intermixed-with-controls. That’s a pain well worth capitalizing on, and/or amortizing across many products.
7. Make a great product for graphics/video/animation/3D/rocket science that boosts productivity 20x.
8. Profit.