${VISUAL}ize the Future

Mike Burns

Do you want to use mvim to edit your commit messages, SQL interactions, command-line prompts, and emails, but can’t seem to get it working? To understand why, and to understand the fix, we first need a bit of history…

In the beginning was the terminal. It was simple, slow, and line-oriented, and so the text editor had to compensate for that. What we now think of as a text editor surely couldn’t work on a 80x25, capital letters only, one line at a time display, and those fancy features were available if you were rich!

Ken Thompson gave us, the collective hivemind of programmers across the world, ed(1) (short for “editor”). It worked on all sorts of terminals, no matter what bizarre state they were in, and allowed for efficient text editing. Moving forward a bit we got ex(1) thanks to Bill “Rewrite unix during breakfast” Joy. The ex (“extended ed”) editor was little more than additional commands for ed; again the same robust works-on-any-terminal technology was in place.

To allow for such crazy experimentation, unix programs started looking to the $EDITOR variable to indicate which editor to use. EDITOR=ed to indicate that you’re into efficient program load time; EDITOR=ex for those who were willing to wait the extra second in exchange for such revolutionary ideas as marks and mappings.

DJ Bill Joy

Then Bill “Kinda a big deal” Joy drops the bomb: ex -v. Visual mode. Terminals that could handle redrawing at high speeds had just come out and ex was updated to support that. Of course this didn’t reliably work and not just because few people had such fancypants terminals, but for those who could, they did.

Hot off the tails of $EDITOR, our unix forefathers added $VISUAL. If this environment variable was set, that editor would be used; otherwise it’d fall back to ye olde $EDITOR.

vi(1) was added as a link to ex; if called with this name it would run in visual mode.

And that’s why we set $VISUAL instead of $EDITOR today.

Here’s some example code that implements the most common pattern (though typically in C): pick an editor, fork, execute the editor, use the result:

require 'tempfile'

def main
  temp_file = Tempfile.new('hello')
  edit(temp_file)
  puts temp_file.read
ensure
  temp_file.close
  temp_file.unlink
end

def edit(temp_file)
  pid = Process.fork do
    exec(editor, temp_file.path)
  end

  Process.waitpid(pid)
end

def editor
  ENV['VISUAL'] || ENV['EDITOR'] || 'vi'
end

main

Methods of interest: #editor encapsulates the idea of finding the right editor, defaulting to the classic vi. #edit forks, which creates a child process; this child process is replaced by the editor, which is given a filename; meanwhile the parent process waits on the child. Back in #main we create a temporary file, call #edit on it, then print out its contents; finally and simply, we cleanup the temporary file.

GUIs

Somewhere betweeen 1973 and 1984 Steve Jobs, Bill Gates, and Håkon Lie invented the GUI, with a little bit of help from Xerox. With this came GUI vim.

GUI vim, typically called gvim (renamed to mvim on OS X), runs in the background. That is, gvim behaves roughly the same as gvim &.

This is done using a common and classic backgrounding mechanism. Here’s another example, also typically written in C: a daemon that writes “goodbye“ to a file every two seconds:

require 'tempfile'

def main
  fork do
    main_task_backgrounded
  end
end

def main_task_backgrounded
  begin
    file = Tempfile.new('goodbye')

    loop do
      file.write("goodbye\n")
      file.flush
      sleep 2
    end
  ensure
    file.close
    file.unlink
  end
end

main

But it also means problems for the $VISUAL code example above. Notice that as soon as the editor terminates successfully, it continues on. This was fine for non-backgrounding editors from ed through vim, but completely breaks for backgrounding editors such as GUI vim and Sublime.

Some real-life examples of where this breaks is git commit:

% VISUAL=gvim git commit
Aborting commit due to empty commit message.

psql, which makes use of the $PSQL_EDITOR variable:

% PSQL_EDITOR=gvim psql
mike=# \e
mike=#

GNU bash, which uses it for ^X^E mode:

% VISUAL=gvim bash
~$ ^X^E

~$

And mutt, started using VISUAL=gvim mutt, will fail to write emails with GUI vim, stating: Aborted unmodified message.

Just One Fix

Luckily GUI vim has a foreground mode that can be enabled with the -f option. Try it on the command line:

% gvim -f

This will hang until you quit gvim. Typically this would be annoying, but in this case it’s exactly what we need.

Ah but wait! If you tried setting $VISUAL to the obvious, it will not work! (I’m not going to show this because I wouldn’t want to leave an example of broken code.) The reason is straightforward, and why the vi command exists at all: $VISUAL and $EDITOR must be set to exec(3)-able programs. No command-line arguments, no shell tricks, no nothing. Just the absolute path to a program.

The solution here is simple. Save this program as gvim_fg. I put it in my ~/.bin directory, which is a helper directory for my rc files.

#!/bin/sh
gvim -f "$@"

Line one is the signature line (“shebang line”) for a POSIX shell script. Line two runs gvim passing the -f flag. It also passes through any command-line arguments; this allows programs to pass filenames or even other flags.

Now we can set $VISUAL to that program:

% VISUAL=$HOME/.bin/gvim_fg git commit
[master deadbee] Defrob the weeblinator
 0 files changed
 create mode 100644 spurtle
%

Do you have something that makes your unix life easier? Show it off with a pull request to our dotfiles repository.