This post is the second in a series on my integration with a VT220-like (VT510) terminal, as my ‘daily driver’. The first post in this series is available here.

Applications

The tasks I am able to perform on my terminal are quite varied. I can

  1. Check my email and file it extremely quickly, using mutt. This includes advanced features like S/MIME and GPG encryption, full-text search (extremely fast!), and dealing with mailing lists sensibly.
  2. Convert my emails to task reminders in my task management tool of choice, OmniFocus.
  3. Read RSS feeds.
  4. Use the web (and other protocols!) in a text-based browser.
  5. Do almost anything else you might think to do on the command line (including editing some of this blog post).

In this post, I will be exploring mutt, for regular email reading, writing and management.

tmux Customisation

First things first, the DEC VT510 is a CRT monitor. It is succeptible to burn-in, leaving marks of any frequently-shown text visible on the monitor, even when the power is off. This can be very distracting!

The contents of my ~/.tmux.conf file alleviates this risk by changing from the default status bar with a light background, to one with a dark (black) background, and displaying only a modified clock in the right hand corner — there is no usual list of windows. To remove the window list in the tmux status bar and perform these other colour customisations, I have the following in my ~/.tmux.conf:

# Customise status bar
set -g status on
set -g status-right "%H%M %d %b"
set-option -g status-left ""

set -g status-bg black
set -g status-fg white

set -g window-status-current-format ''
set -g window-status-format ''

These commands will work in any any tmux environment, not just when used on a VT510 terminal.

Mutt

My full mutt setup is out of scope of this article, however there are a few features that are useful for me in using the DEC VT510 terminal.

Small screen, big world

The first issue to contend with was that I didn’t want to be changing all of my settings just to accomodate my VT510. To make things fit the screen, I didn’t want to be using my luxurious 15” screen with the same settings of an 80 column terminal.

Luckily, mutt allows you add some intelligence to its configuration by referencing shell scripts (however no arguments can be passed). Here’s my terminal-dependent setup in ~/.muttrc:

source '~/.mutt/vt220.sh|'

And vt220.sh itself is:

if [ $TERM = "vt220" ]
then
  echo "set ascii_chars=yes"
  echo "set wrap=80"
  echo "set reflow_wrap=80"
  echo "set pager_index_lines=5"
fi

if [ `tput cols` -lt "75" ]
then
  echo "set sidebar_visible=no"
fi

Special Characters

There are a few settings required to support text-only email reading, particularly on a terminal that doesn’t know a character set such as UTF-8 (modern character sets support advanced characters like emoji). The DEC VT510 supports mainly ASCII.

To ensure that my emails are as readable as possible, I need to filter out undesireable characters - even a “smart” (curled) quote (e.g. “ — These come in single and double varieties, both openening and closing). I have a regex filter that in my ~/.muttrc is enabled by set display_filter="~/.mutt/filter.sh". filter.sh itself follows:

#!/bin/bash

output=$(tee)

# Removal of all smart quotes and weird spaces
output=`echo "$output" | sed "s/[’‘]/\'/g"`
output=`echo "$output" | sed 's/[”“]/"/g'`
output=`echo "$output" | sed -E "s/[[:space:]]+/ /g"`

if [ "$TERM" == "vt220" ];
then
        # Start of a project to remove emoji and replace with 'emoticons'
        output=`echo "$output" | perl -pCSD -e 's/\x{1F600-\x{1F608}/:)/g'`
        output=`echo "$output" | perl -pCSD -e 's/\N{U+1F609}/;)/g'` # Can also be \x{1F609}
        output=`echo "$output" | perl -pCSD -e 's/\N{U+20AC}\s?/euro /g'`
        output=`echo "$output" | perl -pCSD -e 's/\x{1F610}-\x{1F612}/:|/g'`
fi


echo "$output"

This is a work in progress, however fixes a lot of characters that otherwise appear as underscores in my reading of mail.

HTML Alternatives

A lot of email today comes as HTML, but often with a text alternative that mutt and other plaintext programmes can utilise. Often this alternative is just as readible as the HTML (sometimes more so!). Sometimes however, it may be an annoying and terse message telling you only to upgrade your email client to read the HTML email (no chance!). There are ways around this.

In ~/.muttrc, I have:

message-hook . "unalternative_order *; alternative_order text/plain text/enriched text/html"
message-hook '~f bad_sender@domain.com "unalternative_order *; alternative_order text/html"

where bad_sender@domain.com is the name of an inconsiderate sender who does not send useful text email. Looking at you, ABC!

In ~/.mailcap I have:

text/html; w3m -I %{charset} -T text/html; copiousoutput;

This ensures that mutt knows how to deal with HTML, whenever it is selected (including if you manually select the HTML component of an email).

URLs in emails

Occasionally you may come across emails that contain links you’d like to visit. Mutt can parse these for you, using a helper programme called urlview. To extact URLs from emails, add the following to your ~/.muttrc:

macro pager \cb <pipe-entry>'urlview'<enter> 'Follow links with urlview'

This will allow you to press ‘control-b’ (twice in a row, if using tmux) and extract URLs in an email. By default these will be opened by your nearest web browser — if you’re on a VT510 that’s not preferable! To confiure urlview, create the file ~/.urlview:

COMMAND /usr/local/bin/lynx -cfg=~/.config/lynx/lynx.cfg %s

This will launch the text-based browser lynx with the url selected from urlview in mutt. You may remove the -cfg portion of the command — this specifies a configuration file (more on this later).

Task Management

Emails are often the first notification of needing to complete a task, often administrative and easily batched, and whose optimal execution time does not often coincide with their arrival in my inbox. I like to keep a clean inbox, but keep track of things I have to do. To do this, I have a series of scripts that allow me to place tasks (and references to their genesis, email) in OmniFocus. I further have a button in OmniFocus that allows me to select a task and view the email that it came from.

Without detailing them here, these features rely on my mutt setup using offlineimap for keeping a copy of my email locally, and notmuch for fulltext search.

To create a task from an email, I press control-l on my keyboard. ~/.muttrc is configured to execute a script to pipe this for text input

macro index,pager \cL "<enter-command>unset wait_key<enter><pipe-message>mutt-to-omnifocus.py<enter>" \
        "Create OmniFocus task from message"

The script mutt-to-omnifocus.py is the following code. I did not write this, but I have somewhat modified it to support emails with encoded subjects. I have forgotten where I obtained it so I am unable to credit, sorry.

#!/usr/bin/env python

import sys
import os
import getopt
import email.parser
import subprocess
from email.header import decode_header
import re

def usage():
    print """
    Take an RFC-compliant e-mail message on STDIN and add a
    corresponding task to the OmniFocus inbox for it.

    Options:
        -h, --help
            Display this help text.

        -q, --quick-entry
            Use the quick entry panel instead of directly creating a task.
    """

def applescript_escape(string):
    """Applescript requires backslashes and double quotes to be escaped in 
    string.  This returns the escaped string.
    """
    if string is not None:
        # Backslashes first (else you end up escaping your escapes)
        string = string.replace('\\', '\\\\')

        # Then double quotes
        string = string.replace('"', '\\"')

    return string

def parse_message(raw):
    """Parse a string containing an e-mail and produce a list containing the
    significant headers.  Each element is a tuple containing the name and 
    content of the header (list of tuples rather than dictionary to preserve
    order).
    """

    # Create a Message object
    message = email.parser.Parser().parsestr(raw, headersonly=True)

    # Escape special shell characters
    #applescript2 = re.sub("(!|\$|#|&|\"|\'|\(|\)|\||<|>|`|\\\|;)", r"\\\1", applescript)

    # Extract relevant headers
    list = [("Date", message.get("Date")),
            ("From", message.get("From")),
    #        ("Subject", message.get("Subject")),
            ("Message-ID", message.get("Message-ID"))]
    try:
        sub, encoding  = decode_header(message.get("Subject"))[0]

        sub = sub.replace('\n', '');
        pipe = subprocess.Popen(['/Users/joel/bin/item_name.sh', sub], stdout=subprocess.PIPE)
        subject, error = pipe.communicate()
        list.append(("Subject", subject.rstrip('\n')))
    except KeyboardInterrupt:
        print ""
        sys.exit()

    return list

def send_to_omnifocus(params, quickentry=False):
    """Take the list of significant headers and create an OmniFocus inbox item
    from these.
    """

    # name and note of the task (escaped as per applescript_escape())
    name = "%s" % applescript_escape(dict(params)["Subject"])
    note = "\n".join(["%s: %s" % (k, applescript_escape(v)) for (k, v) in params])

    # Write the Applescript
    if quickentry:
        applescript = """
            tell application "OmniFocus"
                tell default document
                    tell quick entry
                        open
                        make new inbox task with properties {name: "%s", note:"%s"}
                        select tree 1
                        set note expanded of tree 1 to true
                    end tell
                end tell
            end tell
        """ % (name, note)
    else:
        applescript = """
            tell application "OmniFocus"
                tell default document
                    make new inbox task with properties {name: "%s", note:"%s"}
                end tell
            end tell
        """ % (name, note)

    # Use osascript and a heredoc to run this Applescript
    os.system("\n".join(["osascript >/dev/null << 'EOF'", applescript, "EOF"]))

def main():
    # Check for options
    try:
        opts, args = getopt.getopt(sys.argv[1:], "hq", ["help", "quick-entry"])
    except getopt.GetoptError:
        usage()
        sys.exit(-1)

    # If an option was specified, do the right thing
    for opt, arg in opts:
        if opt in ("-h", "--help"):
            usage()
            sys.exit(0)
        elif opt in ("-q", "--quick-entry"):
            raw = sys.stdin.read()
            send_to_omnifocus(parse_message(raw), quickentry=True)
            sys.exit(0)

    # Otherwise fall back to standard operation
    raw = sys.stdin.read()
    send_to_omnifocus(parse_message(raw), quickentry=False)
    sys.exit(0)

if __name__ == "__main__":
    main()

This script calls a shell script called item_name.sh, which takes input to determine the task’s name in OmniFocus. The default is Mutt: <email subject>

OS X has Bash 3 available at /bin/bash, which will not work for this script. item_name.sh relies on Bash v5 being installed (typically via homebrew):

#!/usr/local/bin/bash

RESET="\033[0m"
BOLD="\033[1m"
RED="\033[31m"
MUTT="Mutt: $1"
read -e -p "$(echo -e $BOLD$RED"Task Name: "$RESET)" -i "$MUTT" title </dev/tty

echo "$title"

The combination of these two scripts will allow you to enter text to change the name of the task being inserted into OmniFocus, and quickly insert the task into the OmniFocus inbox.

The following AppleScript, when compiled and placed into your OmniFocus script folder (~/Library/Application Scripts/com.omnigroup.OmniFocus3), will allow you to open a mutt instances within iTerm.app (my terminal programme of choice). I have to fiddle significantly with the timings (delay 2 statements, etc) — your mileage may vary on your own system.

tell application "OmniFocus"
  tell front document
    tell content of document window 1 -- (first document window whose index is 1)
      set theSelectedItems to value of every selected tree
    end tell
    
    repeat with anItem in my theSelectedItems
      if note of anItem contains "Message-ID" then
        
        set noteBody to note of anItem
        
        set MessageID to do shell script "echo '" & noteBody & "' | /usr/bin/python ~/bin/message_id.py"
        
        tell application "iTerm"
          activate
          try
            set newTab to true
            set t to the first window
          on error
            set newTab to false
            set t to (create window with default profile)
          end try
          
          tell t
            if newTab then
              set tt to (create tab with default profile)
            else
              set tt to (current tab of t)
            end if
            
            set s to current session of tt
            
            -- Open Mutt
            tell s
              write text "/usr/local/bin/mutt"
            end tell
            
            -- Run "F8" keypress
            tell application "System Events"
              key code 100
            end tell
            
            -- Search for the MessageID
            tell s
              write text "id:" & MessageID
            end tell
            
            -- Wait one second, then hit 'enter' to open the email
            delay 2
            tell application "System Events"
              key code 36
            end tell
          end tell
        end tell
      else
        if note of anItem is "" then
          set dialogText to "No text in note"
        else
          set dialogText to "Note does not appear correctly formatted: missing Message-ID"
        end if
        display dialog dialogText with icon stop ¬
          with title ¬
          "Error" buttons {"OK"} ¬
          default button 1
        
      end if
    end repeat
  end tell
end tell

return

This AppleScript relies on a script called message_id.py:

#!/usr/bin/python
#
#    mutt_flagged_vfolder_jump.py
#
#    Generates mutt command file to jump to the source of a symlinked mail
#
#    Copyright (C) 2009 Georg Lutz <georg AT NOSPAM georglutz DOT de>
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

import optparse
import os
import re
import sys
import types
from subprocess import call
from email.header import decode_header

VERSIONSTRING = "0.1"


def parseMessageId(file):
    '''Returns the message id for a given file. It is assumed that file represents a valid RFC822 message'''
    prog = re.compile("^Message-ID: (.+)", re.IGNORECASE)
    prog2 = re.compile("^Subject: (.*)$",  re.IGNORECASE)
    msgId = ""
    subject = ""
    for line in file:
       # Stop after Header
  if len(line) < 2:
      break
  result = prog.search(line)
  l = len(decode_header(line))
  s = ""
  for i in range(0,l):
    s += decode_header(line)[i][0] + " "
  s = s[:-1]
        result2 = prog2.search(s)
  if type(result) != types.NoneType and len(result.groups()) == 1:
      msgId = result.groups()[0]
  if type(result2) != types.NoneType and len(result2.groups()) == 1:
      subject = result2.groups()[0]
  if subject != "" and msgId != "":
      prog3 = re.compile("([\[\(] *)?(RE|FWD?) *([-:;)\]][ :;\])-]*|$)|\]+ .*$", re.IGNORECASE)
      result3 = prog3.sub('', subject)
      subject = result3
      break
    return (subject.replace('\n',''), msgId.strip("<>"))

subject, msgId = parseMessageId(sys.stdin)
try:
  print msgId
except KeyboardInterrupt:
  print ""
  sys.exit()

Result

After this, you should be able to see your email and integrate it across your other workflows. I’m not posting my inbox (the shame of never being able to reach inbox zero in inescapable!), so please enjoy this image of my RSS reader and a web browser running side by side in tmux, to whet your appetite for the next post in this series. shot of VT220 displaying news feeds on the left hand side of the screen, and a text-only browser displaying google.com on the right

Up Next in the Series

Applications for daily use: RSS, web and terminal security