Setting Up A “lightweight” Archlinux/Awesome WM VM.

Overview

This post is meant to document the various modifications I had to make to get an integrated and functional desktop environment working using Archlinux and Awesome WM. The setup is run on an iMac 5k in a VM which is copied to a surfacebook when traveling.

My goal in setting this up was to create an efficient DE with everything nicely integrated and nothing extra added. If it’s there it’s b/c I wanted it to be.

Basic DE requirements.

  • Tiling WM (chose awesome wm)
  • Beautiful – HiDPI enhanced fonts and elegant (minimal) interfaces.
  • Desktop Notifications
  • Keyring for ssh keys and passwords.
  • Spotlight like searches (i.e., data and metadata indexed in real-time)
  • Dropbox Integration
  • Quick Launch (hotkeys or phrases to quickly run apps or scripts)
  • Detachable remote sessions.
  • Screen captures

Important Applications.

  • Terminal App (Fast)
    • Mate terminal
  • Emacs
    • org mode for todo and mail tracking.
    • mu4e for mail
    • erc for jabber and irc
    • standard programming stuff.
    • Use Spacemacs and Profit.
  • Openvpn

Pleasant Surprises

  • Remote X11 login
  • Decent Sound

Notes

Archlinux AUR Packages

Install pacaur and use it.

Pacaur is a nice way to be able to install from community/extra repositories ala pacman, but also seamlessly fetch additional packages from AUR.

Tiling WM – Awesome

Installed AUR

$ pacaur -S awesome-git
$ pacaur -S obvious-git

Login Applications

In order to have things auto-started from packages you install that require this you should install and use the dex utility. Packages that want to be autostarted on session login add .desktop files to your .config/autostart directory. dex will find these files and execute them as appropriate

Enable dex by adding the following to the bottom of your .config/awesome/rc.lua file.

os.execute(“dex -a -e Awesome”)

Integration with Freedesktop DE software (GNOME, KDE, LXDE, …

Applications and services that come from other standard desktop environment (e.g., GNOME, KDE, et al). that want to be auto-started on session login add .desktop files under the /etc/xdg/autostart directory.

To enable one of these applications or services you need to modify their .desktop files in /etc/xdg/autostart directory. Search for the line OnlyShowIn line and add “;Awesome;” to it if it isn’t there already.

For my instlation I enable the following desktop files:

$ dex -d -a -e Awesome
at-spi-core (reqd by gtk3)
gnome-keyring-pkcs11 (reqd by gnome-keyring)
gnome-keyring-secrets (reqd by gnome-keyring)
gnome-keyring-ssh (req'd by gnome-keyring)
pulseaudio

Desktop Notifications

pacman -S xfce4-notifyd

Awesome WM actually has support for notifications using the naughty package, but it was a bit ugly for me when the notifications included action buttons (they stacked vertically vs. horizontally). I checked out various other daemons listed on the archlinux wiki, and settled on xfce4-notifyd. xfce4-notifyd looked great and supported the general gnome dark theme, didn’t block or stack notifications on top of each other. Some people want this I do not, I do not want to received superfluous notifications so I want to notice them. One major use I have for this is to stick non-expiring notifications for deadlines for TODOs in org mode. Having this allows me to depend on org mode as the keeper of all my notes and reminders.

In order to use an external (to awesome) notification daemon you need to make sure that naughty is not require‘d by your rc.lua config (directly or indirectly). Look out for obvious widgets (e.g., obvious.battery). Eventually it would be nice to figure out how to not have this restriction and perhaps have the naughty API just proxy, and not listen for notifications.

To get customize the look of the notifications you need to use xfce4-notifyd-config

Keyring (GNOME)

I use the gnome keyring. After installing gnome-keyring you need to enable the .desktop files under /etc/xdg/autostart as described above in `Login Applications`. Additionally, you need to rename or remove the /usr/share/dbus-1/services/org.freedesktop.secrets.service file. If you do not do the latter then you will end up with 2 daemons the second of which was not given the password for unlocking from PAM, and so on the first use of the keyring you will be prompted with a pop-up window claiming that the keyring was not unlocked at login and to re-enter your login password.

Startup X11 files

It can be hard to tell what is actually executed during startup you should check: /etc/X11/xinit/xinitrc.d/ as these probably are and may affect your settings (like DPI and font dpi).

In particular xft-settings.sh may be resetting your dpi to 96

HiDPI

This can be quite tricky to get right. The end result is that you need xdpyinfo to show the correct display size (and thus DPI) for your screen resolution. You also need xrdb -query | grep dpi to show Xft.dpi being set to the same value.

The fastest way to do this is to run xrandr --dpi 217 (replace 217 with your actual DPI) somewhere. Here are the other modifications I made.

lightdm

  • display-setup-script can be used to call xrandr and others, primarily make sure your DPI is set correctly also Xft.dpi in your resources
  • xft.dpi in greeter config needs to be set for a nice gtk greeter window
# If inside vmware VM
/usr/bin/vmware-user-suid-wrapper
sleep 4

XRES=$(xdpyinfo | sed  -e '/dimen/!d;s, *dimensions: *\([0-9]*\)x\([0-9]*\) pixels.*,\1,')
YRES=$(xdpyinfo | sed  -e '/dimen/!d;s, *dimensions: *\([0-9]*\)x\([0-9]*\) pixels.*,\2,')
mode=${XRES}x${YRES}
dpi=192  # Whatever the correct value is or calculate by size and res.
x11screename=Virtual1
xrandr --output $x11screename --primary --mode $mode --dpi $dpi >> $tmpf 2>&1

vmware

  • XXX add section on /usr/bin/vmware-user-suid-wrapper, xrandr must be run after this as it messes up the display size.
  • This will be run initially by your login from /usr/x

Beautifully Rendered Fonts (infinality)

Follow these directions.

Don’t forget to update the xft config to your HiDPI setting. This may be being set in /etc/X11/xinit/xinitrc.d/xft-settings.sh. You probably want to just remove that setting and set it elsewhere (perhaps in lightdm fixup.

No scaling setting (i.e., leave 1).

If you’ve done all the above you should nt need any scale adjustment hacks with environment variables or GUI configuration dialogs.

Important Applications

Terminal (mate)

I ended up doing some odd hacks on scripts I found to setup a bunch of color profiles. The important setting though that was hard to find was for setting the “system default monospace font”. To set that to DejaVu Sans Mono 12 point enter the following:

gsettings set org.mate.interface monospace-font-name "'Ubuntu Mono 12'"

Pidgin Dark Background

Changing the background color in pidgin

It took a lot of digging as well as use of a debugger/inspector called gtkparasite, but I am now able to configure pidgin with dark backgrounds. The key is setting custom gtk2 values for interface elements. I created the following file in ~/.purple/gtkrc-2.0.

Here’s an example of what it looks like:

ebb580e8-de07-11e5-8be5-000c298b0326.png

Figure 1: Example Dark Chat Window

Here’s the contents of ~/.purple/gtkrc-2.0 to generate the above.

style "molokai"
{
        text[NORMAL] = "#F8F8F2"
        base[NORMAL] = "#1B1D1E"
}

style "yellowonred"
{
        text[NORMAL] = "#FFFF00"
        base[NORMAL] = "#330000"
}

widget "*pidgin_conv_entry" style "yellowonred"
widget "*pidgin_conv_imhtml" style "molokai"
widget "*pidgin_conv_userlist" style "molokai"

IrcII Colorized Formatting

To colorize output in IrcII one uses the “on” filter to match public and private messages, and specifies an action to take, which should be echo in order to actually output the message. Within this action $0 refers to your nick, $1 to the channel and $2- to the words of the message. The formatting codes are as follows:

| Effect           | Prefix Value | Format        |
|------------------+--------------+---------------|
| All Off          | ^O (\017)    | ^O            |
| Toggle Bold      | ^B (\002)    | ^B            |
| Toggle Underline | ^_ (\017)    | ^_            |
| Toggle Reverse   | ^V (\026)    | ^V            |
| Color            | ^C (\003)    | ^C<fg>[,<bg>] |

For color the <fg> and <bg> are ascii integers the background value is optional. The values for the colors are from mIRC and show in the table below including the ansi values they map to.

Here’s an example set of “on” filters to colorize nicks with yours done differently from others. The ^X characters are actual control characters, so you’ll need to enter them appropriately (e.g., in vim you can do ctrl-v ctrl-c). Add something like these to your .ircrc file.

set colour on
# My messages
on ^send_public * if (C == [$0]) \
    { echo $Z ^B^C10<$(N)>^O $1-  } \
    { echo $Z ^B^C10<$(0)>^O $1- }
on ^send_msg * echo $Z -> ^B^C10*$(0)*^O $1-
# Other's messages
on ^public * echo $Z ^B^C2<$(0)>^O $2-
on ^msg * echo $Z ^B^C2*$(0)*^O $1-

Color value tables:

| Value | mIRC Colour | ANSI color         |
|-------+-------------+--------------------|
|     0 | White       | (15) Light White   |
|     1 | Black       | (0) Black          |
|     2 | Blue        | (4) Blue           |
|     3 | Green       | (2) Green          |
|     4 | Light Red   | (9) Light Red      |
|     5 | Brown       | (1) Red            |
|     6 | Purple      | (5) Magenta        |
|     7 | Orange      | (3) Yellow         |
|     8 | Yellow      | (11) Light Yellow  |
|     9 | Light Green | (10) Light Green   |
|    10 | Cyan        | (6) Cyan           |
|    11 | Light Cyan  | (14) Light Cyan    |
|    12 | Light Blue  | (12) Light Blue    |
|    13 | Pink        | (13) Light Magenta |
|    14 | Gray        | (8) Dark Gray      |
|    15 | Light Gray  | (7) White          |
|       |             |                    |

The following table indicates the mIRC color value to use given the ansi name value.

| ANSI Value  | mIRC |
|-------------+------|
| 0 (black)   |    1 |
| 1 (red)     |    5 |
| 2 (green)   |    3 |
| 3 (yellow)  |    7 |
| 4 (blue)    |    2 |
| 5 (magenta) |    6 |
| 6 (cyan)    |   10 |
| 7 (white)   |   15 |

| Bold Value  | mIRC |
|-------------+------|
| 0 (black)   |   14 |
| 1 (red)     |    4 |
| 2 (green)   |    9 |
| 3 (yellow)  |    8 |
| 4 (blue)    |   12 |
| 5 (magenta) |   13 |
| 6 (cyan)    |   11 |
| 7 (white)   |    0 |

Solarized Colors:

Solarized Name mIRC ANSI
base03 14 Br. Black
base02 1 Black
base01 9 Br. Green
base00 8 Br. Yellow
base0 12 Br. Blue
base1 11 Br. Cyan
base2 15 White
base3 0 Br. White
yellow 7 Yellow
orange 4 Br. Red
red 5 Red
magenta 6 Magenta
violet 13 Br. Magenta
blue 2 Blue
cyan 10 Cyan
green 3 Green

Fix Underline + Color in TMUX (TERM=screen-256color)

When using TMUX normally the TERM environment variable is set to screen-256 color. Under OS X at least the terminfo for this terminal disables color and underline used together. The result is that programs such as emacs will display color but not underline text.

I found the following quick fix for this from Chris Johnsen.

$({ infocmp -x screen-256color;
    printf '\t%s\n' 'ncv@,'; } > /tmp/t && tic -x /tmp/t)

Tmux true color (24-bit RGB) support.

Over the last year or so (2013+) there’s been a few terminal applications that have begun to support 24-bit or true color mode. The application I most wanted to take advantage of this was emacs where I use colorized themes (solarized, zenburn, etc). Eventually I found a diff and improved it some and posted about it here.

But normally I run emacs within tmux so I needed support within that as well. I found a mostly functional diff here; however, this didn’t support the newer “:” delimiter as well as needing some cosmetic changes to apply cleanly to the latest tmux (1.9a).

Here’s the result: tmux-1.9a diff

Happy Colorful Hacking.

Screenshot of emacs in tmux running in iTerm2 with 24-bit color.

Emacs true color (24-bit) in terminal support.

Over the last year or so (2013+) there’s been a few terminal applications that have begun to support 24-bit or true color mode. One terminal that supports true color is iTerm2 (the development version). The application I most wanted to take advantage of this was emacs where I use colorized themes (solarized, zenburn, etc). So I started googling around to find if this was supported and located a diff, unfortunately it did’t work. I started with this as a base and got things working.

Here’s the result:

In order to use this you must set one of two environment variables. For now the best option is to set KONSOLE_DBUS_SESSION=1 (I guess if you are using konsole this is already set for you). It uses the currently more implemented CSI sequence (supported by konsole, iterm, xterm, etc). To use the officially sanctioned way, you should set ITERM_24BIT=1. This latter mode uses colon’s instead of semi-colons in the CSI sequence which is more compatible with terminals that do not support the codes. This latter mode is defined in ITU T.416.

Happy Colorful Hacking.

Screenshot of emacs running over ssh in iTerm2 with full color pallet.

Automatic Switching Between Light and Dark Solarized Modes

I’ve recently switched to using the solarized color theme. At first it took getting used to the lower contrast; however, I was persuaded to keep trying based on the research behind the choice of the colors. I’m now quite happy with it.

With solarized there are 2 mode of operation, light mode and dark mode. These modes have the somewhat obvious meanings, light mode uses a light background with darker text, and dark mode uses a dark background with light text. I like both modes, and find that the dark mode is well suited to nighttime use and likewise the light mode is good for daytime.

So the next question was how to make this automatic. I use iTerm on the Mac so I created 2 profiles 1 for each mode and gave each profile a hot key. But what about when the sun sets or rises and I’ve got windows open? Also I use different colors for the TMUX status bar depending on the mode, I’d like those long running sessions to update as well depending on the time of the day.

In order to update running programs I created a shell script (which also uses applescript if iTerm is running) to update both iTerm and TMUX based on an hour range. I then execute this script once an hour using crontab.

One of the features of solarized is that the only difference between the light and dark modes is the choices for foreground color and background color. So we only have to swap those two values. I also change the color of the text behind the cursor.

Update color mode shell script.

#!/bin/bash

# Set Color Values

darkend=600             # 6am
darkstart=2000          # 10pm
timeval="$(date +%k%M)" # current time

if [ $timeval -gt $darkend -a $timeval -lt $darkstart ]; then
    # Dark Mode
    itermcolors='{{64842, 62778, 56626}, {21257, 26684, 28737}, {60037, 58326, 52284}}'
    tmux_cur_fg=black
    tmux_bg=colour7
else
    # Light Mode
    itermcolors='{{0, 7722, 9941}, {28873, 33398, 33872}, {0, 10207, 12694}}'
    tmux_cur_fg=white
    tmux_bg="#124741"
fi

# Adjust iTerm

if [ -n "$(ps x | grep iTerm | grep -v grep)" ]; then
    osascript <<- EOF
    tell application "iTerm"
        repeat with aSession in sessions of current terminal
            tell aSession
                set theColors to $itermcolors
                set background color to item 1 of theColors
                set foreground color to item 2 of theColors
                set cursor_text color to item 3 of theColors
            end tell
        end repeat
    end tell
EOF
fi

# Adjust TMUX

sessions="$(tmux ls 2>/dev/null | cut -f1 -d:)"
for s in $sessions; do
    tmux set-window-option -t $s -g window-status-current-fg $tmux_cur_fg > /dev/null
    tmux set-window-option -t $s -g status-bg $tmux_bg > /dev/null
done

Crontab entry

0 * * * * /users/chopps/bin/update-color-mode.sh

Automatically Fixing Python flake8 and pylint Errors in Emacs

Having recently installed elpy in my emacs environment and enabled flake8, I was presented with many warnings an errors based on style. At first I just starting disabling them all to avoid the noise and hassle of fixing them. Then I got the idea that it shouldn’t be that hard to create a fixer system to automatically fix at least the most simple stylistic errors. This idea was inspired in part by the automatic fix-up stuff I found while checking out Pycharm.

To create this system I started out with elisp only, and then in an attempt to learn how to use pymacs incorprated that. In this post I’m going to focus on the elisp version.

By checking out the code to display the error on the current line from flymake I derived a pair of functions and an associative array to invoke functions given a certain error found. Then I wrote fixer functions and added them to the array. I also I bind “C-cC-\” to pyfixer:fix-current-line.

Here’s the code with a single fixer function defined. Eventually I plan on hosting this on github so that people may contribute fixers.

(defun pyfixer:add-blank-line (errno errinfo)
  "Add blank line above current line"
  (save-excursion
    (beginning-of-line)
    (newline)))

(setq pyfixer:flymake-fixers
      '(
        ("E301" . pyfixer:add-blank-line)
        ("E302" . pyfixer:add-blank-line)
        ))

(defun pyfixer:fix-error (errdata)
  "Fix the given errdata"
  (if errdata
      (let (errno fixer)
        ;; Handle flake8 or pylint messages
        (if (string-match "\\(^\\([EW][0-9]+\\) \\|\\[\\([EW][0-9]+\\)\\(-.*\\)\\?*\\]\\).*" errdata)
            (progn
              (if (not (setq errno (match-string 2 errdata)))
                  (setq errno (match-string 3 errdata)))
              (setq fixer (cdr (assoc errno pyfixer:flymake-fixers)))
              ;; If we don't have and elisp fixer check pymacs fixers
              (if (and (not fixer) (functionp 'pyfixers-get-fixer-func))
                  (setq fixer (pyfixers-get-fixer-func errno)))
              (if fixer
                  (progn
                    (funcall fixer errno errdata)
                    (flymake-start-syntax-check))
                (message "No pyfixer function for: %s" errno)))))))

(defun pyfixer:fix-current-line ()
  "Display a fix for the current line"
  (interactive)
  (let* ((line-no             (flymake-current-line-no))
         (line-err-info-list  (nth 0 (flymake-find-err-info flymake-err-info line-no)))
         (menu-data           (flymake-make-err-menu-data line-no line-err-info-list))
         (errlist (caadr menu-data)))
    (message "Errlist: %s" errlist)
    (mapcar 'pyfixer:fix-error errlist)))

(global-set-key "\C-c\C-\\" 'pyfixer:fix-current-line)

Now this fixer function is quite simple and doesn’t deal with some cases. For example if a comment immediately precedes the function definition it is not counted as a blank line, but the extra line will be between it and the function definition. To fix this we backup to the first non-comment line before adding any newlines. Additionally, let’s parse the error message to determine exactly how many newlines we need to add. Here’s the revised fixer function:

(defun line-no-commentp ()
  (save-match-data
    (let* ((start (line-beginning-position))
           (end (line-end-position))
           (line (buffer-substring-no-properties start end)))
      (not (string-match "[:space:]*#.*" line)))))

(defun pyfixer:add-blank-line (errno errinfo)
  "Add blank line above current line"
  (save-excursion
    (let ((lines 0))
      (if (string-match "expected \\([0-9]+\\) blank lines?, found \\([0-9]+\\)" errinfo)
          (setq lines (- (string-to-number (match-string 1 errinfo))
                         (string-to-number (match-string 2 errinfo)))))
      (beginning-of-line)
      (previous-line)
      (while (not (line-no-commentp))
        (previous-line))
      (next-line)
      (newline lines))))

Extracting an Environment Variable From a Process

Occasionally I find it necessary to extract the value of an environment variable of a running process. For example to examine the TMUX value of a given process. I’ll use this later to determine if I should signal an emacs process when a TMUX session is reattached. This function depends on /proc/<pid>/environ being supported (apparently a linux thing). Here’s the function.

[Ed: updated version with suggested replacement from KnowsBash reddit-bash comment]

get-pid-env-var () {
    if [ -z "$1" -o -z "$2" ]; then
        echo "usage: get-pid-env-var pid var"
        return 1
    fi
    local key value
    while IFS='=' read -rd '' key value; do
        if [[ $key = "$2" ]]; then
            printf '%s\n' "$value"
            break
        fi
    done < "/proc/$1/environ"
}

And here it is in use.

[22:36:28 ~]$ ps x | grep emacs
24414 pts/1    S+     0:04 emacs -nw .bashrc
26751 pts/4    S+     0:00 grep emacs
[22:36:32 ~]$ get-pid-env-var 24414 TMUX
/tmp/tmux-21995/default,24155,0

Here’s the old version.

get-pid-env-var () {
    if [ -z "$1" -o -z "$2" ]; then
        echo "usage: get-pid-env-var pid var"
        return 1
    fi
    local VAR=$(tr "\0" "\n" < /proc/$1/environ | grep "$2=")
    if [ -z "$VAR" ]; then
        return 1
    fi
    echo ${VAR#$2=}
    return 0
}

A Pymacs Code Example

This is a simple example of using pymacs to fix a flake8 warning. The warning is covering use of = or ! with None. The fixer is invoked with the point on the line containing the warning or error. This is part of a system I’m building to support auto-correction of flake8 and pylint warnings in emacs. As the system matures I plan to host it on github so that others may also contribute to it.

Here is the actual fixer function with a utility function to get the string of the current line. A key to understanding what’s going on with the code is that lisp.function(arg) is the equivalent of (function arg) in elisp.

def get_line_string ():
    """Get the start, end and string of the line the point is on."""
    start = lisp.line_beginning_position()
    end = lisp.line_end_position()
    return start, end, lisp.buffer_substring(start, end)

def fixer_equals_none (unused_error, unused_errtext):
    start, end, line = get_line_string()
    newline = re.sub(r"==\s*None", r"is None", line)
    newline = re.sub(r"!=\s*None", r"is not None", newline)
    if newline != line:
        lisp.delete_region(start, end)
        lisp.insert(newline)

Obviously this function is invoked from somewhere else with the error number and text (e.g., E711 and E711: error text). The following is the code that does that.

fixers = {
    # ...
    "E711": fixer_equals_none,
}

def flymake_fix_current_line ():
    lineno = lisp.flymake_current_line_no()
    errinfo = lisp.flymake_err_info
    errlist = lisp.nth(0, lisp.flymake_find_err_info(errinfo.value(), lineno))
    menudata = lisp.flymake_make_err_menu_data(lineno, errlist)
    did_something = False
    #             caadr (x ((e1 e2 ...))) -> (e1 e2 ...)
    for errtxt in menudata[1][0]:
        if not errtxt:
            break
        try:
            m = re.match(r"([EFW]\d+) .*", errtxt)
        except Exception as ex:
            continue
        if not m:
            print("nomatch on {}".format(errtxt))
            continue
        key = m.group(1)
        if key in fixers:
            did_something = True
            fixers[key](key, errtxt)
    if did_something:
        lisp.flymake_start_syntax_check()

interactions[flymake_fix_current_line] = ''

Finally we have the elisp code that binds a key to the dispatch function.

(pymacs-load "fixers" "fixers-")
(global-set-key "\C-c\C-\\" 'fixers-flymake-fix-current-line)