Speeding Up Emacs and Parsing Emacs Lisp from Emacs Lisp

13 Apr 2013, 08:43 by Giorgos Keramidas

I recently spent a bit of time to clean up all the cruft that my ~/.emacs file and my ~/elisp directory had accumulated. I have been using a multi-file setup to configure my Emacs sessions, since at least 2008. This turned out to be a royal mess after 5+ years of patching stuff without a very clear plan or structure. The total line-count of both my ~/.emacs and all the *.el files I had imported into my ~/elisp directory was almost 20,000 lines of code:

    $ wc -l BACKUP/.emacs \$( find BACKUP/elisp -name '\*.el')
       119  BACKUP/.emacs
        84  BACKUP/elisp/keramida-w3m.el
        90  BACKUP/elisp/keramida-keys.el
       156  BACKUP/elisp/keramida-irc.el
      5449  BACKUP/elisp/erlang.el
       892  BACKUP/elisp/fill-column-indicator.el
       344  BACKUP/elisp/keramida-erc.el
        87  BACKUP/elisp/keramida-chrome.el
        89  BACKUP/elisp/keramida-autoload.el
       141  BACKUP/elisp/keramida-ui.el
        42  BACKUP/elisp/keramida-slime.el
      1082  BACKUP/elisp/ace-jump-mode.el
         2  BACKUP/elisp/scala-mode2/scala-mode2-pkg.el
       907  BACKUP/elisp/scala-mode2/scala-mode2-indent.el
        26  BACKUP/elisp/scala-mode2/scala-mode2-lib.el
       502  BACKUP/elisp/scala-mode2/scala-mode2-fontlock.el
        37  BACKUP/elisp/scala-mode2/scala-mode2-map.el
       808  BACKUP/elisp/scala-mode2/scala-mode2-syntax.el
       111  BACKUP/elisp/scala-mode2/scala-mode2.el
       121  BACKUP/elisp/scala-mode2/scala-mode2-paragraph.el
      1103  BACKUP/elisp/php-mode.el
       142  BACKUP/elisp/themes/cobalt-theme.el
       665  BACKUP/elisp/themes/zenburn-theme.el
       142  BACKUP/elisp/themes/sublime-themes/cobalt-theme.el
        80  BACKUP/elisp/themes/tomorrow-night-blue-theme.el
        80  BACKUP/elisp/themes/tomorrow-night-eighties-theme.el
       115  BACKUP/elisp/themes/tomorrow-theme.el
        80  BACKUP/elisp/themes/tomorrow-night-bright-theme.el
       339  BACKUP/elisp/cmake-mode.el
        95  BACKUP/elisp/keramida-cc-extra.el
      1341  BACKUP/elisp/lua-mode.el
      2324  BACKUP/elisp/markdown-mode.el
       184  BACKUP/elisp/rcirc-notify.el
       167  BACKUP/elisp/keramida-defaults.el
       203  BACKUP/elisp/keramida-hooks.el
        43  BACKUP/elisp/keramida-lang.el
       435  BACKUP/elisp/edit-server.el
       709  BACKUP/elisp/slang-mode.el
        66  BACKUP/elisp/keramida-eshell.el
     19402  total

20,000 lines of code is far too much bloat. It’s obvious that this was getting out of hand, especially if you consider that I had full configuration files for at least two different IRC clients (rcirc and erc) in this ever growing blob of complexity.

What I did was make a backup copy of everything in ~/BACKUP and start over. This time I decided to go a different route from 2008 though. All my configuration lives in a single file, in ~/.emacs, and I threw away any library from my old ~/elisp tree which I haven’t actively used in the past few weeks. I imported the rest of them into the standard user-emacs-directory of modern Emacsen: at ~/.emacs.d/. I also started using eval-after-load pretty extensively, to speed up the startup of Emacs, and only configure extras after the related packages are loaded. This means I could trim down the list of preloaded packages even more.

The result, as I tweeted yesterday was an impressive speedup of the entire startup process of Emacs. Now it can start, load everything and print a message in approximately 0.028 seconds, which is more than 53 times faster than the \~1.5 seconds it required before the cleanup!

I suspected that the main contributor to this speedup was the increased use of eval-after-load forms, but what percentage of the entire file used them?

So I wrote a tiny bit of Emacs Lisp to count how many times each top-level forms appears in my new ~/.emacs file:

    (defun file-forms-list (file-name)
      (let ((file-forms nil))
        ;; Keep reading Lisp expressions, until we hit EOF, and just add one
        ;; entry for each toplevel form to `file-forms'.
        (condition-case err
            (with-temp-buffer
              (insert-file file-name)
              (goto-char (point-min))
              (while (< (point) (point-max))
                (let* ((expr (read (current-buffer)))
                        (form (first expr)))
                       (setq file-forms (cons form file-forms)))))
          (end-of-file nil))
        (reverse file-forms)))

    (defun file-forms-alist (file-name)
      (let ((forms-table (make-hash-table :test #'equal)))
        ;; Build a hash that maps form-name => count for all the
        ;; top-level forms of the `file-name' file.
        (dolist (form (file-forms-list file-name))
          (let ((form-name (format "%s" form)))
            (puthash form-name (1+ (gethash form-name forms-table 0))
                     forms-table)))

        ;; Convert the hash table to an alist of the form:
        ;; ((form-name . count) (form-name-2 . count-2) ...)
        (let ((forms-alist nil))
          (maphash (lambda (form-name form-count)
                     (setq forms-alist (cons (cons form-name form-count)
                                             forms-alist)))
                   forms-table)
          forms-alist)))

    (progn
      (insert "\n")
      (insert (format "%7s %s\n" "COUNT" "FORM-NAME"))
      (let ((total-forms 0))
        (dolist (fc (sort (file-forms-alist "~/.emacs")
                          (lambda (left right)
                            (> (cdr left) (cdr right)))))
          (insert (format "%7d %s\n" (cdr fc) (car fc)))
          (setq total-forms (+ total-forms (cdr fc))))
        (insert (format "%7d %s\n" total-forms "TOTAL"))))

Evaluating this in a scratch buffer shows output like this:

      COUNT FORM-NAME
         32 setq-default
         24 eval-after-load
         14 set-face-attribute
         14 global-set-key
          5 autoload
          4 require
          4 setq
          4 put
          3 defun
          2 when
          1 add-hook
          1 let
          1 set-display-table-slot
          1 fset
          1 tool-bar-mode
          1 scroll-bar-mode
          1 menu-bar-mode
          1 ido-mode
          1 global-hl-line-mode
          1 show-paren-mode
          1 iswitchb-mode
          1 global-font-lock-mode
          1 cua-mode
          1 column-number-mode
          1 add-to-list
          1 prefer-coding-system
        122 TOTAL

This showed that I’m still using a lot of setq-default forms: 26.23% of the top-level forms are of this type. Some of these may still be candidates for lazy initialization, since I can see that many of them are indeed mode-specific, like these two:

    (setq-default diff-switches "-u")
    (setq-default ps-font-size '(8 . 10))

But eval-after-load is a close second, with 19.67% of all the top-level forms. That seems to agree with the original idea of speeding up the startup of everything by delaying package-loading and configuration until it’s actually needed.

10 of the remaining forms are one-off mode setting calls, like (tool-bar-mode -1), so 8.2% of the total calls is probably going to stay this way for a long time. That’s probably ok though, since the list includes several features I find really useful, very very often.

Then vs. Than

09 Jul 2010, 09:28 by Giorgos Keramidas

Scribd has many online books. I try to read at least one book every week. Starting to read a new book is always fun. The sense of anticipation builds up while the search for a new book is still in progress. The novelty of the new text reaches out of ...

A Tiny Lisp Script

25 Feb 2009, 03:18 by Giorgos Keramidas

A tiny Lisp script, noticed first in a recent post in comp.lang.lisp and then ported to SBCL:

    % cat ~/bin/iota
    #!/usr/local/bin/sbcl --script
    (defun iota (count &optional (start 0) (step 1))
      "Build a list of `count' numbers, starting from `start' andincrementing by `step' between
      successive elements ...

Oddmuse Wiki in Greek

24 Nov 2008, 18:43 by Giorgos Keramidas

The Oddmuse wiki engine has been updated by Alex, to include support for the Greek language.

When I started using the Oddmuse Wiki for my personal notes a while back, I was impressed by the easy installation steps, and the collection of supported Oddmuse extensions. Now that the Greek language ...

A bunch of updates for the Greek FreeBSD/doc translations

02 Sep 2008, 15:18 by Giorgos Keramidas

Translations of technical documentation from English to Greek are a relatively difficult task. It takes a certain level of attention to detail and a fairly good command of both languages. Then there is the minor issue of keeping the translations up to date with their English counterparts.

Updating translations (the ...

BSD make on other systems

28 Aug 2008, 02:25 by Giorgos Keramidas

The topic of using a “BSD make” on other systems seems to pop up now and then. I haven’t really finished the automake-based ‘port’ of FreeBSD make that I started some time ago, but seeing that others are interested too is a bit nice. A recent post by Chuck ...

A nice article about dVCS in the Enterprise

13 Feb 2008, 01:02 by Giorgos Keramidas

Bryan W Taylor posted a very intriguing writeup a bit earlier, titled: “The Need for Distributed Version Control in the Enterprise“.

There are a few points of the article that seem a bit controversial. For instance, I am not sure I totally agree with the comments abouts “feature scoped” development ...

Mercurial teaser

11 Nov 2007, 18:23 by Giorgos Keramidas

How fast can you push changesets to your colleagues if you are using a client-server based SCM?

Mercurial (which is a distributed SCM), when changes are pushed over an NFS-shared filesystem, can push 24 changes, with 81 patches, affecting 54 files, in less than 0.3 sec:

    gker@freya ...

Automated workspace updates with Mercurial

06 Sep 2007, 15:34 by Giorgos Keramidas

When using a distributed SCM, it is often very useful to be able to automatically sync a “reference” workspace with a remote, “parent” workspace. This way, even when offline, a local clone of the parent workspace is available.

Having a local clone of the “reference tree” is useful in many ...