Emacs+Python

This blog is very useful for pythoner under emacs...................

resource:http://pedrokroger.net/2010/07/configuring-emacs-as-a-python-ide-2/

-------------------------------------------------------------------------------------

Emacs is a huge beast. It canread email, play tetris, act as a file manager,display google maps, and evenedit videos. It has support for many, many programming languages and has manyfeaturestoprogramming. Unfortunately, emacs doesn’t have a full programming environment for python out-of-the-box.

In this post I’ll show how to configure emacs to write Python programs. We want to have not only basics things like syntax highlighting, but also code completion, easy access to Python’s documentation, ability to check for common mistakes, run unit tests, debugging, and a good interactive programming environment.

This setup is based onipythonandpython-mode, but it’s also possible to userope,ropemacs, and theauto complete modeas we can seehere. I didn’t have much luck with ropemacs in the past but I’ll try it again in the future.

Tools installation and configuration

First we should install the tools we need.

• python-mode. There are two python modes;python.elandpython-mode.el. Althoughpython.elcomes with GNU Emacs 22, I recommend you usepython-mode.elsince it has support for ipython. (There’s sometalkabout merging the two modes, but I don’t know it’s current status). Download and installpython-modeand put the following in your.emacs:

    (require 'python-mode)
    (add-to-list 'auto-mode-alist '("\\.py\\'" . python-mode))
  

• ipython. Ipython is a powerful python shell with advanced features and nice code completion. Check [ipython’swebsite]for documentation and screencasts. If you’re using debian or ubuntu, install it with:

    sudo apt-get install ipython
  

To configure ipython, edit~/.ipython/ipy_user_conf.pyand add your options. I like to useipy_greedycompleterso it will complete things like “foo”.TAB.

• ipython.el. It allows you to use python-mode with ipython. Download and installipython.eland put the following in your.emacs:

    (require 'ipython)
  

• lambda-mode. This is only for aesthetics. When you installlambda-mode.elit will display a lambda character (λ) when you typelambda:

The full name “lambda” will still be there, emacs is only displaying it differently. I like lambda-mode because it makes the code look shorter, neater, and cooler. Download and installlambda-mode.eland put the following in your.emacs:

(require 'lambda-mode)
(add-hook 'python-mode-hook #'lambda-mode 1)

I had to set the variablelambda-symbolto the following, but YMMV:

(setq lambda-symbol (string (make-char 'greek-iso8859-7 107)))

There are other ways to have pretty lambdas and pretty symbols in emacs, check theEmacs Wikipage to see which one you prefer.

• anything. We will useanythingfor code completion. Download andinstallanything.el,anything-ipython.el, andanything-show-completion.el, and add the following to your.emacs:

    (require 'anything) (require 'anything-ipython)
    (when (require 'anything-show-completion nil t)
       (use-anything-show-completion 'anything-ipython-complete
                                     '(length initial-pattern)))
  

• Change comint keys. Comint is a minor-mode to deal with interpreter commands in an emacs buffer. It’s used by many modes, including ipython. Comint uses M-p and M-nto cycle backwards and forward through input history. I prefer to use the up and down arrow keys:

    (require 'comint)
    (define-key comint-mode-map (kbd "M-") 'comint-next-input)
    (define-key comint-mode-map (kbd "M-") 'comint-previous-input)
    (define-key comint-mode-map [down] 'comint-next-matching-input-from-input)
    (define-key comint-mode-map [up] 'comint-previous-matching-input-from-input)
  

• pylookup. We will usepylookupto search python’s documentation within emacs. First install the python documentation locally. If you use debian or ubuntu you can install it with one command:

    apt-get install python2.6-doc
  

If the python documentation is not available as a package for your system, download it fromthepython websiteand unzip it somewhere in your computer. Then downloadpylookupand follow the instructions in the README file. Run pylookup.py to index the database by running:

./pylookup.py -u file:///usr/share/doc/python2.6-doc/html

This command will generate a database file (pylookup.db). Naturally, you need to put the correct path for the python documentation. Next you need to add the following to your.emacs. Again, replace the values to match your system.

(autoload 'pylookup-lookup "pylookup")
(autoload 'pylookup-update "pylookup")
(setq pylookup-program "~/.emacs.d/pylookup/pylookup.py")
(setq pylookup-db-file "~/.emacs.d/pylookup/pylookup.db")
(global-set-key "\C-ch" 'pylookup-lookup)

• autopair. A common feature in modern editors is to insert pairs of matching elements such as parenthesis, quotes, and braces. There are many ways to do this in emacs and installingautopair.elis probably the easiest one. You can configure it to work in all modes and exclude the modes you don’t to have it activated. For instance, I have it configured to work globally, except in the lisp-mode, where I useparedit:

    (autoload 'autopair-global-mode "autopair" nil t)
    (autopair-global-mode)
    (add-hook 'lisp-mode-hook #'(lambda () (setq autopair-dont-activate t)))
  

I also have the following to in my.emacsto make autopair work with python single and triple quotes:

(add-hook 'python-mode-hook
          #'(lambda () (push '(?' . ?')
                              (getf autopair-extra-pairs :code))
 (setq autopair-handle-action-fns
      (list #'autopair-default-handle-action
            #'autopair-python-triple-quote-action))))

• pep8 and pylint. These scripts are good to check your program for style errors and common mistakes. Install pylint and pep8:

  apt-get install pylint pep8

and download and installpython-pep8.elandpython-pylint.elto integrate them with emacs and add the following to your.emacs:

(require 'python-pep8)
(require 'python-pylint)

• delete trailing space. When you type the return key in python-mode it’llinvokepy-newline-and-indent. As the manual says, it “deletes the whitespace beforepoint, inserts a newline, and takes an educated guess as to how you want the new line indented.” This is great and I love this feature, but one unintended consequence is that you may end up with blank lines with trailing spaces. Python doesn’t care, but the pep8 tool will complain. Put the following in your.emacsto delete the trailing spaces when saving a file:

    (add-hook 'before-save-hook 'delete-trailing-whitespace)
  

• ipdb package. Theipdb packagemakes it easy to set breakpoints in the ipdb debugger. It’s easy to install it usingeasy_install(duh :-P):

    easy_install ipdb
  

• reimport package.reimportis a replacement for Python’sreloadfunction that reloads all modules and sub-modules consistently. It’s useful for long-running programs and interactive development. Install it witheasy_install:

    sudo easy_install reimport
  

• Templates.Yasnippetis a textmate-like templates for emacs.Download Yasnippetand pet the following in your.emacs:

    (require 'yasnippet-bundle)
    (yas/initialize)
    (yas/load-directory "~/.emacs.d/my-snippets/")
  

You can also installflymaketo check the syntax on the fly. I find it distracting, so I don’t use it.

All right, these are the tools we need! Now we can go on and use these features.

Using python-mode

So now we’re ready to go. Open a python file, saytest.py, typeC-c !(py-shell) to start the python interpreter. Go back totest.py(withC-x o) and typeC-c C-c(py-execute-buffer) to send the whole buffer to the interpreter. In the following picture we can see how it works; it’s neat to be able to write code in one file and test it in the interpreter without the whole edit-compile-run-test cycle:

Code completion

Code completion is very useful and I find a must to be able to complete methods for both built-in and user-defined classes. For instance, if you type “os.path.” andM-tabemacs will show all methods available in os.path:

This completion method usesanythingand you can useC-nandC-pto navigate through the completion items (when you’re done type return). This works for used-defined methods and functions too:

Access to documentation

Python has excellentdocumentation and it’s a huge bust in productivity to be able to access the documentation without leaving the editor. By using pylookup we can typeC-c hto search any term in the python documentation. For instance, if we type “os.path.basename” and thenC-c hto invoke the documentation, pylookup will open the documentation page foros.path.basenameusing the default browser configured in emacs (usually firefox):

However, many people prefer to usew3m inside emacsto browse technical documentation. With w3m we can read the documentation without leaving emacs, making it easy to copy code snippets and not needing to switch to an external program (to much switching can kill focus). Here’s the same documentation in the above picture but using w3m this time:

If you search for something with more than one possibleanswer, pylookup will ask you to choose among some possibilities, for instance, if I search for “print” I’ll get many options to choose from:

Code quality

Thepep8 scriptchecks if your code follow thePEP8 style guide. You can run it inside emacs by typingM-x pep8. The key M-x ` will jump to the place in the source code where the next error occurs:

You can run pylint inside emacs withM-x pylint, it behaves in a similar way to pep8:

Unit tests

It’s possible to run unit tests inside emacs if we use thecompilation mode. Type M-x compile and replace what will appear (most likely “make -k”) with your favorite python program to run unit tests. As with pep8 and pylint (they both use the compilation mode), you can type M-xto navigate to the next error. In the following screenshot we can see thattest_square_of_100_is_1000` is not correct:

Debugging

There aremany waysto debug python code. I like to use ipdb, ipython’s debugger, because it has advanced features like code completion and syntax highlighting. With the-pdbflag Ipython will go directly to the debugger when an exception is raised, instead of just printing a stack trace. (you may want to edit~/.ipython/ipythonrcto make the change permanent).

To set breakpoints in the regular python debugger, you need to add the following lines to your code:

import pdb; pdb.set_trace()

With theipdb packagewe can set breakpoints in a similar way if using ipdb:

import ipdb; ipdb.set_trace()

I like to make emacs highlight the lines that set breakpoints, so I can easily visualize where the breakpoints are and remember to remove them from the code after I’m done debugging. I use the following code to accomplish that:

(defun annotate-pdb ()
  (interactive)
  (highlight-lines-matching-regexp "import pdb")
  (highlight-lines-matching-regexp "pdb.set_trace()"))

(add-hook 'python-mode-hook 'annotate-pdb)

Notice in the following screenshot how the line that sets the breakpoint is highlighted and the program execution stopped at the breakpoint. I used the commandn(next) to advance to the next statement:

I have a function to add breakpoints mapped toC-c C-tso I can set breakpoints easily:

(defun python-add-breakpoint ()
  (interactive)
  (py-newline-and-indent)
  (insert "import ipdb; ipdb.set_trace()")
  (highlight-lines-matching-regexp "^[ ]*import ipdb; ipdb.set_trace()"))

(define-key py-mode-map (kbd "C-c C-t") 'python-add-breakpoint)

Templates

For textmate-like templates I useYasnippet(be sure to watch itsdemo). With it you can define templates easily and quickly to fill things for you, like a new-style class with documentation and an__init__method. Python doesn’t have much boilerplate, but using yasnippet can help you to write code even faster.

Finding your way in the source code

Emacs allows us to navigate through source code by usingtags. When you see a function call you can jump to its definition withM-.(find-tag) and jump back withM-*(pop-tag-mark). Another useful command istags-query-replace, to rename functions, methods, etc. To use it we need to generate a TAGS file. Emacs comes with theetagscommand but I recommendexuberant tags:

sudo apt-get install exuberant-ctags

Usually I put the following code in a Makefile to generate a TAGS file for a project:

ctags-exuberant -e -R --languages=python --exclude="__init__.py"

Dealing with multiple files

As we have seen, we can program in an interactive style in python-mode by sending the buffer to the python interpreter withC-c C-c. This works well for single file scripts and libraries, but no so well for more complex modules with sub-modules. For instance, if you openfile4.pyin themainmodulebellow and execute it withC-c C-cit’ll fail if it depends on other submodules.

mainmodule
|--- __init__.py
|--- submodule1
     |--- __init__.py
     |--- file1.py
     |--- file2.py
|--- submodule2
     |--- __init__.py
     |--- file3.py
     |--- file4.py

We can define a master file using thepy-master-filevariable; python-mode will execute the file set in the variable instead of the current buffer’s file. This variable can be set as afile variable, but I prefer not to pollute every single freaking .py file in a module with a file variable, so I use adirectory variableinstead. Another advantage of using directory variables in this case is that each member of a team can set thepy-master-fileto reflect their file systemlayout. Create a file called.dir-locals.elinmainmodule’s root with the following content:

((python-mode . ((py-master-file . "/path/to/interactivetest.py")
 (tags-file-name . "/path/to/TAGS"))))

(Note that I also define the tags’ filename, so emacs’ll automatically load it when I read a python file located in this directory.)

To make this work I use a master file calledinteractivetest.pyin the module’s root to re-import the module with thereimportpackage. This file is not really part of the module and I don’t even check it under version control. Now, every time we hitC-c C-c, regardless of what file we are changing, python-mode will executeinteractivetest.pyagain, and update the whole module. For instance, this is what I have in theinteractivetest.pyfor my aristoxenus library:

import reimport
import aristoxenus
reimport.reimport(aristoxenus)

But I may add a few things in theinteractivetest.pyfile to make testing things interactivelyeasier and to be able to save it between coding sessions:

from aristoxenus.parse import humdrum
foo = humdrum.parse_string("**kern\n4C")

Of course this is not a substitute for unit tests, but it’s nice to havefooavailable in the REPL so I can inspect it and quickly see what’s going on.

Conclusion

As you can see, we can have a nice programming environment for python in emacs. It may look like a lot of work, but in reality we only need to download and configure already existing packages. However, it would be nice to have a full-fledged development environment for python likeslime.

I didn’t mention many things like theemacs code browser, thegit interface, and thewindow session manager. They are not particularly specific to python, but they contribute to turn emacs into a powerful tool for development. Maybe I’ll mention then in another post.

I intend to make a screencast in the future showing these features, so stay tuned. Meanwhile, what do you use to develop python code?