Stepping through code in IPython

I spend a lot of time in the Python shell - specifically, IPython. Like many Python programmers, I find it invaluable for delving into the structure of objects, exploring their members, running their methods, and so on. It's really the dynamic language's answer to the really good IDE support you'd get in a more static language like Java.

One of the things it's really useful to be able to do in the shell is to import a module and then step through the code in the debugger. Now, you can do this simply by importing pdb, but then you don't get the nice IPython-enhanced version that you get when you're dropped into the IPython debugger via the %debug magic.

Obviously, there must be a way of getting that IPython debugger manually within IPython itself - and there is:

from IPython.core.debugger import Pdb
ipdb = Pdb()

Now you can do with the ipdb object anything you would previously have done with pdb, except more snazzily.

The other thing I always forget is how to actually call an imported function and start in debug mode. Most of the time, when debugging running code, I just do import pdb;pdb.set_trace() and leave it at that (I do this so frequently I have a snippet in vim for it). But to call a function from the shell and start debugging straight away, you need another function: runcall().

This takes the function and its parameters, as separate arguments. So to start the function foo(bar, baz, quux) in the debugger you would do ipdb.runcall(foo, bar, baz, quux).

Now I understand why this is: otherwise you would be calling the debugger on the result of the function, which isn't what you want. But it's still a bit annoying to have to remember to do that. So, I decided to write an IPython magic script that translates the function call syntax into the separate-argument version - while still accepting the latter. I haven't written any magics before, so this is an experiment with the syntax.

Drop this into ~/.ipython/extensions and do %load_ext step from IPython. Now you can do %step foo(bar, baz, quux) and step through your code with joy.

import re

from IPython.core.magic import Magics, magics_class, line_magic
from IPython.core.debugger import Pdb

ipdb = Pdb()

@magics_class
class StepMagic(Magics):

  @line_magic
  def step(self, params):
    # params might either be
    # foo, bar, baz
    # or
    # foo(bar, baz)
    # we need to determine if there is a comma before the opening paren.
    comma_pos = params.find(',')
    paren_pos = params.find(')')
    if comma_pos > -1 and (comma_pos < paren_pos or paren_pos == -1):
      param_list = params.split(',')
    else:
      # Match everything up to first open paren, and everything inside parens.
      # We use lazy repetition on the first group to ensure that it copes when
      # the expressions within the parens are themselves calls.
      match = re.match(r'(.*?)\((.*)\)', params)
      if match:
        func, args = match.groups()
        # TODO: this doesn't work when args consists of lists/tuples.
        param_list = [func] + args.split(',')
      else:
        # Assume it's a single expression
        param_list = [params]

    evaluated_params = [self.shell.ev(p) for p in param_list]
    ipdb.runcall(*evaluated_params)


_loaded = False

def load_ipython_extension(ip):
    """Load the extension in IPython."""
    global _loaded
    if not _loaded:
        plugin = StepMagic(shell=ip)
        ip.register_magics(plugin)
        _loaded = True

As I note in the code, there's a bug: if your parameters are themselves lists or tuples, this will fail, as it will split the elements of the list into separate arguments. There are probably various ways of dealing with that, from better regexes all the way to messing with ASTs, but it's good enough as far as it goes.