Converting strings to methods

During my internship, I am switching gears and fixing bugs in the imaplib standard library. I seemed daunting at first, reading the RFC documentation. However, I feel like I am getting the hang of it.

Anyway, today I thought I go through my journey on how I figured out how to call methods with a string command, since I've been reading a lot of code that does that, particularly in the imaplib unit tests.

Let me explain the problem statement.

Let's say you have a networking protocol and depending on the message, you want to call different methods. Here, I am using a subset of commands from the IMAP protocol.

Let's say you are a server, getting strings with different commands. Here are some examples.

b"CATZ2 LOGIN 'lita@fake_email.com' 'this_is_my_password'"
b"CATZ2 SELECT INBOX"
b"CATZ2 SEARCH ALL"

# Note the 'b' prefix means this is a byte literal, not string literal.
# There is no difference in 2.7 but matters in Python 3.3.

The above represents commands that the client could send you through the IMAP protocol. Let's quickly break down what these commands means.

CATZ2 is an identifier called a "tag". This tag is generated by the client and helps the server keep track of all their clients. This tag must be the same throughout the client and server interaction. I like to think of a tag to a IMAP server as a cookie to a browser (although it is not completely the same, as the cookie is generated by the server).

LOGIN, SELECT and SEARCH are the actual command names. Anything following that are arguments to the commands or nested commands.

The three commands basically logins into your email, selects the Inbox mailbox, and searches for all the mail inside of Inbox, returning the ids of all the messages. It doesn't actually give you your messages quite yet. You need to call the FETCH command for that.

I had to make a mock server to write unit tests for the imaplib. I had to write a simple parser to parse the client's commands. Here, we will go through the different methods of doing this. Unfortunately, I won't go over how to implement the IMAP commands. I've also simplified my code in order to highlight using strings to call functions.

The naive way is to write an if-statement, like so.

def login(args):
    print("LOGIN")

def select(args):
    print("SELECT")

def parse_command(cmd):
    splitline = cmd.decode('ASCII').split() # We need to decode the string
    tag = splitline[0]
    cmd = splitline[1]
    args = splitline[2:]
    if cmd == 'LOGIN':
        login(args)
    elif cmd == 'SELECT':
        select(args)
    ...
    else:
        raise ValueError("Command doesn't exist")

This is not very elegant. We definitely can do better.

Another way is to create a dictionary, where the keys are the string names and the values are the methods themselves. I've seen this used in the Turtle module and various other places.

def login(args):
    print("LOGIN")

def select(args):
    print("SELECT")

def search(args):
    print("SEARCH")

str_to_func = {'LOGIN': login,
               'SELECT': select,
               'SEARCH': search}

def parse_command(cmd):
    splitline = cmd.decode('ASCII').split() # We need to decode cmd as
                                            # they are byte literals
    tag = splitline[0]
    cmd = splitline[1]
    args = splitline[2:]
    if cmd in str_to_func:
        str_to_func[cmd](args)
    else:
        raise ValueError("Command doesn't exist")

This is a lot better! The parse command is a lot shorter due to the fact that there isn't a long chain of if statements.

But our mock server will probably be a class. And what if we want to add new commands in the future? What if other developers wanted to subclass our class and add their own custom commands? What if we need to add special features for different tests? Each time, they would have to know to override or add to the str_to_func hash table and register their new methods.

class BaseServer(object):

    def __init__(self):
        self.str_to_func = {'LOGIN': self.login,
                            'SELECT': self.select}

    def login(self, args):
        print("LOGIN")

    def select(self, args):
        print("SELECT")

    def parse_command(self, cmd):
        splitline = cmd.decode('ASCII').split()
        tag = splitline[0]
        cmd = splitline[1]
        args = splitline[2:]
        if cmd in str_to_func:
            self.str_to_func[cmd](args)
        else:
            raise ValueError("Command doesn't exist")

class CustomServer(BaseServer):
    def __init__(self):
        super().__init__() # You can do this in Python 3! Otherwise you
                           # the syntax is super(CustomServer, self)
        self.str_to_func["CATZ"] = self.catz

    def catz(self):
        print("LOLCATZ")

Sure, you can write a registration method. However, it would be great if we didn't have to keep track all the methods at all.

Well, it turns out we don't have to! When you create a class, Python creates a dictionary of attributes for it's interpreter. We can use this to our advantage to convert the string commands to a method call. We can use the hasattr and the getattr methods to keep track of commands for us!

class BaseServer(object):

    def login(self, args):
        print("LOGIN")

    def select(self, args):
        print("SELECT")

    def parse_command(self, cmd):
        splitline = cmd.decode('ASCII').split()
        tag = splitline[0]
        cmd = splitline[1]
        args = splitline[2:]
        if hasattr(self, cmd.lower()):
            getattr(self, cmd.lower())(args)
        else:
            raise ValueError("Command doesn't exist")

class CustomServer(BaseServer):

    def catz(self, args):
        print("LOLCATZ")

The class object is already managing what methods exists and what doesn't. Why don't we use that to our advantage? This way, when someone subclasses the BaseServer, they don't need to worry about registering the new commands.

The downside to this is that the method names must match up with the string names. So if you wanted the login method to be called authenticate, you have to go back to using your own dictionary.

Hopefully, these ideas are useful. I am still learning as I read through the source code of Python's standard libraries. If you know of other ways to do this, please comment!

Comments !