Ronge

The current design

Kiwi’s design has gone through a number of iterations, and each iteration has reduced the amount of code, bettered the design, and brought us one step closer to a beta release. Kiwi’s current design has served us well, but we are beginning to near the end. We could trod forward, but things no longer fit together properly. Every time a simple code is changed, it brings with a multitude of changes in other components. Adding each feature adds code maintenance that is a bore, and is, frankly, unproductive. I’ve got some ideas, and with the help of others I’d like to hash out some more that fix this lack of design in the model and controller. Let me be clear, much of the code is quite good, but there are a few parts that are clunky. It’s those parts that I want to address.

First, let’s peek at the current Kiwi design. The application centers around a few key components: the database, MailCore, the threaded core, the model objects, and the controllers. In order to make sense of this all, let’s look at what happens when we click on a folder in the Kiwi UI.

Note: If this bores you to death, skip down to “So what’s right?”

1. We select a folder in the account and folder outline view. The outline view is bound to a tree of CTFolder objects which are held in memory, so the selection causes the corresponding CTFolder object that we’ll call theFolder to return its list of messages.

2. theFolder first creates an action to get a list of the messages from the server. Once the action is created, it is enqueued with the thread core. Next, if the folder doesn’t have the list of messages stored in memory, it fetches them from the CoreData database. Once the CTFolder object has a list of messages in memory it returns them, so the possibly incomplete list can be shown immediately in the UI while the actual list is loaded from the server.

3. Within the thread core a thread which is blocked waiting for an action to complete, unblocks when the retrieve message list action arrives in the queue. The thread contacts the server using the MailCore library, and pulls the resulting list of message headers from the server. The thread calls the specified callback it was given with its list of message headers.

4. When theFolder built the action, it specified a callback which was to be invoked by the thread when it finished. The same CTFolder object has now been called by the thread, letting it know that the message list is available from the server. This callback now makes sure that both the in-memory store, and the database are in sync with the message list passed to the callback.

5. After the message lists have been synchronized with the memory store and the database, the UI is told to update and display the new messages.

So if we look at this from a higher level this is what it looks like:

So what’s right?

Briefly, there are a number of things that I think are properly design. Separating out the IMAP, MIME and SMTP code into MailCore has drastically cleaned up the code base. I think it was one of the best design decisions made, and we should continue to add functionality to MailCore when appropriate.

Using the canonical producer-consumer model, and basing interaction around a queue with callbacks has given us a solid threading architecture. Despite a few minors changes that will be put in the long term basket, I think this bit of design should stay.

Constructing the UI via bindings is also a good practice, it has removed most boilerplate code. We’ve tried constructing the UI with bindings, and without, and with bindings is clearly the winner.

So what wrong?

Kiwi probably sounds solidly designed, but the jumble of synchronization code presents a problem. When the thread calls the callback, the callback has to synchronize the database with the server, and the in memory store. All the sync code amounts to a ball mud.

The UI is bound to a series of objects which are kept in memory, namely CTDatabase, CTAccount, CTFolder and CTMessage. Every time a folder is opened, a series of CTMessages are brought into memory from the database which the user interface binds to. Consequently, when the database is updated, it is necessary that the changes in memory are kept in sync with the database.

This is the problem with the current design, there is too much synchronization spaghetti code. How can we solve this? Well, that’s a really good question, and that’s really the purpose of writing this. I’ve been thinking about this the past few days and I believe it’s first necessary to separate out any code that synchronizes the database and the server. The code that resides in the callback should take care of just server to database sync.

What do we do about the memory store to database sync? Ideally, we should get rid of having to manually manage it. How can we do this? Well, instead of binding to CTAccount, CTDatabase, and CTMessage, we can bind the interface to the database directly and let bindings take care of keeping the UI in sync with the database.
I believe this would be ideal, but…there is a gotcha.

When a user updates something, like sets a message to read, we don’t want the read flag to be set in the database. Rather, we want it to issue an action for one of the threads to set the read flag on the server. Once the thread is complete, it will execute a callback that will update the read flag in the database. The question is, how do we make it so that doing edits in the user interface causes actions to be sent to threads, not edits to the database? Can this be done with NSArrayController, NSObjectController, etc. subclasses?

How can we allow bindings to take care of keeping the user interface in sync with the database, but override the default behavior so that when an edit is made we can modify the data on the server and then update the local database?

Other necessary changes

Here’s a few other changes that need to be made, but somewhat resolved.

• Move account information out of the CoreData DB. All the information in the database is a local cache of what’s on the server, except for the account information. It should be possible to delete this cache without causing any harm, and by putting account information in another file this will be possible. A nice side effect, is that during development we can continue to remove the cache and not have to type in account settings repeatedly. What I have in mind here, is that the account information can be moved to a plist, and we can keep an account record in the DB that just contains the account name.

• Get rid of all the unsightly [obj setValue:@”” forKey:@”“] and [obj valueForKey:@”“] CoreData code. I’m beginning to use mogenerator so that classes and accessor methods are generated automatically.

• Queue code should be self contained. Rather than passing around queue instances, as is currently done, the queue can be a singleton object which is given the action and an identifier. Using the identifier the queue “master” will find the appropriate queue. Because certain threads can only open connections to certain folders, the queue “master” will route actions to the appropriate thread queue.

Why not use the current design?

Why go through all this design work anyway? Why not continue to use the existing design, and forge ahead? Briefly, less code means fewer bugs, easier maintenance, and a gentler learning curve. We can press forward with muddy model objects, but I believe that fixing things now will save us a lot of headaches in the future.

By restructuring existing code, we can also make the code easier to test. Good codebases have as much testing code as actual code, and by this measure Kiwi needs a lot more testing code. By decoupling components, we make it easier to add unit tests throughout the project.

I am a firm believer in beautiful, simple code and architecture. If we keep Kiwi in line with these principles it may take longer to build, but the result will be first rate. As I was revising this, I found a footnote by Jon Bentley in Programming Pearls that illustrates his focus on simplicity:

Problem 6 describes a class exercise that I graded on programming style. Most students turned in one-page solutions and received mediocre grades. Two students who had spent the previous summer on a large software development project turned in beautifully documented five-page programs, broken into a dozen functions each with an elaborate heading. They received failing grades. The best programs worked in five lines of code, and the inflation factor of sixty was too much for a passing grade. When the pair complained that they were employing standard software engineering tools, I should have quoted Pamela Zave: “The purpose of software engineering is to control complexity, not to create it.” A few more minutes spent looking for a simple program might have spared them hours documenting their complex programs.

And with that note, let’s find a way to simplify.