Software Engineering

The core routines of the Handler superclass have been in place for a week (with under a week left to deadline), but I produced no useful work this week. Eventually I realized it was because I was fretting about the quality of the code in the Handler. It works, sure, but it works in a way that I've come to view as lacking, and which is unacceptible in this project.

It has no engineering in it. It's just sketched out and working, but it ONLY minimally works, and there is NO error handling, and... It's simply not on par, quality and reliability-wise with the Lexer and DOM code (not that I'm pretending they're perfect).

To fix this, I sat down and added a huge new section to the status codes list. Now that the Handler works, this list (section 3000 of it) is the spec for making it properly engineered code. It's also the roadmap for a test suite. I believe I might do my first bit of actual test-driven development, contra my more usual "Okay, this new little chunk works; time to write a test for it" method. (What's a good, marketable term for that, anyway? Would know-nothing trendwhore coders buy into "Test-backed Microdevelopment"? Would management pay $3500 to send their dev leads to a 2-day seminar on TBM?)

Lexer again?

Working on the Handler made me realize that the Lexer needed some teensy-tiny functionality adds. They are in now, and tested already. In addition to making the calls to the handlers cleaner, they should actually make error reporting nicer in at least some cases.

Away we go

I admit it: I was worrying that my holiday burst of inspiration and work ethic was an abberration which wouldn't soon be repeated. Having overhauled the Lexer and DOM and gotten them both up to 100% test coverage, I couldn't seem to get going on actual new code.

Sure, I'd spend a couple of hours every day or two of the last week-ish writing down bulleted lists or flowcharts of logic flow in my trusty quadrille notebooks, but I was frozen up as far as coding went. It just didn't all gel in my head yet.

Saturday night I felt like I'd gotten it all (where "it all" is the interactions between the Grammarian and its flock of subclassed Handler modules) worked out, and part of me was very much wanting to get going on the code, but another part of me was decidedly disinterested and just wanted to play video games. So that's what I did Saturday and most of Sunday, but I wasn't enjoying it all that much, because the part of me that wanted to be coding was busily yelling at the rest of me that I was a gigantic slacker who was going to completely fail on a self-imposed deadline and new year's resolution and god I just suck so bad why do I even bother talking to me?

I was exceptionally grumpy on the drive back to Atlanta.

But just before bedtime I finally managed to scrap together just enough code in the Handler superclass to write the first few tests for it. And then I subclassed it into the beginnings of the Comment handler, and wrote a handful more tests for that.

The game, once more, is afoot.

Today's Progress

The DOM is up to 98.9% coverage, with the remaining misses waiting on specific fixes to come along later.

I've started on a series of small, lightweight introductions to Carrot. That's way harder than the design and coding.

And finally, not something I did but something I realized today: It's time to start designing the language, and not just the compiler.

Planning, Testing

I've been working (slowly) on pulling coverage up. Just this morning, Lexer finally hit 100% across the board. The DOM still has some way to go, mostly in the coverage of repetitive checks on statuses and the like.

I haven't written much Handler code yet. Despite having some good ideas about things, once I started to code those ideas I realized that they might be good but they were also complex. At this layer, modules can't stand alone anymore (unlike the Lexer and DOM, which are dumb functional units that sit below everything, operating free of external context). Now things are starting to pass data back and forth in ways that affect each other's operation. Given that, I've been doing a lot of work in a quadrille notebook and rather little in Emacs, but it's all good and it's very valid progress.

The thing I'm happiest about is the (apparent) power of the framework I'm building. Though there are many, many, many things left to be done, it is becoming easier and easier to see how to implement things I once thought would be very difficult. I'm almost ready to believe this was all a good idea :)

Rewrites complete

The DOM and Lexer mdifications are complete, barring a tiny bit of Devel::Cover-fueled test writing on the DOM. I've also written a wrapper around D::C and upload its report to the Carrot site.

This weekend work on Handlers begins.

Resolution

In the past 4 days I've driven all over the state and still managed to get tons of work done on Carrot. What started as minor fixup on the DOM module ended up being a wholesale, in situ rewrite. The resulting code is much more regular and has a lot less special cases, and should be generally more maintainable in the future. My tendancy to view writing test suites which are as complete as possible as a sort of software engineering minigame paid off bigtime in this effort Even though it needed many (repetitive) modifications itself, the existing test suite was invaluable for turning up regression errors as well as plain old dumb mistakes made while coding in the pre-dawn hours.

Though there is a small amount of cleanup left before this effort can really be called "done", it's very close, and I know what comes next: The Grammarian and its attendant Handlers, which are pretty well planned out.

I'm only going to make one resolution for 2008 -- we'll see if there's a need for others to follow. I hereby resolve to have a public Carrot grammar validator in operation by midnight, 1February, 2008.