When To Choose Code Generation Over Reflection

One of the design decisions that has eluded me for quite some time now is the question of whether to choose code generation over runtime introspection (i.e. reflection). The current consensus today is to avoid code generation. I guess it's because too many of us have been burned by either inflexible systems or ultimately unmaintainable nightmares. However, after recieiving my share of difficulty working with today's crop of frameworks, I'm beginning to seriously doubt the overuse of reflection.

Michael Rettig wrote an article a long time back about "Reflection vs. Code Generation". When I first read the article, I didn't truly appreciate what he was saying. However, after contemplating about the issues, I returned to the article and finally it made perfect sense to me. One reason why you should remember those articles that don't makes sense to you.

Another article that I didn't have time to really read in detail was Howard Ship's line precise reporting explanation. Howard says that feedback is a key aspect in building frameworks, in conjunction with simplicity, efficiency and consistency. Howard's method is to always record information as to the source of configuration parameters. That way if an exception does occur, you can quickly pin-point the source of the problem.

The difficulty with overusing reflection from complex scenarios is that it is painfully difficult to discover the source of an error. This may not be a problem of reflection per se. Using reflection is an extra indirection away from using configuration parameters. The key missing ingredient of most systems is the lack of traceability. Ever wonder why there's this new movement in trying to avoid using configuration files?

One of Tom Rettig's problems, as he explains in his article, is that systems with reflection don't report the source of the error as precisely as systems that use code generation. However, the difference is not only in less traceability. Employing code generation allows you to leverage the full syntactic and semantic capabilities of the underlying language. In simpler terms, that means you can use the debugger effectively. Using reflection doesn't mean you can't use the debugger, it just makes using the debugger more difficult.

So, if you ask me, when should one use code generation over reflection? The obvious answer would be when you want better performance, however the enlightened answer would be, when you want better traceability. Tom Rettig writes:

As described above, the number one limitation to using runtime reflection is "Do not make a simple problem complex." With reflection, this is unavoidable. Coupling reflection with recursion is one serious headache; reviewing the code is a nightmare; and determining exactly what the code is doing is an intricate process. The only way to truly determine the code's behavior is to step through it, as it would behave at runtime, with sample data. However, to do this for every possible data combination is nearly impossible. Unit testing code helps the situation, but still cannot quell the fears of a developer who sees a possibility for failure.

In general, use code generation when you have to work with complex recursive structures in combination with a complex object model. Don't over complicate your life by having to simultaneously deal both a complex runtime structure and a complex object model. Divide and conquer, if you can reduce the variability of the solution by freezing one structure through code, then you have a better chance at solution.

However, I've got to ask, has anybody put together a good methodolgy for working with code generation?