DRY versus WET

You've got to have either a Snappy Acronym or Fancy Vague Phrase (FVP) in order be credible. So when I decided to write about what I think is wrong with DRY, I came up with 'DRY versus WET' - but I didn't know what WET meant.

DRY - Don't Repeat Yourself - is one of the Mantras of the Current Age of Programming Wisdom.

In case you don't know what a mantra is, it's a sound or phrase which you repeat over and over again while meditating. Meditating is something you do with your mind which brings peace and quiet . . . by avoiding all thought.

The problem I have with DRY is that you have to Think in order to write Good Code. In other words, you need Well Examined Thought - that's where WET comes from. [So now I've got an Acronym that Actually Means Something].

For Example:

Take a look at the Rails source code to see what over-DRY-ness does. After a while you'll see that (almost) every piece of code which could be repeated is turned into a method call. Even stuff which isn't more than a fraction of a line! This is Really DRY - so It Must Be Good. Right? Wrong!

This is Spaghetti [see The New Spaghetti for more ranting . . I mean 'detail'].

Historical Note: Spaghetti code originally referred to the overuse of GOTO (or branch) statements in FORTRAN or Assembler programs, but any program which has a high ratio of Control Transfer statements to Useful Work is really Spaghetti - because it reads like all the code was written on the sides of noodles and then all mixed up.

So . . . Don't DRY up. Get WET!


Think it's worth doing a book about?

The New Spaghetti

Back before Structured Programming - which everyone now thinks is as natural as breathing - we had a thing called 'Spaghetti Code'.

You wrote Spaghetti Style by using lots of conditional and unconditional GO TO statements to re-use every bit of code you'd written. It was the privative precursor of DRY [Don't Repeat Yourself].

In those days it kind of made sense - for a couple of reasons:

1. we didn't know any better

2. memory was tiny - we measured big machines in KiloBytes and wrote programs using 'overlays' [in case you don't know, an overlay is a chunk of executable code which 'overlays' another chunk in memory. We used to do that because (a) we didn't have enough memory to do the job and (b) there wasn't any automatic memory management]

The result of Spaghetti Style is that nobody could ever figure out how the programs worked and modifying them was next to impossible.

What brought this up?

I've been trying to 'learn Rails' - which is a pretty large Ruby application. Rails has close to 10,000 methods defined, about 1,500 classes, and God only knows what else. The Vocabulary is huge - and so it's a daunting task to say the least.

On top of the huge vocabulary, there it's DRY.

That seems to mean that every chunk of code which is used two or more times is ripped out and turned into a method. The result is that in order to do anything, somewhere near one and a half gazillion methods are called. In order to figure out how almost anything works you have to work through a call stack which is makes the old FORTRAN Spaghetti look like a 2nd grader's maze.

Why is this happening?

I'm guessing, but I think there are three factors driving this result:

1. DRY - This is a misunderstanding. DRY is a hip way of saying 'we want to maintain a single point of control so that we don't have to repeat bug fixes in the code'. DRY shouldn't be a mantra, because it's not a Principal of Good Programming. It's a method to implement an idea which is good programming.

Prior to formulating 'Single Point of Control' as DRY, programmers used some judgement in the size of the methods/functions they wrote. Typical size was 5 to 25 lines or so - enough to do useful work and small enough to be easily understood. Rubyists seem to be addicted to one-line methods which call other methods which call other methods which ... - but they don't Repeat!

2. Agile Development - This seems to go hand in hand with Test Driven Development. Both sound like good ideas, and they seem to work well for 'throw away' applications - which is what most web sites are. After crawling through the Rails Association Code, I've come to the conclusion something critical has been lost in the Agile-ness of the Development: There's no Design Phase.

Test Driven Development has proven you don't have to have an overall plan to build code which works. You can build any twisted mess and if you continuously test, it will pass the test.

But it won't make sense, won't be cohesive, will probably be inefficient, and it will be hell to modify or learn.

Everything has to be designed it is going to work well and be maintainable. That's true for Buildings, Cars, Motorcycles, Bridges, and, believe it or not, Software.

3. Ruby itself.

I love Ruby - almost - because it's a beautiful language and it makes it easy to otherwise hard things. The fact that Ruby is a scripting language means we get instantaneous feedback - and I love that too.

But both of those things - along with Ruby's openness which allows all kinds of self-modifying code [which seems to be confused with meta-programming] makes it easy to get the illusion of 'doing something' when the programmer is actually just churning different implementations with little impact on improving the goals of the application being developed.

Does that make sense? If not, read it again.

Activity != Progress

It takes discipline and experience to handle as powerful a tool as Ruby is without creating a mess.

Don't believe me?

Just take a look at the source for Mongrel and compare [or rather Contrast it] with a bunch of the Rails code. Zed Shaw writes awesome, easily understood code. Somebody in the Rails Core doesn't.

Hey Ruby: Where's the Main Program?

So as I get deeper into Ruby and Rails, I've found I've been going absolutely nuts! trying to understand how the programs work.

I think I finally figured out my problem. (That's right, it's a problem with ME for a change!)

There's no 'main' in a Ruby program.

Most executables - as in /usr/local/bin/foo - are two to ten line Ruby scripts which set up some environment variables and 'require' the 'real' script. The 'real' script usually 'require's a bunch of other scripts, instantiates an object, and calls what appears to be a random method.

That's it.

Step back and contrast this with C, Java, Python*, FORTRAN, COBOL, the Boot Sequence of a computer, ... Need I go on? Everyplace else there is (or pretty much is) a well known function, procedure, 'program', file or something where execution starts for every program which is written. Even Javascript kicks off with an 'onload' function (except in JQuery, but I think that's really a wrapper).

Notice I've left out Lisp, Scheme, Smalltalk and a whole bunch of other languages.

I don't know much about them - from a practical point of view - except that they are interpreted, don't (to my knowledge) have a 'main' entry point, and allow programmers to modify the base system so that it is unique to their program.

Now, I know someone might argue that all Ruby programs start by firing up the interpreter and loading the environment, so I'm wrong - as usual. But that's not what I'm talking about.

'main' is the start of the logic of the program. Knowing this gives (the generic) you a place to start unraveling the logic of a program. This seems like a 'good thing' (trade mark), but is it?

The whole idea of 'main' was really invented so that other programs (loaders) would know where to start executing a program. This is a 'convenience' with compiled programs: programs where the source code is processed into (possibly virtual) machine instructions and then further processed into an executable image saved in a file. This executable image is then loaded, source code agnostically, into the execution environment of a computer and run. It's a simple fact that the 'loader' needs a place to start and it doesn't know from beans about the source code - hence the well known 'main'.

You don't need this for interpreted, dynamic languages, but the S/W community seems to have carried this structure forward.

I think Ruby represents a significant break from this architecture.

Ruby 'programs' seem to be a bunch of symbiotic objects, executing in a common (probably specialized) runtime environment. Various behaviors (programs) can be achieved by instantiating different objects and calling appropriate 'starting' methods. This is similar to, but much more fluid than, writing a program within the constraints of a published API for, say, Windows, or a purchased library.

I like it.

I don't know how to understand it easily yet, nor do I think anyone really knows how to document it yet, but I think it's an advance.

---

* Python really doesn't have a 'main' either, but it has taken the 'flavor' of 'main' stipulating that code can be conditionally executed if it is 'run' as though it were the 'main' program AND a chunk of code is included of the form:

if __name__ == '__main__':
  some code

This chunk typically goes at the bottom and is encouraged when the code in the file can server either as the basis for a stand-alone program or an importable unit to another OR for testing.

Code Efficiency in Ruby

I got interested in the qualify of Ruby code in Rails when I noticed what appears to me to be a useless method in the ActiveRecord code. Specifically, ActiveRecord::Base.save is a public method which calls the private method ActiveRecord::Base#create_or_update. That doesn't make a lot of sense to me, because it could be replaced by making 'create_or_update' public and then aliasing 'save' to it.

So I decided to check to see what the superfluously method call costs.

I wrote a test which performed a simple task [incrementing an instance variable by a random number between 1 and 10] using five (5) different ways of accessing the instance variable and invoking the action.

The class definition is at the bottom of this post.

I then ran these methods 10,000,000 times using four different ways of invoking the methods:

• directly calling the methods - e.g. foo.inc_instance_variable()


• invoking via the method's 'call' attribute - e.g. foo.inc_instance_variable.call()


• invoking via the method via 'send' - e.g. foo.send('inc_instance_variable')


• invoking via 'eval'ing the string - e.g. eval 'foo.inc_instance_variable'

The Precent Results are simply the run time divided by the minimum run time for all tests converted to a percent increase.

Here's the summary:

• Invoking via an Alias doesn't cost anything


• unnecessarily accessing an instance Variable via an accessor slows down about 20%


• The unnecessary method/function call slows down by about 25%


• Combining the unnecessary call with accessor access slows down about 45% - so the effect is linear


• Invoking by the 'call' method slows it down by about 13%


• Using 'send' slows down about an additional 45%


• Using eval slows the process down by something on the order of 400%, but the effect is not linear, so 'eval' must be doing some additional mucking about.

So, what's the point? None, if you're satisfied with glacial execution speeds.

On the other hand, it's something you should know if you are writing critical code and have to make choices about how to implement it.

As usual, your mileage may vary. The full program code is at http://www.clove.com/downloads/method-call-timing-tests.rb.

Here are the detailed Percentage Results

Percent Results for direct method of invocation
foo.inc_var_as_instance                   0.99
foo.inc_var_as_instance_alias             0.00
foo.inc_var_as_method                    19.89
foo.inc_var_as_func_and_instance         25.28
foo.inc_var_as_func_and_method           45.17

Percent Results for call method of invocation
foo.inc_var_as_instance                  14.06
foo.inc_var_as_instance_alias            12.93
foo.inc_var_as_method                    32.53
foo.inc_var_as_func_and_instance         42.19
foo.inc_var_as_func_and_method           59.52

Percent Results for send method of invocation
foo.inc_var_as_instance                  42.05
foo.inc_var_as_instance_alias            46.02
foo.inc_var_as_method                    60.65
foo.inc_var_as_func_and_instance         75.57
foo.inc_var_as_func_and_method           94.03

Percent Results for eval method of invocation
foo.inc_var_as_instance                 393.89
foo.inc_var_as_instance_alias           410.94
foo.inc_var_as_method                   420.03
foo.inc_var_as_func_and_instance        458.10
foo.inc_var_as_func_and_method          577.84

Here are the raw timing results:

Result using Direct Calls
foo.inc_var_as_instance   7.070000   0.040000   7.110000 (  7.306318)
foo.inc_var_as_instance_alias   7.000000   0.040000   7.040000 (  7.155283)
foo.inc_var_as_method   8.390000   0.050000   8.440000 (  8.585970)
foo.inc_var_as_func_and_instance   8.780000   0.040000   8.820000 (  8.950350)
foo.inc_var_as_func_and_method  10.160000   0.060000  10.220000 ( 10.378127)

Result using .call
foo.inc_var_as_instance   7.990000   0.040000   8.030000 (  8.204646)
foo.inc_var_as_instance_alias   7.910000   0.040000   7.950000 (  8.061320)
foo.inc_var_as_method   9.280000   0.050000   9.330000 (  9.502992)
foo.inc_var_as_func_and_instance   9.940000   0.070000  10.010000 ( 10.249010)
foo.inc_var_as_func_and_method  11.170000   0.060000  11.230000 ( 11.439927)

Result using 'foo.send '
foo.inc_var_as_instance   9.950000   0.050000  10.000000 ( 10.220120)
foo.inc_var_as_instance_alias  10.220000   0.060000  10.280000 ( 10.462063)
foo.inc_var_as_method  11.250000   0.060000  11.310000 ( 11.514329)
foo.inc_var_as_func_and_instance  12.290000   0.070000  12.360000 ( 12.583919)
foo.inc_var_as_func_and_method  13.590000   0.070000  13.660000 ( 13.891333)

Result using 'eval '
foo.inc_var_as_instance  34.550000   0.220000  34.770000 ( 35.387702)
foo.inc_var_as_instance_alias  35.740000   0.230000  35.970000 ( 36.670451)
foo.inc_var_as_method  36.390000   0.220000  36.610000 ( 37.296297)
foo.inc_var_as_func_and_instance  39.050000   0.240000  39.290000 ( 40.033456)
foo.inc_var_as_func_and_method  47.420000   0.300000  47.720000 ( 48.638035)

Here's the class definition:

class Foo
attr_accessor :var

def initialize
 @var = 0
end

# access the instance variable directly
def inc_var_as_instance
 @var = @var + 1 + rand(10)
end
# access instance variable directly, but us an alias
alias_method :inc_var_as_instance_alias, :inc_var_as_instance

# access instance via accessor method, even though inside class instance
def inc_var_as_method
 self.var = self.var + 1 + rand(10)
end

public
# add an additional method call to accessing via direct access to instance variable
def inc_var_as_func_and_instance
 inc_var_as_instance
end

# add an additional method call to accessing via accessor
def inc_var_as_func_and_method
 inc_var_as_method
end
end