ORM or Not? Part Two - Definitely Not

Maybe that's a little too harsh - but I don't think so.

Here's the background:

One of the biggest recurring programming problems I've had to deal with in site and application development has been - (drum roll) - developing the database. Not populating it - that's just boring. Developing and refining the data definitions.

The "Conventional Wisdom" promulgated by Software Gurus is basically this:

1. Databases are Good With Data
2. Data Objects Are Good With Data
3. A Programmer Must Define a Mapping Between these Two Good Things and then All Will Be Good

Not my experience.

Software Gurus don't write application. Software Gurus write Books and (occasionally) toy examples. So they really don't understand that Good does not generalize because the Good that Databases are with data is all about safety and accessing it in great and small piles. The Good that Data Objects are about is manipulation, fine structure, and flexibility.

My experience with Object Relational Data Mapping - which is just a fancy phrase for how you get the data from the database into an object and back - is this: when I Did it Their Way, I had to hand-coordinate both my Objects and my Database.

So when I wanted to add a field or change a field or change a datatype, I had to do everything twice.

Now Rails has Active Record - which is a stupid Software Guru name for an ORM which creates the Data Objects automatically (my drivers license is an active record - meaning it's current so I can legally drive), but that doesn't 'solve the problem' it just moves it. [maybe Rails 3 is smarter,
I dropped out just as Rails 2 was coming out]

Specifically, the Rails solution moved the problem to 'migrations' which turned out to be fragile. I know: I tried it.

So, a couple of years ago I proposed Not building an ORM, but rather loosening the coupling between the Database and the Data Objects.

Here's what I've done:

1. Defined an PHP class which implements a data abstraction which includes most of the types of data and methods I want in my CMS [see http://www.yasitekit.org] and knows how to map those data types into database fields. It also knows how to create the database structures, create, update, delete, and retrieve those values.
2. Defined a PHP class which makes it easy (relatively speaking) to create objects which are instances of the data abstraction class. These classes come with a bunch of methods which I've found useful as well as a management class which automatically provides interactive data administration.
3. Some automated analysis tools which work with this stuff so it's possible - and relatively painless and safe - to add, delete, and modify the derived objects (point 2) and reload the database. All done by hacking the derived PHP objects, but never ever touching SQL.
4. As a side benefit, I threw together a database adaptor which abstracts the 8 essential database operations [create/destroy database;create/drop table;insert,update,delete,select] at a level high enough that we never have to muck with SQL. This makes it possible to augment it with non-SQL databases - such as Mongodb. In it's present form it allows painless migration between sqlite, mysql, and postgresql (it currently handles 5 different PHP database interfaces - automatically figuring out what's available - yada yada yada)

So, at this point, I feel more than comfortable saying No to ORM's.

BTW - if you go to the YASiteKit site, you may be a little dissapointed because all of this good stuff is not available yet. But It Will Be - Real Soon Now (no kidding). And a whole lot more.

What is on the site is a pretty-much full set of documentation - both about the system design and about how each of the parts work [I write doc as I go, so it's not all pretty - but it's useful (I know because I use it)].

So take a look. If you have any comments - let me know.

Also, if you like it enough to want to help - let me know sooner - I'm almost ready to turn it loose to see if it gets any traction.

Timing Tests

OK - I'm not a timing test expert. I'm not a program profiling expert. etc etc etc

I know timing tests are 'hard to do right'. I know that there are all kinds of consideration. I know that 'to do it right, you have to . . .'

But I don't really care about 'doing it "right"' according to some picky standard.

What I do care about is not writing really slow code.

For that, the rules are simple:

Rule 1: Don't do things that take a long time

Rule 2: If you have to do repeat something a lot, check out alternative ways to do it and pick the method which is both clear to read / understand and takes the least time.

Rule 1 - expanded:

You do this by knowing how long things take and which operations are blocking. You don't need to be accurate, because for most things, 'takes a long time' is measured in orders of magnitude.

Here are the relevant cases:

1. Monolithic program doing in-memory data access/processing
   
2. Multi-threading/parallel processing/whatever - any form of parallel processing which is executed inside a single process context. Here you tend to lose because of blocking and communication - one thread needs to access shared data and so blocks reads, etc OR one thread needs results from another to continue OR etc.
3. Self generating code - aka Metaprogramming. This is a cool way to impress your friends, but it costs orders of magnitude in performance. The idea is to write code which traps function calls to functions which don't exist, then parse the function name and build a function 'on the fly' to do the task encoded in the function name, dynamically build the call sequence, execute the function and return the result. It's not hard to do, but it's pretty much unnecessary (almost all the time) and really slows things down, because parsing strings takes a lot of repetitive work. That's why we have compilers!
   
4. Disk reads are much longer - at a minimum they require a context switch as you make a system call. Then it depends on file size, caching in the operating system, memory size, etc. The Rule is: for repeated reads, try to do only Once and cache the result in a variable
5. Run a subprocess - this requires a context switch, process invocation, lots of disk reads, etc etc followed by receiving result, parsing it, etc etc. Much more expensive than disk, but less expensive than Network reads.
   
6. Network reads are longest - not only do they require a system call, you typically have to run another process someplace. If that process is on a different host, then the cost is astronomical relative to in-memory and disk i/o.

So Rule 1, says - if you don't really need to do the Slow Thing, then don't. And Do the Slow Thing as seldom as you can get away with.

Rule 2 - expanded

Right now I'm writing a lot of PHP (don't groan, it seemed like a good idea at the time) and in this code I have lots of places where I need to do things based on the value of a string. For example, I'm writing a lot of PHP5 objects where I put guards on attribute access so that I can find spelling errors (my High School English teachers understand why I need to do this).

So I have lots of functions that look like:

function __get($name) {
if (in_array($name, array('foo', 'bar', 'baz'))) {
  return $this->$name;
}else {
  throw new Exception("$name is not a valid attribute name");
}
}

or

function __get($name) {
if (($name == 'foo' || $name == 'bar' || $name == 'baz'))) {
  return $this->$name;
}else {
  throw new Exception("$name is not a valid attribute name");
}
}

or

function __get($name) {
switch ($name) {
  case 'foo':
  case 'bar':
  case  'baz':
    return $this->$name;
  default:
    throw new Exception("$name is not a valid attribute name");
}
}

I do this a lot, so I need to know which one is the fastest. I don't need to know precisely, I just need to know 'more or less'.

To do this, I need to build a test case and run it to get some timing numbers.

The test case doesn't have to be perfect, but it does need to put the emphasis on the differences between the three different methods. It also has to be large enough to be able to distinguish run times between the methods.

In this case, I built the three functions each with about 150 alternatives and the built a list of trials which would fail about 1/2 the time. I then executed each function a bunch of times.

How many is the right bunch? I'm lazy, so I start small for the number of repetitions and then crank it up until the total run time per method is around 10 to 60 seconds.

Here's what I got:

• switch: 49.7731 seconds
  
• in_array method: 86.3004 seconds
  
• if with complex conditional: 57.0134 seconds

Guess which method I'm going with.

[guess how I'm going to refactor a lot of my code (sigh - I should have tested first)

Why are our Programming Languages so Bad?

I just took a quick look at Scala and Lua . . . and I don't think either one is a winner, but for different reasons.

The Scala guys seem to have fallen into the arcane syntax trap - with lots of critical information inferred. I think I agree with this blog post from January, 2008: Scala is probably not a readable language.

Scala seems partially motivated by the Java mistake of believing that 'more required words make more readable code'. That just makes the programs bulkier, not clearer. Other parts of the motivation seem to be to try to find a syntax which supports functional programming, OOP, and everything else which might be fun.

On the other hand, Lua doesn't have a rich enough structure. I think I'd rather write in C than something like Lua. Lua isn't OOP, it isn't a functional language, it doesn't even support structures sufficientlyl, let alone objects. I've had a lot experience working in awk - which is nice, but does not have good support for complex data objects, which makes the programs difficult to maintain and . . . this could get boring fast, so forget it. The bottom line is: languages which don't support a decent object model slow me down too much to bother with.

I've been writing a lot of Python lately - after burying myself in a PHP project for about a year and a quarter. I also write a lot of shell script, HTML, CSS, and whatever. I don't write Ruby anymore - but I don't want to get into that now. Before this I wrote a lot of C, Pascal, awk, etc and - believe it or not - about a half a ton of FORTRAN. So I've written a lot of code in some pretty awful languages.

The simple fact is that none of the modern languages are any good. They all suck.

Why is that?

We really know a lot about language design by now - or at least we should.

One of the things which really irritates me is that every damned language uses different syntax for the same semantic concept. I don't think I know of two languages which implement if ... else-if ... else ... the same way. [else-if is spelled 'elif', 'elsif', 'elseif', or 'else if' or doesn't exist].

It is now a fact that programmers work in multiple languages. These stupid, unnecessary syntax inconsistencies make life hell.

I'm starting a list of what should be in a modern language:

1. It should be syntactically as small and clear. Don't use words where symbols are clear: i.e. don't use 'begin' and 'end' - curly braces work just fine.
2. No reserved words. We don't need them. Somebody - I think it was Kernighan - pointed out that a decent parser can determine the meaning of a word based on the structure of the sentence, so that 'for' can mean one thing in one context and something else in another.
   
3. It has to support objects with (at least) single inheritance. I think Ruby got that mostly right. I think Python got it wrong by supporting multiple inheritance. I like Ruby's idea of Modules because it allows excellent code reuse. It gets us the utility inheritance promises without the head aches.
   
4. Scoping has to be correct from the start. Block scoping the way C does it is right. Matz got that wrong in Ruby - I hear they're changing it again in 1.9. Python still doesn't have it right yet, but I think it's getting closer.
   
5. Global variables are Evil. Crockford is right: They should not exist.
6. Variable declarations are a pain - but less painful than tracking down spelling errors which the language accepts. I've lost a lot of time needlessly hunting down misspelled words in PHP that would have been caught by simply requiring variable declarations so the compiler could catch them. So, variable declarations are Good.
7. String handling is Good. If you don't think it's a necessity - go write some string handling in C for a few years.
   
8. Dynamic - aka Duck - Typing is Good - we need it. I wasted too many years living without polymorphic stuff to ever go back [read: writing in C, pass a pointer and figure out what it is inside the function and hope to hell you don't screw up].
9. Static Typing is Good - we need it too. I've wasted too many years finding bugs which could be caught by a decent type system.
10. Functions need to be 1st class things. In fact, everything needs to be.
11. Closures are Good - we need them.
12. Functional programming is good - We need it.
13. Imperative Programming - Structured style [like Dykstra told us] is good - we need it too.
    
14. Operation overloading is Good - We need it. Everything should be overloadable. I think Ruby got that right as well. Python has been incrementally getting there for years.
15. Self modifying code can be a good thing - but it's hard to do right and rarely needed. I think it's better to support it directly rather than trying to discourage it. This is in spite of the crap the Ruby community loves to write [they call it meta-programming, but it's not really meta programming - it's automatically generated code] For some reason they think that self-modifying, self-generating code is intrinsically a good thing - but then I used to do a lot of stupid stuff when I was younger too. I think this is a result of lack of experience - especially in maintenance - and a lot of incompetence. It sure makes Rails a mess.
    
16. Exceptions are good - We need them. Error handling is always an issue and good, clean exception generation and handling support makes it easy to include it.
17. Interpreted is Good. It makes writing code much faster. Must have a REPL.
    
18. Compiled is Good. We always need speed. Only the stupid say 'speed doesn't matter'. It always matters, but very rarely at the expense of clarity.
19. Do we need an IDE? I don't think so, but I don't know. I just write in TextMate on my Mac. I used to write in Emacs - and still use vi from time to time. I've tried Eclipse, but just couldn't deal with it. I think this is a non-issue except that the Language should support development without and IDE.
20. Reflection - aka inspection - aka whatever - is Good. Python has that pretty right. All functions and classes take an optional documentation string which you can print in the interpreter by typing print foo.__doc__. It saves lots and lots of time paging through documentation. There is also a builtin function called dir() which generates a sorted list of all the attributes of it's argument. Ruby has that wrong - I don't know how many times I wrote foo.methods.sort to try to remember the name of a method when hacking Ruby.
21. Automatic Document Generation is Good - but the current systems stink. They are like a tail wagging a dog: Code is always more important that comments - and all documentation is comments. Why? Comments don't execute, so they always have bugs and eventually diverge from the meaning of the code [See Brooks: Mythical Man Month where he points out that it's not possible to keep to separately maintained files in sync] Documentation needs to be unobtrusive, compact, and easy. I have no idea how to do this - yet.
22. To be Continued

Any Things to add to the list?

Documentation - It's Part of the Design Process

I was documenting some code I just wrote to create a packing list for an internet store we'll be opening soon. It's kind of a hairy problem to come up with a semi-optimal allocation of goods into boxes - but absolutely necessary to do right if you want to accurately estimate shipping and keep costs down.

It took me about a week to really get my head around the problem so that I could solve it. The actual solution now looks pretty obvious - but it won't in six months or a year.

So I document all my code - profusely, but compactly.

A Strange Thing happened while I was writing the doc: I found and corrected a whole bunch of bugs - both coding bugs, 'bugs of omission', and design bugs. That's when I realized that I use the very act of documenting code as part of my design iterations.

So, I started thinking about what I do and why I do it the way I do - which is (as usual) a bit contrary to 'accepted best-practice'.

First of all, there are basically Three Types of documentation in code:

• The Code Itself
• Comments
• Doc

The Code: My code is semi-self documenting. I do the usual stuff: identifiers are semantically related to their use [function save_data() - etc]; use of white space; consistent breaking of long lines and breaks at binary operators; consistent indentation.

But you can do more.

Simple, direct, and consistent code does wonders for readability. Consistency in within-loop logic [use if elseif elseif ... or switch case case default or if () { stuff; continue;} if () { stuff; continue;}]. I don't think it matters what style you use as much as consistency and simplicity.

Small is usually good as well. We used to talk about 'locality of code' meaning that our code was written in chunks which weren't 'too long.' 'Too long' usually really meant 'it fits in the window of my editor' so that you can stare at it and understand the whole thing. There's also 'too short', but the Ruby on Rails guys have the market on that with their obsessive DRYness.

Anyway, the Code itself is always the final authority on what the program does - so it's the most authoritative documentation.

Do yourself a favor and make it as readable as possible - you might have to fix it later.

Comments: Helpful hints which make the Code more readable. Some (IMHO) IDIOT came up with the idea that Comments Must be Set Off with Big Boxes of Stars or something like that.

Why?

Well, I'll tell you.

We used to write programs on Paper and then punch them into Punch Cards and run them through a Card Reader and then get a Big Wide Printout on Greenbar paper and do our debugging at a desk. We'd page through this Paper with a pencil and write revisions to our programs. The paper was 14 inches wide [132 columns] and 11 inches long [60 lines at 6 lines per inch with 1/2 inch top and bottom margins].

In those days it made sense to Mark Off the comment blocks. It made them easy to spot - both for someone reading the code and for the Boss as he walked by your desk. [You'd get demerits for not commenting your code, so it was a good idea to make it easy for him to see].

Things have changed. You can easily flip around in an on-screen editor, but you can't very well write on it. It works a lot better if the information in the editor window is very dense. The way to do that is to write meaningful comments with as little excess white space as possible.

Now I use blank lines to separate logically disjoint chunks of code and preceed them with comments - if appropriate - but I don't waste a lot of screen space blocking off comments.

I think we need a very high Signal to Noise ratio - and big blocks of comment markers are just noise.

I don't use PHPDoc because those guys seem to think that the Documentation and the Comments are More Important than the Code itself. I don't. I want as much Code on the screen as I can get if it is blocked out so that the logic is obvious and commented as needed.

The Doc: The Doc is narative description of what the code does, how it does it, how to use it, calling parameters, etc etc. Can't live without it.

I've experimented with all kinds of methods. I used to write UNIX style man pages - and they are still a really good format. But now days, everything has to be in HTML, so I've changed.

There are a lot of Documentation Project things - like PHPDoc - but I don't use them.

I think they're too complex and they are usually specific to one language. Besides, they just aren't necessary for good, clear documentation.

Like many people, I work in several languages [one reason I'm bald] so I need a multi-lingual documentation system. I also want something which doesn't take up a lot of space in my program files and which is really simple. Here are my 'Must Haves':

• Must allow the Doc to be imbeded in the same file with the code
• Must produce HTML
• Must be flexible
• Must be very Simple

I couldn't find anything which didn't take multiple days [or weeks] to learn, so I wrote up something to pull Textile makeup out of a simple text file and that's what I use.

Frankly, I'm smart enough and write well enough to describe what some code does. So are you. Just pretend you're writing for yourself about a year from now when you have to find a bug and fix it in record time - or you'll be fired.

What kind of Doc do you want then? Clear, correct, self-contained without a lot of references jumping around, etc etc. Write something which will save your butt when the crunch comes and you're fine.

Design?

So what does this have to do with Design?

As I write my Doc - part 3 of the Code Documentation - I find myself thinking about how the whole thing works and how understandable it is. It can get kind of contemplative.

Another thing which happens is that I find myself wondering why in the hell I wrote something and what it really does - so I go through the logic again.

There's a different perspective and attitude when reading code to understand and clearly document it than there is when writing it initially. At least there should be.

When I write code, I get involved in the microscopic detail, in solving the problem, in creating the logic and the loops.

When I read code to document it, I get involved in the overall strategy of the solution and the clarity of the logic. It's just different.


Anyway, that's how I think about Documenting code now days.

Oh, if you're interested in my document extractor - I'm planning on dumping on the download page of http://www.clove.com. It's written in Python, so it should run about anyplace.

ORM's or not?

In my quest for a CMS I can not only stand but want to embrace, I've spent the last couple of weeks messing with Drupal.

Drupal is impressive - at the very least in scope and participation. I like quite a bit of it. In contrast with Rails, Drupal releases seem to be Designed rather than Grown - which is a very strong plus. [I'm not sure I can stand it, but that's another story.]

But I'm not writing to Praise Drupal, but to make an observation about Object Relational data Mappers.

Drupal through 6 doesn't have an ORM. They have two 'database adapters' and opt to use SQL. Most of the SQL used for CRUD operations is standard, so that works just fine. I'm told there is discussion in the development forums about stuffing a layer of abstraction in the database code - which I read as 'create an ORM.'

Looking over the landscape littered with the corpses of ORMs in almost every conceivable (computer) language, should give pause - at least it does to me.

Why do we ORM?

Well, we have to have the data tables in the Database match up with the Attributes in the Objects, don't we?

Well, No we don't.

There's no reason for the Data Tables to model our 'business process'. That's a knee jerk reaction taught as gospel in C.S. courses.

Couple of Facts:

• Databases are really good at holding, retrieving, searching, and sorting vast quantities of data. Programming Languages aren't.
  
• Programming Languages are really good manipulating, displaying, inputing, validating, mashing, spindling, and folding data. Databases aren't.

Now for a Web Site, much of the content doesn't have a long life, is retrieved by simple, well known searches, ordered by well known sorts, and is displayed in small chunks.

It seems to me that there much of the data used in a web site could be modeled exclusively in the Programming Language and then serialized in XML or JSON for storage in the database. That separates the representation of the data in the application from the representation in long term storage and obviates the need for an ORM.

I like that idea so much, I'm building one now just to see how it works in practice.

Stay Tuned

DRY versus WET

You've got to have either a Snappy Acronym or Fancy Vague Phrase (FVP) in order be credible. So when I decided to write about what I think is wrong with DRY, I came up with 'DRY versus WET' - but I didn't know what WET meant.

DRY - Don't Repeat Yourself - is one of the Mantras of the Current Age of Programming Wisdom.

In case you don't know what a mantra is, it's a sound or phrase which you repeat over and over again while meditating. Meditating is something you do with your mind which brings peace and quiet . . . by avoiding all thought.

The problem I have with DRY is that you have to Think in order to write Good Code. In other words, you need Well Examined Thought - that's where WET comes from. [So now I've got an Acronym that Actually Means Something].

For Example:

Take a look at the Rails source code to see what over-DRY-ness does. After a while you'll see that (almost) every piece of code which could be repeated is turned into a method call. Even stuff which isn't more than a fraction of a line! This is Really DRY - so It Must Be Good. Right? Wrong!

This is Spaghetti [see The New Spaghetti for more ranting . . I mean 'detail'].

Historical Note: Spaghetti code originally referred to the overuse of GOTO (or branch) statements in FORTRAN or Assembler programs, but any program which has a high ratio of Control Transfer statements to Useful Work is really Spaghetti - because it reads like all the code was written on the sides of noodles and then all mixed up.

So . . . Don't DRY up. Get WET!


Think it's worth doing a book about?

The New Spaghetti

Back before Structured Programming - which everyone now thinks is as natural as breathing - we had a thing called 'Spaghetti Code'.

You wrote Spaghetti Style by using lots of conditional and unconditional GO TO statements to re-use every bit of code you'd written. It was the privative precursor of DRY [Don't Repeat Yourself].

In those days it kind of made sense - for a couple of reasons:

1. we didn't know any better

2. memory was tiny - we measured big machines in KiloBytes and wrote programs using 'overlays' [in case you don't know, an overlay is a chunk of executable code which 'overlays' another chunk in memory. We used to do that because (a) we didn't have enough memory to do the job and (b) there wasn't any automatic memory management]

The result of Spaghetti Style is that nobody could ever figure out how the programs worked and modifying them was next to impossible.

What brought this up?

I've been trying to 'learn Rails' - which is a pretty large Ruby application. Rails has close to 10,000 methods defined, about 1,500 classes, and God only knows what else. The Vocabulary is huge - and so it's a daunting task to say the least.

On top of the huge vocabulary, there it's DRY.

That seems to mean that every chunk of code which is used two or more times is ripped out and turned into a method. The result is that in order to do anything, somewhere near one and a half gazillion methods are called. In order to figure out how almost anything works you have to work through a call stack which is makes the old FORTRAN Spaghetti look like a 2nd grader's maze.

Why is this happening?

I'm guessing, but I think there are three factors driving this result:

1. DRY - This is a misunderstanding. DRY is a hip way of saying 'we want to maintain a single point of control so that we don't have to repeat bug fixes in the code'. DRY shouldn't be a mantra, because it's not a Principal of Good Programming. It's a method to implement an idea which is good programming.

Prior to formulating 'Single Point of Control' as DRY, programmers used some judgement in the size of the methods/functions they wrote. Typical size was 5 to 25 lines or so - enough to do useful work and small enough to be easily understood. Rubyists seem to be addicted to one-line methods which call other methods which call other methods which ... - but they don't Repeat!

2. Agile Development - This seems to go hand in hand with Test Driven Development. Both sound like good ideas, and they seem to work well for 'throw away' applications - which is what most web sites are. After crawling through the Rails Association Code, I've come to the conclusion something critical has been lost in the Agile-ness of the Development: There's no Design Phase.

Test Driven Development has proven you don't have to have an overall plan to build code which works. You can build any twisted mess and if you continuously test, it will pass the test.

But it won't make sense, won't be cohesive, will probably be inefficient, and it will be hell to modify or learn.

Everything has to be designed it is going to work well and be maintainable. That's true for Buildings, Cars, Motorcycles, Bridges, and, believe it or not, Software.

3. Ruby itself.

I love Ruby - almost - because it's a beautiful language and it makes it easy to otherwise hard things. The fact that Ruby is a scripting language means we get instantaneous feedback - and I love that too.

But both of those things - along with Ruby's openness which allows all kinds of self-modifying code [which seems to be confused with meta-programming] makes it easy to get the illusion of 'doing something' when the programmer is actually just churning different implementations with little impact on improving the goals of the application being developed.

Does that make sense? If not, read it again.

Activity != Progress

It takes discipline and experience to handle as powerful a tool as Ruby is without creating a mess.

Don't believe me?

Just take a look at the source for Mongrel and compare [or rather Contrast it] with a bunch of the Rails code. Zed Shaw writes awesome, easily understood code. Somebody in the Rails Core doesn't.

DSL Design - that's Domain Specific Language

A Domain Specific Language - for those who don't know - is a bunch of functions named so that writing function calls 'reads' like natural language. It seems to be sprouting wildly in Ruby - most probably because Ruby doesn't require parentheses around function arguments and Ruby programmers are kind of rebels anyway.

For example:
rabbit_jumps_in_the_hole :hole_size => 10

Even if you don't understand Ruby Syntax, you know what the function call does.

Think of it this way:

1. Function/methods are really 'verbs'.
   
2. Function Options are 'adverbs'
3. Object Identifiers are 'nouns'
4. Object Attributes are 'adjectives'
5. Class Names are 'class names' [gottcha!!!!]

I think we should think of learning one of these DSL things the same way we think about learning a new Language.

This can be either a good thing or very bad.

Size Matters

Which is easier to learn: A language with 10 verbs or one with 1,000?

Just for the heck of it, I recently tried to get a count of the 'verbs' in Rails 2.0.2. I ran 'egrep -r 'def [a-z]' on all lib directories and came up with:

• actionmailer/lib 694
  
• actionpack/lib 1393
  
• activerecord/lib 1134
  
• activeresource/lib 125
  
• activesupport/lib 577
  


• Total 3923

In contrast, the Merb Framework is much smaller:

• merb 713
• merb.rb 19
• tasks.rb 0


• Total 732

Of course, this isn't fair because Merb doesn't come with an ORM [Object Relational Mapper library (bunch of database access functions - for those really out of it)] [or Active Record Pattern Implementation, for those . . . - well, you know who you are], so you have to add that in.

But Merb gives you a choice of ActiveRecord - with it's 1,100 verbs; DataMapper - with about 500 methods; or Sequel - with about 600.

So learning Merb should be easier than Rails because the vocabulary is about 1/3 to 1/2 the size.

Synonyms are Bad

Which is easier to Learn: a language with one word for each concept or with two or more?

A programming language or environment isn't meant for composing poetry, novels, or movies. It's supposed to precisely express a procedure. Period. It should be concise. That makes it easier for Programmers to understand.

Case closed. DSL's should be concise, singular, and boring - but very, very accurate.

Corollary:

The Rails Inflector is a mistake in every possible way:

• It Expands rather than Tightens the vocabulary of the Rails DSL
• It injects confusion because Programmers now have to worry about singular and plural forms depending on context
• It doesn't work:
  • 'XMLClass'.underscore -> 'xml_class'
  • 'XMLClass'.underscore.camelize => 'XmlClass'
  • 'slave'.pluralize == 'slaves'
  • 'slave'.pluralize.singularize == 'slafe'
• It wastes lots of cycles doing it - machine, programmer, and learning

Distance is Good

I'm talking about the distance between words. For example frog is very close to frogs but far from toads. That makes it easier to tell a frog from a toad in print than in real life.

Good DSL design should not only use expressive and concise identifiers, but should also keep them far apart, especially when the referents do significantly different things.

Again, picking on Rails, the methods update_attribute(attribute) and update_attributes(attributes) are very close together, but one bypasses attribute Validation. Can you tell which one by the names? Don't you think it's important to know?

DSL is Not Documentation

Most DSL seem to grow more or less organically. The Ruby universe is filled a lot of apparently useful packages with virtually no documentation. Almost all of them have fairly reasonable API documentation - which allows 'one' to learn what each of the 'verbs' in the DSL do, but that's like learning to drive a car by reading Glossary of the Parts! It Just Don't Work.

It's hard as hell to learn a system without some sense of what the thing is supposed to be doing and how it's put together.

Don't belive me?

Figure out a Car from stuff like this:

Wheel - 1. circular object in contact with ground; 2. circular object interfacing driver to directional controls.
Nut - 1. Device for attaching wheel; 2. driver in other automobile; 3. nutritious snack
etc.

That's API doc and that's what you've got when all there is is the DSL.

'nuff for now

Is Java Bad for You?

How do Heavily Constrained programming environments - such as Java, C#, and friends - effect our thinking and creativity?

I think the goal of heavy constraints and requirements started out as a way to get better code by automatically checking as much stuff as possible mechanically. It all started with compile time type checking and has extended into things I don't want to know about.

Anyway, the result is that it's hard to write code with these tools. From what I hear - and I don't do Java, C#, and friends - the 'programmers' spend most of their time figuring out what API's and Design Patterns to use. I don't find that fun at all.

I usually spend most of my time trying to better understand the problems I'm trying to solve and creating software structures which mimic the nature of the problem. After I've coded and tested one of these structures, I call it a 'solution'.

In other words: I don't use Design Patterns and don't think in terms of API's.

Does that make me a dinosaur?

I don't think so.

I gravitate toward unrestricted programming environments. My first introduction was SCO Xenix around 1986 or 87. I became really excited as I realized how easy it was to do mundane tasks by stringing filters together in pipelines. I could accomplish more useful work in 1/10th the time [or so it seemed] than I could writing special programs to do the same thing.

In addition to being faster, it was more fun. I spent more and more of my energy solving problems rather than conforming to code writing rules.

The same thing happened when I discovered Python - and now to a similar extent Ruby. Scripting languages with good support for dynamic strings, arrays, hashes and objects are wonderful. They handle all the details of what I need - as a coder - to do the job.

How do I keep from hurting myself when the Programming Environment doesn't keep tabs on me?

Well, it's not a problem. I just test as I go and keep rewriting my code so it works, is more succinct, and tighter. [I guess you call that Refactoring now - we used to call it rewriting]

The facts are that Anyone can write bad code - and restrictive frameworks don't stop them. Anyone can also write good code - if they take the time to learn how and pay attention to what they are doing. And restrictive languages don't help with that either.

When I need a solution - I just think one up (or two or three) up and try it out. It's easy to write the code, change it, test it, and refine it. In a verbose, API laden environment like Java, that cycle isn't so easy - or at least is a heck of a lot more verbose.

I suspect that restrictive environment programmers get dulled down by the drudgery of just writing the code and learning all the API's. There is so little room for creativity that they lose it - creativity is something you have to practice and cultivate.

As a result, they get used to solving problems by applying packages and patterns. They don't really design: they apply old designs to new problems and hope that they work. [BTW, that's the reality behind Anti-Patterns]

So, I guess it makes sense: if everything you do is a cut-and-paste of something somebody else thought up, you would apply that to Design as well.

God, that's boring!

If I'm write, then Design Patterns are a result of Boring Programming Tools which create Bored, Dull Programmers and more Bad Code.

I don't think that's a good thing.

What do you think?

Design Patterns and the Fall of S/W

One nice thing about having a blog nobody reads is that I can say anything I want without worrying about it biting my butt.

I hate Design Patterns.

It's that simple.

I hate the guys who promote them.

Most of all, I hate the s/w industry - especially the programmers - for being duped by these guys.

And I'm qualified.

I received an engineering education and am a self taught computer 'something'. I'm not exactly a programmer, although I've written an awfully lot of code in a variety of environments - all the way down to machine code on microprocessors up to hokey database/user interface stuff. I've done device drivers and created my own little languages using lex & yacc and 'in the raw' in C, Python, and awk. And I've been doing this over 40 years - so I've earned the right to be a grouch.

The Design Pattern guys are the current generation of Yordon (sp?), Codd(sp?), and Bouch (sp?): Consultants who watch other people create software while telling them how to do it right. None of them actually do anything, but the sure create a lot of bad advice.

I remember buying three (3!) books by Peter Codd(sp?) on Object oriented programming and design only to find out that he admitted that he didn't really know anything about it. The idiot had gotten excited about the idea, so he and his group spent a year puttering with it and writing books - probably giving lectures and doing expensive consulting with BIG companies at the same time.

It takes years of experience doing something to understand the concepts. [things move quickly, but our brains take a while to catch up]. Hell, it takes 10 years plus to design, implement, and knock most of the bugs out of any programming language.

The Design Pattern guys are the absolute worst! The claim that they are defining these things to add clarity to the software process and they they write the vaguest crap imaginable. Don't believe me?

Grady Bouch: "In the world of software, a pattern is a tangible manifestation of an orginization's tribal memory." - CoreJ2EE Patterns (introduction) [clipped from PHP 5 Objects, Patterns, and Practice - by Matt Zandstra]


Merrian Webster Dictionary: "1. an ideal model. 2. something used as a model for making things. 3. Sample"


Which is clearer? If you like Bouch - then you need to join a consulting company and stop pretending to write code.

I'm starting to froth at the mouth, so it's time to simplify things. Here's a simple procedure to see for yourself.

1. go to a book store and pick up one of Martin Fowler's many books on patterns. WARNING: Do not buy the book.

2. Select a pattern at random and read the first paragraph describing it.

3. Answer yourself honestly: Do I know what this 'pattern' is well enough to describe it in a single sentence?

if No - read the rest of the description and try again

if still No - replace book on shelf.

if Yes, please send that sentence to me.

Thanks,
Mike