"I have a mind like a steel... uh... thingy." Patrick Logan's weblog.

Search This Blog

Friday, January 25, 2008

Dynamic Languages: Should the Tools Suck?

A comment to my previous "oh my god" post just came through. It is the typical "static typing is good because it enables good tool support like refactoring and finding methods". Mmmm.

First let me agree that tool support for Python and Ruby truly suck. They do. I still love the languages and would use them in a heartbeat before any static language. But the tool support sucks. But not because the languages are "dynamic".

To make my point a bit more concrete, here is a snapshot of a Smalltalk image I just created...

The aqua colored window is a "method finder". I entered the text "#(1, 2, 3, 4). 2. true", which is an array of numbers, the number 2, and the boolean true.

The method finder then found five methods that when applied to the array and the number, return true. It found those for me by considering potential candidate messages and executing the method code for me. The matches are listed in that window's pane just below the text I entered.

I then selected from that list the method "windowReqNewLabel:" and the method finder brought up the green window which is a system browser. It set the browser to display the code for that selected method. (By the way the browser has a complete kick-ass refactoring engine available. But that's for another post maybe.)

In the system browser for that method I clicked on "senders". This brought up a menu consisting of that method's selector as well as all the other message selectors that are used in that method. I chose that method's selector because I hypothetically want to know all the places this application sends that message. (Otherwise I could find all the senders of messages that this method also sends.)

Doing so brought up the front, light blue, window. This is the "Senders of windowReqNewLabel:" window. This window lists the four places that message is sent in this application. You can see I have selected the third occurrence, and the code for that sender is displayed in the bottom pane of that window.

As you can see, tools for dynamic languages only suck when they have not been implemented. On the other hand everything you see in this screenshot has been in Smalltalk for decades.

Decades of better tools than you can find for Java, C, Scala, you name your favorite statically typed language.

I am sick and tired of people whining about how dynamic languages cannot support useful tools. They can, and they are *better* than yours. This example is the tip of an iceberg I just captured in a matter of seconds. That first method finder "search by example" completes in a split second.

Can your static tools do that? Maybe, which is why I won't go around claiming that they can't, because I am ignorant of that, just as you are ignorant of what dynamic language tools can do.

Begin Update

An anonymous commentator points out that one of us is being obtuse. How can a dynamic language possibly refactor the name of a message when there are three implementations of that method but only one of them should be renamed.

I will quote from the documentation for the very first usable refactoring tool ever. (This very first one ever was written in Smalltalk and is the one that still is used by Smalltalkers.) The quote...

One of the key features of Smalltalk that allows us to perform the sorts of analyses and source code transformations necessary for refactoring is that it has reflective facilities, that is, the ability for the language to examine and modify its own structures. For instance, the Refactoring Browser examines the context stacks of executing programs to determine runtime properties such as callers of a particular method or cardinality relationships between components and aggregates. Refactoring can be performed in the absence of reflective facilities, but then requires a separate, metalanguage that can be used to manipulate the original program. Having a single language simplifies the process...

Some preconditions of refactorings are not simple to compute from the static program text. With Smalltalk's dynamic typing, determining the type of a particular variable can be extremely difficult and time consuming. Moreover, there are analyses that defy static analysis in any language, such as cardinality relationships between objects.

Since, performing these types of analysis statically is difficult or impossible, the Refactoring Browser computes some of the preconditions by performing the analysis dynamically rather than statically. By observing the actual values that occur, the Refactoring Browser can perform correct refactorings. As an example, consider the rename method refactoring. To correctly rename a method, all calls to that method must be renamed. This is difficult in an environment that uses polymorphism to the extent that Smalltalk does. Smalltalk also allows dynamically created messages to be sent via the perform: message. If an application uses this approach, any automatic renaming process has the potential of failure. Under these conditions, guaranteeing the safety of a rename is impossible.

The Refactoring Browser uses method wrappers to collect runtime information. These wrappers are activated when the wrapped method is called and when it returns. The wrapper can execute an arbitrary block of Smalltalk code. To perform the rename method refactoring dynamically, the Refactoring Browser renames the initial method and then puts a method wrapper on the original method. As the program runs, the wrapper detects sites that call the original method. Whenever a call to the old method is detected, the method wrapper suspends execution of the program, goes up the call stack to the sender and changes the source code to refer to the new, renamed method. Therefore, as the program is exercised, it converges towards a correctly refactored program.

The major drawback to this type of refactoring is that the refactoring is only as good as your test suite.

Except that good test suites are never a major drawback. They are actually the only thing that can save your application over time, dynamic or static.

And so you see good refactoring tools, good (dynamic) languages, and other good things go hand-in-hand with good tests and good test coverage. Working incrementally helps you make only small changes at a time, testing well points out just the few things you just broke, good coverage makes sure you find everything, and refactoring tools introspecting the code that your good tests cover help automate those changes and bring them up to a higher level of expression for you.

No single tool or practice is in itself sufficient. They work together for the good of the whole. Should I assume your refactoring tool for your static language works so well as this? I believe method wrappers, etc. take a fair bit more machinery and end up not so expressive in static languages. At best they require more code than with dynamic languages.

You do not have to compromise your very simple language just so your tool can work. Dynamic languages and good tools work for you. Static declarations -- they appear to work for you until they don't. They're not a compromise, they are a bloody surrender, somewhat quoting Guy Steele from some long ago Lisp memo I don't fully recall.

Oh my god here I go again. I know you love your static languages and type theories. Good on you. I don't.

End Update

Thursday, January 24, 2008

Experiment, Programmers and Engineers

Cole Porter wrote a song all programmers should embrace...

Make it your motto day and night.
And it will lead you to the light.
The apple on the top of the tree
Is never too high to achieve,
So take an example from Eve,
Be curious,
Though interfering friends may frown.
Get furious
At each attempt to hold you down.
If this advice you always employ
The future can offer you infinite joy
And merriment,
And you'll see
One thing we programmers don't do enough, or at least don't discuss enough, is experimentation. Every application we write is somewhat new, perhaps completely new. Something about the technology is usually new as well. Often something about the team itself is new.

Every new venture or project can be aided by thinking about your assumptions and your hypotheses about how it might succeed and what might interfere. Each of these, in various ways, can be handled by treating it as an "experiment" of sorts -- how do you propose to address these hypotheses? What if something turns out to be other than what you expected?

Of course your sponsors may be annoyed at best if they believe all you are doing is "playing". These experiments have to be time-boxed, well chosen to address risk and value to the sponsor, and well structured to make sense of the results.

I recently responded to just such a situation someone is grappling with on the REST yahoo group... I quote myself...

Re: REST and ESB

> present one of three alternatives:
> 1) REST + centralized ESB (e.g. Mule)
> 2) REST + centralized Message Queue / Mediator (e.g. ActiveMQ)
> 3) REST + decentralized, "powerful" end-points

A big advantage of the REST/HTTP approach is that many kinds of
systems can participate. Reducing to one of these three appears to be
an artificial constraint. Why make that decision now, up-front?

> (5) have short "time to deployment"
> (4) Performance is a issue
> (4) is a point-to-point service
> (3) is data-centric service
> (3) need multiple adapters
> (3) can have high concurrency of users
> (2) have long running processes
> (2) needs high level of Compensation
> (2) needs integration with legacy data
> (2) have complex business logic
> (1) needs low latency

You may want to choose one or two of your services, one or more of
your "REST + X" options, and two or three of your capabilities listed
here, and conduct time-boxed experiments, say over the next six weeks,
to investigate the most important or promising combinations of these.

These time-boxed experiments will provide a much greater wealth of
real experience than you would obtain trying to make a decision
through discussions on this list alone. Even though there are
incredibly bright people on this list, your own experience over six
weeks will tell you more than you could possibly imagine by sitting
and thinking alone.
I've been a programmer for a while. This approach has served me well going back into the early 1980s, when there were not nearly so many books, blogs, and frameworks to rely on as today. I often long for those days just for that reason.

Around 1985 when I was a programmer at Data General working on internal tools for designing electronics, another group was developing this new thing (at least to us) -- a "relational database" -- could we use it for electronic circuit data? We conducted an experiment. Of course we found the implementation was fairly slow, even for those days. We also found our first occurrence of the "O/R mapping" problem.

In 1995 I wrote in the usenet comp.object list...

st...@projtech.com wrote:
>At the risk of agreeing with RCM, I say integrate early and
>often--just make sure you do so from the bottom layer up.

I look at risk management in two dimensions:

1. The risk of providing the correct behavior.
2. The risk of any technology used to implement those behaviors.


A. Prioritize the known required behaviors.

B. Prioritize the proposed technologies according to which behaviors
   they will be applied and the certainty of the use of the technology.

C. Prioritize resources (people, tools, money, etc.) toward both risks
   as best meets the overall risk:

    i. Prioritize toward verticle prototypes of the most important
       behaviors. Make them as independent of any specific technologies
       as possible.

   ii. Prioritize toward horizontal prototypes of the most critical
       technologies: i.e. the ones used in the most important behaviors
       that are also the least understood.

  iii. As horizontal prototypes prove the technologies, plug them into
       the verticle prototypes to prove their combination.
Therefore, experiment. Wisely.

Wednesday, January 23, 2008

Math and Software Development

Via Reg Braithwaite, Ravi Mohan questions the arrogance of software developers calling themselves "engineers". Ravi's argument is they do not do math...

If you don’t use mathematics in your day to day work, you aren’t an engineer...

The title “Software Engineer” is (most of the time) a particularly deceptive one.

Certainly "logic" is a form of math and is a central aspect of software development.



Cost / benefit trade-off analysis and assessment, i.e. kinds of "forecasting", "budgeting", "accounting".

These are all mathematical, and significant aspects of software development.

Not that I care. I'm "just a programmer".

Tuesday, January 22, 2008

Oh my god here I go again

I quit participating in "static language" vs. "dynamic language" debates for the most part a few years ago. A few pieces have been getting my goat lately, though. There was that piece on "gradual typing" (I won't find it for you).

Now this linked from Lambda the Ultimate...

When scripts in untyped languages grow into large programs, maintaining them becomes difficult. A lack of types in typical scripting languages means that programmers must (re)discover critical pieces of design information every time they wish to change a program. This analysis step both slows down the maintenance process and may even introduce mistakes due to the violation of undiscovered invariants. This paper presents Typed Scheme...
The problem is I've seen large programs in all kinds of languages become difficult. Two things can preserve large programs... many good tests, and familiarity. I've seen that work.

Maybe good type systems can help maintain large programs. I don't know, since I've never participated in developing a large program with a *good* type system.

I do know that a good type system is *not necessary* for maintaining large programs. I suspect it would not help as much as many good tests and familiarity.

I await the comments showing up in my inbox. No promises to publish them.

Blog Archive

About Me

Portland, Oregon, United States
I'm usually writing from my favorite location on the planet, the pacific northwest of the u.s. I write for myself only and unless otherwise specified my posts here should not be taken as representing an official position of my employer. Contact me at my gee mail account, username patrickdlogan.