"I have a mind like a steel... uh... thingy." Patrick Logan's weblog.

Search This Blog

Monday, February 21, 2005

I'm a Broken Record

Update: Luke Gorrie comments...

Can you point me towards any examples that have gone "all the way" with a production language implementation based on translation to Scheme? The approach looks really appealing but I'm becoming skeptical of it.
I don't know. Kelsey went far enough to get a PhD. Feeley (with Erlang to Scheme) went far enough to "prove" the concept. Neither could be considered close to production worthy. The closest thing I can cite would be the compilers for Fortran, Ada, Pascal, C, and Prolog for the Lisp Machine. I'm not sure if they compiled to Lisp or to the Lisp Machine assembly language (which could be seen as a kind of Lisp as much or more than a typical "machine" language.)

I guess the point is not that this is proven, but that today, in 2005, why has it not been explored? People are still beginning VM projects left and right in the 1970's approach. Where does that leave us? Why not try something else, at a higher level, to *see* if it works?

End Update

As per James Robertson, Sriram Krishnan and some folks are building a new VM. It sounds ambitious and two minutes of looking around did not clarify for me why they want to build something from scratch. So rather than suggest they shouldn't, I'll try to be more positive and suggest some reading material for someone interested in these things, then speculate on what I'd do, then admit I have no intention of doing such a thing myself but you might listen to me anyway. Of course I think these ideas are worth considering, unless you *really* want to work on the lower level bits.

Right now, we're looking to copy from existing VMs as much as we can since we really don't have time to design a lot of stuff. This doesn't mean that we don't want to innovate - just that we don't want to reinvent every wheel. I'm personally looking forward to figuring out how to implement continuations inside the VM itself.
You could do worse than to look at the source code for Gambit Scheme and read the 20+ years of technical reports on compiling Scheme. By Sriram's second post, it looks like he is on this trail...
Everything worth inventing has been invented by the Lisp guys 2 decades ago. People like Guy Steele have done some really cool work
I guess if I wanted to work on a language project it would go more like this... start with the implementation of Gambit Scheme because you probably won't do better with less than 2-5 years of research and development. That's probably optimistic, based on the fact that Gambit has a decade or so of concentrated effort from a very bright person, and there's decades of work and discussions that went into Marc Feeley's own thought processes. You'd better talk with those guys because you'll be recreating their mistakes at best.

Then I'd build the translators for the languages I want to run to compile from source to Scheme. Along the way I'd do something interesting to demonstrate the payoff of using what's been done so well already: e.g. I might write translators for partial definitions of a couple languages , perhaps subsets of Smalltalk, Python, and Ruby, and then combine Scheme, Smalltalk, Python, and Ruby in a simple continuation-based modal web server based on Chris Double's tutorial.

The would demonstrate a better way of programming web applications on a well-designed kernel, supporting multiple popular languages. It would also demonstrate the value of continutations in languages that do not have them, e.g. Python (other than Stackless, I think). Pehaps others would want to add to the language fidelity. Another benefit of this approach then is that those folks don't have to understand my VM design, or track its quality, or track its design changes. All they need is to understand Scheme and write a translator from their language to Scheme, or enhance one of the initial prototypes.

GNU Smalltalk is fairly simple... a simple initial translator to Scheme does not seem too hard. I'm not sure what starting points to use for Python or Ruby. But I'm not proposing to do this... only suggest it to someone. I think it would be useful but it's not high on my personal list.

Just some thoughts if there's a Masters student looking for an interesting project.


Swaroop C H said...

I guess you missed the part where he says

"This is *very* similar to what the Parrot (www.parrotcode.org) guys are doing and in fact, I've been lurking on their lists for a few weeks now. If I had had the choice, I would have built upon their code- but my problem is that my college doesn't like people modifying existing code. So working on Parrot or Rotor was out of the question.Bummer again."

I have had my own share of problems with clueless college lecturers, so I know how he feels. You wouldn't believe the hard time I had to convince my own lecturer who insisted that MySQL is 'a small utility in Linux, not a real database'!!

- Swaroop

Anonymous said...

"You wouldn't believe the hard time I had to convince my own lecturer who insisted that MySQL is 'a small utility in Linux, not a real database'!!"

Well, perhaps opinions may vary about whether MySQL is a "real database" or not:


Anonymous said...

Can you point me towards any examples that have gone "all the way" with a production language implementation based on translation to Scheme? The approach looks really appealing but I'm becoming skeptical of it.

For example ETOS (the Erlang->GambitScheme compiler) is nicely written but extremely far from being a practical Erlang compiler. I'm guessing Kelsey's is similar: faster on benchmarks that what they compile the operating system with but probably not complete enough to actually write an operating system with.

I'm very curious to know whether the ETOS-style approach scales to practical compilers that people can really use.

Sriram said...

I've pretty much 70% of the stuff from readscheme.org already :-)

As for the translators, I'm sure they're a viable road. But one of the goals of this project was to learn about internals of a VM as well - so think of us as trying to attempt the Minix of the VM world :-)

Ian Bicking said...

You mean, tried it like Guile? That was the whole premise of Guile, that you could translate other languages into Scheme. They never even got as far as translating elisp (which should have been the easy part) so that they could move Emacs to Scheme.

That might not really be fair -- honestly, I don't know if anyone cared enough to try, Guile was born of a flame and never had a real champion. I tried translating Tcl, which basically "worked", but I don't know if you could call it a success, as it wasn't really *useful*. An accurate translation just isn't very compelling. Someone also tried translating Python to PLT Scheme, and they ended up giving up and just embedding Python in the Scheme interpreter.

And that doesn't even deal with the object mismatch issues. You don't actually have to resolve those issues if you just want to take advantage of the performance of the underlying implementation, but if you want to leverage it for interlanguage communication, translating it all to Scheme doesn't actually accomplish a whole lot.

The underlying implementation also really has to be designed with other languages in mind. Guile was crappy to debug Scheme in, but it offered absolutely nothing for any other language built on top of it. Can you annotate your Scheme source with information that allows you to reconstruct the native language environment when there's an error? Many languages deal poorly with incorrect programs, but Scheme implementations tend to be particularly bad, as is any implementation that comes out of an academic background (rather than being designed by a programmer that actually wants to *do* things with their implementation).

From that perspective, I know PLT Schema / mzScheme offers a lot. But there might be real compromises they made in the process, and I don't know if Gambit offers anything. I'm highly sceptical of any academic projects, as I they tend to be mismotivated.

tweedledeetweedledum said...

This blog is awesome! If you get a chance you may want to visit this inventory software site, it's pretty awesome too!

Anonymous said...


cash at home said...

Hello There, My name is Donald you have a great blog here! I'm definitely going to bookmark you! I have a 105 from home make money ways
site. It pretty much covers 105 from home make money ways
related stuff.

Come and check it out if you get time :-)

Anonymous said...

Hey, you have a great blog here! I'm definitely going to bookmark you!

TS said...

Nice Blog!!!   I thought I'd tell you about a site that will let give you places where
you can make extra cash! I made over $800 last month. Not bad for not doing much. Just put in your
zip code and up will pop up a list of places that are available. I live in a small area and found quite

Blog Archive

About Me

Portland, Oregon, United States
I'm usually writing from my favorite location on the planet, the pacific northwest of the u.s. I write for myself only and unless otherwise specified my posts here should not be taken as representing an official position of my employer. Contact me at my gee mail account, username patrickdlogan.