Reverse engineering the DLTK Python plugin example

Now that you've got Eclipse [link to installing eclipse be], I bet you wanna start kicking some ass, right?

Maybe you've been kicking some ass already, and just wish that you could kick a little more. Specifically, "if that thing would only do what you want it to do".

To that end, we're going to focus on the tools you use to do what you do, and in this case, Eclipse plugins. Yay!

Anyways, I'm enamored of some of the features in some of the editor plugins. Really swell stuff.

I use CFEclipse [link] for editing coldfusion files. It's pretty swell, but I wanted to make it sweller. Better than bee's knees. Or at least have occurrence marking capabilities.

So I took a stab at it, having taken a stab at a couple other plugins, and, as with the other plugins, saw ways to do things different. Not exactly originality, as I got the ideas from other places with similar constraints (like when trying to add key-binding to Subclipse, and looking to the CVS plugin for implementation ideas).

Anyways, a lot of this stuff is about abstraction, oddly enough. Instead of coding 5 specific bits of code, write one general one-- or, true-er; one fragile implementation of something core can become a lot of "work around" coding, sorta.

Anyhow, the less code the better, and CFE looked like there was a good bit of code we could get rid of.

I'm leaning towards getting rid of a lot, like a lot a lot, and relying on other plugins for some stuff we currently handle ourselves, but that's another story.

After looking around, I was liking DLTK for a base IDE type deal, and Mark Drew provided a little starting point-- He'd walked through part of the DLTK IDE how-to walk-thru, and put it in the source repository for people to play with.

Since the code for the example python IDE was available (and even though the real Python DLTK plugin is out now, and I could have used it, or a language really really close to CF for a basis, perhaps) I downloaded it, and sorta strolled around.

Mark Mandel had done some work on an ANTLR coldfusion parser, and the Python plugin used a ANTLR parser, so "I was like, hmmm... I wonder if I can just plug this other parser in?"

And because Eclipse JDT is so awesome, I was able to find out pretty quick. Ha ha! (4hrs+ quick)

What I did was, look for patterns, basically. (Good reason to use something nice and specific when naming stuff-- "PythonExampleParser" was pretty easy to refactor to "CFMLParser", roughly.)

Eclipse would ask me "rename all this crap, dude?" and since it was just the example, I was like, "sure".

Before I did that tho, I'd consolidated the 4 example plugins into two plugins-- a "core" and a "ui". I did that by just dragging the packages from one plugin to the other, and copy/paste the various bits of XML from each plugin into the respective plugin.

The idea was to keep the UI (eclipse presentation type stuff-- font colors or whatnot) apart from the core (I've since gone further, and created a plugin for the CF parser, but for dicking around, I could have just copied them all into one plugin, really-- compartmentalizing stuff was kinda an exercise, sorta.).

Annywho, after I consolidated the 4 plugins to 2, I then refactored all the "PythonExample"s to "CFML"s, which was pretty easy since JDT is so "aware".

Then I fired the plugin up in my usual manner: I'm editing the plugin.xml file for the plugin (any one-- they all fire up the same runtime application) with the Plugin Manifest Editor (right-click, "open with", if you got it but it's not a default-- probably comes with PDE, which you'd have to have by now to do any of this anyways), and on the "overview" screen, there's a "Testing" section, under which there's a nice little "Launch an Eclipse application in Debug mode".

I click on that link, and Eclipse fires up a test instance using all the open plugin projects. That's totally configurable, BTW-- under "Run" (top of screen) there's "Run Configurations" where you can configure the Eclipse Application instance that your code runs in-- use this to disable the loading of any plugins that you don't want running in that test/debug instance.

As the Python parser was still parsing, I was like "cool, now to try shoving the CF parser in"-- I was actually aiming for a quick switch, but it wasn't that pretty. Heh.

So I dropped in the CFMLParser, basically. Of course it didn't have the same kind of AST as the python parser (abstract syntax tree, apparently used to map parsed things to other things), so simply using the python stuff was out-- and the python stuff had all the DLTK stuff, for the outline view and whatnot. Sorta. Hell, look at the code [link], you'll see what I mean. Most of the DLTK stuff's in the parser, actually. Well, grammer file, I guess, technically. It's that grammer file that builds the parser and lexer.

Well now, this is getting long...

Luckily, I think that's pretty much it. This is piss poor, as far as a learning aid, I reckon, but maybe with a little editing or something... eh.

So, DLTK is an dynamic language IDE API, basically. It's got some generic-ish stuff you can use to do things like populate the outline view, create custom searches, and apparently mylyn integration (although I haven't explored that aspect yet), etc..

That generic-ish stuff was in the python parser itself (at least some of it), so to get the CF parser to talk to DLTK I think that's what we've got to do-- define functions components and whatnot, map them to the DLTK equivalents.

It really doesn't matter what happens now... I'm just amazed that things could work that simply. By "work", I mean that when I fired up that test instance, and opened a .cfm file with the "CFML Example Editor" *grin*, I saw output from the parser in the console of my main eclipse instance. "entering cfif, leaving cfif". Did a little more refactoring, put in the more abstract cf parser, that uses dictionary files... "isColdfusion: true" started coming out...

Eh. Maybe I should just do a screencast of the basic deal.

And I mean "doesn't matter" because in a few days, I've sorta kinda picked up just an amazing chunk of knowledge. When I first looked at that CFML.g file, or the python_v3.g parser grammer file... wow. I was like? WTF? Ya know?

Then it turns out it's like sorta, maybe, regular expression chaining or something. (Rough analogy, for sure). There's nice GUIs that make it obvious why X does Y or whatever.

Now I'm off onto another entry tho. Guess I'll cut this out, right about...