In 2008, I attended the Microsoft Professionals meeting (Atlanta) where
Shawn Wildermuth gave a presentation on Domain specific languages or DSL's. The concepts were an introduction into a Microsoft product development that used the code names "OSLO", "M", and "Quadrant" that was previewed in part at the PDC
2008 (Professional Developer Conference) which has mostly morphed into something else by PDC2009. Shawn produced three
articles for MSDN on the topics of "OSLO" and "M". In the 2008 presentation I attended, Shawn introduced the premise that we use DSL's already today; SQL, XAML, Make, Ant/Nant, MSBuild, regular-expressions, even musical notation are all examples of DSL's. From that talk, the idea of software development DSL's was planted - even though the technologies that had prompted his discussion have been morphed into new forms within Microsoft - the conclusion from this argument is that a set of text statements ( or specific symbols like music or regular expressions) can be declared and used to achieve a desired result - and within programming - results in how executing code processes those results. A concise grammar and syntax, that declares what you want to do but not how you do accomplish that - regular expression syntax is a great example. Software development using a DSL, is an idea that continues to cross my path. I thought I would try to capture some of the variations I have seen in how various developers and architects approach the concept of creating and using DSLs.
Martin Fowler wrote the book on
Domain Specific Languages where he describes the 3 types of DSLs.
There are some early Oslo software preview
videos out on the Microsoft site that show the language M being used to generate a DSL grammar within the tool called Quadrant. This is what Martin Fowler referred to as a "DSL workbench"- where the tool builds the DSL. Martin Fowler identified two other main categories of DSL; the External DSL, and the Internal, or Embedded DSL. External DSL's are often quite complex - and rarely come to general usage. Internal, or Embedded DSL's are abstractions over an API, or framework, within an application.
The idea of a DSL being used was briefly touched on in the Jan 2009 DotNet Rocks
episode with
Aslam Khan a software architect that lives in South Africa. The company he works with uses this to achieve
Domain Driven Design (DDD). "Write a story that describes a problem", and understand that little piece, to understand the domain. "Something that describes it completely - that does not bleed into the other", which build up a dependency graph, it exists in context to the other things - behind each story. A story about the problem - the domain. The more you reveal incorrect assumptions in the dependency graphs and correct them, you don't have to prune source code trees -so you get smaller source code trees, and designing API's around the stories - means changing the granularity of the stories - not modules. In the podcast (just after 38:41) they describe a misunderstood story - that would have produced an incorrect system design - but writing out a story, that the users can understand - allows for corrections can be made before hand. Understanding the user's vocabulary, and you can write it in those terms, then you have the basis to build on(this goes back to the Eric Evan's Domain-Driven-Design
"ubiquitous language"). It helps other people later on to understand the language of the application. Aslam Khan uses a Wiki to capture that language - and refers to the "Wiki as code" (about 44:44 in the podcast). He refers to it as "a DSL captured in business terms - that cannot be executed - but at that point you can start writing code". After listening to that podcast(in 2009), I went to Aslam's website - and discovered his
posting about using Cucumber and driving it down into
Rspec. That led to my reading of a post about Ruby and the latest on
Ruby testing - using cucumber. (There is a language called
Gherkin used with Cucumber to create a
Ruby testing framework). The dynamic nature of Ruby - and the concept of Metaprogramming - became connected to the idea of how one could produce a DSL. Aslam Khan stated it, on his blog, as "the Code is the data" behind a DSL, and he encouraged learning about AST's (Abstract Syntax Trees) which to me connected directly to .Net Expression Trees - the basis of Microsoft Dynamic Language Runtime(DLR) in .Net 4.0. DSL's can become Expression Trees which in turn can become running code in the DLR. At least, in theory, that would be one Dynamic-Language approach to implementing a DSL(There may be some limitations on how that can occur).
During 2010, I picked up a copy of Orin Eini's(a.k.a
Ayende Rahein) book called "
DSL's in Boo - Domain Specific Languages in .Net". (he did an
interview in March 2010 with DotNetRocks on this topic).Ayende is known in the .Net development world for RhinoMocks( a mocking framework), his work on NHibernate(his company makes a profiler for NHibernate and EntityFrameworks), and now his RavenDB - document database project. Somehow, he also managed to have written a book on DSL's - the guy is prolific at producing content - whether you see his coding output(RhinoMocks,RavenDB), his blog, and now his book. In his book, he describes how he builds his own DSL's using the .Net language
Boo as the foundation, as it gives him a parser,lexer, and great extensibility points, that compile into .Net code - that is not based on the DLR. He shows how the Boo language allows for defining your own expressions that can then be compiled - statically. Boo is based on the released
ECMA standard for the CLI. Interestingly -both the Boo Language, and the ECMA CLI standard, had recent updates(at this writing) Boo 0.9.4 was updated Jan 2011, and the ECMA Standard December 2010. Boo has some capacities that make the language very powerful. It can be an object oriented language, and it can do functional composition (where functions are first class citizens like those in F#), it has '
Syntactic Macros', and it has an extensible compilation pipeline - which is what the creator, Rodrigo B. de Oliveira, wanted to have in the first place, and it can run on multiple platforms (Windows CLR and Linux Mono). Boo has its roots based on Python Syntax and looks like it has some great potential. To use Boo that as the basis for creating a DSL seems like a very interesting choice - here it would be a statically typed implementation of a DSL. This could be a way of generating a concise Internal, or External, DSL that works for the users and programmers alike within a given domain. Ayende's book gets into the details of using boo for creating your own DSL.
I found a few entries that related how Powershell could be used at the basis of a DSL. There is a blog
post from Keith Hill on how Powershell's syntax flexibility allows you to create a internal DSL. Doug Finke, a Powershell MVP, posted something that went back to
the Oslo starting point from a Powershell point of view. Doug Finke shows implementing an OSLO grammar syntax using Powershell, based on the concepts originally introduced in that early video of the M Language example. So from Dynamic Languages or Scripting tools we can find approaches to creating DSL's today. Doug Finke connected the concepts of DSL, DRL, and Boo; all the ideas together in his post
"Creating a unified programming model on top of .NET" He cites "Trends and Directions in programming languages" from Anders Hejlsberg. DSL's could certainly be part of that mix in creating solutions.
Jeffrey Snover, the architect of Powershell, had a discussion in 2008 on DSL's with Martin Fowler and Neal Ford from Thoughtworks. (
recording on Channel9). In this talk they made the point that a DSL can be an External DSL, an Internal DSL or a 'workbench' with tools wrapped around it. Martin Fowler made the point that not many successful DSL's have been done starting from the business side and working toward the API - but that is where the concept is the most powerful, they made the point that it is usually an abstraction of an existing API that simplifies the underlying API - and that DSL can be replaced with the word 'Framework' and mean the same thing. Jeffrey Snover also
posted that Powershell could be used to create DSL's by limiting the syntax. The key piece he refers to is the
data segment SupportedCommands - which he alludes to as a way to limit a DSL done in Powershell.
Karl Prosser referred to the Powerboots library for Powershell as a DSL in a
blog post. It is a Powershell set of commands that are specific to using WPF from within Powershell - which stays with the concept that it is limited to that specific domain.
Sometimes the use of a Fluent API - can be used to write code that is very readable and expressive - such that it becomes an internal DSL within the general purpose language(Steven Hoisee
blog has some coverage on this). I came accross a paper on how JMock was developed and how they worked toward incorporating a Fluent API as part of their own
Embeded DSL within the framework.
Martin Fowler, who took the time to write the book on DSL, has a post on
BusinessReadableDSLs. Where the idea is different than business-writeable DSLs. That was the impression I got from Aslam Khan's use of Cucumber - that the business could read the DSL and confirm if it was correct or not.
Martin Fowler invokes the comparison to COBOL - which was supposed to be an english way of telling the computer what to do - and begs the question - how is this one different? My impression is that the a correct implementation of a DSL is declarative - it describes what you want to do, now how you do it - and having programmed in COBOL - it is an imperative approach, not a declarative one.
Developing a DSL, based on the Jeffrey Snover video, leads me to see that in a DSL you have vocabulary and you have grammar that must be decided on - and the idea of keeping it small seems to be important. Just enough to keep and simple - and no simpler.
Jeremy Miller(Creator of the IOC container - StructureMap, an MVC framework called FubuMVC, and a testing tool called StoryTeller) did a
presentation on creating Internal DSL's at Oredev in Oslo - Nov 2010.
He contributed to the Fluent NHibernate project, and incorporated the use of an Internal DSL in these projects - and in the presentation he goes over some of the difficulties of using and developing an Internal DSL using the C# 3.0 language features. He touches on the separation of the Runtime and Configuration model in building a DSL, and that testing an internal DSL for configuration is very difficult - you can test the runtime model more easily, and then make sure your configuration model works correctly. (the video is a bit grainy - and a few of the questions, that came from the audience, are not heard - but the concepts that were used in creating an internal DSL are shown pretty well.)
From
this set of touch points - on the various flavors of DSL's, it is
something that can be done in a variety of ways. The simplest form is the
internal or embedded, perhaps as a Fluent API approach. This simple form of a DSL can be
for a small team that produces repetitive code, or business rules that
are needed in in either a static or dynamic fashion - but still being an
internal abstraction - within a small domain. Then there is an external
DSL - and that would need to have a the same considerations as a full
blown computer language - as the publishing of an External DSL approach tends to get the syntax set in
stone, unless you can absorb breaking changes later on.
I find
this an interesting research topic, that may prove to be a practical
solution to some future area of work. Along these lines - I recently purchased a book that looks promising;
Terrence Parr's book "
Language Implementation Patterns - Create your own Domain Specific and General Programming Languages". Terrence Parr developed
ANTLR and that tool is used in the Boo compiler. I will have more reading and learning to do in this area - and, perhaps at some point, the payoff will be when I can see an area where a DSL can be used to invoke parts of a framework. Ideally this would be in terms of the users, a readable Business DSL, or an API syntax that developers for an application can use that brings a higher level of abstraction and makes code more readable - and perhaps more declarative. The Jeremy Miller video shows that is not a simple task, however, he shows that it can be used to develop some powerful solutions too, and that I find to be valuable.