Groovy DSL Workshop

Getting Started

In this workshop we will learn how to create domain specific languages (DSL) in Groovy. We will create builder-pattern style DSL with nested blocks which perfectly suits the Groovy language. All the code exercises are written in Java so no previous knowledge of Groovy is required.

The builder pattern is used to create complex objects with constituent parts that must be created in the same order or using a specific algorithm. An external class controls the construction algorithm. — The Gang of Four

We will use yUML diagrams (data class diagrams) as a reference domain model.

yUML Diagram’s Model

This workshop is based on Groovy DSL Builders series which can be used as reference documentation:

Software Requirements

To get started you need to have installed following:

Java Development Kit 8

Project is built using Gradle build tool but it’s using Gradle Wrapper so you are not required to install it manually nor you don’t have to be afraid of version clash.

You can use IDE of your choice but IntelliJ IDEA provides so far the best Gradle and Groovy integration even in the free Community version. Import the project as Gradle project get the best developer experience.

Project Archive

To get started download exercise project archive file and unzip it. Run following command to get all dependencies downloaded:

./gradlew dependencies

You can execute tests by running following command:

./gradlew test

There are several sub-projects called step-xx in the archive where xx is number of the exercise. Each of the exercises is covered by a single test which will pass only if you have successfully finished all the tasks. Each sub-project contains ideal solution of the previous exercise so in case of you get stuck or you want to skip an exercise for any reason you get everything you need to keep coding. Of course you can continue working on your own code. In that case, you can simply copy the test from the next step into your current sub-project.

Introduction

In this workshop, we are going to create DSL for yUML.me diagrams. yUML helps to create simple UML diagram online.

YUML Diagram’s Syntax Example

[note: You can stick notes on diagrams too!{bg:cornsilk}]
[Customer]<>1-orders 0..*>[Order]
[Order]++*-*>[LineItem]
[Order]-1>[DeliveryMethod]
[Order]*-*>[Product]
[Category]<->[Product]
[DeliveryMethod]^[National]
[DeliveryMethod]^[International]

Its simplified data model is shown in the following diagram:

YUML Diagram’s Model

yUML Diagram’s Model

Although we are implementing Groovy DSL the DSL code will be written in Java. This approach will help us to reach greater audience and help us avoid some common pitfalls more easily. See The Resignation: Rewriting the Groovy DSL builder into Java for further reference on writing Groovy DSLs in Java.

First, we should get familiar with the data model in cz.orany.yuml.model package.

Diagram represents the diagram which can contain notes, types and their relationships.

Diagram.java

public class Diagram {

    private Collection<Note> notes = new LinkedHashSet<>();
    private Collection<Type> types = new LinkedHashSet<>();
    private Collection<Relationship> relationships = new LinkedHashSet<>();

    // boilerplate
}

Note is a simple box placed next to the diagram which can have it’s text and color.

Note.java

public class Note {

    private String text;
    private String color;

    // boilerplate
}

Relationship represents relationship between two types. Each relationship has it’s type, source and destination. Relationships can be bidirectional. You can specify cardinality and title for the source and destination

Relationship.java

public class Relationship {

    private RelationshipType type = RelationshipType.ASSOCIATION;
    private boolean bidirectional;
    private Type source;
    private String sourceCardinality;
    private String sourceTitle;
    private Type destination;
    private String destinationCardinality;
    private String destinationTitle;

    // boilerplate
}

RelationshipType.java

package cz.orany.yuml.model;

public enum RelationshipType {
    ASSOCIATION, AGGREGATION, COMPOSITION, INHERITANCE;
}

Type is a data class determined by it’s name.

Type.java

public class Type {

    private String name;

    // boilerplate
}

Following example shows how verbose is creating the diagram without any DSL improvements:

Creating Diagram using Constructors

Diagram diagram =  new Diagram()

diagram.notes.add(new Note(
    text: 'You can stick notes on diagrams too!',
    color: 'skyblue'
))

Type customer = new Type(name: 'Customer')
Type order = new Type(name: 'Order')
Type lineItem = new Type(name: 'LineItem')
Type deliveryMethod = new Type(name: 'DeliveryMethod')
Type product = new Type(name: 'Product')
Type category = new Type(name: 'Category')
Type nationalDeliveryMethod = new Type(name: 'National')
Type internationalDeliveryMethod = new Type(name: 'International')

diagram.types.add(customer)
diagram.types.add(order)
diagram.types.add(lineItem)
diagram.types.add(deliveryMethod)
diagram.types.add(product)
diagram.types.add(category)
diagram.types.add(nationalDeliveryMethod)
diagram.types.add(internationalDeliveryMethod)

diagram.relationships.add(new Relationship(
    source: customer,
    sourceCardinality: '1',
    destinationTitle: 'orders',
    destination: order,
    destinationCardinality: '0..*',
    type: RelationshipType.AGGREGATION
))

diagram.relationships.add(new Relationship(
    source: order,
    sourceCardinality: '*',
    destination: lineItem,
    destinationCardinality: '*',
    type: RelationshipType.COMPOSITION
))

diagram.relationships.add(new Relationship(
    source: order,
    destination: deliveryMethod,
    destinationCardinality: '1'
))

diagram.relationships.add(new Relationship(
    source: order,
    sourceCardinality: '*',
    destination: product,
    destinationCardinality: '*'
))

diagram.relationships.add(new Relationship(
    source: category,
    destination: product,
    bidirectional: true
))

diagram.relationships.add(new Relationship(
    source: nationalDeliveryMethod,
    destination: deliveryMethod,
    type: RelationshipType.INHERITANCE
))

diagram.relationships.add(new Relationship(
    source: internationalDeliveryMethod,
    destination: deliveryMethod,
    type: RelationshipType.INHERITANCE
))

The initial version is very verbose and there is definitely lot of place for the improvement in future steps.

1. Self-managed Content

The most of the verbosity of the initial example originates from manual handling of diagram’s content - relationships, notes and types. Let’s make Diagram class responsible for managing these items. Let’s create methods note, type, relationship which will internally create or reuse existing parts of the diagram:

Diagram Responsible for Content Management

Diagram diagram =  new Diagram().with {
    note('You can stick notes on diagrams too!','skyblue')

    relationship(type('Customer'), RelationshipType.AGGREGATION, type('Order')).with {
        sourceCardinality = '1'
        destinationTitle = 'orders'
        destinationCardinality = '0..*'
    }

    relationship(type('Order'), RelationshipType.COMPOSITION, type('LineItem')).with {
        sourceCardinality = '*'
        destinationCardinality = '*'
    }

    relationship(type('Order'), type('DeliveryMethod')).with {
        destinationCardinality = '1'
    }

    relationship(type('Order'), type('Product')).with {
        sourceCardinality = '*'
        destinationCardinality ='*'
    }

    relationship(type('Category'), type('Product')).with {
        bidirectional = true
    }

    relationship(type('National'), RelationshipType.INHERITANCE, type('DeliveryMethod'))
    relationship(type('International'), RelationshipType.INHERITANCE, type('DeliveryMethod'))

    it
}

Methods note, type and relationship are now responsible for creating or referencing the particular content of the diagram. The example uses default Groovy method with simplify accessing diagram’s parts properties. In the next step, we are going to replace method with with our own implementation.

Tasks

Implement methods note, type and relationship in Diagram class to manage diagram content. Methods must not create duplicates for the same input.
Make Diagram01Spec passing.

You can find solution here.

2. Basic Java DSL

In this step we are going to replace calling with method with custom methods. As we write your DSL code in Java we start with implementation using Consumer functional interface which will pass the instance being configured (e.g. Relationship) into the single absract method accept. As Consumer is functional interface we can use Java lambda syntax. After that, we will get closer to desired tree-like structure of the code.

Java Version of the DSL

private Diagram buildDiagram() {
    return Diagram.create(d -> {
        d.note("You can stick notes on diagrams too!", "skyblue");

        d.aggregation("Customer", "Order", r -> {
            r.source("1");
            r.destination("0..*", "orders");
        });

        d.composition("Order", "LineItem", r -> {
            r.source("*");
            r.destination("*");
        });

        d.association("Order", "DeliveryMethod", r -> {
            r.destination("1");
        });

        d.association("Order", "Product", r -> {
            r.source("*");
            r.destination("*");
        });

        d.association("Category", "Product", r -> {
            r.bidirectional(true);
        });

        d.type("National").inherits(from).type("DeliveryMethod");
        d.type("International").inherits(from).type("DeliveryMethod");
    });
}

You can see there are many new methods accepting lambda expressions. They all simplifies calling the base method relationship:

Methods Accepting Consumer

public Relationship relationship(
        String source,
        RelationshipType type,
        String destination,
        Consumer<Relationship> configuration
) {
    Relationship relationship = new Relationship(type(source), type, type(destination));
    configuration.accept(relationship);
    relationships.add(relationship);
    return relationship;
}

Declaring inheritance is handled using a fluent DSL. Fluent DSL uses a feature called Command Chains which allows to skip the parentheses.

Method inherits is just a very simple method of Type which returns instance of InheritanceBuilder which handles rest of the DSL statement.

Method inherits

public InheritanceBuilder inherits(DiagramKeywords.From from) {
    return new InheritanceBuilder(diagram, this);
}

InheritanceBuilder is a helper class to specify the rest of the information. From is just an enum with single value FROM which is passed into the DSL from static import.

Class InheritanceBuilder

package cz.orany.yuml.model;

public class InheritanceBuilder {

    private final Diagram diagram;
    private final Type type;

    public InheritanceBuilder(Diagram diagram, Type type) {

        this.diagram = diagram;
        this.type = type;
    }

    public Relationship type(String parent) {
        return diagram.inheritance(type.getName(), parent, c-> {});
    }
}

Tasks

Implement methods association, aggregation, inheritance and composition accepting Consumer in Diagram class for building different kinds of relationships.
Implement static method create in Diagram class which is accepting Consumer to build the whole diagram.
Practise implementing the fluent DSL for creating inheritance (feel free to copy & paste it from the snippets above)
Make Diagram02Test passing.

You can find solution here.

3. Basic Groovy DSL

It is quite easy to add groovy DSL extension once we have methods accepting functional types. Here is an example of the desired DSL in Groovy.

Groovy Version of the DSL

@CompileStatic                                                                      (1)
private static Diagram buildOrderDiagram() {
    Diagram.build { Diagram d ->                                                    (2)
        note('You can stick notes on diagrams too!', 'skyblue')

        aggregation('Customer', 'Order') {                                          (3)
            source cardinality: '1'                                                 (4)
            destination cardinality: '0..*', title: 'orders'
        }

        buildOrderTypes(d)                                                          (5)

        association('Category', 'Product') {
            bidirectional true
        }

        type 'National' inherits from type 'DeliveryMethod'
        type 'International' inherits from type 'DeliveryMethod'
    }
}

1	The goal is to give the static compiler enough hints to be able to compile the code statically
2	New static method in the `Diagram` class which delegates the closure to the instance of `Diagram` class and which passes the same instance of `Diagram` class as a single parameter of the closure
3	New method in the `Diagram` class which helps building a `Relationship`
4	New method in the `Relationship` class which accepts named arguments and helps specifying the cardinality and title
5	The closure parameter `Diagram d` can be used extract parts of the diagram definition to separate method

It is a good practise not to mix Java API and Groovy API together so let’s use Groovy Extension Modules to add additional methods to our current Java API.

Let’s create a extension module descriptor text file called org.codehaus.groovy.runtime.ExtensionModule in src/main/resources/META-INF/services/

Extension Module Descriptor

moduleName=yuml-extra
moduleVersion=0.1.0
extensionClasses=cz.orany.yuml.groovy.DiagramExtensions
staticExtensionClasses=cz.orany.yuml.groovy.DiagramStaticExtensions

Obviously we need create two new classes as well - DiagramStaticExtensions and DiagramExtensions. We will use annotations described in The Aid: Using the annotations for static compilation to add a hints for the static compiler and also the IDE.

Static Extension Class

package cz.orany.yuml.groovy;

import cz.orany.yuml.model.Diagram;
import cz.orany.yuml.model.DiagramContent;
import cz.orany.yuml.model.DiagramKeywords;
import groovy.lang.Closure;
import groovy.lang.DelegatesTo;
import groovy.transform.stc.ClosureParams;
import groovy.transform.stc.SimpleType;
import space.jasan.support.groovy.closure.ConsumerWithDelegate;

public class DiagramStaticExtensions {

    public static Diagram build(                                                        (1)
        Diagram self,                                                                   (2)
        @DelegatesTo(value = Diagram.class, strategy = Closure.DELEGATE_FIRST)          (3)
        @ClosureParams(value=SimpleType.class, options="cz.orany.yuml.model.Diagram")   (4)
        Closure<? extends DiagramContent> definition                                    (5)
    ) {
        return Diagram.create(ConsumerWithDelegate.create(definition));                 (6)
    }

    public static DiagramKeywords.From getFrom(Diagram self) {                          (7)
        return DiagramKeywords.From.FROM;
    }
}

1	Add a new static method called `build` …
2	… to the `Diagram` class
3	`@DelegatesTo` tells the compiler and IDE which object will be send as a delegate
4	`@ClosureParams` tells the compiler and IDE which object will be passed as closure parameters
5	Return value of the closure should be the common type of the return values of every possible DSL statement to prevent unfinished statements such as `type 'Type' inherits from`
6	`ConsumerWithDelegates` is a helper class from Groovy Closure Support library which helps using closures with methods which accepts common Java functional interfaces such as `Consumer`
7	Shortcut to `from` keyword so no static imports are required in Groovy code

Please, pay attention that the name of the static method added to Diagram class is different (build) then the one accepting Consumer (create). There is a bug in groovy which prevents overloading static methods. The restriction does not apply on the instance methods.

You may notice that in the example we also used a new methods source and destination accepting map as a parameter. We can define additional checks to this methods using annotations mentioned in The Navigation: Using the annotations for named parameters.

Extension Class

public class DiagramExtensions {

    private static final String CARDINALITY = "cardinality";
    private static final String TITLE = "title";

    public static Relationship relationship(
            Diagram diagram,
            String source,
            RelationshipType type,
            String destination,
            @DelegatesTo(value = Relationship.class, strategy = Closure.DELEGATE_FIRST)
            @ClosureParams(value = SimpleType.class, options = "cz.orany.yuml.model.Relationship")
            Closure<Relationship> configuration
    ) {
        return diagram.relationship(source, type, destination, ConsumerWithDelegate.create(configuration));
    }


    public static Relationship source(                                                  (1)
            Relationship self,                                                          (2)
            @NamedParams({                                                              (3)
                    @NamedParam(value = CARDINALITY, type = String.class),              (4)
                    @NamedParam(value = TITLE, type = String.class)
            })
                    Map<String, String> cardinalityAndTitle                             (5)
    ) {
        return self.source(
                cardinalityAndTitle.get(CARDINALITY),
                cardinalityAndTitle.get(TITLE)
        );
    }

    // rest of the methods
}

1	Add a new method called `source` …
2	… to the `Relationship` class
3	`@NamedParams` aggregates the possible named parameters
4	For example parameter `cardinality` of type `String`
5	The argument itself is still just plain old `Map`

Tasks

Using a static extension class, add a static method build accepting closure to the Diagram class
Using a static extension class, add a shortcut keyword from to the Diagram class (i.e. getFrom method)
Using an extension class, add methods for creating different types of relationships to the Diagram class which accept closures
Using an extension class, add methods source and destination which accepts named parameters to specify cardinality and title into the Relationship class
Make Diagram03Spec passing

You can find solution here.

4. [Bonus] Designing for Extendability

This is and advanced exercise. Feel free to ignore it or just read through it if you don’t feel comfortable trying to implement this step.

A good DSL is open and provides enough extension points to allow developers to extend the original functionality. Please, read The Extension: Designing your builder DSL for extendability to better understand the mechanism of creating extendable DSL.

For example, some developers would like to add support for stereotypes and properties.

Groovy Version of the DSL

@CompileStatic
private static Diagram buildDiagramStereotypesAndProperties() {
    Diagram.build { Diagram d ->
        note('You can stick notes on diagrams too!', 'skyblue')

        aggregation('Customer', 'Order') {
            source cardinality: '1'
            destination cardinality: '0..*', title: 'orders'
        }

        buildOrderTypes(d)

        association('Category', 'Product') {
            bidirectional true
        }

        type 'National' inherits from stereotype 'DeliveryMethod'                   (1)
        type 'International' inherits from stereotype 'DeliveryMethod'

        type('Customer') {
            property 'string', 'name'                                               (2)
        }
    }
}

1	Declaring a stereotype
2	Declaring type’s properties

Adding support for stereotype is pretty simple. Stereotype is just a type surrounded by << and >>. We can achieve this directly in the extension class.

Stereotype Extensions

public static Type stereotype(Diagram self, String name) {
    return self.type("<<" + name + ">>");
}

public static Relationship stereotype(InheritanceBuilder self, String name) {
    return self.type("<<" + name + ">>");
}

Adding support for the properties requires more steps. First we need to declare an interface for helper objects which can gather the metadata for diagram.

Diagram Helper

package cz.orany.yuml.model;

import java.util.Map;

public interface DiagramHelper {

    Map<String, Object> getMetadata();

}

These helpers can be added to the diagram and they simplifies access to the generic metadata map:

Diagram Helpers and Metadata

public static Diagram create(Consumer<Diagram> diagramConsumer) {
    Diagram diagram = new Diagram();
    diagramConsumer.accept(diagram);
    diagram.postprocess();                                                          (1)
    return diagram;
}

public Map<String, Object> getMetadata() {                                          (2)
    return metadata;
}

public <H extends DiagramHelper, R> R configure(                                    (3)
        Class<H> helper,
        Function<H, R> configurationOrQuery
) {
    H helperInstance = (H) helperMap.computeIfAbsent(helper, (h) -> {
        try {
            return (DiagramHelper) h.newInstance();
        } catch (InstantiationException | IllegalAccessException e) {
            throw new IllegalArgumentException("Cannot instantiate " + h, e);
        }
    });

    return configurationOrQuery.apply(helperInstance);
}

void postprocess() {                                                                (4)
    for (DiagramHelper helper : helperMap.values()) {
        metadata.putAll(helper.getMetadata());
    }
}

1	Method `postprocess` is called when the build phase of the DSL is finished
2	Metadata are just a generic map with string keys
3	Method `configure` instantiates or reuses existing helper instance and executes some code on helper instance and returns the result
4	Method `postprocess` iterates all the existing helpers and gathers the metadata

Now we can create a helper to store the metadata for given type:

Properites Helper

package cz.orany.yuml.model;

import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;

public class PropertiesDiagramHelper implements DiagramHelper {

    private static final String METADATA_KEY
            = "cz.orany.yuml.properties.PropertiesDiagramHelper.typeToProperties";

    private final Map<String, Map<String, String>> typeToProperties                     (1)
            = new LinkedHashMap<>();

    public PropertiesDiagramHelper addProperty(String owner, String type, String name) {(2)
        Map<String, String> properties = typeToProperties.computeIfAbsent(
                owner, (key) -> new LinkedHashMap<>()
        );

        properties.put(name, type);

        return this;
    }

    @Override
    public Map<String, Object> getMetadata() {
        return Collections.singletonMap(METADATA_KEY, typeToProperties);                (3)
    }

    @SuppressWarnings("unchecked")
    public static Map<String, String> getProperties(Diagram diagram, Type type) {       (4)
        Map<String, Map<String, String>> typeToProperties = (Map<String, Map<String, String>>) diagram
                .getMetadata()
                .computeIfAbsent(METADATA_KEY, (key) -> new LinkedHashMap<>());
        return typeToProperties.computeIfAbsent(type.getName(), (key) -> new LinkedHashMap<>());
    }
}

1	Properties for each type are held in a map
2	Adds a property to the given type
3	Returns the metadata provided by this helper - the map of the types and their properties
4	Returns the properties of given type

Now we have finally everything to add the method for adding type’s properties. This is also implemented as extension method:

Property Extensions

public static Type property(Type typeDefinition, String type, String name) {
    typeDefinition.getDiagram().configure(PropertiesDiagramHelper.class, (h) ->
        h.addProperty(typeDefinition.getName(), type, name)
    );
    return typeDefinition;
}

The last step is to update the class responsible for printing the diagram to take the properties into an account:

YUML Priter with Properties

private String print(Diagram diagram, Type type) {
    Map<String, String> properties = PropertiesDiagramHelper.getProperties(diagram, type);

    if (properties.isEmpty()) {
        return String.format("[%s]", type.getName());
    }

    String propertiesFormatted = properties
            .entrySet()
            .stream()
            .map((e) -> e.getKey() + ":" + e.getValue())
            .collect(Collectors.joining(";"));

    return String.format(
            "[%s|%s]",
            type.getName(),
            propertiesFormatted
    );
}

Tasks

Using an extension class, add a method stereotype to the Groovy DSL
Using an extension class, add a ability to specify type’s properties to the Groovy DSL
Update the printer class to print types' properties
Make Diagram04Spec passing.

You can find the solution in the step-05/src/main/java/ source codes.

5. [Bonus] API Clean Up

This is and advanced and quite stereotype exercise. Feel free to ignore it or just read through it if you don’t feel comfortable trying to implement this step.

The current version of the DSL allows calling all the operations of the diagram objects inside the definition. It is a good practice to split the methods for reading and the methods for defining the diagram into two separate interfaces.

See the following Diagram and DiagramDefinition classes:

Read Operations

package cz.orany.yuml.model;

import java.util.Collection;
import java.util.Map;
import java.util.function.Consumer;

public interface Diagram {

    static Diagram create(Consumer<DiagramDefinition> diagramConsumer) {
        return DefaultDiagram.create(diagramConsumer);
    }

    Collection<Note> getNotes();

    Collection<Type> getTypes();

    Collection<Relationship> getRelationships();

    Map<String, Object> getMetadata();

}

Definition Operations

package cz.orany.yuml.model;

import java.util.function.Consumer;
import java.util.function.Function;

public interface DiagramDefinition {

    default Note note(String text) {
        return note(text, null);
    }

    Note note(String text, String color);

    TypeDefinition type(String name);

    default TypeDefinition type(String name, Consumer<TypeDefinition> configuration) {
        TypeDefinition type = type(name);
        configuration.accept(type);
        return type;
    }

    default Relationship association(String source, String destination, Consumer<RelationshipDefinition> configuration) {
        return relationship(source, RelationshipType.ASSOCIATION, destination, configuration);
    }

    default Relationship aggregation(String source, String destination, Consumer<RelationshipDefinition> configuration) {
        return relationship(source, RelationshipType.AGGREGATION, destination, configuration);
    }

    default Relationship inheritance(String source, String destination, Consumer<RelationshipDefinition> configuration) {
        return relationship(source, RelationshipType.INHERITANCE, destination, configuration);
    }

    default Relationship composition(String source, String destination, Consumer<RelationshipDefinition> configuration) {
        return relationship(source, RelationshipType.COMPOSITION, destination, configuration);
    }

    Relationship relationship(String source, RelationshipType type, String destination, Consumer<RelationshipDefinition> configuration);

    <H extends DiagramHelper, R> R configure(Class<H> helper, Function<H, R> configurationOrQuery);
}

As we are developing for version Java 8 and later we can move some methods directly into the DiagramDefinition class. All the consumers now uses *Definition interfaces only.

Tasks

Separate the read and definition roles of the diagram using interfaces
Make Diagram05Spec passing

You can find the solution in the step-99/src/main/java/ source codes.