Groovy in the Clouds Vladimír Oraný

Gaelyk 2.0 is finally out. Some of the new features were already listed on this site.

The most important features are:

• Groovy 2.0 support and removing GaelykCategory
• Search DSL
• Optinal routes parameters
• Better parameters conversion
• Better coercion performance
• Returning QueryResultList and QueryResultIterator from datastore.execute{} and datastore.iterate{} methods
• and many others …

Gaelyk 2.0 is already available on Maven Central or you can download it from Gaelyk Download Page.

What’s new in Gaelyk 2.0 Session at Gr8Conf

If you are attending Gr8Conf EU next week, I’ll be glad meeting you at my session What is new in Gaelyk 2.0 where all the new features will be shown.

Gaelyk Spock and core plugins

Gaelyk related projects were relased as well:

• Gaelyk Spock 0.4
• Gaelyk Resources Plugin 2.0
• Gaelyk Bootstrap Resources Plugin 2.0
• Gaelyk Console Plugin 2.0
• Gaelyk Datastore Viewer Plugin 2.0

Thank all the great people helping with this release!

Gaelyk Query DSL makes quering Google App Engine Datastore a lot simpler but sometimes it isn’t working as expected.

Variable naming conflict

Some of problems origins from using the variable or property names which are already taken. Names of the properties in where clause are converted to String automatically. What property we will be querying?

Variable name conflict

def count = 123
// a lot of code here so you've already forgotten you have such a variable

def maxCount = 100

datastore.execute {
    from Item
    where count <= maxCount
}

This query will be translated as 123 <= 100 which probably isn’t what you wanted.

Binding variable name conflict

Previous example was quite obvious but what if we have following query:

Binding variable name conflict

datastore.execute {
    from Item
    where users > 10
}

Instead of getting result of items having more than ten users we get empty result set. It’s because users is Gaelyk’s shortcut to UserService so the where clause is translated to something like UserService@xyz123 > 10 which obviously returns no results.

If you run into name conflict just use good old String as property name in the where clause such as where 'users' > 10.

Entity pitfalls

@Entity annotation adds sevral useful methods to the POGO class such as findAll which resemble their Grails counterparts. But don’t get confused. The syntax of using such methods differs slightly. find, findAll or count method are just shortcuts to Query DSL!

Using findAll method

@Entity class Item {
    int count
}

Item.findAll { count == 10 }

The query listed above will return all the items since the condition is ignored because the where keyword is missing.

Using findAll method with where

@Entity class Item {
    int count
}

Item.findAll { where count == 10 }

We have added the where keyword to the findAll method but now we’ll always get empty result because all field of @Entity class are unindexed by default. You need to mark the field @Indexed to use it in queries.

Using findAll method with where on indexed field

@Entity class Item {
    @Indexed int count
}

Item.findAll { where count == 10 }

Summary

Using Query DSL you should basically always take care about two important things:

1. Property names are always converted to String, this is may make mess if the variable with the same name is already defined (even in Gaelyk shortcut bindings). Use String constants instead of property names in case of any possible name conflict.
2. All properties you’re querying must be indexed. When using @Entity annotations all the properties are unindexed by default.

In Gaelyk you can already save query created by query dsl using datastore.query{...} and than excute it using datastore prepare function but this method only support a subset of dsl statements - namely select, from, where, and and sort. Corection using as or setting limit or offset is simply ignored. In Gaelyk 2.0 you can use datastore.build{...} method to create instance of QueryBuilder.

Let’s show the difference between datastore.query and datastore.build methods:

Query method example

Query query = datastore.query {
    from 'Comment' as Comment
    where author == 10
    sort by crate desc
    limit 10        
}

PreparedQuery pq = datastore.prepare query

// The limit is gone! We need to set it again
FetchOptions.Builder options = withLimit(10)

// set the offset if there is page param specified
if(params.page){
    options = options.offset((params.page as int) * 10)
}

def comments = pq.asList(options.build())

// and we also get only entities not comments
for(Entity e in entity){
    // which needs to be coerced manually
    Comment comment = e as Comment
    process comment
}

When using datastore.query method a portion of query parameters where erased, expecially loosing coercion is very confusing. On the other hand if you use datastore.build instead than all the information are kept.

Build method example

QueryBuiler dsl = datastore.build {
    from 'Comment' as Comment
    where author == 10
    sort by crate desc
    limit 10
}

// set the offset if there is page param specified
if(params.page){
    dsl.offset((params.page as int) * 10)
}

def comments = dsl.execute()

// no need to coerce manually
for(Comment in comments){
    process comment
}

Google App Engine is full of hidden traps. One of them is the need to restart long running queries when the query expired causing following exception to be thrown:

The requested query has expired. Please restart it with the last cursor to read more results

In Gaelyk 2.0 you can now tell your query to restart automatically if this happen. Restarting happen under the hood so you can use for loops flawlessly.

To enable automatic restarting of queries add restart automatically statement to your query dsl. Currenty, you can only use this option on iterate method, because it doesn’t make much sense with execute method.

Simple restarting query

def comments = datastore.iterate {
    from Comment
    limit 50000
    restart automatically
}

for(comment in comments){
    // would throw exception randomly for long running queries
    process comment
}

You can use restart automatically with select all and select keys query types. It also works with coerced queries using as Type keyword e.g. form 'Comment' as Comment.

Today, Google App Engine Team said farewell to deprecated classes in version 1.7.6 causing sevral sites crashed. Even the application was deployed with older SDK the AddResponse class just disappeared. Yes, we all were warned in that release notes but who cares about the warnings. I wish they have mentioned when the new version will be released too so we’ll be extra careful today.

For Gaelyk users there are two choices to fix your application:

• Switch to Gaelyk 2.0 snapshot unless version 2.0 is finally out
• Build Gaelyk 1.3 snapshot fixed by Michael Migdol and use it

Both solutions was discussed in this thread. If you use GaelykCategory class directly in your code, 1.3 snapshot is better option for you because the class is no longer present in the code base of 2.0 snapshots. The category class was broken into serval separate extension modules.

If you’re using plain Java in your Google App Engine application, just switch to latest 1.7.6 SDK and compliation errors will guide you.