Saturday, March 15, 2014

How you can benefit from Groovy Shell

This is a post about the Groovy Shell and how it can help you with your daily work (as long as you are working as software developer). You can benefit from the Groovy Shell no matter what programming language(s) or technologies you are using. The only real requirement is that you are able to write (and read) small pieces of Groovy code.

Getting started
I think the purpose of the Groovy shell is best described by the official documentation:
The Groovy Shell, aka. groovysh is a command-line application which allows easy access to evaluate Groovy expressions, define classes and run simple experiments.
The Groovy Shell is included in the distribution of the Groovy Programming language and can be found in <groovy home>/bin. To start the Groovy Shell simply run groovysh from the command line:
GROOVY_HOME\bin>groovysh
Groovy Shell (2.2.2, JVM: 1.7.0)
Type 'help' or '\h' for help.
--------------------------------------------------------------------
groovy:000>
Within the shell you can now run Groovy commands:
groovy:000> println("hu?")
hu?
===> null
groovy:000>
It supports variables and multi line statements:
groovy:000> foo = 42
===> 42
groovy:000> baz = {
groovy:001> return 42 * 2
groovy:002> }
===> groovysh_evaluate$_run_closure1@3c661f99
groovy:000> baz(foo)
===> 84
groovy:000>
(Note that you have to skip the def keyword in order to use variables and closures later)

A few words for Windows Users
I can clearly recommend Console(2) which is a small wrapper around the awkward cmd window. It provides Tab support, better text selection and other useful things.
Unfortunately the Groovy 2.2.0 shell has a problem with arrow keys on Windows 7/8 in some locales (including German). However, you can use CTRL-P and CTRL-N instead of UP and DOWN. As an alternative you can use the shell of an older Groovy Version (groovysh from Groovy 2.1.9 works fine).

So, for what can we use it?
The most obvious thing we can do is evaluating Groovy code. This is especially useful if you are working on applications that make use of Groovy.
Maybe you know you can use the << operator to add elements to lists, but you are not sure if the operator works the same for maps? In this case, you can start googling or look it up in the documentation. Or you can just type it into Groovy Shell and see if it works:
groovy:000> [a:1] << [b:2]
===> {a=1, b=2}
It works!
You are not sure if you can iterate over enum values?
groovy:000> enum Day { Mo, Tu, We }
===> true
groovy:000> Day.each { println it }
Mo
Tu
We
===> class Day
It works too!

It is a Calculator!
The Groovy Shell can be used for simple mathematical calculations:
groovy:000> 40 + 2
===> 42
groovy:000>
groovy:000> 123456789123456789 * 123456789123456789123456789
===> 15241578780673678530864199515622620750190521
groovy:000>
groovy:000> 2 ** 1024
===> 179769313486231590772930519078902473361797697894230657273430081157732675805500963132708477322407536021120113879871393357658789768814416622492847430639474124377767893424865485276302219601246094119453082952085005768838150682342462881473913110540827237163350510684586298239947245938479716304835356329624224137216
groovy:000>
As you can see Groovy can work well with numbers that would cause overflows in other programming languages. Groovy uses BigInteger and BigDecimal for these computations. By the way, you can verify this yourself very quickly:
groovy:000> (2 ** 1024).getClass()
===> class java.math.BigInteger
Note that Groovy math tries to be as natural as possible:
groovy:000> 3/2
===> 1.5
groovy:000> 1.1+0.1
===> 1.2
In Java these computations would result in 1 (integer division) and 1.2000000000000002 (floating point arithmetic).

Do more
Maybe you need the content of a certain web page? This can be easily accomplished with Groovy:
groovy:000> "http://groovy.codehaus.org".toURL().text
===> <!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8"/>
    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>
    <meta name="description" content="Groovy Wiki"/>
    ...
Maybe you only want the <meta> tags for some reason?
groovy:000> "http://groovy.codehaus.org".toURL().eachLine { if (it.contains('<meta')) println it }
    <meta charset="utf-8"/>
    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>
    <meta name="description" content="Groovy Wiki"/>
    <meta name="keywords"
    <meta name="author" content="Codehaus Groovy Community"/>
===> null
I am sure you were in a situation where you needed the url encoded version of some text:
groovy:000> URLEncoder.encode("foo=bar")
===> foo%3Dbar
Of course you do not need to remember the exact class and method names. Just type in the first characters and then press tab to get the possible options:
groovy:000> URL
URL                       URLClassLoader            URLConnection             URLDecoder                URLEncoder
URLStreamHandler          URLStreamHandlerFactory
It works with methods too:
groovy:000> URLEncoder.e
each(            eachWithIndex(   encode(          every(           every()

Customize it
To truly benefit from the Groovy Shell you should customize it to your needs and provide functions that help you in your daily work. For this you can add your custom Groovy code to $HOME/.groovy/groovysh.profile (just create the file if it does not exist). This file is loaded and executed when groovysh starts.

Let's assume you want to decode a piece of Base64 encoded text. A viable approach is to start googling for an online Base64 decoder. An alternative is to add a few lines to your groovysh.profile to accomplish the job:
encodeBase64 = { str ->
  return str.bytes.encodeBase64().toString()
}

decodeBase64 = { str ->
  return new String(str.decodeBase64())
}
Now you can use the encodeBase64() and decodeBase64() functions within Groovy Shell to do the job:
groovy:000> encoded = encodeBase64('test')
===> dGVzdA==
groovy:000> decodeBase64(encoded)
===> test
This approach might be a bit slower the first time you are using it but you will benefit from it the next time you need to encode/decode a Base64 message. Note that autocomplete also works on your own  methods, so you do not need to remember the exact name.

Another example function that can be useful from time to time is one that computes the MD5 hash from a passed string. We can use Java's MessageDigest class to accomplish this task in Groovy:
import java.security.MessageDigest

md5 = { str ->
  // thanks to https://gist.github.com/ikarius/299062
  MessageDigest digest = MessageDigest.getInstance("MD5")
  digest.update(str.bytes)
  return new BigInteger(1, digest.digest()).toString(16).padLeft(32, '0')
} 
To compute a MD5 hash we then just have to call the md5() function:
groovy:000> md5('test')
===> 098f6bcd4621d373cade4e832627b4f6
But what if we want to compute the MD5 value of a file?
If the file is not that large getting the content of it is as simple as this:
new File('test.txt').text
We just have to pass this to the md5() function to compute the md5 hash of the file:
groovy:000> md5(new File('test.txt').text)
===> a4ba431c56925ce98ff04fa7d51a89bf
Maybe you are working a lot with date and times. In this case it can be useful to add Joda-Time support to your Groovy Shell. Just add the following lines to groovysh.profile:
@Grab('joda-time:joda-time:2.3') import org.joda.time.DateTime
import org.joda.time.DateTime
If you run groovysh the next time Joda-Time will be downloaded using Grape. Additionally the Joda DateTime class is imported so it can be used in Groovy Shell without prefixing the package name:
groovy:000> new DateTime().plusDays(42)
===> 2014-04-22T22:27:20.860+02:00
You commonly need to convert time values to/from unix timestamps?
Just add two simple functions for it and you no longer need your bookmark for an online converter:
import java.text.SimpleDateFormat
dateFormat = new SimpleDateFormat('yyyy-MM-dd HH:mm:ss')

toUnixTimestamp = { str ->
  return dateFormat.parse(str).getTime() / 1000
}

fromUnixTimestamp = { timestamp ->
  return dateFormat.format(new Date(timestamp.toLong() * 1000))
}
Usage in Groovy Shell:
groovy:000> toUnixTimestamp('2014-04-15 12:30:00')
===> 1397557800
groovy:000> fromUnixTimestamp('1397557800')
===> 2014-04-15 12:30:00
Maybe you want to execute a command on a remote machine?
You only need another simple function to accomplish this task with Groovy Shell:
ssh = { cmd ->
  def proc = "ssh -i keyfile user@host $cmd".execute()
  proc.waitFor()
  println "return code: ${proc.exitValue()}"
  println "stderr: ${proc.err.text}"
  println "stdout: ${proc.in.text}" 
}
Usage:
groovy:000> ssh 'ls -l'
return code: 0
stderr:
stdout: total 1234
-rw-r--r-- 1 foo foo 7678563 Oct 28  2009 file
drwxr-xr-x 4 foo foo    4096 Mar  1 17:07 folder
-rw-r--r-- 1 foo foo      19 Feb 27 22:19 bar
...
In case you did not know: In Groovy you can skip parentheses when calling a function with one or more parameters. So ssh 'ls -l' is the same as ssh('ls -l').

Conclusion
Before I switched to the Groovy Shell, I used the Python shell for nearly the same reasons (even if I was not working with Python at all). Within the last year I used a lot of Groovy and I quickly discovered that the Groovy Web Console is a very valuable tool for testing and prototyping. For me the Groovy Shell replaced both tools. It is clearly a development tool I do not want to miss.

I think it is really up to you how much you let the Groovy Shell help you.

Share this post using Facebook, Twitter or Google+

8 comments:

  1. Very beneficial article. The groovysh is a very cool tool that comes with groovy and even Java developers can be benefited from this.

    ReplyDelete
  2. Please note that your ssh example is vulnerable to cmd injection if an attacker has control over any part of $cmd

    ReplyDelete
    Replies
    1. You are absolutely right. This is definitely not the recommended way to do stuff on a production system. An alternative solution would be to write small functions for every task that needs to be done on the remote system (e.g. restartServer(), ..).

      Delete
  3. Nice writeup. I like the Groovy Console for these types of tasks.

    Quick question - is that a typo or a double import for the Joda Time example? Also, do you mean Groovy Console in the section talking about Joda Time or Groovy Shell? Does the Groovy console somehow use the Groovy Shell - so those imports would be available for the Groovy Console?

    ReplyDelete
    Replies
    1. Thanks for your comment. In the Joda Time section I accidentally wrote "Groovy Console" instead of "Groovy Shell". I corrected this.

      Unfortunately the double import of DateTime is no typo. A single import didn't work for me. The first one is needed to place the @Grab annotation while the second import statement actually imports the class. The import statement seems somehow special in Groovy Shell :/

      I think the Groovy web console uses GroovyShell internally. The source can be found on GitHub: https://github.com/glaforge/groovywebconsole/blob/master/war/WEB-INF/groovy/executor.groovy

      Delete
  4. Consider also this for Grapes:
    http://tech.puredanger.com/2010/03/01/groovy-grape-groovysh/
    In case you care a lot about groovysh, I need feedback on this:
    http://groovy.329449.n5.nabble.com/groovysh-variable-declarations-td5717853.html

    ReplyDelete
  5. Groovy is kinda new to me, and when I try to customize the groovysh with groovysh.profile in place, I encountered this :

    "ERROR [org.codehaus.groovy.tools.shell.Groovysh] Unable to load user-script, missing 'load' command".

    After searching and experimenting with all kinds of ideas I can come up with, I think I finally find something that might be the culprit.

    I run groovysh with -d, and the log tells everything(part of the log is omitted):
    ❯ groovysh -d
    DEBUG [org.codehaus.groovy.tools.shell.BufferManager] Created new buffer with index: 0
    DEBUG [org.codehaus.groovy.tools.shell.BufferManager] Buffers reset
    DEBUG [org.codehaus.groovy.tools.shell.Parser] Using parser flavor: rigid
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: ?
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: :help
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: :display
    ...
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: .
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: :load
    ...
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: :register
    DEBUG [org.codehaus.groovy.tools.shell.CommandRegistry] Registered command: :doc
    DEBUG [org.codehaus.groovy.tools.shell.Groovysh] Terminal (jline.UnixTerminal@65373267)
    DEBUG [org.codehaus.groovy.tools.shell.Groovysh] Supported: true
    DEBUG [org.codehaus.groovy.tools.shell.Groovysh] ECHO: (enabled: false)
    DEBUG [org.codehaus.groovy.tools.shell.Groovysh] H x W: 61 x 180
    DEBUG [org.codehaus.groovy.tools.shell.Groovysh] ANSI: true
    ERROR [org.codehaus.groovy.tools.shell.Groovysh] Unable to load user-script, missing 'load' command

    I also went to the repository to take a look at the source code(OK, I admit, the page of source code is the very first entry that got pop up in google. :) ):

    private void loadUserScript(final String filename) {
    assert filename

    def file = new File(userStateDirectory, filename)

    if (file.exists()) {
    def command = registry['load']

    if (command) {
    log.debug("Loading user-script: $file")

    // Disable the result hook for profile scripts
    def previousHook = resultHook
    resultHook = { result -> /* nothing */}

    try {
    command.load(file.toURI().toURL())
    }
    finally {
    // Restore the result hook
    resultHook = previousHook
    }
    }
    else {
    log.error("Unable to load user-script, missing 'load' command")
    }
    }
    }

    So I suspect that the "registry['load']" part of the source code should be replaced with "registry[":load"]".

    Anyhow, I can do nothing about it(that will scare my pants off if I can).

    So frustrating and disappointing. Please help me with this.

    Well, I guess the easiest way might be to point out where I got wrong.

    ReplyDelete
    Replies
    1. FYI, the goovy version with this possible issue is 2.3.0, and 2.2.2 is working totally fine.
      The thing is, homebrew does not allow me to specify a version. I have to modify the formula to get the 2.2.2.

      Delete