What was I Looking For?

 
My exploration of Play! Scala was actually prompted by the Product Owner I am currently working with. He recently took up Ruby on Rails with the intent to be able to do rapid prototyping of concepts for customers. The prototypes help him determine the customer’s real wants/needs so that he can communicate them more easily to his teams, and get a better end product. So far the concept works quite well.
 
So why Rails? Well, our code-base is primarily Java, so non-Java code inhibits the temptation to keep a prototype. Rails also allows very fast prototyping. You can be doing basic CRUD moments after the rubber hits the road, allowing prototypes to even have limited functionality to more fully test user’s expectations.
 
However, as I am a strong proponent of Scala’s extensibility and power, I was curious; why Rails?
 
Is it possible for Scala to provide the same ability to build prototypes extremely quickly?
 
 

What I Found

 
Play! Scala has some real strengths:

• Development lifecycle very similar to a dynamic language – edit, and refresh
• Type-safe code without having to declare types often (type inference)
• Type-safe templates based on the Scala language
• Very fast to go from nothing to running

 
Unfortunately, it’s not ready:

• “Automated Testing” is done by clicking a button in the browser. I want CI.
• Had a hard time getting CRUD running for Scala
• Documentation could use some work

 
Interestingly, CRUD likely would have worked, however I had already tore out the ScalaQuery JDBC implementation in favor of my preferred Squeryl.
 
A more thorough tutorial/example would have likely got my CRUD working before I attempted something like Squeryl.
 
In fact, a good start-to-finish tutorial application (like a basic Blog), complete with CRUD and authentication, could probably mature the basic “rails” (pun totally intended) of Play! Scala to the point that it could directly compete with Ruby on Rails for rapid prototypes.
 
Of course, for scalability Scala has many other options that are almost as fast; I’ll take Lift any day, but I’ve also heard amazing things about Scalatra, Spray, and a few other frameworks.
 
 

Final Thoughts

More/Better Tutorials.
 
Maybe integrate into SBT instead of having a separate executable if Play! Scala is “true” Scala.
 
Deliverable WARs instead of Play Apps to take advantage of a more well-known deployment lifecycle.
 
Easier hooks into testing, with the intent to enable automation for CI.
 
Lots of potential here. Given a little more maturity, Play! Scala may become a real option in my mind. For now, it’s a very interesting project that just has a little further to go.

Why fly when a car will do?

Don’t get me wrong, I drive regularly. Occasionally I even take long trips by the pavement when the tarmac is too pricey. However, there’s a limit. Flying is faster, safer, and more convenient. When cost (both price and time-overhead) allows, I’d prefer to be in the air.
 
In a way, that’s where the comparison breaks down. While Maven is generally faster, safer, and far more convenient than Ant, nobody is charging you exorbitant prices to use it. Both are free, it’s up to you to pick which.
 
Unlike real air travel, Maven also avoids long up-front overhead costs. To fly for real I have to head to the airport early, get through security, sit on my hands for an hour, then jump on the plane and wait another twenty minutes before takeoff. Maven doesn’t have that. I’m running in about a half-dozen lines, and a couple directories.
 
Amusingly enough, I’m still talking about Maven and Ant here. I’ve personally moved on to SBT and I’m loving it. One quick command-line call, and I’ve got my project rolling. I can customize where I want to whatever extent I want because I get to use real code to do so. For me, it’s like moving from the incredible velocity of a jet plane, and strapping myself inside a rocket bound for wherever I want to go.
 
Still, I’m baffled every now and then when I look back down on earth to see someone driving their car because “it’s the way it’s always been done”.

Unfortunately, it has a few drawbacks.

I was introduced to Scala through Mike Nash, a coworker of mine who dove into the language first. While learning, I would regularly poke fun at his occasional Scala-isms while he tried to write Java code. For example:

public class ServiceTest {
    @Test
    public void testSummation() {
        List<Integer> intsToSum = List(1, 2, 3);  // Uh... that's not Java mike...
        int sum = service.sum(intsToSum);
        assertEquals(6, sum);

        // At least he didn't intsToSum.reduceLeft(_+_) ;P
    }
}

Even as I slowly slipped into the same habit, I would still tease, but today I realized that the real frustration isn’t the occasional Scala-ism. The biggest pain-point I’ve found so far in learning Scala is having Scala in my personal toolkit, but not having it in our team’s toolkit.

Scala is a JVM language. It would compile into the WAR-file and be indistinguishable to our customer. It’s statically typed (which I like), but still uses type-inference to avoid the verbosity of most static-type languages. It’s extremely expressive, complete with the power of the functional and the object oriented paradigms. It’s all those things, and more, and I can’t use it.

case class ProductFamilyCategoryWithChildren(category : ProductFamilyCategory, families : List[ProductFamily]);

public class ProductFamilyCategoryWithChildren {
    private ProductFamilyCategory category;
    private List<ProductFamily> families;

    public ProductFamilyCategoryWithChildren(ProductFamilyCategory category, List<ProductFamily> families) {
        this.category = category;
        this.families = families;
    }

    public ProductFamilyCategory getCategory() { return category; }
    public List<ProductFamily> getFamilies() { return families; }
}

That’s 1 line of Scala to 12 of Java. There are 111 characters of Scala to 456 of Java. I stare at both, swallow my pride a little, delete the Scala version, and move on…

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

Motivation

Recently in order to gain confidence in our web application, our team decided it pragmatic to run a series of tests over a deployed version of the application complete with production-like database. A simple suite of non-destructive tests we could run in any environment was quickly put together.
 
Unfortunately one of our simplest tests quickly began causing us trouble. The test emulated a search spider, crawling every link it found on the entire site in an exhaustive graph traversal complete with cycle-detection. Unfortunately, the test ran for over 24 hours without completing.
 
While we’ve made other optimizations to improve performance of the test (such as excluding sufficiently similar pages), the topic of today’s post is the concurrency we introduced in order to help take the edge off test time.
 
 

Distributing Concurrent Tests

Our goal was relatively simple. We wished to run a very simple test across many thousands of URLs:

def testSingleUrl(nextUrl : String) = {
      client.open(nextUrl)
      client.statusCode should equal(200)
}

 
 
If either of the calls inside this test fail, an exception would be thrown. This was an acceptable way of detecting test failure. However, the test needed to do a bit more – traverse the graph:

  def testUrl(nextUrl: String): Unit = {
      if (wasVisited(nextUrl)) return

      val client = new HtmlUnitRunner
      client.open(nextUrl)
      client.statusCode should equal(200)

      registerVisitedUrl(nextUrl)

      JListWrapper(client.currentPage.getAnchors())
              .map(_.getAttribute("href"))
              .filter(isLinkValid(_))
              .foreach((a: String) => {
                      markForTesting(nextUrl)
              })
  }

Of course, this was called by a method which pushed and popped from a stack, and “markForTesting” pushed a new link onto that stack. This code worked great sequentially, but we wanted it to operate concurrently in order to minimize testing time. For this, we employed Akka’s actors:

class ConcurrentTest extends Actor {
  def receive = {
    case name: (() => Unit) =>
      try {
        name()
        self reply(true)
      } catch {
        case t: Throwable =>
          t.printStackTrace()
          self reply(false)
      }
    case _ =>
      println("Actor received an unrecognized message")
      self reply(false)
  }
}

ConcurrentTest, as you can see, is the driver behind an individual run of the test method:

val testRunner = actorOf[ConcurrentTest].start()
val result = testRunner !!! (() => testUrl("/index.html")
// Other code can go here
result.get

 
 
Assuming the methods called within testUrl are thread-safe (which we also did using Actors), this will run a single test using a second Thread, and allow us to continue on with our business. However, since there’s only a single Actor here, we only have one Thread with which to process URLs. This means that we’re still effectively opening each link sequentially.

We need a Thread pool, and Akka is glad to provide one:

class Master(nofWorkers: Int) extends Actor {
  val workers = List.fill(nofWorkers)(actorOf[ConcurrentTest].start())
  val router = Routing.loadBalancerActor(new CyclicIterator(workers)).start()
  val answers = ListBuffer[Future[Boolean]]()

  def receive = {
    case name: (() => Unit) => answers += router !!! name
    case "result" => {
      Futures.awaitAll(answers.toList)
      self reply(answers.foldLeft(true)(_ && _.result.get))
      workers.foreach(_.stop())
      router.stop()
      self stop()
    }
  }
}

 
 
So far everything I’ve presented as code came very naturally. In fact, minor modifications for the purpose of blogging notwithstanding, we used the code I’ve presented so far to test several links across our site very successfully, in a fashion very similar to the following:

val masterRunner = actorOf(new Master(concurrentActors)).start()
val productList = fetchAllProductsFromDatabase

productList.map(productToUrl(_))
        .foreach(url => masterRunner ! (() => testSingleUrl(url)))

((masterRunner !! "result").getOrElse(false).asInstanceOf[Boolean]) should equal(true)

However, when we attempted to apply the same methodology to our crawl test, things didn’t work as well as we’d hoped:

    val masterRunner = actorOf(new Master(concurrentActors)).start()

    // def testUrl as seen above

    def markForTesting(nextUrl : String) {
            masterRunner ! (() => testUrl(nextUrl))
    }

   def fullTest() = {
            val baseUrl = "/index.html"
            masterRunner ! testUrl("baseUrl")

             ((masterRunner !! "result").getOrElse(false).asInstanceOf[Boolean]) should equal(true)
    }

 
 
The theory was that “result” would wait for all of the answers to come back before returning. Unfortunately, that’s not quite the sequence of events the Actor sees. In reality, after digging, we figured out what messages the actor received:

1. (() => testUrl(“/index.html”)) occurs, which is quickly sent to a ConcurrentTest runner
2. “result” comes next, as it takes a second or two for the runner to open the test
3. (() => testUrl(_)) is received for several other URLs as links are scraped off the first page

“result” doesn’t actually wait for all the answers to come back, as it has no way of knowing how many answers are actually required. For that matter, we aren’t sure of that number either, as the test is meant to be dynamic. Instead, “result” simply compiles the answers it has so far, and then shuts down all of the actors. This means we get a “yes” or “no” about “/index.html”, but all of the other URLs are still sitting in the mailbox of Master when it’s shut down. Uh-oh!

So how do we know when we’re done? Mailbox sizes. We added a new match case to Master which would calculate if it believed the tests to be done yet:

class Master(nofWorkers: Int) extends Actor {
  ...
  def receive = {
    ...
    case "done" => { // Added this case, the rest of "Master" remained unchanged
        Futures.awaitAll(answers.toList)
        val overallSize = workers.map(worker => worker.mailboxSize).foldLeft(0)(_ + _) + router.mailboxSize
        self.reply(overallSize == 0 && self.mailboxSize == 0)
    }
  }
}

 
 
This code is different than “result” in that it actually attempts to detect if the tests are done by:

1. Waiting for all currently outstanding test-methods to complete
2. Counting any pending messages in the router and worker mailboxes.
   (This should always be zero, as we’ve waited for all answers to return, but it’s still safer to be sure)
3. Counting any pending messages on the master
4. Return “true” if the total pending messages is 0, otherwise “false” as more tests have to run

This algorithm will work for us because when we run Futures.awaitAll it runs every outstanding test to completion. Any URLs found on the pages to be tested are checked against previously-visited URLs, and added to Master’s queue if they are new. Since Master is still processing “done” those tests will stay on the queue and “mailboxSize” will return a positive non-zero number. If, however, no new links are encountered, then there will be no tests waiting on the Master queue, and our “done” operation will detect 0 pending tests.

In use:

val masterRunner = actorOf(new Master(concurrentActors)).start()
def markForTesting(nextUrl : String) = masterRunner ! (() => testUrl(nextUrl))
// def testUrl as seen above

def fullTest() = {
    val baseUrl = "/index.html"
    masterRunner ! testUrl("baseUrl")
    while((masterRunner !! "done").getOrElse(false).asInstanceOf[Boolean] == false) {
        Thread.sleep(1000);
    }
    ((masterRunner !! "result").getOrElse(false).asInstanceOf[Boolean]) should equal(true)
}

Now we sleep our thread, asking the master if it’s completed its job once every cycle, until the master claims all of its workers have completed their work and no new work is pending for the master to distribute.

Feedback as to other potential approaches is very welcome, I find the entire topic of concurrency and job distribution very interesting.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

DSL Exploration: Our Tale So Far

For those interested in the whole DSL Exploration “mini-series”, we began with DSL Exploration & Operator Notation.

To quickly summarize our tale so far, I decided to explore the creation of a DSL to expose some of Scala’s nuances. My project of choice was codifying the LOLCode language as a Scala DSL. So far we have created a syntax for outputting data to the screen.

Working syntax so far:

U SEEZ "Hello World" !
U SEEZ "Hello World" !!

Learned Lesson 3: Operation Interpretation

So far we’ve been doing all of our work in the Scala REPL. While extremely powerful, the REPL does have a few known limitations – it sometimes interprets a line differently than it would have been interpreted from a source file. Today we’ll discover one such situation, and explore the reason.

Let us start with a Scala source-file which will act as our basis:

class Printable(output:String) {
  var bang = false;
  def ! = { print(output); }
  def !! = { println(output); }
}
class OutputStream {
  def SEEZ(output: Printable) : Unit = println(output.toString)
}
val U = new OutputStream

U SEEZ "Hello"!
U SEEZ " World"!!
U SEEZ "Goodbye"!
U SEEZ " World"!!

The code above is equivalent to what we were writing into the REPL in previous posts, but let’s store that in a file ScaLol.scala and run it:

tcalder:bin tcalder $ scala ScaLol.scala
/.../ScaLol.scala:10: error: value ! is not a member of java.lang.String
U SEEZ "Hello"!

Clearly not what the REPL was telling us. At first glance, it now appears as if it’s completely changed its operational precedence, and that’s true… but only sort of.

After trying various combinations of the code (none of which helped), I eventually gave up and started reading the Scala Reference again.

Let’s review what we’ve said so far:

• Methods with one argument can be used as an infix operator
• Methods with no arguments can be used as a postfix operator
• Unary operators for !, +, -, and ~ can be used if equivalent “unary_*” methods are defined
• Postfix operators always have lower precedence than infix operators
• Infix operators have an order of precedence which includes letters being lower than special characters

Reasoning briefly about our code, the REPL clearly evaluates the “!” as a postfix operator, which gives it the lowest precedence and results in:

U.SEEZ("Hello").!()

Yet in ScaLol.scala we appear to have attempted to run “!” on String. Since Postfix have low precedence, we just be attempting to evaluate “!” as an Infix operator. This would make sense since special characters have a higher operational precedence:

U.SEEZ( "Hello World".!(.....?) )

Now if you’re like me, you’re wondering what on earth it could possibly be trying to fill in as the argument to this strange new Infix “!”. The answer is, it’s trying to use the next line as a continuation on the current:

U.SEEZ("Hello".!(U)).SEEZ("World"... // Or some rough equivalent

From the ScalaReference:

Scala is a line-oriented language where statements may be terminated by semicolons or newlines. A newline in a Scala source text is treated as the special token
“nl” if the three following criteria are satisfied:

1. The token immediately preceding the newline can terminate a statement.
2. The token immediately following the newline can begin a statement.
3. The token appears in a region where newlines are enabled.

“U” can start a statement, and “!” can terminate one, since the REPL clearly didn’t get upset at either of these. Newlines are enabled in our code – there’s a lot of logic in the Reference that explains why if you’d like to explore it.

Based on this, we can conclude that our “\n” after the “!” is, in fact, a “[nl]“.

The second piece of the puzzle that we need is the following (summarized) syntactic definitions, expressed in the Scala Ref using Extended Backus–Naur Form:

PostfixExpr ::= InfixExpr [id [nl]]
InfixExpr ::= PrefixExpr
                     | InfixExpr id [nl] InfixExpr
PrefixExpr ::= [‘-’ | ‘+’ | ‘~’ | ‘!’] SimpleExpr

In specific, draw your attention to the similarity between the definition of “PostfixExpr” and the second definition of “InfixExpr”, and pay special attention to our friend “nl”.

The following is a (rough) breakdown of what we’ve actually attempted:

U SEEZ "Hello World" !
U SEEZ "Goodbye World" !

SimpleExpr id SimpleExpr id nl SimpleExpr id SimpleExpr id

PrefixExpr id PrefixExpr id nl PrefixExpr id PrefixExpr id

InfixExpr id InfixExpr id nl InfixExpr id InfixExpr id

Bingo, there’s our culprit. The second “id” in the expression is our “!” operator. Based on the formal specification, “InfixExpr id nl InfixExpr” can be simplified as a single “InfixExpr”, and since “!” has higher precedence than “SEEZ”, it will clearly be evaluated first.

As a final proof, try saving the following code as “ScaLol2.scala” and running it. Note how we use two new-lines in-between “SEEZ” calls. This results in two “nl” entities instead of one, which prevents the compiler from interpreting an InfixExpr where we didn’t intend one:

class Printable(output:String) {
  def ! = { print(output); }
  def !! = { println(output); }
}
class OutputStream {
  def SEEZ(output: String) : Printable = new Printable(output)
}
val U = new OutputStream

U SEEZ "Hello"!

U SEEZ " World"!!

U SEEZ "Goodbye"!

U SEEZ " World"!!

The result on my machine:

tcalder:bin tcalder $ scala ScaLol2.scala
Hello World
Goodbye World

That’s all for this week! I hope today’s post proved interesting, it was far deeper than I ever expected to go, which was exactly what I was hoping for.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

DSL Exploration: Our Tale So Far

For those interested in the whole DSL Exploration “mini-series”, we began with DSL Exploration & Operator Notation.

To quickly summarize our tale so far, I decided to explore the creation of a DSL to expose some of Scala’s nuances. My project of choice was codifying the LOLCode language as a Scala DSL. So far we have created a syntax for outputting data to the screen.

Working syntax so far:

U SEEZ "Hello World"

Learned Lesson 2: Operation Precedence

Now that we’ve successfully defined “U SEEZ”, the LOLCode spec allows us to add a bang at the end of a sentence to “cancel” the newline. While this feels a little backwards, we can attempt it.

Our goal:

U SEEZ "Hello World" // Includes newline
U SEEZ "Goodbye"! // No newline

My first plan of attack was to make “SEEZ” return an object that has the “!” method, which would do a “print” instead of a “println”. Fortunately, before attempting this solution I realized that if I were to do so there would be no method call to trigger the non-bang “println”.

Attempt two, maybe we can allow SEEZ to accept a different object. That object can have a “!” method. Here’s what I’m aiming for:

U.SEEZ( "Hello World".!() )

 
We’ll also need an implicit to turn Strings into this special new object. Here goes.

scala> class Printable(val text : String, val bang : Boolean) {
     | override def toString() = if (bang) {text} else {text + "\n"}
     | def ! = new Printable(text, !bang)
     | }
defined class Printable

scala> class OutputStream {
     | def SEEZ(p : Printable) = print(p.toString())
     | }
defined class OutputStream

scala> val U = new OutputStream
U: OutputStream = OutputStream@e208506

scala> implicit def stringToPrintable(s : String) : Printable = new Printable(s, false)
stringToPrintable: (s: String)Printable

scala> U SEEZ "Hello World"
Hello World

That’s good news, we’ve managed to maintain the old behaviour, so let’s give our new “!” method a try:

scala> U SEEZ "Hello World"!
:10: error: value ! is not a member of Unit

Foiled again! So what happened here? As “println” returns type “Unit”, which is the Scala equivalent to “void”, we can venture a guess that the “!” method was actually run on the result of “println”:

U.SEEZ("Hello World").!()

 
As I’m sure you’ve guessed, the issue at hand is operator precedence. Let’s experiment a little:

scala> class MyObj(val v : String) {
     | def in(m : MyObj) = {println("[" + v + ".in(" + m.v + ")]"); new MyObj(v + m.v) }
     | def post() = {println("[" + v + ".post()]"); this}
     | def unary_+ = {println("[" + v + ".pre()]"); this}
     | }
defined class MyObj

scala> implicit def stringToMyObj(s : String) : MyObj = new MyObj(s);
stringToMyObj: (s: String)MyObj

scala> +"a"
[a.pre()]
res2: MyObj = MyObj@7987b796

scala> + "a" in "b"
[a.pre()]
[a.in(b)]
res3: MyObj = MyObj@3cd41115

scala> + "a" in "b" post
[a.pre()]
[a.in(b)]
[ab.post()]
res4: MyObj = MyObj@6dcc55fb

scala> "a" in "b" in "c"
[a.in(b)]
[ab.in(c)]
res5: MyObj = MyObj@69cad977

scala> "a" in "b" post
[a.in(b)]
[ab.post()]
res6: MyObj = MyObj@37b24706

scala> "a" post
[a.post()]
res7: MyObj = MyObj@7ea4b9da

scala> "a".post in "b"
[a.post()]
[a.in(b)]
res8: MyObj = MyObj@7e28388b

scala> "a" post in "b"
:1: error: ';' expected but string literal found.
       "a" post in "b"
                   ^

 
 
We seem to be allowed to omit all the dots and brackets when we chain infix operators, but we’re unable to chain an infix operator to the result of a postfix operator unless the postfix used a dot to call. The postfix call at the end of a chain also seems to apply to the result of the previous infix call.

Note: By using the ‘dot’ in ["a".post in "b"] we are no longer using an ‘operator’. Explicit method calls are referred to as “function applications” in the reference

According to the Scala Reference:

• If there are several infix operations, operators with higher precedence bind more closely than operators with lower precedence
• If there are consecutive infix operations with operators of the same precedence, all operators must have the same associativity
• Postfix operators always have lower precedence than infix operators

While prefix operators aren’t mentioned, they appear to have a higher precedence than infix operators via experimentation:

scala> class MyObj(val v : String) {
     | def +(m : MyObj)  = {println("[" + v + ".in(" + m.v + ")]"); new MyObj(v + m.v) }
     | def unary_+ = {println("[" + v + ".pre()]"); this}
     | }
defined class MyObj

scala> implicit def stringToMyObj(s : String) : MyObj = new MyObj(s);
stringToMyObj: (s: String)MyObj

scala> +"a" + +"b"
[a.pre()]
[b.pre()]
[a.in(b)]
res21: MyObj = MyObj@144f1ada

The end result of this discussion is another compromise. After experimenting with various notations, I gave up and returned to my original plan of a “!” method on an object returned by “SEEZ”:

scala> class Printable(output:String) {
     | def ! = { print(output); }
     | def !! = { println(output); }
     | }
defined class Printable

scala> class OutputStream {
     | def SEEZ(output: String) : Printable = new Printable(output)
     | }
defined class Printable

scala> val U = new OutputStream
U: OutputStream = OutputStream@608d116e

scala> U SEEZ "Hello World"!
Hello World
scala> U SEEZ "Hello World"!!
Hello World

scala>

Stay tuned, next time we’ll attempt to put the pieces together and run HelloLol.scala

Why make a DSL?

I’ve heard it said that the real meat of learning Scala is done by people who want to expose a DSL. So far I’ve found learning Scala fairly easy. Sometimes I’m a little unclear on rules, like when I can leave out brackets and when I can’t, but when that happens I can just re-add the brackets and carry on.
 
The same flexibility isn’t quite so true for people who wish to create a good DSL. A great deal of effort goes into carefully constructing the syntax. Scala is such an incredibly extensible language that, with a little patience, you can create code that doesn’t look at all like “Scala”, and yet still compiles.
 
In order to get a taste of this learning, I decided I was going to attempt a DSL.
 
 

Starting into DSL Exploration

I decided my DSL of choice was going to be LOLCode.
 
Based on the (priceless) LolCats meme, LolCode presented itself as the perfect project to build a DSL around:

• Some rigidity in sticking close to the community consensus on syntax
• Enough flexibility to allow me to explore a few different approaches
• Different enough from Scala to present a challenge in implementation
• There’s a specific “feel” for the DSL we can aim for

 
 

Learned Lesson 1: Operator Notation

I decided to start where everyone starts, “Hello World”. System output is one of the most basic pieces of functionality for a language. The spec lists “VISIBLE” as the keyword for println:
VISIBLE "Hello World" // Prints hello world with a newline
VISIBLE "Goodbye"! // Prints goodbye, but with no newline

 
This seemed easy enough. We’ll start by ignoring the bang-syntax:
scala> def VISIBLE(text : String) = {println(text)}
VISIBLE: (text: String)Unit
 
scala> VISIBLE "Hello World"
:1: error: ';' expected but string literal found.
       VISIBLE "Hello World"

 
Uh-oh.
 
Just starting in, we’ve already ran into a lesson on operator syntax. There are three kinds of Operator Notation:

• Unary (prefix) Notation
  • Only works for +, -, !, and ~
  • Method must be no-args
  • Must omit the dot – ie: !.true would not work, but !true would
• Postfix Notation
  • Method must be no-args
  • May omit the dot and brackets if desired, but not required.
• Infix Notation
  • Method must be single-argument
  • May omit the dot and brackets if desired, but not required. ie: “a” + “b” or “a”.+(“b”)

 
In my case, “VISIBLE” would be a good Unary operator on some string-like object, but it is not a valid unary operator. For it to be an infix operator, I need some other “object” for it to be Infix of. Looks like I’ll have to compromise on this one.
 
Reading around the site, I discovered that while the spec listed “VISIBLE” as the “println” equivalent, “U SEEZ” was the community’s preference.
 
U SEEZ "Hello World"
U SEEZ "Hello World"!

 
This is convenient, as commenters on LolCode pointed out, as “U” can be seen as the I/O similar to Java’s “System.out”, and “SEEZ” is simply a “print” method on any output stream.
 
So let’s wrap up today’s post with one last crack at the non-bang notation:
scala> class OutputStream {
     | def SEEZ(output: String) = println(output);
     | }
defined class OutputStream
 
scala> val U = new OutputStream
U: OutputStream = OutputStream@4be03c55
 
scala> U SEEZ "Hello World"
Hello World

 
Worked like a charm. Stay tuned, as I continue to explore the process behind creating a DSL, and what it reveals about the language.

---



]]> http://codequirks.com/2011/05/09/scalearning-4-dsl-exploration-and-operator-notation/feed/ 0 criassk facebook twitter digg dzone stumbleupon reddit delicious facebook twitter digg dzone stumbleupon reddit delicious http://codequirks.com/2011/04/15/the-fallibility-of-tests/ http://codequirks.com/2011/04/15/the-fallibility-of-tests/#comments Fri, 15 Apr 2011 20:30:31 +0000 criassk http://codequirks.com/?p=226 Read more]]>


---

Recently I posted about Test Flow and Method Contracts. The key takeaway was that we can use tests to prove the contracts our system lives up to.

We all create a logical representation of our software in our minds in order to reason about it. Tests (either manual, or automatic) allow us to prove the rules of that representation to be true.

Today I’ll talk about what it means when there are mistakes in those tests. For the purposes of this discussion, “tests” may be manual or automatic, but examples are given in code for clarity.
 
 

Test Fallacy vs Test Absence

The first distinction I want to make is between incorrect tests, and missing tests.

Test Fallacy is when a test exists, but creates a logical fallacy within our reasoning system.

For example:

// {new Cache} r = cache.has(x) {r == false}
@Test
public void has_shouldReturnFalse_whenItemHasNotBeenAddedToCache() {
    //setup
    Cache c = new Cache();
 
    //execute
    boolean b = c.has("random");
 
    //assert
    assertTrue(b);
}


Clearly the code disagrees with the test name. Since the name of the test is more likely to correspond to the rule I have in my head that I believe the system follows, this test could be extremely destructive to my system.
 
 
Test Absence is when a test is absent, but we assume behaviour anyways. This is a logical mistake on our part, but it leads to bugs in the system.

I gave an example of this in my post Test Flow and Method Contracts where we “forgot” to define the behaviour of “has” when the cache is empty.

The problem with test absence is that we often assume the behaviour exists, and is “reasonable” by our own subjective measure. That creates a problem – what is “reasonable”?  
 

The Danger of Test Fallibility

While tests are fallible, either through a mistake or through absence, it’s useful to keep in mind why that is dangerous.

A professor for my logic class said something one class that stuck with me:

Logical fallacies don’t just create problems in logical systems, they destroy them. If you consider any false statement to be true, anything can be proven.

Let’s say, for example, that 2 = 3. I’ll prove I’m the Queen of England.

How? Well 3 – 2 = 1, but 3 – 3 = 0. Since 2 and 3 are equal, that implies 1 and 0 are equal.

Furthermore, 2 – 1 = 1, but 2 – 0 = 2. Since 1 and 0 are equal, so too are 1 and 2.

Now the Queen and I are, in fact, two distinct people. But 2 = 1, which means the Queen and I are one. So you see, I am in fact the Queen of England.
(Paraphrased from memory)

Honestly, I probably would have chosen to prove I was Metallica (the entire band), but if one admits that 2 = 3, the rest of the logic is difficult to argue against. With one false “fact”, we destroyed our entire reasoning system.

I’m sure many of us can relate this back to software. Try to recall a time when you received a bug complaint from a user, and after tracking down the bug you stared in awe at your computer screen. “That shouldn’t even be possible…”

The real problem? These “glitches” in our reasoning systems are inevitable. We’re all humans, we will make a mistake. One way or another, you will eventually have a bug in a test, or assume behaviour exists that really doesn’t.
 
 

How do we Address Test Fallibility?

This brings me back around to the original topic – how do we avoid this fallibility?

We automate our test suite.

It seems strange to say automation will “cure” all of our problems reasoning about systems, and that’s because it won’t. It’s simply the best way to combat the problem.

Automation addresses Test Absence by:

• being cheaper (time) than manual testing
• eliminating any reasons to skip tests
• growing as a regression suite over time as we think of new test cases

If you forget a test, simply add it and it will never be forgotten again.

Automation addresses Test Fallacies by:

• being repeatable
• being correctable (you can’t prevent humans from re-making mistakes)
• being easy to read, examine, and reason about

If you make a mistake in a test, simply fix it and the mistake will not reoccur.

In summary, automating your tests doesn’t guarantee that your tests will be good, but it allows you to confidently improve your tests over time. Good test allow you to reason correctly about your system.

---



]]> http://codequirks.com/2011/04/15/the-fallibility-of-tests/feed/ 0 criassk facebook twitter digg dzone stumbleupon reddit delicious facebook twitter digg dzone stumbleupon reddit delicious http://codequirks.com/2011/04/14/scalearning-3-is-scala-hard-to-learn/ http://codequirks.com/2011/04/14/scalearning-3-is-scala-hard-to-learn/#comments Thu, 14 Apr 2011 21:48:22 +0000 criassk http://codequirks.com/?p=199 Read more]]>


---

Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

Is Scala Hard to Learn?

If you want to skip the explanation, my answer is no.

To elaborate, I’ve heard a great deal of “buzz” about how complicated Scala is. I’ve heard it said that it will become an academic language, and that average programmers won’t be able to pick it up. Unfortunately, the blogosphere is so cluttered with rebuttals to the idea that it’s hard to find anyone actually critiquing Scala.

Some examples of pro-scala posts:

Is Scala Complex? (Martin Odersky)
Is Scala Too Complex? (Eric Daugherty)
Is Scala More Complicated than Java? (Vassil Dichev)

Some examples critiquing the complexity:

My Verdict on the Scala Language (Doug Pardee)
Is the Scala 2.8 Collections Library a Case of the Longest Suicide Note in History? (oxbow_lakes)

Since a lot has been said already, I won’t go into any in-depth tirade about software complexity. I will, however, draw on my experience:

For me, Scala has been easy to learn after a few years of Java because:

• Type system that’s similar to Java
• Java interoperability allows me to ease myself in
• It’s possible to write near-Java syntax, and have it be valid Scala
• Many Scala keywords have direct analogs in Java (ie: public final String s vs val s : String

 
 
So far I’ve done two “projects” in Scala.

One was a Lift (version 1.x) application that talked to a back-end set of Jersey services. The native XML handling was a dream-come-true, but the mapping of Lift 1.0 made me think Scala was much harder than it is.

The second project was an experimental conversion of a Spring MVC application using JSP views into a Lift 2.x website. I left the backing services in Java, used Spring to get the dependencies, but rewrote the front-end to be Lift. The mapping I hated in 1.x had been completely overhauled, and was actually very pleasant to work with.
 
 
That brings me to the take-away of this post.
 
 
A large part of the “complexity” people see in Scala isn’t the language. It’s the libraries and frameworks.

Scala, as a language, is very simple and very powerful. Overall, while I don’t know every feature, I feel as if I know enough to be able to understand most of the code I read, and colleagues familiar with Scala seem to be able to read the code I’m writing.

I find Scala very easy to learn. Some libraries have been a challenge… but then, so is learning the exact magic incantation for Java’s file IO.

---



]]> http://codequirks.com/2011/04/14/scalearning-3-is-scala-hard-to-learn/feed/ 5 criassk facebook twitter digg dzone stumbleupon reddit delicious facebook twitter digg dzone stumbleupon reddit delicious http://codequirks.com/2011/04/13/scalearning-2-reducing-string-join-to-a-line/ http://codequirks.com/2011/04/13/scalearning-2-reducing-string-join-to-a-line/#comments Wed, 13 Apr 2011 16:29:32 +0000 criassk http://codequirks.com/?p=177 Read more]]>


---

Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.

String Joining

I recently found myself writing a SQL generator that took a domain-specific query object and turned it into raw SQL. We had to write our own, as we needed to optimize several parts of the query, or face the rage of the customer when things were a little sluggish.

(It’s generally bad for business when the customer goes Hulk on you.)

As happens often in generating SQL, I found myself wanting to store a list and later convert it into a comma-separated string.

Java – First Attempt

StringBuilder result = new StringBuilder();
List items = Arrays.asList("a", "b", "c", "d");
boolean first = true;
for(String i : items) {
    if(first) first = false;
    else result.append(", ");
 
    result.append(i);
}
System.out.println(result.toString());


Sloppy. Three lines in the for loop, a local variable hanging around. It’s readable, but having to read 5-7 lines just to realize it’s a string-join seems excessive. Try again!

Java – Second Attempt

StringBuilder result = new StringBuilder();
List items = Arrays.asList("a", "b", "c", "d");
for(String i : items) {
    result.append(", ").append(i);
}
System.out.println(result.toString().replaceFirst(", ", ""));


Shorter, no local variable, but now we’re doing “unwork” to get rid of the leading comma. That’s an easy detail to miss, so fewer lines but not much more readable.

(I’m open to cleaner implementations! Comment with what you prefer.)

Scala

In Scala (and many other languages, ie: PHP) this is a simple one-liner.


val items = List("a", "b", "c", "d");
println( items.reduceLeft(_ + ", " + _) )
println( items.mkString(", ") )


Two ways, both one line. Both are readable, and I don’t have a real preference between them.

While readability and line count aren’t always so tied, the fewer lines a person has to read the faster they can read them. While I could easily write a method for Java to join Strings, this is just another example of how Scala seems to actively try to make my life easier.

---



]]> http://codequirks.com/2011/04/13/scalearning-2-reducing-string-join-to-a-line/feed/ 0 criassk facebook twitter digg dzone stumbleupon reddit delicious facebook twitter digg dzone stumbleupon reddit delicious