Sunday, December 30, 2012

A Issue with Using db4o in Scala Programs

In the other article I showed that we can use db4o in Scala programs. I also mentioned that there are some limitations: db4o cannot correctly store Scala objects with fields of the type scala.collection.mutable.Map and scala.collection.mutable.Set. This article discuss this issue in detail.

The Scala class Programmer (Listing-1) has a field named skillMap whose type is scalca.collection.mutable.Map. The map is used to record the programmer's skills and respective skill levels.

Listing-1 

package tutorial.db4o

class Programmer (val name: String, var phone: String) {
    private val skillMap = scala.collection.mutable.Map[String, Int]()
    
    def skills = skillMap.keySet
    
    def hasSkill(skill : String) = skills.contains(skill)
    
    def skillLevel(skill: String) : Option[Int] = {
        if (hasSkill(skill)) Some(skillMap(skill))
        else None
    }
    
    def addSkill(skill: String, level: Int) {skillMap(skill) = level}
}

In the program in Listing-2, we create an instance of the Programmer class and store it in the db4o database. In the program in Listin-3, we retrieve the object stored in db4o, and print out the programer's name, phone number, skills and corresponding levels. However, the execution runs into an exception.

Listing-2 - A Program to store an object into db4o

package tutorial.db4o

import com.db4o.Db4oEmbedded
import Adapter._

object AddProgrammers extends App {
    // a connection to the tutorial (db4o) database
    val connection = Db4oEmbedded.openFile("tutorial.db4o")
    
    // create a Programmer instance and store it in the database
    
    val steve = new Programmer("Steve", "513-206-3276")
    steve.addSkill("Java", 6)
    steve.addSkill("Perl", 3)
    connection.store(steve)
    
    connection.commit()
    
    connection.close()
}

Listing-3 - A Program to retrieve an object from db4o

package tutorial.db4o

import com.db4o.Db4oEmbedded
import Adapter._
import com.db4o.query.Predicate

object ReadProgrammers extends App {
    // a connection to the tutorial (db4o) database
    val connection = Db4oEmbedded.openFile("tutorial.db4o")
    
    // Retrieve programmers with the name Steve from the database
    val rs = connection.query{programmer: Programmer => programmer.name == "Steve"}
        
    // Print out each programmer's name, phone number, and skills
    rs.foreach{programmer =>
        println("Programmer name: %s, phone number: %s, skills:".format(programmer.name, programmer.phone))
        programmer.skills.foreach{skill => println("    %s at level %d".format(skill, programmer.skillLevel(skill).get))}
    }
    
    connection.close()
}

Output of execution of the program in Listing-3:

Programmer name: Steve, phone number: 513-206-3276, skills: Exception in thread "main" java.lang.NullPointerException at scala.collection.mutable.HashTable$$anon$1.(HashTable.scala:159) at scala.collection.mutable.HashTable$class.entriesIterator(HashTable.scala:157) at scala.collection.mutable.HashMap.entriesIterator(HashMap.scala:45) at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:190) at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:45) at scala.collection.mutable.HashMap$$anon$1.foreach(HashMap.scala:99) at tutorial.db4o.ReadProgrammers$$anonfun$2.apply(ReadProgrammers.scala:17) at tutorial.db4o.ReadProgrammers$$anonfun$2.apply(ReadProgrammers.scala:15) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) at scala.collection.immutable.List.foreach(List.scala:76) at tutorial.db4o.ReadProgrammers$delayedInit$body.apply(ReadProgrammers.scala:15) at scala.Function0$class.apply$mcV$sp(Function0.scala:34) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App$$anonfun$main$1.apply(App.scala:60) at scala.App$$anonfun$main$1.apply(App.scala:60) at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59) at scala.collection.immutable.List.foreach(List.scala:76) at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:30) at scala.App$class.main(App.scala:60) at tutorial.db4o.ReadProgrammers$.main(ReadProgrammers.scala:7) at tutorial.db4o.ReadProgrammers.main(ReadProgrammers.scala)

Clearly, the Programmer object was not stored correctly in db4o. If we replace the Scala Map by a Java Map (e.g. java.util.HashMap), the programs will work correctly. It is however a awkward work-around to use Java map instead of Scala map in a Scala program.

Sunday, December 23, 2012

Using db4o in Scala Programs

This is a tutorial on using db4o in Scala programs. db4o is an object database management system with both Java and .NET APIs. We can also use db4o in Scala programs (with some limitations) as shown in this tutorial. Scala has many advantages over Java, as also shown in this tutorial. The readers of this tutorial are not assumed to have any knowledge about db4o. However, readers who have experience with Java programs using db4o can appreciate the elegance of Scala over Java more. The readers are expected to have basic knowledge about both Java and Scala programming.

It is a good practice to manage library dependency using Maven. (If you use Eclipse, and are interested in using Maven to manage classpath in Eclipse, you may be interested in this article of mine on this subject.) To manage db4o library dependency using Maven, include the snapshot repository below in the pom.xml file:

<repository>
    <snapshots>
        <enabled>true</enabled>
        <updatePolicy>never</updatePolicy>
        <checksumPolicy>fail</checksumPolicy>
    </snapshots>
    <id>db4o</id>
    <name>DB4O</name>
    <url>http://source.db4o.com/maven</url>
    <layout>default</layout>
</repository>

Also include the following dependency on the db4o-full-java5.jar in the pom.xml:

<dependency>
    <groupId>com.db4o</groupId>
    <artifactId>db4o-full-java5</artifactId>
    <version>8.0-SNAPSHOT</version>
</dependency>

(This tutorial includes both Java and Scala programs. If you use Eclipse and want help on working with Eclipse projects mixing Java and Scala, you may be interested in this article of mine on this subject.)

A Java Program Using db4o 

This tutorial starts with a Java program using db4o, to highlight the advantages of Scala. In Listing 2, the Java program opens a db4o database and retrieves Programmer objects with name "Joe" from it. The Java class Programmer is in Listing 1. A programmer has a name and a phone number. The name is not mutable and the phone number is. The query is awkward because a Java method cannot take another method as parameter so we have to wrap the match method in an anonymous class extending the  Predicate class (Line 12 - 17). The shortcoming can be overcome in Scala programs since Scala is also a functional programming language and can take a function as function parameter.


Listing 1 - The Programmer Java Class 

package tutorial.db4o.java;

public class Programmer {
    private String name;
    public String phone;
    
    public Programmer(String name, String phone) {
        this.name = name;
        this.phone = phone;
    }
    
    public String getName() {
        return name;
    }
}

Listing 2 - A Java Program Using db4o 

package tutorial.db4o.java;

import com.db4o.Db4oEmbedded;
import com.db4o.ObjectContainer;
import com.db4o.ObjectSet;
import com.db4o.query.Predicate;

public class ReadProgrammer {
    public static void main(String[] args) {
        ObjectContainer connection = Db4oEmbedded.openFile("tutorial.db4o");
        
        ObjectSet<Programmer> rs = connection.query(new Predicate<Programmer>() {
            @Override
            public boolean match(Programmer programmer) {
                return programmer.getName().equals("Joe");
            }
        });

    }
}

Adding Scala Objects into db4o 

Now let's look into the Scala world. In Listing 3 is the Scala counterpart of the Java Programmer class. An instance of the Scala class Programmer also has a immutable name and a mutable phone number. It is obvious that the Scala version is much simpler.

Listing 3 - The Scala Class Programmer 

package tutorial.db4o

class Programmer (val name: String, var phone: String)

In Listing 4 is a Scala program that creates three instances of the Programmer class and adds them into the db4o database. A Java version will be similar. There is no advantage in Scala programs over their Java counterparts in regard to storing objects into a db4o database, and deleting them from it.

Listing 4 - Add Objects into a db4o Database
 
package tutorial.db4o

import com.db4o.Db4oEmbedded
import Adapter._

object AddProgrammers extends App {
    // a connection to the tutorial (DB4o) database
    val connection = Db4oEmbedded.openFile("tutorial.db4o")
    
    // create three Programmer instances and store them in the database
    connection.store(new Programmer("Steve", "513-206-3276"))
    connection.store(new Programmer("Bob", "513-376-2521"))
    connection.store(new Programmer("Joe", "513-536-8093"))
    connection.commit()
    
    connection.close()
}

Retrieving Objects from db4o

In Listing 5 is a Scala program that retrieves all programmer instances in the db4o database and prints out their name and phone number. Contrast this program with the one in Listing 2, we can see that the query in the Scala version (Line 11) is much simpler than their Java counterpart (Line 12 - 17). The query method in the Scala version takes an anonymous function as its parameter. The anonymous function, on its turn, takes a Programmer object as its parameter and returns a Boolean value. If it returns true, the programmer object will be included in the query results, or excluded from it if it returns false. In Listing 5, the anonymous function always returns true regardless the programmer object passed in since we want to retrieve all programmer objects in the db4o database.

Listing 5 - Retrieve Objects from a db4o Database 
 
package tutorial.db4o

import com.db4o.Db4oEmbedded
import Adapter._

object ReadProgrammers extends App {
    // a connection to the tutorial (db4o) database
    val connection = Db4oEmbedded.openFile("tutorial.db4o")
    
    // Retrieve all programmers from the database
    val rs = connection.query{programmer: Programmer => true}
    
    // Print out each programmer's name and phone number
    rs.foreach{programmer =>
        println("Programmer name: %s, phone number: %s".format(programmer.name, programmer.phone))
    }
    
    connection.close()
}

If we run the program in Listing 5 without running the program in Listing 4 before, it prints nothing, because there is no programmer object in the database. If we run the program in Listing 4, then run the program in Listing 5, it will print:

Programmer name: Joe, phone number: 513-536-8093
Programmer name: Bob, phone number: 513-376-2521
Programmer name: Steve, phone number: 513-206-3276

Updating Objects in db4o

In Listing 6 is a Scala program that at first retrieves programmer objects with name "Joe" from the db4o database, and changes their phone number from whatever to 513-111-1111.  

Listing 6 - Update Objects in a db4o Database

package tutorial.db4o

import com.db4o.Db4oEmbedded
import Adapter._

object UpdateProgrammers extends App {
    // a connection to the tutorial (db4o) database
    val connection = Db4oEmbedded.openFile("tutorial.db4o")
    
    // Retrieve programmers with name "Joe" from the database
    val rs = connection.query{programmer: Programmer => programmer.name == "Joe"}
    
    // Change the programs' phone number
    rs.foreach{programmer =>
        programmer.phone = "513-111-1111"
        connection.store(programmer)
    }
    connection.commit()
    
    connection.close()
}
After we run this program, if we run the program in Listing 5 again, the output will be:

Programmer name: Joe, phone number: 513-111-1111 
Programmer name: Bob, phone number: 513-376-2521 
Programmer name: Steve, phone number: 513-206-3276 

We can see phone number of Joe has been updated.

In db4o, the method to store a brand new object and the method to store a updated object is exactly the same one.

Deleting Objects from db4o 

In Listing 7 is a Scala program that deletes every programmer objects in the database.

Listing 7 - Delete Objects from a db4o Database 

package tutorial.db4o

import com.db4o.Db4oEmbedded
import Adapter._

object DeleteProgrammers extends App {
    // a connection to the tutorial (db4o) database
    val connection = Db4oEmbedded.openFile("tutorial.db4o")
    
    // Retrieve all programmers from the database
    val rs = connection.query{programmer: Programmer => true}
    
    // delete every programmers
    rs.foreach{programmer =>
        connection.delete(programmer)
    }
    connection.commit()
    
    connection.close()
}

The Adapter

The magic to simplify the query method is an adapter class shown in Listing 8.  It is this class that has the query method that takes a predicate (i.e. a function returning a Boolean value) rather than an anonymous class extending the Predicate class. In the company object we also defined a implicit method that converts (i.e. wraps) an ObjectContainer into an Adapter object. With this implicit method, and a import statement import Adapter._, we will be able to call the new version of query method on an ObjectContainer object, as shown in Listing 5, 6, 7 (The compiler will do the conversion behind the scene). These are the real power of the Scala programming language.

Listing 8 - An Adapter Class

package tutorial.db4o

import com.db4o.ObjectContainer
import com.db4o.query.Predicate

class Adapter (connection: ObjectContainer) {
    def query[T](predicate: T => Boolean) : List[T] = {
        var results : List[T] = List[T]()
        val objectSet = connection.query(new Predicate[T]() {
            override
            def `match`(entry: T) = {
                predicate(entry)
            }
        });
        
        while (objectSet.hasNext()) {
            results = objectSet.next :: results
        }
        
        results
    }
}

object Adapter {
    implicit def objectContainer2Adapter(connection: ObjectContainer) = new Adapter(connection)
}

Conclusion 

We can use db4o in Scala programs. The functional programming nature of Scala greatly simplifies the query API. There are, however, some limitations on storing Scala objects in db4o databases. My experience shows that db4o does not support Scala collections. In other words, if a Scala class has Scala Map and Set as fields, db4o will not be able to store them correctly. I am going to discuss more details in this regard in my next post on this subject.