The Problem

JJTree is a part of JavaCC is a parser/scanner generator for Java. JJTree is a preprocessor for JavaCC that inserts parse tree building actions at various places in the JavaCC source. To follow along you need to understand the core concepts of parsing. Also review basic JJTree documentation and samples provided in JavaCC distribution (version 4.0).

JJTree is magically powerful, but it is as complex. We used it quite successfully at my startup www.moola.com. After some the basic research into the grammar rules, lookaheads, node annotations and prototyping I felt quite comfortable with the tool. However, just recently when I had to use JJTree again I hit the same steep learning curve as if I have never seen JJTree before.

How to write a tutorial that gets you back in shape quickly without forcing the full relearning?

The Solution

Here I capture my notes in a specific form that I do not have to face that same learning curve again in the future. You can think my approach as layered improvement to a grammar that follows these steps:

• get lexer
• complete grammar
• optimize produced AST
• define custom node
• define actions
• write evaluator

I always start simple and need to go more complex - this is exactly how I will document it. In each example I start with a trivial portion of grammar and then add some more to it to force specific behavior. New code is always in green. Let's hope this save all of us the relearning.

Reorder tokens from more specific to less specific

The token in TOKEN section can be declared in any order. But you have to pay very close attention to the order because the matching of tokens starts from the top and down the list until first matching token is found. For example notice how "interface" or "exception" are defined before STRING_LITERAL. If we had defined "interface" after STRING_LITERAL "interface" would never get matched,  STRING_LITERAL would. 

TOKEN : {
	  <INTERFACE: "interface" >
	| < EXCEPTION: "exception" >
	| < ENUM: "enum" >
	| < STRUCT: "struct" >
	
	| < STRING_LITERAL: "'" (~["'","\n","\r"])* "'" >
	| < TERM: <LETTER> (<LETTER>|<DIGIT>)* >
	
	| < NUMBER: <INTEGER> | <FLOAT> > 
	| < INTEGER: ["0"-"9"] (["0"-"9"])* >
	| < FLOAT: (["0"-"9"])+ "." (["0"-"9"])* >
	| < DIGIT: ["0"-"9"] >
	| < LETTER: ["_","a"-"z","A"-"Z"] >
} 

The ordering is the same reason why we can't just use "interface" inline in the definition of productions. The STRING_LITERAL will always match first.

Remove some nodes from final AST

Some nodes do not have any special meaning and should be excluded from the final AST.  This is done by using #void like this:

void InterfaceDecl() #void : {
}{ 
	ExceptionClause()
	| 
	EnumClause()
	|
	StructClause()
	|
	MethodDecl()
}

Add action to a production

You will definitely need to add actions to the production for your parser to be useful. Here I capture the text of the current token (t.image) and put it into jjThis node that will resolve to my custom node class TypeDecl. You bind a variable "t" to a token using "="; the action itself is in curly braces right after the production and can refer to current token as "t" and current AST node as "jjtThis".

void TypeDecl() : {
	Token t;
}
{
	<VOID>
	|
	t=<TERM> { jjtThis.name = t.image; } ("[]")?}
}

Here I further set isArray property to true only if "[]" is found after the <TERM>:

void TypeDecl() : {
	Token t;
}
{
	<VOID>
	|
	t=<TERM> { jjtThis.name = t.image; } ("[]" { jjtThis.isArray = true; } )?}
}

Multiple actions inside one production rule

Just as we have seen earlier you can access values of multiple token in one production rule. Notice how I declare two separate tokens "t" and "n". Here:

void ConstDecl() : {
	Token t;
	Token n;
}
{
	LOOKAHEAD(2)
	
	t=<TERM> { jjtThis.name = t.image; } "=" n=<NUMBER> { jjtThis.value = Integer.valueOf(n.image); }
	|
	<TERM>
}

Lookaheads

There are certain points in complex  grammars that might not get parsed unambiguously using just one token look ahead. If you are writing high performance parser you might need to rewrite grammar. But if do not care about performance you can force lookahead for more that one symbol.

JJTree generator will give you a warning about ambiguities. Go the the rule it refers to and set lookahead of 2 or more like this:

void EnumDeclItem() : {}
{
	LOOKAHEAD(2)
	
	<TERM> "=" <NUMBER>
	|
	<TERM>
}

Node return values

It is possible to return nodes from the productions, just like function return values. Here I am declaring the ASTTypeDecl will be returned.

ASTTypeDecl TypeDecl() : {
	Token t;
}
{
	<VOID>
	|
	t=<TERM> { jjtThis.name = t.image; } ("[]" { jjtThis.isArray = true; } )?}

	{ return jjtThis; }
}

Once you start having a lot of expressions in one production it is better to group them together so return statement applies to all of them. The above example will actually result in a bug due to a fact that the return statement is attached to one branch of "|" production and not to both branches. We can easily fix the issue using parenthesis to force order of precendence:

ASTTypeDecl TypeDecl() : {
	Token t;
}
{
	(
		<VOID>
		|
		t=<TERM> { jjtThis.name = t.image; } ("[]" { jjtThis.isArray = true; } )?}
	)

	{ return jjtThis; }
}

Build abstract syntax tree as you go

After you have all production return values you can build AST tree on the fly while parsing. Just provide found overloaded add() methods in the ASTInterfaceDecl class and call them like this:

void InterfaceDecl() #void : {
	ASTExceptionClause ex;
	ASTEnumClause en;
	ASTStructClause st;
	ASTMethodDecl me;
}
	ex=ExceptionClause() { jjtThis.add(ex); }
	| 
	en=EnumClause() { jjtThis.add(en); }
	|
	st=StructClause() { jjtThis.add(st); }
	|
	me=MethodDecl() { jjtThis.add(me); }
}

Use <EOF>

Quite often you can get your grammar written and start celebration when you notice that part of the file is not being parsed... This happens because you did not tell the parser to read all content till the end of file and it feels free to stop parsing at will. Force parsing to reach end of file by demanding <EOF> token at the top most production:

void InterfaceDecl() #void : {
}{ 
	ExceptionClause()
	| 
	EnumClause()
	|
	StructClause()
	|
	MethodDecl()
	|
	<EOF>
}

The Final Word

JJTree works incredibly well. No excuse to regex parsing no more... Don't even try to convince me!

Drop me a line if you need help with JJTree - will be glad to share the experiences with you.

References

1. The JavaCC FAQ by Theodore S. Norvell

The highlights are:

• Unit testing database code is a bit of a funny problem. Most developers can pretty easily get their heads around unit testing a piece of Java code using interfaces and mock objects. When it comes to database code or DAOs, it suddenly becomes particularly difficult.
• Most large development projects go like this: The database guys update the schema. The developers write the code. The developers need a particular data set to exercise the various use cases so they add it to the schema. It all becomes a bit messy.
• The database schema generally is not version controlled, as it is constantly being redefined using DDL statements run by the DBAs.
• Bob [the developer working next to you] didn’t actually run your test because he was too busy with his own and the test suite isn’t clean anyway because everyone is falling over each other.
• In Ruby on Rails a developer’s workspace has multiple environments by default - development, test and production. You develop against the development schema. When you run unit tests the schema from development is copied into the test database with no data in it. The framework imports version controlled sets of test data. Whenever a test is run, it is guaranteed that the database will be in this state. The tear down step cleans out your changes.
• You need multiple database schemas [and consequently - database instances] in order to unit test your db code. Pause and re-read that line. It’s not negotiable. Probably two per developer. One with sample data to use while you work on the user interface. The other, a temporary one for unit testing. A whole development team using the one schema does not work. Most projects do it, but that doesn’t mean that it’s a good idea.
• The full DDL for the database is kept in version control. After each change, the full database DDL is dumped and checked in. No UPDATE TABLE statements. Ever. This way you are guaranteed that if you ever want to get a baseline of your system, you can also rebuild the database as it existed at this time. I worked on a very large telecoms project with a huge development team, and this worked.
• The test data for your environments is stored in version control - at the very least, as dumps of insert statements.
• To test your database code, refresh your test schema [and test data] with the one from version control. Now run your test cases. Gorgeous! No tripping over other people, and your tests are guaranteed to work the same each time.
• So why is unit testing databases so difficult if it doesn’t have to be? Most of the time it involves process change and getting out of bad habits, not just a tool. And change means convincing people. Generally, managers do not understand what benefit there is in multiple database schemas, as it is seen to increase complexity and therefore risk, and DBAs like to have full control over what is going on on their servers. The topic of databases and processes is also a great one for religious zeal.

He did not address one issue that I hear a lot: how will we test the production bugs if we don't have the production data. The answer is quite simple - you connect to a snapshot of the production database and propose a fix. After bug is fixed, you throw the snapshot away and go back to mock databases. You can even capture the test case against the bug by creating new mock database that has just enough data to reproduce the bug.

Another common problem that is harder to solve is about the test cases with large number of database records. How do you test the speed of the SELECT or JOIN statement in a table with 100 million rows? One can import up to 100,000 records using bulk insert reasonably fast, but not millions. This problem may be here to stay.

The Context

I am going over various aspects of Scala programming language. The basic language features are very attractive so is it's ability to natively interop with all of my other Java code. In couple of talks and presentations I felt a vibe that Scala Actors are magical things that make concurrency easy. Some of the statements went as far as to say that Scala Actors can help us to easily take advantage of multi core processors that are coming in the near future.

Is this true? What kind of exciting stuff can be done with Scala Actors? Is there something that I can get done in Scala that I can't get done in Java? I will talk about all of it here. You will also learn how Doug Lea became a grandfather of all Scala Actors. As a supplement to this discussion you can download a complete Eclipse project that has both Scala and Java code discussed.

Ping Pong Messaging in Scala

As a newbie to Scala I have looked around for the examples of Actors. One clear illustrative Scala Actors tutorial was finally found credits to Philipp Haller (who is the contributor to scala.actors package itself). There are basically two actors Ping and Pong that exchange messages. The Ping Pong messaging example in Scala almost exactly as Philipp Haller is provided in Appendix A of this post. This example is quite self explanatory and shows how Scala Actors are typically used. Please read the entire original post for more details on how Scala Actors work if you have trouble following this abridged example.

The only thing I would change is to declare the message classes directly and  separately from Actors. Currently it reads "pong ! Ping", so we are sending an object Ping to an object pong, but the object Ping is the Actor in itself... So we are in fact sending Actors to other Actors as messages, which feels weird to me. It helps me personally (and hope some of you too) to separate Actors from messages and simply say "pong ! PingMsg".

Ping Pong Messaging in Java

I found that re-implementing something in the language  I already know helps me to get a good feel for the real issues. Can the same be done in  Java? Sure. Yes, there is much more verbosity (30% more lines of code). Yes, elegance is lost. Yes, it is not cool. However, it is quite possible. In about 1 hours I was able to reproduce quite similar Java Actors implementation shown in Appendix B.

To complete the proof of concept I had to build couple of supporting classes. There are base classes needed: the Actor class and the Message class. The complete runtime, consisting of one thread rather than a full blown connection pool, is implemented in the MessageBus class. In Java example you see the explicit calls to the message bus initialization; the same exists inside Scala Actors code, but invoked implicitly. You can also see how I declare message classes directly.

Scala Actors use Doug Lea's Fork and Join Tasks (FJT)

Originally written by Doug Lea and released into the public domain FJT library was developed as early as 1999 - almost 10 yeas ago! The introduction to FJT package can be found here. FJT classes are not parts of Actors, but they are distributed all along in the package scala.actors of scala-library.jar. Why are they there?

It turns out that Scala Actors execute message passing using FJT classes written by Doug Lea. This is what is going on when you try to send a message to Actor asynchronously using "pong ! Ping":

• Actor class (Scala) has new operator defined "!" that allows asynchronous message passing to this actor instance
  

  	def !(msg: Any) {
      		send(msg, Actor.self)
    	}	
    	

• Actor class (Scala) defines send() method that passes a message and the continuation (aka future, aka completion point as I like to call it) into the Scheduler (Scala)
  

  	def send(msg: Any, replyTo: OutputChannel[Any]) = synchronized {
  	    ...
  	    Scheduler.execute(new Reaction(this, continuation, msg))
  	    ...
  	}
  	

• The Scheduler class (Scala) queues the tasks and calls FJTaskScheduler2 class (Scala) that further uses FJTaskRunnerGroup class (Java) to get some threads and execute the task
• Control returns to the client and there seems to be no way to wait for the reply

This is all that happens from the sender perspective. Quite simple conceptually, but syntactically elegant ("pong ! Ping") due to Scala's partial, closure and lambda function capabilities.

Now the receiver (or the Actor) perspective. There two different ways to process messages when you are an Actor: the receive() and the react(). When receive() is called the message is pulled out of the queue and execute in the caller thread. When react() is called the message is pulled out of the queue and given to the above scheduler for execution by one of the FJT threads.

There is one (hopefully just one) thing that I don't know. Can a sender wait for the response to a message sent asynchronously via "pong ! Ping"?

The Final Word

This quick look into Scala Actors is very illustrative. Here are my personal takeaways:

• 1000 lines of code can change the world
• source code is there - go and read it when in doubt on how things work
• pay attention to fundamentals; threading is not to be taken lightly; read Doug Lea's book if you have anything to do with threading
• remember that Scala does not use magic, but relies on conventional good software engineering; rational software engineering can be conducted in any language
• you can't make "pong ! Ping" work in Java; Scala can definitely do DSL's or at least significantly reduce verbosity over Java
• Scala is Java

This is not all. There are concerning comments that I have to make now.

First of all, personally, I would like to see a cleaner separation between Scala as language (and a compiler) and Scala as collection of libraries. The Actors in Scala are really not a part of the core Scala language, but rather a library and it would be better to provide scala-actors.jar and scala-lang.jar as separately versioned artifacts.

Secondly, if you remember Win32 API you realize that asynchronous messaging is not new. The whole Windows GUI subsystem is based on message passing. Just go to MSND and search for postMessage or peekMessage if you need a refresher. I had to write a lot of Win 32 API code back in 1997. I still have grey hairs from the thread affinity of HWND handles, MsgWaitForMultipleObjectsEx and other things of that nature. All of it makes me think that Scala Actors are not sufficient for enterprise scale asynchronous messaging. Once again Scala has a good set of language features and very good performance. But the libraries that come with Scala, including Actors, have to be reviewed carefully to ensure that they meet your enterprise level of you quality & maturity expectations. For some things I will still pick Doug Lea's 1000 lines of century old code...

Appendix A. Source code for PingPongScala.scala

import scala.actors.Actor;
import scala.actors.Actor._;

package com.oy.shared.scample.scala {

	case object Ping
	case object Pong
	case object Stop
	
	class Ping(count: int, pong: Actor) extends Actor {
	  def act() {
	    var pingsLeft = count - 1
	    pong ! Ping
	    while (true) {
	      receive {
	        case Pong =>
	          if (pingsLeft % 1000 == 0)
	            Console.println("Ping: pong")
	          if (pingsLeft > 0) {
	            pong ! Ping
	            pingsLeft -= 1
	          } else {
	            Console.println("Ping: stop")
	            pong ! Stop
	            exit()
	          }
	      }
	    }
	  }
	}
	
	class Pong extends Actor {
	  def act() {
	    var pongCount = 0
	    while (true) {
	      receive {
	        case Ping =>
	          if (pongCount % 1000 == 0)
	            Console.println("Pong: ping "+pongCount)
	          sender ! Pong
	          pongCount = pongCount + 1
	        case Stop =>
	          Console.println("Pong: stop")
	          exit()
	      }
	    }
	  }
	}
	  
	object PingPongScala extends Application {
		def run(){
		  val pong = new Pong
		  val ping = new Ping(100000, pong)
		  ping.start
		  pong.start
		}
	}

}

Appendix B. Source code for PingPongJava.java

package com.oy.shared.scample.java;

import com.oy.shared.scample.actor.Actor;
import com.oy.shared.scample.actor.Message;
import com.oy.shared.scample.actor.MessageBus;

class PingMsg extends Message {
	final Ping sender;
	public PingMsg(Ping sender){
		this.sender = sender;
	}
}

class PongMsg extends Message {
	final Pong sender;
	public PongMsg(Pong sender){
		this.sender = sender;
	}
}

class StopMsg extends Message {
	
}	

class Ping extends Actor {
	private Pong pong;
	private int pingsLeft;
	
	public Ping(int count, Pong pong) {
		this.pong = pong;
		this.pingsLeft = count - 1;
	}
	
	public void start(){
		super.start();
		
		PingMsg pingMsg = new PingMsg(this);
		pong.postMessage(pingMsg);
	}
	
	public void react(Message msg) {
		if (msg instanceof PongMsg) {
			if (pingsLeft % 1000 == 0) {
				System.out.println("Ping: pong");
			}
			if (pingsLeft > 0) { 
				PingMsg pingMsg = new PingMsg(this);
				pong.postMessage(pingMsg);
				pingsLeft -= 1;
			} else {
				System.out.println("Ping: stop");
				pong.postMessage(new StopMsg());
				stop();
			}
		}
	}
}

class Pong extends Actor {
	private int pongCount;
	
	public void react(Message msg){
		if (msg instanceof PingMsg){
			PingMsg pingMsg = (PingMsg) msg;
			if (pongCount % 1000 == 0) { 
	            		System.out.println("Pong: ping " + pongCount);
			}
			
			PongMsg pongMsg = new PongMsg(this);
			pingMsg.sender.postMessage(pongMsg);
	        	pongCount = pongCount + 1;
	        
	        	return;
		}
		
		if (msg instanceof StopMsg){
			System.out.println("Pong: stop");
			stop();
			
			return;
		}
	}
}

public class PingPongJava {
	public static void run(){
		MessageBus mbus = new MessageBus();
		mbus.start();
		
		Pong pong = new Pong();
		Ping ping = new Ping(100000, pong);
		ping.start();
		pong.start();
		    
		mbus.stopWhenActorsFinish();
	}
}

The system encountered an error while trying to process your request. If you would like to help us out in figuring out the problem, please report the problem using our Report Problem Form.

org.springframework.dao.DataIntegrityViolationException: SqlMapClient operation; SQL [];   
--- The error occurred in UserProfile.xml.  
--- The error occurred while applying a parameter map.  
--- Check the addProfile-InlineParameterMap.  
--- Check the statement (update failed).  
--- Cause: com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: 
Duplicate entry '183531' for key 1; nested exception is com.ibatis.common.jdbc.exception.NestedSQLException:   
--- The error occurred in UserProfile.xml.  
--- The error occurred while applying a parameter map.  
--- Check the addProfile-InlineParameterMap.  
--- Check the statement (update failed).  
--- Cause: com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '183531' for key 1
	at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.
translate(SQLErrorCodeSQLExceptionTranslator.java:292)
	at org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:212)
	at org.springframework.orm.ibatis.SqlMapClientTemplate.insert(SqlMapClientTemplate.java:397)
	at com.dzone.user.dao.ibatis.IBatisUserProfileDAO.addProfile(IBatisUserProfileDAO.java:55)
	at sun.reflect.GeneratedMethodAccessor720.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:301)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
	at $Proxy22.addProfile(Unknown Source)
	at com.dzone.user.managers.UserManager.addProfile(UserManager.java:66)
	at sun.reflect.GeneratedMethodAccessor687.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:301)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
	at $Proxy23.addProfile(Unknown Source)
	at sun.reflect.GeneratedMethodAccessor687.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:301)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:198)
	at $Proxy24.addProfile(Unknown Source)
	at com.dzone.user.acegi.DrupalUserDetailsProvider.createRemoteUserAndProfile(DrupalUserDetailsProvider.java:94)
	at com.dzone.user.controllers.RegisterController.onSubmit(RegisterController.java:98)
	at org.springframework.web.servlet.mvc.SimpleFormController.processFormSubmission(SimpleFormController.java:267)
	at org.springframework.web.servlet.mvc.CancellableFormController.processFormSubmission(CancellableFormController.java:140)
	at org.springframework.web.servlet.mvc.AbstractFormController.handleRequestInternal(AbstractFormController.java:265)
	at org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
	at sun.reflect.GeneratedMethodAccessor274.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:301)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
	at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:50)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy29.handleRequest(Unknown Source)
	at org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:874)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:808)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:476)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:441)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:153)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:91)
	at com.caucho.server.dispatch.ServletFilterChain.doFilter(ServletFilterChain.java:103)
	at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:39)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at com.dzone.utils.filters.WhosOnlineFilter.doFilter(WhosOnlineFilter.java:148)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:738)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:265)
	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:107)
	at org.acegisecurity.intercept.web.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:72)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
	at org.acegisecurity.ui.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:110)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:229)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:286)
	at org.acegisecurity.util.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:275)
	at org.acegisecurity.util.FilterChainProxy.doFilter(FilterChainProxy.java:149)
	at org.acegisecurity.util.FilterToBeanProxy.doFilter(FilterToBeanProxy.java:98)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at com.dzone.utils.filters.DZThemeOverrideFilter.doFilter(DZThemeOverrideFilter.java:46)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at com.dzone.utils.filters.DZThemeFilter.doFilter(DZThemeFilter.java:42)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at com.dzone.utils.filters.DZCharacterEncodingFilter.doFilterInternal(DZCharacterEncodingFilter.java:77)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:75)
	at com.caucho.server.dispatch.FilterFilterChain.doFilter(FilterFilterChain.java:87)
	at com.caucho.server.webapp.WebAppFilterChain.doFilter(WebAppFilterChain.java:181)
	at com.caucho.server.dispatch.ServletInvocation.service(ServletInvocation.java:266)
	at com.caucho.server.hmux.HmuxRequest.handleRequest(HmuxRequest.java:435)
	at com.caucho.server.port.TcpConnection.run(TcpConnection.java:602)
	at com.caucho.util.ThreadPool$Item.runTasks(ThreadPool.java:690)
	at com.caucho.util.ThreadPool$Item.run(ThreadPool.java:612)
	at java.lang.Thread.run(Thread.java:619)
Caused by: com.ibatis.common.jdbc.exception.NestedSQLException:   
--- The error occurred in UserProfile.xml.  
--- The error occurred while applying a parameter map.  
--- Check the addProfile-InlineParameterMap.  
--- Check the statement (update failed).  
--- Cause: com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '183531' for key 1
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeUpdate(GeneralStatement.java:91)
	at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.insert(SqlMapExecutorDelegate.java:447)
	at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.insert(SqlMapSessionImpl.java:82)
	at org.springframework.orm.ibatis.SqlMapClientTemplate$9.doInSqlMapClient(SqlMapClientTemplate.java:399)
	at org.springframework.orm.ibatis.SqlMapClientTemplate.execute(SqlMapClientTemplate.java:209)
	... 82 more
Caused by: com.mysql.jdbc.exceptions.MySQLIntegrityConstraintViolationException: Duplicate entry '183531' for key 1
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931)
	at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2934)
	at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1616)
	at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1708)
	at com.mysql.jdbc.Connection.execSQL(Connection.java:3255)
	at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1293)
	at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:867)
	at org.apache.commons.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:168)
	at com.ibatis.sqlmap.engine.execution.SqlExecutor.executeUpdate(SqlExecutor.java:81)
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.sqlExecuteUpdate(GeneralStatement.java:200)
	at com.ibatis.sqlmap.engine.mapping.statement.GeneralStatement.executeUpdate(GeneralStatement.java:78)
	... 86 more

The Context

I am researching Scala because it might just be the next big things. My specific interests are in using Scala to replace the aging parts of an existing large Java systems with their Scala alternative and hopefully more elegant implementations.

There are not enough examples out there today on interoperability with Java and we need more good examples. So I decided to write some classical Java code using Scala. For this exercise I implement Servlet Filter (javax.servlet.Filter) in both Java and Scala side by side. The dozen lines of code below is sufficient to fully Scala enable your favorite Java application server! This is not a rocket science, but the complete example never hurts. It helped me to get my feet wet. I hope it shows you the availability and non intrusiveness of Scala and gets you excited.

You can download complete Eclipse 3.1.0 Servlet Filter in Scala project that has both Java and Scala source code for servlet filters.Get the project, get Tomcat 5.5.26, start Tomcat, run build.xml, drop created classic-scala-filter.war and classic-java-filter.war to the %CATALINA_HOME%/webapps folder and access the servlet filters: java (via http://localhost:8080/classic-java-filter/) and scala (via http://localhost:8080/classic-scala-filter/).

Servlet Filter in Java

package com.oy.shared.scample.java;

import java.util.Date;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;

public class ClassicJavaFilter implements Filter {

	public void init(FilterConfig filterConfig) throws ServletException {
		System.out.println("> ClassicJavaFilter: init()");
	}
	
	public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) 
		throws java.io.IOException, ServletException {
		 
		System.out.println("> ClassicJavaFilter: doFilter()");
		   
		response.getWriter().write("> The time now is " + new Date());
	}
	 
	public void destroy(){
		System.out.println("> ClassicJavaFilter: destroy()");
	}
}

Servlet Filter in Scala

package com.oy.shared.scample.java

import _root_.java.util.Date;

import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.FilterConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;

class ClassicScalaFilter extends Object with Filter {

	@throws(classOf[ServletException]) 
	def init(filterConfig: FilterConfig) : Unit = {
		println("> ClassicScalaFilter: init()");
	}
	
	@throws(classOf[_root_.java.io.IOException]) 
	@throws(classOf[ServletException]) 
	def doFilter(request: ServletRequest, response: ServletResponse, chain: FilterChain) : Unit = {
		println("> ClassicScalaFilter: doFilter()");
		
		response.getWriter().write("> The time now is " + new Date);
	}
	
	def destroy() : Unit = {
		println("> ClassicScalaFilter: destroy()");
	}

}

Compiling Scala from Ant

<target name="compile-scala">
	<taskdef resource="scala/tools/ant/antlib.xml">
		<classpath>
		<pathelement location="${scala.home}/lib/scala-compiler.jar"/>
		<pathelement location="${scala.home}/lib/scala-library.jar"/>
		</classpath>
	</taskdef> 
	
	<path id="cp-scala">
		<pathelement location="${scala.home}/lib/scala-compiler.jar"/>
		<pathelement location="${scala.home}/lib/scala-library.jar"/>
		<pathelement location="${tomcat.home}/common/lib/servlet-api.jar"/>
	</path>
	
	<mkdir dir="build/scala"/>
	
	<scalac srcdir="src-scala" destdir="build/scala" classpathref="cp-scala" force="changed">
		<include name="**/*.scala"/>
	</scalac> 
</target>

The Final Word

Scala is absolutely brilliant. If runtime penalties are bearable - expect wide adoption everywhere.

[scalac] scala.tools.nsc.FatalError: object scala not found.
[scalac] at scala.tools.nsc.symtab.Definitions$definitions$.getModuleOrClass(Definitions.scala:363)
[scalac] at scala.tools.nsc.symtab.Definitions$definitions$.getModule(Definitions.scala:332)
[scalac] at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackage(Definitions.scala:34)
[scalac] at scala.tools.nsc.symtab.Definitions$definitions$.ScalaPackageClass(Definitions.scala:35)
[scalac] at scala.tools.nsc.symtab.Definitions$definitions$.init(Definitions.scala:664)
[scalac] at scala.tools.nsc.Global$Run.<init>(Global.scala:449)
[scalac] at scala.tools.ant.Scalac.execute(Scalac.scala:584)
[scalac] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:275)
[scalac] at org.apache.tools.ant.Task.perform(Task.java:364)
[scalac] at org.apache.tools.ant.Target.execute(Target.java:341)
[scalac] at org.apache.tools.ant.Target.performTasks(Target.java:369)
[scalac] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1216)
[scalac] at org.apache.tools.ant.Project.executeTarget(Project.java:1185)
[scalac] at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:40)
[scalac] at org.eclipse.ant.internal.ui.antsupport.EclipseDefaultExecutor.executeTargets(EclipseDefaultExecutor.java:32)
[scalac] at org.apache.tools.ant.Project.executeTargets(Project.java:1068)
[scalac] at org.eclipse.ant.internal.ui.antsupport.InternalAntRunner.run(InternalAntRunner.java:423)
[scalac] at org.eclipse.ant.internal.ui.antsupport.InternalAntRunner.main(InternalAntRunner.java:137)

The solution was quite easy to find. The example for <scalac /> task typically starts with <taskdef >:

<taskdef resource="scala/tools/ant/antlib.xml">
    <classpath>
        <pathelement location="${scala.home}/lib/scala-compiler.jar"/>
        <pathelement location="${scala.home}/lib/scala-library.jar"/>
    </classpath>
</taskdef>

And then you call <scalac /> like this:

<scalac srcdir="src-scala" destdir="build/scala" classpathref="my-classpath" force="changed">
    <include name="**/*.scala"/>
</scalac>

It turns out that the error above will show up if my-classpath does not have scala-compiler.jar and scala-library.jar specified on it. Because when you specify classpath parameter to <scalac /> task it overrides class path defined inside <taskdef /> rather then adds to it. So just add scala jars to your own class path.

This is it.

Several of my attempts to scale PHP, JavaScript, Ruby and Python did not go anywhere. The lack of strong types was a huge obstacle that I could not mentally overcame for anything over 3000-5000 lines of code. I consequently ignored Scala when it started to popup everywhere in the blogs because I assumed that it would be more of the same thing as those other languages.

But Scala is different - Scala has static typing. If you are programming in Java now and if you are looking to achieve higher levels of abstraction for existing or new large Java products or if you want to build domain specific languages (DSL) embedded in Java you got to look at Scala.

Scala is Java. Scala code compiles to Java bytecode, it uses JVM as a runtime and has full interoperability with Java. You can Scala code from Java and back, quite transparently and with minimum or no performance implication. There is no nee to learn the limits of another runtime. Everything you know about Java, JVM, garbage collection, class loading, threading, sockets and so on still applies. All you hardware, deployment scripts and application server licenses will continue to work. Read my other post explaining how to add Scala support to Apache Tomcat.

Scala is Statically Typed. Scala uses all the same primitive and basic types as Java. The types must be declared at designed time and resolved at compile time. The type inference can still be used so you don't have to declare types in many cases, but the typing is static.

Scala is Functional. Scala is functional having both functions as first class objects and variety of ways to limit or fully avoid side effects. The ability to control side effects is very important for large projects. With functional features of Scala there are now many more opportunities to control side effects cheaply and naturally.

Scala Simplifies Embedded DSL creation. Scala supports various forms of compact syntax and includes support for writing new control structures. The syntax can be verbose or compact - you choose. And you get to keep static typing and compile time error checking.

Bill Venners has a book on Scala. The book is a very good overview of the language and has enough Scala examples to see the power of the language and its relevance to Java. What is missing are rich examples on interoperability on Java to Scala that go both ways: from Java call Scala and call Scala from Java. Even if both of these languages run in the same JVM with the same bytecodes, we need to figure out some cookie cutter and simple ways to glue existing Java libraries and frameworks to new Scala extensions as well as the other way around.

How beautiful that I don't have to relearn everything I already know and can simply start doing new things all in the conform of Eclipse and Java!

My project

I have finally decided to get new dedicated hardware for my pet projects. This is not a conventional server, workstation or a desktop - this is pure computational machine. All I really want is:

• huge amount of RAM
• fast processor
• fast bus
• fast disk

I don't really need anything else because my pet projects are typically about pushing the limits of computation. I don't care about other bell and whistles like sound or video card quality. Just give me memory and CPU... Well, I do care about one more thing - the price in hope that $700-800 should do the trick.

One quickly discovers that motherboards that support above 8G of RAM are quite rare and expensive. Also typical 32 bit OS (Windows or Linux) can't handle memory above 4G. This means that I have to get into 64bit computing.

I don't want to pay for 64bit Windows Advanced Server and I will go with Linux. While any 64bit OS today will have issues something tells me that 64 bit Linux is safer than Windows. Most of my pet projects are in Java with MySQL InnoDB. I do hope that both of these are stable on 64bit Linux.

My new hardware

I have bought this new barebone system from http://www.portatech.com. I took almost two weeks and US$799 to get it. Very typical barebone system vendor, nothing fancy, no complaints. Here it is:

Fast CPU = AMD Athlon 64 X2 6000+ (Dual Core)

8GB of Memory = 4 sticks of 2GB Dual Channel Capable PC6400 DDR2-800 Memory Modules

Fast Front-Side Bus Motherboard = Mfr Part Number: M2A-VM(GREEN) * CPU: Socket AM2 support AMD Athlon64 FX/ Athlon64 X2/ Athlon64/ Sempron processors; AMD Cool 'n' Quiet Technology; AMD64 architecture enables simultaneous 32-bit and 64-bit computing; AMD Live! Ready; FSB 2000/1600 MT/s * Chipset: AMD 690G & ATI SB600 * Memory: 4x 240pin DDR2-800/667/533 DIMMs, Dual Channel, un-buffered ECC and Non-ECC, Max capacity 8GB * Slots: 1x PCI-Express x16 slot; 1x PCI-Express x1 slots; 2x PCI slots * IDE/SATA: 1x ATA-133 channels, 4x SATA2 ports support RAID 0, 1 and 10 * Audio: ALC883 6-channel High Definition Audio CODEC * Video: Integrated ATI Radeon X1250-based Graphics Controller, shared upto 256MB memory * LAN: Intergated PCI-Express Gigabit Ethernet Controller * Ports: 10x USB 2.0 ports (4 rear, 6 by headers); 2x PS/2 ports; 1x Parallel port; 1x VGA port; 1x DVI-D port; 1x RJ45 LAN port; Audio I/O jacks * Power Connector: 1x 24pin main power, 1x 4pin CPU power * Form Factor: Micro ATX, 9.6 x 9 inch / 24.5 x 22.9 cm * Package: Retail * RoHS Compliant

SATA II (300 MB/s) Disk = 160GB 7200RPM SATA II U300 - Hard Drive

Ubuntu 6.06 & 7.10 desktop or server

Ubuntu installer started, but failed to begin installation because it could not find a CD disk. It was kind of funny because it was actually running from the CD disk, but just could not re-mount it to read the rest of the install packages. The CD/DVD drive in my system is on SATA interface, which I guess is the root of the problem.

Fedora 8

Fedora 8 failed to even to boot the install image. Right after BIOS message goes away DVD boot was initiated couple of messages got printed on the console and the thing prints "Failed to start" and dies.

CentOS 4.6 x86_64

CentOS 4.6 run and installed everything! I was so happy, but... The installation run fine from beginning to the end, but two major problems came up right after that.

First of all, the installation onto SATA disk did not work. It did recognize the disk, partitioned and installed onto it, but would fail to boot. The exact text of the error I do not have, but it looked similar to this:

	ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen
	ata4.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0 cdb 0x0 data 0 out

The same message would repeat several times for about 5 minutes and system would eventually slowly boot. Maybe this is "normal", but not for me...

Second problem was that the network card was not detected. At this point I gave up. I could have looked for the driver and so on, but I really did not want to kernel compilation.

openSUSE 10.3

openSuse 10.3 installer started and run fine. By this time I was already frustrated and ready to give up. And then I noticed how much different this installer was. It actually had dedicated hardware detection screen that I used to review my hardware before the installation. Here I saw that my SATA and IDE drives were properly discovered. The network card, audio, video - it all was properly detected. Great job!

Another thing I likes is the ability to fully configure installed packaged, view package dependencies and so on. Other installers I tried give you this opportunity as well, but not with the level of detailed I found in SUSE.

After all said and done openSUSE installed and worked great. Installation takes long time, much longer than CentOS, but I did not care. Finally, I was able to install and fully configure the hardware without any fiddling with it. Linux works, man!

Running MySQL 5.0.45 on 64 bit Linux

Installed and run great. I was able to use 7GB of RAM for MySQL without a problem using --innodb-buffer-pool variable. The data files were placed both on SATA and IDE disk and worked fine on both. No complaints here.

64bit Java 1.5.0_13 SDK from Sun

Installed and run great. No complaints here.

Running Eclipse on 64bit Linux

Eclipse did not run on 64bit Linux exiting with the Exit code=13. Here it the error message:

	JVM terminated. Exit code=13
	/oy/au_node/jdk1.5.0_13/bin/java
	-Dosgi.requiredJavaVersion=1.5
	-Xms40m
	-Xmx256m
	-jar /oy/au_node/eclipse-java-europa-fall2-linux-gtk/eclipse
		/plugins/org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar
	-os linux
	-ws gtk
	-arch x86
	-showsplash
	-launcher /oy/au_node/eclipse-java-europa-fall2-linux-gtk/eclipse/eclipse
	-name Eclipse
	--launcher.library /oy/au_node/eclipse-java-europa-fall2-linux-gtk/eclipse
		/plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.0.2.R331_v20071019/eclipse_1021.so
	-startup /oy/au_node/eclipse-java-europa-fall2-linux-gtk/eclipse
		/plugins/org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar
	-exitdata 90002
	-vm /oy/au_node/jdk1.5.0_13/bin/java
	-vmargs
	-Dosgi.requiredJavaVersion=1.5
	-Xms40m
	-Xmx256m
	-jar /oy/au_node/eclipse-java-europa-fall2-linux-gtk/eclipse
		/plugins/org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar  

I have looked around and many people report that Eclipse's SWT (Standard Window  Toolkit) uses native code that is 32 bit. I am sure there are "official" workarounds, but I will simply not run Eclipse for now.

Measuring performance with SciMark

SciMark 2.0 is a Java benchmark for scientific and numerical computing. It uses several numeric methods to measure the composite MFlops score. I run SciMark for both the server and the client JVM's from the command line like this:

	java -server -cp scimark2lib.jar jnt.scimark2.commandline
	java -client -cp scimark2lib.jar jnt.scimark2.commandline

New system score (client JVM performs better) is quite respectable:

	SciMark 2.0a
 
	Composite Score: 709.5588785447046
	FFT (1024): 560.3086048901283
	SOR (100x100):   969.0495393131292
	Monte Carlo : 82.16573593238223
	Sparse matmult (N=1000, nz=5000): 529.7979030420823
	LU (100x100): 1406.4726095458013
	
	java.vendor: Sun Microsystems Inc.
	java.version: 1.5.0_12
	os.arch: amd64
	os.name: Linux
	os.version: 2.6.22.5-31-default

For comparison, my M1210 Dell XPS laptop score (server JVM performs better) is here:

	SciMark 2.0a

	Composite Score: 477.50727979704214
	FFT (1024): 330.2235173390862
	SOR (100x100):   708.9606526769663
	Monte Carlo : 30.68535111475739
	Sparse matmult (N=1000, nz=5000): 367.86977640512555
	LU (100x100): 949.7971014492753
	
	java.vendor: Sun Microsystems Inc.
	java.version: 1.4.2
	os.arch: x86
	os.name: Windows XP
	os.version: 5.1

The Final Word

One thing I would recommend if you are going into 64bit is to choose a proven motherboard. In the old days the only thing the motherboard had was a CPU, memory and extension slots. So nothing could really go wrong. These days, when motherboards come with all the things soldered on including RAID controller, you got to pay attention to it. So many things can go wrong... You will be flashing that BIOS till the end of the days...

It looks like Linux on 64bit is accessible, stable and mature OS. I finally have huge amounts of RAM and speed I need for my projects. And all of this can be had for $799 and accessed remotely via VNC and SSH from anywhere. Beats the hell out of my programmable calculator way back from 1987...  Very very cool!

I am very interested in models for knowledge representation. Below is an excerpt from a very good condensed introduction to the frame based representation systems. Just to set the record straight - I did not write any of the following text myself. I have compiled it from the original materials by Peter D. Karp and Thomas Gruber from Artificial Intelligence Center at SRI International. They talk specifically about Lisp, but all the concepts are the same for other implementations.

I hope they forgive me for recopying it in hope that more casual readers get exposure to the topic.

Frames

A frame representation systems (FRS organizes) knowledge in an ``object-oriented'' manner, which means that facts are associated with the objects mentioned in the facts. A frame is an object with which facts are associated. Frames have the required property of being named; that is, the FRS maintains a mapping from names to frame objects. These frame objects denote entities in the conceptual world, in the usual sense of a symbol denoting an object in the universe of discourse. In some languages one may denote an object by an expression rather than a symbol. For example, the expression (+ 2 3) denotes the integer 5. However, in the Generic Frame Protocol, a frame is accessed by a name which is a Lisp symbol, and each frame has a unique identifying name. When a frame is renamed all of the facts associated with the previous name are then associated with the new name. Operationally, then, it does not matter whether a renamed frame is the ``same'' frame or a new one with all the same associations.

A frame exists in the finite storage of a knowledge base. Whether the entity denoted by the frame exists in the world is an orthogonal question. The existence of a frame in a knowledge base is determined by the functions that map from names to frames. There is no assumption about how the information associated with frames is stored in Lisp memory.

Slots

Information is associated with a frame via slots. A slot is a mapping from a frame to a set of values. A slot is also known by a name. For example, the fact that Fred's favorite foods are potato chips and ice cream might be represented by a slot called favorite-food on the frame called fred, where that slot has the values potato-chips and ice-cream. Slots and frames share a common namespace. That is, a symbol might name both a frame and a slot, both of which denote the same entity in the world. This is because in some FRSs, slots are reified as binary relations and denoted by frames of the same name. When this is the case, each application of the slot to a frame is a specialization of the same binary relation. For example, the symbol favorite-food might be the name of a frame which stands for all of the mappings between entities like Fred and their favorite foods. However, in GFP, slots may be used polymorphically; the frame/slot relationship is described by both the slot name and the name of the frame to which the slot has been applied. There is no assumption that a slot mapping a particular frame to some values has any relationship to a slot of the same name applied to a different frame. However, good software engineering practice suggests that slots of the same name should be used to represent at least analogous relationships when applied to two similar frames. We will see that this convention is enforced by some FRSs when we describe inherited slots.

A slot value can be any Lisp object (symbol, list, number, string, etc). When the value of a slot on a frame is a symbol there is no requirement that this symbol name a frame, although this is a conventional interpretation in some applications.

Slots can be viewed as binary relations. However, in many FRSs there is no facility for supporting relations of higher arity. To accommodate these systems, the Generic Frame Protocol imposes no format for expressing higher-arity predicates.

Knowledge bases

A knowledge base or KB is a collection of frames and their associated slots and values. KBs are also named using symbols. KBs share a namespace with frames and slots, the name of a KB could be the name of a frame that denotes it (although this is not a requirement). The Generic Frame Protocol provides functions for storing and retrieving KBs to secondary storage, but a KB is simply a named collection of frames regardless of how it is encoded or stored.

Template & Own slots

A template slot is a description of a slot that is associated with a class Frame, but that applies to all instances of that class. In many FRSs template slots are presented as if they were literally slots. But they are actually a way of specifying, in one place, slots for all the instances. For example, if we wanted to say that all instances of the class female-person have a slot called gender with the value female, we could define a template slot called gender for the female-person frame and give it value female. Then if we created an instance of female-person called mary, and we asked for the value of the slot gender on mary, we would be told that her gender is female.

What would we get if we asked for the gender slot of female-person? The question is ambiguous, because we could mean the template slot on the frame female-person viewed as a class, or the slot on the frame viewed as an instance (e.g., of a class of classes). To disambiguate these senses we make a clear distinction in the protocol between template slots and own slots, which are the normal slots on an instance. We know from common sense that the gender could not be an own slot on the female-person frame, because that would be specifying the gender of a class! Instead, we say that gender is a template slot on female-person with the template slot value female. For mary, an instance of female-person, the value female is an inherited value for the own slot gender. We say that the value is known by monotonic inheritance from the female-person class, to distinguish it from nonmonotonic inheritance described in the next section.

An own slot value of an instance can be inherited from a template slot of any class that is a type for the instance, not only direct types of the instance. If mary were a direct instance of single-working-mother, which is a subclass of single-mother, which in turn is a subclass of female-person, then the value of the gender slot for mary would still be inherited from the class female-person.

Classes and instances

A class is a set of entities, which are called the instances of the class. An entity can be an instance of many classes, which are called its types, and a class can be a type of many classes. A frame denoting a class is called a class frame, and a frame denoting an entity that is an instance of a class is called an instance frame. A class can also be an instance, i.e., an instance of a class of classes (a metaclass). The relation that holds between a instance and a class is a primitive, akin to set membership. With this primitive we can define the sub relation that holds between classes. A class is a sub of class if all instances of are also instances of . In other words, all instances of the subclass are instances of the superclass, and the superclass may have other instances.

Facets

Facets are annotations on slots. Facets have values, which, like slot values, may be any Lisp data structure. Since slots are identified by a slot name and a frame, facets are identified by a facet name, a slot name, and a frame. Some facets pertain to the values of a slot; for example, a facet can be used to specify a slot constraint or a method for computing the value of a slot. Other facets are about the slot itself, such as documentation. The Generic Frame Protocol provides a set of a standard names for a few facets, and applications may assert other facets as well. For example, to assert the value-type restriction on favorite-food, one could assert that the facet :value-type has value edible-food on the template slot favorite-food of class frame human.

FRSs have a variety of ways of specifying facets. Some systems allow for uniform nesting of facets (i.e., facets on slots, facets on those facets, etc). Systems that represent slots as reified binary relations (slot frames) may represent slot constraints using slots on global slot frames. The Generic Frame Protocol only provides a notation for a single level of facets on slots, and makes no requirement that a facet on a slot has any relationship to a slot on a frame that represents a binary relation. In fact, only those facets that describe slot constraints have any semantics in the protocol. All other facets are simply annotations for applications to use as needed.