Single point of configuration

In a software project, especially java, one would employ various third party libraries to ease and speed up development.

However, every library would likely have it's own configuration file or approach to configuration.

Let's look at a typical java project. A web based project would have a web configuration file (XML). It is highly likely to use logging, maybe configured with a property file. For sure there is a database connectivity, and since there is password involved, you might want to define it in the application server. And if you are using Spring, you might have a bunch more XML files to configure.

So that is like at least 3-4 files to configure.

Now let's say you are a very modular person. You like to have things configurable too. So you had a configuration file for the LDAP settings, one file for the security policies, one file for default email templates, one file for mail servers.

Now, imagine that a new developer comes along, and need to update the configuration of a bunch of things. The changes would be spread among various files.

Or imagine a fresh deployment, which require the configuration of a database connectivity, LDAP, email server. 3 files instead of one. Or maybe it is not a new server for production, but just a test server.

I was brought to the wordpress system. To configure a new copy of wordpress, all one need is to configure just one file. The file would contain all the settings. I did not look into it, but it woul also make sense for plugins to expose their settings in there too.

A conclusion I draw from this? Try to centralize an application configurations at a single location. Try to make it a facade for all the subsystems, even though they might be third party plugins, libraries, or systems. This helps in management of settings, and reduce learning curve for new developers or users.

Making audit data friendly

Auditing is a facinating topic to me. So much so that I'm writing another blog post on it.

So, let's assume that we have a UI screen that allows a user to update his particulars. And once the user submits, the data are pushed out to be updated.

The logical thing drawn here is that there should be an audit entry indicating the user initiated an update. Simple, easy. So far so good.

Now, what if the update looked simple, but was not? The user might have updated his name, password and telephone number. And assume that the password has to be pushed to an external legacy security system. The telephone has to be pushed to the company address book system. The name to the company address book system, as well as the HR system.

Each update could fail independently of each other. And since it actually push updates to external system, we really should audit them too.

So now we actually have four audit entries. One for the user clicking submit, one for the secuity system update, one for the telephone sytem, and one for the HR system.

But how do we know the other three updates belong to the same user initiated update action?

The problem gets worse when you had another background workflow that does similar update. So you had a system initiated update, a user initiated, and a bunch of interleaved updates. They could be on the same user, or different. The focus here is the audit log. Imagine how painful it would be to decipher which update belong to which action!

We need to group all related actions. In the example above, the user initiated action could generate an action transaction id. And all the sub-actions should be tagged with the same action transaction id. This allows us to view and perhaps even search the audit log by action transactions, giving us greater control.

And as a random thought, perhaps audit entries could be catagorized with detail level too.

Let's extend the update example furthur. Assume that an approval workflow is involved. There will be audit entries for request submission, approvals, escalations and resolution due to timeout, provisioning, and so on.

We could have a level of audit for request workflow. These are request submissions, approvals, escalations, etc.

Next, a level for provisioning. The target systems being updated, and the data changes.

Depending on what one want to view, they could view a 'business action' audit, or a 'technical' audit.

This is something I noticed missing in the few systems customization project I worked in. And we usually had to build this in. And I always wonder why they never think to build this in.

I'll probably be still pondering on auditing in business applications. I hope to be involved in creating an audit friendly application development someday.

Audit on XML data

I had to implement a feature recently for a particular requirement. Due to the nature of the product, a particular value had to be serialized into XML data before being persisted to pass through a particular adapter layer code. And just before persistence, the code would deserialize the XML data so that they can be written to the corresponding database tables.

The database is not storing XML data, to be clear. And the reason we had to do the serialization and deserialization is because the boundary of the adapter code accept only int, boolean, string, and list of strings. We needed list of objects. So our only escape route was list of strings.

It was definitely not pretty during auditing.

There is an in-built auditing feature. When data pass through the adapter, all old and new values are audited.

But now that the data are in XML, we get well.. a long string of XML. It was not too clear which value actually changed. One had to copy both old and    new XML string to some text file, do some formatting, and then do a diff on them.

The application was rather smart. Before passing the values to the adapter, they would first do thier own diff. If there were no changes, the adapter would not receive the values. The audit would not log these unchanged values too.

This, however, pose a problem with my xml data at times. Sometimes, the original value came out with a particular order of the attributes. And when saving, the order might change (perhaps I added/removed, or there was something different in the serialization). The attribute values remain the same, yet the order change, resulting in the application thinking that a change had occurred.

Imagine the frustration when you had done a diff on two values, and realized that the values were the same!

Auditing at the level of xml string is a bad idea. It always has been, and will be. If it was not for the in built functionality, and limited time, we probably would have reworked that part of the application.

Someone has to own the project

Working on the most recent project was a huge pain.

As mentioned, it was half on-site and half off-site.

The pain was beyond communication. It seems that at every moment, there is always something new to do.

And these are not new or changing requirements. They are hidden in the current code, unimplemented.

The lead on the project love to use a UI driven approach, or top-down as some would call it. Mock up the UI with dummy API calls. Nothing wrong with that.

Except when I was going to start changing some UI to use a new logic/design, I notice that some of the data the UI was displaying were hardcoded dummy data!

Now, you might think that time was saved, since I was going to update it to the new UI. But what was worrying was that when I was going to update this, I asked question on this part, and the lead was unsure if it was displaying data from the backend or otherwise!

It was never clear who owned a module. Sometimes a few of us would be working on the same library. It was never clear to me whose duty end and whose begins. And this might result in unimplemented logic.

And like many project, the requirement is constantly changing. And many of the business logic are still convinently locked in the mind of only a few. There is constant fear that not everything is implemented.

All I need is the confidence that someone has everything under control. He knows what had been done, and what not. And he can decide when something is done. That's what we lack.

Sounds like a good project manager is needed.

Programming: Feeling burdened

Recently I had been giving much thoughts into the programming languages like C++ and Java. I mentioned only these two because those are the ones I had been working in most of the time.

I felt somewhat burdened.

When the project grows beyond a certain number of classes and methods, it felt like a pain trying to even look at the source code. Even though the files were structured to be pleasant to read, there is this weird growing sense that something's not right.

I remembered Uncle Bob mentioned in his book, 'Clean Code', that the sequence of methods placed in the source file matters. It had to flow right. Perhaps that is my main problem.

I tried to keep my public methods as a block, and my private methods as another block. It did not work, since you had to jump from the public down to private when understanding code. Then I had all related private methods right below each public method. However, now, at a glance I cannot know what all the public methods of a class are (except through perhaps a smart IDE filter).

This is just wrong! Why are we still concern with ordering of methods in a source file! We should be working on a higher level of abstractions on them! We should be looking at only method blocks where they matter! The IDE should be smart enough to filter them out!

(I still miss VisualAge for Java from IBM).

But that's not all that made me feel burdened. I feel slow when developing on these. Perhaps its not the languages themselves. It could be my approach. I am very much a TDD guy. But recently, much affected by all the readings from 37signals on Getting Ready, etc. I wonder. Perhaps I am over TDD? Or are they even necessary?

Even Kent Beck, in his recent post on his JUnit Max, confessed that he did not take a TDD approach. There are plenty of code that were not test-covered.

It seemed that, in a rush to deliver, we had to sacrifice test coverage.

But of course, they do not apply to all scenarios. The reason I am feeling these are because, well, when I am working on my own hobby projects. I felt that I am slow in crunching out and moving towards the goal.

Lastly, a part of the burden is due to the language (definitely). The syntax are really to verbose. Lack of closures, excessive need to declare private variables with accessors. Constructors which simply set values. Lack of named parameters, which result in the need of parameter objects. There are just no lack of complains.

Perhaps I should move to a lighter, more agile language. Maybe even dynamic language.

Working offsite

My earlier post mentioned that I had been on an overseas assignment.

There was this week, where I had the luxury of working offsite.

Except, it was no pleasure. It became more like a nightmare.

I had no ability to checkin my work into their subversion repository. I am constantly in the fear of being out-of-sync. On the second day we had a resolution. They would check out the entire repository. I would merge my code. And then send it back to them. I have no idea how the other co-worker working offsite would be doing.

And of course, for communication, we had google group talk. But it felt slow and unresponsive. I could understand. When I was on site, the google talk client could not work. I had to use the browser based gmail. And I would not be notified of new chats unless I switch over to check it.

And for me, communication was a huge problem. Because the requirements were not fix, and comprehensive enough. There were many doubts, and many things to clarify. I felt my productivity go down to a snail. So much so that while waiting for responses... I was really out of tasks to do. I went to read a book I had just borrowed from the library instead.

There was also the timezone problem. Thailand is an hour behind Singapore. And they usually start work at around 930. Which means 1030 for me in Singapore. And they work late. Well not too late nowadays. Around 1900~2000 Bangkok time. And that would be 2000~2100 for me in Singapore.

Which is why, after the second day, I started working from home. No point sticking to the Singapore timezone.

I do envy how 37Signals got their style to work. It did not work for me.

Software Developer Career Path in Singapore

If you are a software developer newly graduated in Singapore, what are your possible route of career advancement?

This is actually a topic that has been a constant frustration for me.

Outlined is a natural progression of a typical software developer in Singapore:

1. Developer -> Team Lead -> Project Manager.

Short and simple to understand. Most software developers would follow this natural progression, and is probably the only model they are familiar with. Project Manager is where the money is, which is of course essential for our day to day expense, especially in this country of high cost of living (I think we are ranked like 10? in the world).

Notice also that in this progression, a person develop less and less, but manage more and more.

Now what about those who do are not good in managing? Or uncomfortable in managing project and people?

In the current situation, they will just... try to manage. This path is a very natural model here, as I stressed. And in fact, for some, this is seen as progress, or promotion. It is just NATURAL. If you don't go up the chain, you are seen as well.. incompetent. Irregardless of how beautiful and perfect your code is. Or how knowledgable. Or how good at debugging you are. If you don't go up the chain, you are simply SECOND CLASS. Your pay and bonus will definitely be lower, because there is likely a cap at each stage you are in. Even worse, maybe the highest pay a developer can get will still be lower than the lowest pay a Team Lead get. Simply because a Team Lead manage, which means additional responsibility (this is true). But even if he suck at managing, he can probably still get more pay than the most component developer in the company.

But actually, there is an alternative route of progress, which is not that common in Singapore.

2. Developer -> Senior Developer (-> Architect) -> Consultant
*(-> Architect) represents an optional progression

The developer could be not good at managing, or simply uncomfortable with managing. This represents an attractive alternative route. All experience, built up since the time of development, are still utilized. In fact, the developer is free to, and strongly encouraged to, engage in active development at every stage of progress. Even when he is an Architect.

In this other route, the skills of a developer is capitalized and maximized.

But why is this model not popular in Singapore?

The answer is very simple. Singapore Companies do not recognize software development as a high quality skill. This arise when you have management who have no experience in the software development industry or domain, who had never done any software development before. They see software developers as expandable works, who can be easily replaced as and when. Every software developer, to them, has the same amount of output, and contribute evenly to the development of a project. Software developers can be added to a project anytime, which can result in a directly proportional speed increase.

Of course, we cannot deny the contributions by our Software Development Industry itself on this misconceptions. Recognizing Software Development as an engineering discipline gives people the perspective that the development lifecycle is predictable, and easily controlled. The original waterfall approach does not help, but it is widely adopted (agile and scrum is the right approach, but still need some time to properly displace waterfall in most companies). And then, the cheap outsourcing alternative is not helping as well. To the management, why pay 2x for a senior developer, when you can hire 2 junior developer? And of course, 4 foreign developer for that matter?

Which is why as software developer, we should recognize which route we are comfortable with. And identify companies that support the selected route.

IBM, Sun, and other MNC would be more keen in supporting both route of advancement, while most local companies like ST, NCS, would be more keen in supporting route 1. I am actually lucky to find a local company that support route 2. But still, the best approach is definitely to work overseas ;)

Audit and Security are not meant to be interceptors

I was working on an administrative web interface for a user management product, and was trying to find out how best to do audit and security.

I should throw up the conclusion first: Audit and Security are not meant to be interceptors.

And now for the journey to that conclusion.

We are in the year 2009. Java annotations are on the rampant. Every problem looks like a nail to that hammer. And it is no surprise that there are many projects which implement audit and security annotations, and run them through interceptors like AOP.

Which I tried as well. It worked great initially. Marking methods to be auditable and securing methods with roles were simple, and does not obstruct code logic.

Until the security got more 'realistic' and requring the audit to be 'useful'.

In the original concept, a user could be an administrator, and able to modify any users. In the updated requirement, a user could manage multiple group of users. No longer that simple to just check by a single role a user has. The role would be dynamic too. To support multiple dynamic groups after deployment, the role name would be built up from the name of the group of users. And annotations for role name support only strings that can be determined from compile time!

Granted, a more complicated security annotation might work. Something along the line of annotating a method with a security policy instead of a role name...

But let's move on to audit first. The most common audit approach is probably marking methods as auditable, and then record all input arguments and output result of that method.

That seemed to me to be a rather... technical audit rather than a business audit trail. Translation of the audit information to business information is not so easy. And there are definitely times when the required audit information might not be part of the input arguments nor output result. And this might violate the SRP to some people, but it's entirely possible that a single method is in fact, multiple events and require two audit entry instead of one.

My point here is probably this, that for a given business scenario, there are definitely clear requirements on what audit information to capture. And these information usually go beyond the simple 'method invocation with arguments, results, or exceptions'.

I'm definitely not against annotations and interceptors. They are excellent idea, and work great for many cases. My grip is that audit and security functional requirements are never that trival, in the sense that many out of the box interceptors will fulfill the requirements. And there is just too much hype in audit and security interceptors as the holy grail.

Danger of being overly focused (to a skillset)

I started playing with XCode, Objective-C and iPhone Application Development recently (which was about 3 days ago).

Some background on my skillset first.

I started out learning Pascal and Foxpro, and then proceed to C++ with Visual C++. Along the way I picked up VB, and then Java. Then I worked with Java professional for about 3 years, and then out came C# (within that 3 years), which I then decided to make a jump for it, but ended up with VB.Net and Visual Studio for about a year. And then I decided that Java had a better career opportunity back then. So it was back to Java and Eclipse for another 2+ years.

And this is where I am now. Which is to say that I am strongly comfortable with Java and Eclipse (but seriously, Eclipse is an excellent IDE).

During the time I read up on Ruby and Python, but resisted developing with them, blaming the lack of an IDE with the quality of Eclipse.

And now that I want to try out iPhone Application Development, I had to pick up Objective-C and XCode.

I started reading up on both topics. And with every read, Objective-C felt like an excellent language. Named parameter was a big plus for me. Categories were a good way to slice up code into separate files (much like .Net's extension methods). Other than that, much was the same as old time (to me). XCode itself was a good tool, with code completion support (though I have yet to read anything on refactoring, and not much books talk about unit testing though they are available online).

And then came the time of actual usage. I was confident that with my experience in C++ I could tame that Objective-C beast. And the tool did seem good.

I was wrong. Not on the account of Objective-C. That was pretty easy. I am a firm believer that language syntax are simply decorations. And so I had concentrated on honing the more important skills of writing clean code, design patterns, building stronger foundation. But development still take a while, because I was so used to Java. Coding in Objective-C requires some mindset changes. Recognizing new symbols.

XCode was a much bigger killer. Code completion works, but code 'suggestion' was a different key. I had to retrain my instincts. I have yet to figure out unit testing, but probably will complain when it get working, citing it as different and weird compared to Eclipse (but that is definitely due to my overly used to Eclipse, not because of XCode). Checking for compilation errors was different and had to get used to. And looking at the logs while debugging was... well just wonder why I had to switch my view in XCode. Perhaps there is a different project layout view (I was sure I read something on it), and I should try that.

So why so much pain? I became overly reliant on a single tool and language. And that is definitely a very dangerous thing. Regardless if you read up on different tools or languages, if one does not actually try using them, they will not be able to operate under different environment. And as developers, we MUST be as flexible as possible. One can never know what tools one might use in the future of their career. (I am currently cursing on using Visual Studio in Windows XP VM on my Mac... GRRRR).

And after years of using Java, a garbage collecting language, I had to revert back to a non-garbage collected language. More care is put into each line, and there is always a wondering thought on if I should release the memory of something returned from a system call, or not if I am returning them as return arguments. It takes some getting used to.

I probably should have tried out Ruby and Python more often, and C++ too, with a variety of tools. Life would be better.

Stop anticipating requirements

As developers, we want an easier life. We worry and fear of unstable client requirements. We think of any possible ideas the client might come up with, and implement hooks for them in our current code (as long as they are not too tough, and even that is relative to a developer's skill).

Half the time, we are right, and we felt good about saving ourselves a lot of hell by putting in the hooks early.

The other half, the requirements never come. But we still felt good because we WILL be saving ourselves a lot of hell by putting in the hooks early.

But in general? The extensions and hooks we introduced made the system more complex. Code does not read that naturally, hidden within implementations of Template Pattern, nested one within another. The runtime structure of an application is so remotely different from the static structure of an application (class diagrams), that documentation no longer seem to be useful. In fact, they seem to be talking about totally different applications! The growing unnecessary hooks only add to more documentation, which seemed to scare every other developer away except yourself. And we wonder why.

But what could be worse? Well, that the requirements for a new feature came. And it is totally in conflict with the hook we planned. Well, we could argue that the hook was not well designed, and some change would be necessary, and then we could wait for the 'real' requirements to come. So for now, we add a hook to the hook, happy that we have added even more flexibility. And while we are at it, we thought of another possible client requirement. So we add another hook too.

And the story goes on. Hooks on hooks on hooks. on hooks. Half of them probably would never be used.

So what exactly went wrong? Obviously, the problem is us.

We tried to act like we are the client. Except we are not. We are probably the worse candidate to act like the client, since in most cases we are developing for a problem domain of which we are neither expert, nor even remotely familiar with.

Which is why as developers, we should always keep our solution simple. A simple solution is a flexible solution. Add hooks only if the actual client requirement comes. The value added service we provide here is faster delivery of solution, instead of a monolith and extensible solution but would require months to add new features.

Remember, in most cases the clients are the domain experts, not us. They may not always know what they want, but they do know what they do day-to-day. Spend time with them, understand their need. Do not give in to their desires. And do not give them something that they do not need.

Towards Better Code: No side effects, shorter methods, exit early, and log every invocations

Recall earlier that I had mentioned that I learnt through logs of existing program/framework.

It's weird, but I cannot emphasize enough about the importance and need to log.

I have been using a rather new approach to logging. Well, not exactly to logging, but a new approach combining logging and coding style.

First off, I keep my methods short. For example, given the following algorithm:

public void login(final String username, final String password) {
  // authenticate
  // check if user container is active
  // check if user is active
}

It will most likely evolve into the following code:

public void login(final String username, final String password) {
  authenticate(username, password);
  verifyUserContainerIsActive(username);
  verifyUserIsActive(username);
}
void authenticate(final String username, final String password) {
  final boolean valid = checkUserCredentials(username, password);
  if (!valid) throw new AuthenticationException();
}
void verifyUserContainerIsActive(final String username) {
  final Container container = getUserContainer(username);
  if (container.isInactive()) throw new UserContainerInactiveException();
}
void verifyUserIsActive(final String username) {
  final User user = getUser(username);
  if (user.isInactive()) throw new UserInactiveException();
}

That's just a rough idea. Basically, each method is composed of many more methods, which serve to describe what each step do. Benefits? Eliminate the need to comment complicated algorithms. Comments run the highest risk of being out of sync with the actual code intent.

Also, notice the use of exceptions in the inner methods. I had started out with an alternative idea, that the inner methods would return true/false values, which would. It basically create the following nasty code block:

public void login(final String username, final String password) {
  if (isValidCredentials(username, password)) {
    if (isUserContainerIsActive(username)) {
      if (isUserIsActive(username)) return;
      throw new UserInactiveException();
    }
    throw new UserContainerInactiveException();
  }
  throw new AuthenticationException();
}

Of course, the better way is as follows:

public void login(final String username, final String password) {
  if (isNotValidCredentials(username, password)) throw new AuthenticationException();
  if (isUserContainerNotIsActive(username)) throw new UserContainerInactiveException();
  if (isUserIsNotActive(username)) throw new UserInactiveException();
}

Personally, I still felt that my first example up there produce a much more readable version of the algorithm, which communicates the intent much better.

But the basic idea/rule here is, exit early. What is most important is to communicate the basic intent of the code. The other possible cases are, well, exceptions (pun intended), and should not clutter the original intent of the code.

And finally, on to logging.

Now, I log all entry and exit of a method, as well as method arguments and method return values. Ideally this should be done with AOP, but I am not doing that yet.

What benefit does this logging bring me?

I can actually, from tracing the method entry logs, form a very clear view of how a request pass through the entire system. An example of the above log might be as follows:

enter: login {username:kentlai, password:xxx}
enter: authenticate {username:kentlai, password:xxx}
exit: authenticate
enter: verifyUserContainerIsActive {username: kentlai}
exit: verifyUserContainerIsActive
enter: verifyUserIsActive
exception caught: UserInactiveException

With shorter methods, logging every method invocations, I can trace the flow very accurately. Sure, there is an increase in the size of logs, but the return in such investment is easier debugging. Without even hooking up to the debugger.

With the logged method arguments, and knowing which method is causing a problem (from the logs), I can actually write a quick unit test to fix any potential problem found.

But there is one last magic elixer in this combination. No side effects.

Basically, each class should not have mutable state. The only mutable state should be from external storage (eg. database). And if a function need to use any values from such storage, they should, themselves be encapsulated into methods of their own, with the return values logged.

Why? This allow you to put together a complete picture, the context and environment of which a method was executed under. When every variable becomes known and can be supplied once again to a method during unit testing, a method can be guaranteed to work consistently and constantly as expected, which allows for more robust code.

Funny it took me so long to realize such principals.

Where can Twitter get money?

Twitter has recently annouced that they might offer a premium twitter service at the end of the year. But before then, there were endless thoughts and advices on how to run twitter. How various company should buy twitter, why twitter should sell, and wonders on how long twitter will go down the deadpool.

Except seriously, twitter has nothing to be afraid about. Various attempts have been made to create alternative microblogging services, tools, and software, but they did not seem to gain the same amount of traction and adoption rate.

Twitter is an infrastructure. Messaging delivery infrastructure.

The rate of growth is unbeliverable fast. The ecosystem of tools and services around it is huge as well.

People are using it for real time news and search. Recently, if I have to find information, I use twitter search first before I even try google.

Companies are monitoring twitter for keywords to do direct sales and product support. In fact, they are more responsive than me sending them an email for support!

Imagine if twiiter goes bankrupt and into the deadpool. Imagine all the companies that will be affected.

Which is why all the alternatives to twitter keep coming up. And the advices to business models and selling to a rich company like microsoft or google.

People are afraid of losing such an important tool. And people do recognize its importance!
And so the investments come. People who are betting on a future with twitter. They might want direct paybacks on the long term. Or they might have interest in the twitter ecosystem. Or they might be using twitter and am convinced how it will help them in the long run.
That is an interesting growth/survival strategy.
Start with something free. Make it easy to use, and easy to create tools and services around it (the API). As long as it gains popularity with the netizens, and the ecosystem starts to grow, investments (or donations) will come.
But that's easier said than done I guess.

The next developer is probably not as good

Of course, this is an unfair judgement call, since you probably had not even met the next developer.

But this is something my colleague kept trying to hammer into me, which has its valid point.

Why does he say so?

Well, simply because my code is really, rather hard to dive into.

We are actually working on an in-house project, where we want to set up a simple framework, and have the future development 'outsourced' if possible. We had a quick stint with an intern, but it took really long for him to get into.

There are various technology used.

Spring DI. Spring JDBC. Spring LDAP. Spring JavaMail.

Apache Quartz. Apache Struts 2. Apache Tiles. Apache Velocity. Apache Maven.

Logback. JUnit. Mockito. Selenium.

All rather simple to learn and use... to me that is.

He made an interesting point. That most developers do not learn new stuff on their own. They are component with Java, but perhaps as a language. They do not go experiment with the bleeding edge. They are comfortable with the basics. Some of them are probably still in the plain old Servlet and JSP era! Annotations? What are those? How do we use them?

I came initially from the 'productive rock star' perspective. Anything that keeps the stack lean and mean, and allow me to quickly deliver business values, the better it is.

But given his perspective, I honestly doubt a newcomer can easily pick it up. I will have to explain Maven, set it up for him. Explain unit testing and Selenium. Mocks. What Spring DI is. Where are the Struts localization files. How Tiles work. Logging with logback. Struts interceptors. Struts validation... blah blah blah.

Which brings me back to reiterate the same point in a different word from him again. Assuming that there are component developers in the company. But being in a company, there are definitely programmers of different level and standards. Being on the bleeding edge might also mean that the knowledge of using it being constraint to the 'elite'. The rest 'inferior' developers might find it harder to pick up. Or maybe even impossible and never.

And as we saw in the phrase, "It is all very much good to place all decision making in the hand of the perfect man, but what if the perfect man has a bellyache?"

We are using Java. Why Java? Sure, it takes more line to do the same work in Java than probably in Ruby or Scala. But it is 'clear and simple', and 'mass-market'. It is very much easier to get java developers here than the others.

So right now I'm taking a step back. Annotations might have to go. Various Struts specific concepts might have to go too. An abstraction layer of the above mentioned stack might have to be developed, such that we can have a consolidated framework and usage pattern, along with documentations on how to use that.

That was his idea, but it definitely has merit. Given a one to one mapping (even though it is a thin redirection layer), we can simply tell the incomer to use 'our libraries', complete with tutorials and example. It promotes a standardization of company code, and probably is much more helpful in the long run.

Learning new frameworks through logs and reading source code

I was having lunch with a colleague, and commenting on how slow another colleague was in learning new frameworks. I wondered if it had to do with experience, since he was relatively new. But still, the rate he was picking up was unimaginable (to me at least).

I cited how I had to help in resolving a compilation and deployment problem, which by just googling a similar pattern of the error message (since the exact message would not yield that close a match with our project names, user folders, etc in it) gave me a hit on the first page (maybe not the first result, but at least it was on the first page).

I wondered if it had to do with the educational system, where they are more spoon-fed and focused on passing exams. Back in the older days we had a genuine desire to learn new stuff. We would borrow and devour books (figuratively speaking)

And then I mentioned about how I went about learning Struts2 and Tiles for this current project.

Which led me to notice the approach I took to learning it (as well as the previous JAX-RS framework).

1. Reading official documents and tutorials. The framework usually have some form of documentations. Even if they are incomplete or few, they are still a source of learning.

2. Google. For something you want to be done, there is a high chance that someone else has done it as well.

Usually these two would be sufficient, but recently, I go a step further.

I just came back from an IDM project. Sun Identity Management product. It was heavily customized. And at times, we had to deal with insufficient documentations, and find out if something is a product bug or a customization done wrong.

And how we do that is to turn on log tracing, and trace the calls, and then check the product source code (decompiled).

After the project, I had just unknowingly do that whenever I hit a problem in using a framework.

I fire up the logging, and find out which method call I had to use, and go through to look at the source.

In fact, it helped me understand the framework much better.

Of course, some would argue that reading source is hard. But I did not find it that way. I could still find my way around it, even though it might take a while.

Let me share my latest experience.

I was using Struts2 with Tiles, and I ended up with a problem of wildcard matching of definition names, and cascading attributes. The attribute was not found in the nested definition.

First, I tried to check the documents. They cited examples of wildcards, and then of cascading attributes, but not together. So I was not able to determine if I was using it correctly.

Next I tried to google. It took a while, but I finally managed to find someone with the same problem. Apparently it was a bug the guy reported, and he was asked to produce a JIRA (not sure if he did), and it was dated April 2009.

But this did not solve my problem, since it was a deliverable project. So I had to fix it myself. I dug through the code, and realized that the problem was in ResolvingLocaleUrlDefinitionDAO.replaceDefinition, where the definition was not duplicating the cascading attributes, only the local.

Now to figure out how to fix it. I just need a custom DAO. From my experiences with the various frameworks, a good framework usually allow you to provide custom components, defined in a property file or web.xml. There was no property to provide a custom DAO (I traced through the DAO creation path by code), but there was a way to provide a custom DefinitionsFactory, which create the DAO.

So I did that, and solved my bug.

If I had gave this task to that new colleague, I doubt that the bug can be resolved. Somehow, decompiling might not have crossed his mind.

I am trying to find out, what was the reason of the difference.

Was it because I was placed in a situation where I had to read the source and debug and fix?

Was it because I had more years of experience working? (approaching 6 this year)

Was it because of inadequate googling skills? (That proved to be a problem for another of our ex-colleague, who was not that good in English)

Was it because I am used to reading logs?

Was it because I am comfortable reading source code?

But in any case, learning new framework and reading source code does give me various new ideas on how to design and develop. A case in point: if it is a framework, most components should be allowed to be defined and replaced by the consumer of the framework, just like the Tiles case above.

Digging into Jersey JAX-RS: 3. Request flow

Logging

Now it is time for a more complete feel of how a request flow through the system. And to do that, we will enable logging for Jersey.

Jersey uses java.util.logging as the logging framework, so enabling logging for it is easy. Just throw in a logging.properties into the WEB-INF/classes with a log level of ALL for com.sun.jersey.

(However, I did not manage to view my log output in tomcat launched via Eclipse. Somehow the console displays only levels up to FINE, and I needed levels up to FINEST... so I launched it via the normal tomcat outside of Eclipse.)

So I made a request for the xml content, and then view the log file.

... and there was nothing logged about the request.

Guess it's back to the code for manual mental tracing :(

Request Flow

The entry point of jersey is of course, the filter we set up, com.sun.jersey.spi.container.servlet.ServletContainer. 

I first looked at the code of com.sun.jersey.spi.container.servlet.ServletContainer#doFilter, and I go... ???

It is part of a filter chain, but however, it does not involve the next filter in the chain! It simply.. consumes the current request and is absolutely sure that it will handle the request. That does not seem to play well with static content though, unless my filter does not catch /*.

But let me put aside that for now. So it flows through to com.sun.jersey.spi.container.servlet.WebComponent#service, where the application context, user principal, form input are set up, and request/response are stored in the local thread storage context (though why I am unsure).

Next would be com.sun.jersey.server.impl.application.WebApplicationImpl#handleRequest, which applies the request filters. 

com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule#accept is triggered, which tries to match the input path and process it. This is where the com.sun.jersey.server.impl.template.ViewableRule mentioned in the previous post comes into play. But there are others as well. 

It is highly likely, in our example, that com.sun.jersey.server.impl.uri.rules.ResourceObjectRule/ResourceClassRule are matched. They are created in com.sun.jersey.server.impl.application.WebApplicationImpl#processRootResources, where all located resources/singletons (that was scanned by jersey runtime in the web.xml configuration) annotated by Path are attached to the root rule.

Eventually, the com.sun.jersey.server.impl.uri.rules.HttpMethodRule comes into play, which dispatch the request to the java method of the resource.

And after the com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule#accept is completed, the response filters are applied to the response, and the output is written out.

javax.ws.rs.ext.MessageBodyWriter & javax.ws.rs.ext.MessageBodyReader

So where do these message body reader/writers come into play?

In com.sun.jersey.spi.container.ContainerResponse#write, the right message body writer is located via best media type match. 

And in com.sun.jersey.spi.container.ContainerRequest#getEntity, the right message body reader is located as well via best media type match.

It is interesting to note however, that the com.sun.jersey.spi.container.ContainerRequest#getEntity result is not cached internally. Which means that multiple calls of it might result in an error. I would assume that it is currently carefully handled.

Summary

It was an interesting exploration, and there are a lot of magic involved (magic to me. I just did a touch and go on purposes of most classes). I was however, rather disappointing in the servlet/filter implementation, in that it does not give the next in chain a chance to handle the request if it was not handled by jersey itself.

Now I am rather curious as to how the other implementations are. I ought to take a look too. And what will I do next after the look? Probably try to code up a website on pure jax-rs if possible, reverting to jersey's helper if unable to.

Digging into Jersey JAX-RS: 2. custom message body writers

I decided to start with something simple, like a service that simply returns a java POJO object.

So with that in mind, I created an EntryPoint that mapped to the root path of /.

Simple Service

@Path("/")
public class EntryPoint {
     @GET
     @Produces(MediaType.APPLICATION_XML)
     public Data getData() {
          final Data data = new Data();
          data.add("key1", "value1");
          data.add("key2", "value2");
          return data;
     }
}

And I tried to access the web application with my Safari, deployed to tomcat automatically with eclipse.

A message body writer for Java type, class me.kentlai.jaxrs.services.Data, and MIME media type, application/xml, was not found.

Hmm.. I wonder what are all the message body writers registered in the system. Other than the com.sun.jersey.server.impl.template.ViewableMessageBodyWriter in jersey-server, the following were found in jersey-core.

com.sun.jersey.core.impl.provider.entity.StringProvider
com.sun.jersey.core.impl.provider.entity.ByteArrayProvider
com.sun.jersey.core.impl.provider.entity.FileProvider
com.sun.jersey.core.impl.provider.entity.InputStreamProvider
com.sun.jersey.core.impl.provider.entity.DataSourceProvider
com.sun.jersey.core.impl.provider.entity.RenderedImageProvider
com.sun.jersey.core.impl.provider.entity.MimeMultipartProvider
com.sun.jersey.core.impl.provider.entity.FormProvider
com.sun.jersey.core.impl.provider.entity.FormMultivaluedMapProvider
com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$App
com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$Text
com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General
com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$App
com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text
com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General
com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$App
com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$Text
com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General
com.sun.jersey.core.impl.provider.entity.ReaderProvider
com.sun.jersey.core.impl.provider.entity.StreamingOutputProvider
com.sun.jersey.core.impl.provider.entity.SourceProvider$SourceWriter

Ok to be fair, there are various ways to serialize xml. And various xml libraries. There are a few in-built xml providers, but they take in JAXB elements of DOM elements (as far as I can see. I am not an xml guru).

It is a good chance to experiment with my own provider. An xstream writer provider.

Custom XStream Message Body Writer

This is not an entry about how to use xstream. There are much better articles out there for xstream, so I will skim over the details.

I annotated my POJO with xstream annotations, to make the xml output prettier.

The xstream writer provider was rather simple to write. All it had to do was match the xml media type, and write the object out with an xstream instance. I am not using any xstream annotation here yet. Note also that I annotated it with javax.ws.rs.ext.Provider and com.sun.jersey.spi.resource.Singleton. The first allows the class to be picked up when Jersey scans the package for providers, and the second requests jersey to create only a single instance of the writer for the entire web application. This is a better option as readers/writers should be stateless and can be reused.

@Provider
@Singleton
public class XStreamMessageBodyWriter implements MessageBodyWriter<Object> {
     public long getSize(final Object t, final Class<?> type, final Type genericType,
               final Annotation[] annotations, final MediaType mediaType) {
          return -1;
     }
     public boolean isWriteable(final Class<?> type, final Type genericType,
               final Annotation[] annotations, final MediaType mediaType) {
          return mediaType.isCompatible(MediaType.APPLICATION_XML_TYPE) ||
               mediaType.isCompatible(MediaType.TEXT_XML_TYPE);
     }
     public void writeTo(final Object t, final Class<?> type, final Type genericType,
               final Annotation[] annotations, final MediaType mediaType,
               final MultivaluedMap<String, Object> httpHeaders,
               final OutputStream entityStream) throws IOException, WebApplicationException {
          xstream.processAnnotations(type);
          xstream.toXML(t, entityStream);
     }
     private final XStream xstream = new XStream();
}

And accessing the redeployed page gives me the following:

<data>
     <entry key="service1">
          <value>path1</value>
     </entry>
     <entry key="service2">
          <value>path2</value>
     </entry>
</data>

Cool! Now how about json?

Custom JSON Message Body Writer

Now change the javax.ws.rs.Produces to produce MediaType.APPLICATION_JSON instead.

A message body writer for Java type, class me.kentlai.jaxrs.services.Data, and MIME media type, application/json, was not found

That was expected. Now, to handle Json specifically with a JSON message body writer using json-lib.

@Provider
@Singleton
public class JSONMessageBodyWriter implements MessageBodyWriter<Object> {
     public long getSize(final Object t, final Class<?> type, final Type genericType,
               final Annotation[] annotations, final MediaType mediaType) {
          return -1;
     }
     public boolean isWriteable(final Class<?> type, final Type genericType,
               final Annotation[] annotations, final MediaType mediaType) {
          return mediaType.isCompatible(MediaType.APPLICATION_JSON_TYPE);
     }
     public void writeTo(final Object t, final Class<?> type, final Type genericType,
               final Annotation[] annotations, final MediaType mediaType,
               final MultivaluedMap<String, Object> httpHeaders,
               final OutputStream entityStream) throws IOException, WebApplicationException {
          entityStream.write(serializer.toJSON(t).toString().getBytes());
     }
     private final JSONSerializer serializer = new JSONSerializer();
}

Accessing the page gives me the following:

{"data":[{"key":"key1","value":"value1"},{"key":"key2","value":"value2"}]}

End of this post

This is a simple post, with nothing overly hard to try nor implement.

Next up I will probably try to trace the flow of request through the components up to the response.

Update: My bad, I did not notice that if I annotated my POJO with a JAXB @XmlRootEntity/Type, it will trigger the JAXB writer.

Digging into Jersey JAX-RS: 1. setting up

So I started with a maven web application in Eclipse to do my test drive of jersey.

I added the repositories, and the jersey-server dependency (https://jersey.dev.java.net/source/browse/*checkout*/jersey/tags/jersey-1.0.2/jersey/dependencies.html)

I started out with a filter, instead of the servlet. I always prefer filters. I also added the following initialization parameters:

com.sun.jersey.config.property.packages: me.kentlai.jaxrs
com.sun.jersey.config.feature.Redirect: true
com.sun.jersey.config.feature.ImplicitViewables: true

com.sun.jersey.config.property.packages

I'm probably dumb, because I had a hard time finding out documentations for what these fields actually do. I do know that com.sun.jersey.config.property.packages tells jersey the package to find the resource classes, but all examples I saw specified only a single package.

Does it work for multiple packages? It did say packages..

The documentation did mention that the search is for the declared package and all sub-packages, so that would be fine. But how about two different packages?

I check in the source file. Multiple packages is detected by having the com.sun.jersey.config.property.packages. So how can I go about inserting multiple values?

So apparently, the documentation can be found in com.sun.jersey.spi.container.servlet.ServletContainer. Use ; as seperators.

I noticed two other properties mentioned:

com.sun.jersey.config.property.resourceConfigClass
javax.ws.rs.Application

I will probably look at them in a later date.

In conclusion: The value for com.sun.jersey.config.property.packages can be a ; seperated string value, as a list of packages to scan in. The scan works for sub-packages too.

com.sun.jersey.config.feature.Redirect

What is this? After doing a grep through the source, I found the documentation in com.sun.jersey.api.core.ResourceConfig. So apparently, what it does is this (when set to true)

request uri -> /test/abc
path declared -> /test/abc/

And the client will be redirected to /test/abc/. Hmm, seems no particular strong reason for me to turn that on. I'll keep it to false (default value) then.

I saw two other properties of interest in com.sun.jersey.api.core.ResourceConfig

com.sun.jersey.config.feature.NormalizeURI & com.sun.jersey.config.feature.CanonicalizeURIPath

They have a default value of false, and they seem to be about normalizing uri. For example, com.sun.jersey.config.feature.NormalizeURI would turn the request uri of a/b/../c into a/c. I'm not sure if it affects the routing, but I would not mind it turned on. Maybe some people like to check for intrusion attacks?

I'm not too sure what the second one does, but from what I understand after digging into the code, it seems to turn the request uri of ///a/b/c into /a/b/c. Guess it can be turned on as well.

I definitely need enlightenment on why some people might want this to be false.

It is also interesting to note that for com.sun.jersey.config.feature.CanonicalizeURIPath to take place, com.sun.jersey.config.feature.NormalizeURI must be true too.

The code to handle the normalization is found in com.sun.jersey.server.impl.container.filter.NormalizeFilter.

A filter?

Filters

Jersey supports filters! They are implemented in the two interfaces com.sun.jersey.spi.container.ContainerRequestFilter, com.sun.jersey.spi.container.ContainerResponseFilter, such that if your custom filter can care only before processing, or after processing, or if inclined, both. There are quite a few custom filters defined.

So where/how are they registered?

They seemed to be located by com.sun.jersey.server.impl.container.filter.FilterFactory (and it turns out that there are other filters like resource filter too), which the com.sun.jersey.server.impl.application.WebApplicationImpl will query and execute in order of declaration. Additional filters can be defined in the following init-params, which are executed before the apparently in-built filters:

com.sun.jersey.spi.container.ContainerRequestFilters
com.sun.jersey.spi.container.ContainerResponseFilters
com.sun.jersey.spi.container.ResourceFilters

The documentation of these can once again be found in com.sun.jersey.api.core.ResourceConfig. But in short, they are either a single string value, or a list of string values, which are fully qualitifed class names. It is interesting to note that the documentation and code will create these as singletons. That is, if you have a single class specified in all three filters, they are created as a single instance.

But where are the in-built filters defined? I had to dig deep, and found a com.sun.jersey.spi.service.ServiceFinder class. It is a way to locate services 'exported' by jars, as long as they have a folder META-INF/services in the jar, with the class files. So, opening the jersey-server jar, I found a single file with this value:

file: com.sun.jersey.spi.container.ContainerRequestFilters
values: com.sun.jersey.server.impl.container.filter.NormalizeFilter

There are more files in there, but for my current purpose, knowing this is sufficient.

There are more filters available too, like com.sun.jersey.api.container.filter.GZIPContentEncodingFilter, com.sun.jersey.api.container.filter.LoggingFilter, etc.

com.sun.jersey.config.feature.ImplicitViewables

This has to be on for jersey to resolve your jsp view implicitly. I'm curious on a few things.

1. The example had the jsp files in the root of the webapp. Convention had them inside WEB-INF. There should be a way to change that.

2. Is jsp the only option? Does it actually expose the ability to do other views like freemarker/velocity? There is mention of a com.sun.jersey.spi.template.TemplateProcessor, but how is it hooked up?

The magic seems to be in two places: com.sun.jersey.server.impl.model.ResourceClass and com.sun.jersey.server.impl.template.ViewableMessageBodyWriter.

The com.sun.jersey.server.impl.template.ViewableRule seems to be responsible for matching additional subpaths to the original resource. There is an additional match of empty/null subpath, and a match of an additional segment.

Eg, resource A with path /a/b. A request comes in as /a/b. The implicit view will be triggered. A request /a/b/c will be processed. But a request /a/b/c/d will not be processed by the resource A.

It iterates through the template processors from com.sun.jersey.spi.template.TemplateContext (it is injected, but what are the values?). And it only accepts HTTP GET method. What is the implications? I guess it would mean that if a request came in without a properly matched resource/path, it should not be redispatched to an implicit view. I am not too sure what it might really mean until I made experiments with HTTP POST.

A subclass com.sun.jersey.server.impl.template.TemplateFactory is created in com.sun.jersey.server.impl.application.WebApplicationImpl. This will then be injected into the com.sun.jersey.server.impl.template.ViewableRule. So there are two ways to have the com.sun.jersey.spi.template.TemplateProcessor hooked up. Services (as mentioned above), and providers. Providers seems to be classes marked with the annotation javax.ws.rs.ext.Provider.

Except that com.sun.jersey.server.impl.container.servlet.JSPTemplateProcessor is not marked with it! Mysterious..

But it turns out I missed out another way template processors can be injected. Via singleton instances registered with com.sun.jersey.api.core.ResourceConfig. And the creation and registration is done in com.sun.jersey.spi.container.servlet.WebComponent, which is the parent class of the jersey filter/servlet we usually registers.

Now on to com.sun.jersey.server.impl.container.servlet.JSPTemplateProcessor. It actually mentions, in the class, a configuration property used to lookup JSP files! com.sun.jersey.config.property.JSPTemplatesBasePath. Woo hoo!

Now back to com.sun.jersey.server.impl.template.ViewableMessageBodyWriter. Now that I am used to how things work, I check to see if there is a file javax.ws.rs.ext.MessageBodyWriter in the META-INF/services of the jar. Sure enough, there is one, with the entry com.sun.jersey.server.impl.template.ViewableMessageBodyWriter. And this is how it is injected into the application.

Looking at the writeTo code of com.sun.jersey.server.impl.template.ViewableMessageBodyWriter. It iterates through the template processors from com.sun.jersey.spi.template.TemplateContext again. It asks each template processor to resolve a path (which is the fully qualified class name, with . replaced with /, and append with 'index' if the path had been empty.

But wait.

The iteration does not stop with the first resolved template! All resolved template are written to! What could this mean? Potentially we could register two com.sun.jersey.spi.template.TemplateProcessor, and if both matches, could write its output out to the output stream? It does seem to indicate that..

To clarify more on that, we have to find out more about this com.sun.jersey.server.impl.template.ResolvedViewable mentioned in the class. It is the only exception to iterating over the available template processors. Who would set this?

Turns out we have to go back to look at com.sun.jersey.server.impl.template.ViewableRule. In its accept request check, it actually stops at the first resolved template processor, and set the response.

Things might be getting more confusing than required.

I will just dig into com.sun.jersey.server.impl.container.servlet.JSPTemplateProcessor first.

When asked to resolve a template, it actually attempts to locate the resource via the servlet context. If there is a perfect match, the template string is returned immediately. Otherwise it appends '.jsp' and try again. If there is still no match, a null string is returned. Pretty straightforward.

And writeTo basically commits the status/headers out to the response first, and then does a forward with the dispatcher to the new jsp.

While I'm here, I might as well find out what variables are exposed to the dispatched jsp. These are what I found and suspect what they mean (I gotta test it to find out)

1. _basePath: Original path to the resource

2. resource: Resource that handled the request

3. it: The resulting data

4. _request: Original request instance

5. _response: Original response instance

End of this post

This has been a long post (for my standard). I had approached this as a development and exploration diary, to pen down my discovery. All I had done so far has only been to figure out initialization parameters, and how the implicit view might flow.

I should stress that this is not meant to be an introduction on how to use Jersey. This is a post on understanding how Jersey works (even internally), in the hope to be able to use it more effectively. This is an exploration process, and as such, some of what was mentioned might be even wrong, different, or changed in future versions. And some of the exploration might be incomplete, but they are sufficient for the understanding I desired.

Coming up in the next post, I will follow up with a simple, barebone jersey web application with implicit views.

Jersey JAX-RS RI: An amazement

I took a longer harder look at Jersey, a JAX-RS Reference Implementation, and I was totally blown away.

Past

A background on the path to amazement.

I had heard about JAX-RS a while back. I think it was when I was playing around with Restlets. But I dismissed it off as I thought it was for web services.

I went around, playing with Wicket and Spring as well. I'll just round up my feelings. They might be outdated conclusions though.

Restlet: When I played with it, it was very much in the pre-annotations craze days. I had to add my restlets to an application router manually, and I had to extend Resource and Finder classes. I did not find out how to integrate with jsp at that time, so I was not sure if it supported it then (I did see freemarker/velocity support). But somehow, I did notice myself repeating a lot of code for different resources.

Wicket: It was a pleasant experience. Testing was easy too, and I felt very confident when developing the web application. There was no doubt, as my html and code are tested together. It was quite heavily session-based though, and turning off session/reducing the usage was harder than it was. Just when you felt used to a desktop-oriented way to developing with Wicket, you had to unlearn that and code session-less page.

Spring MVC: I tried it out with annotations, and it was truly wonderful. The resulting controller did not feel like a web-based java artifact, other than a minor hint of request/response/session here and there. Testing was easy too. But somehow, I am quite unsure of how generation of content other than html can be achieved for selected paths/resources. It felt like a tool for power user, actually.

Now

So fast forward to now. I actually took a look at Jersey again. This time I actually took a longer, harder look.

The code were really clean and simple. It was annotation-driven, with @Get and @Post and @Consume and @Produce. I just came from the Spring playground, and this felt really attractive. It was how Restlet should had been (and which they are now, as far as I know. I heard they currently had JAX-RS implementation within Restlet).

I knew I had to give it a go.

So I pondered about an implementation. I wonder if I had to go with html pages accessing JAX-RS with javascript. I wondered if I could get them to spit out html. I could, but if I had to make it annotation-driven, it would take quite a bit of effort. Then I wonder about which javascript framework would be suitable.

And as I was just going through the links on the jersey site, I saw this post: http://macstrac.blogspot.com/2009/01/jax-rs-as-one-web-framework-to-rule.html

Hmm.. so apparently, Jersey has this feature known as Implicit/Explicit views. That was really fascinating! And then I followed the link: http://blogs.sun.com/sandoz/entry/mvcj

Really very neat, it integrates with JSP! Not that I am a JSP fanboy though. I will get to that point later.

But anyway, all this is good, but I did noticed the issue James Strachan brought up, regarding Safari preferring Xml to Html. I also noticed that the Jersey implementation was registered as a Servlet with /* . That would not play well with static content. I thought to myself, that I probably could hack around a filter version later.

Today

And now, I saw on Paul Sandoz's blog about Jersey 1.0.2. Ok ok I know it was released in February. But I only got to know about it now. It added filter support, as well as ImplicitProduce!

I got excited. I decided to take a closer look at the implicit view now.

The documentation was sparse. There was nothing much on it. I had to download the samples to look at it. I checked out the bookstore sample.

The code was confusing. There was nothing much mentioned. I decided to run it.

And then it all made sense. And I was blew away.

The code was clean. Even annotations were kept to a minimal. It introduced some interesting way of resolving paths beyond the standard Path annotation.

I am really very impressed. I am going to give it a test drive now.

Summary

But to sum it up first, JAX-RS might have just become my preferred choice of web development framework (if it does not disappoint me in my test drive).

It is a very simple model. A very simple concept. Which means a very low entry barrier. I probably can introduce someone to JAX-RS development faster than Spring MVC. It would be even faster if there was a flow diagram of what components a request would go through, and what actors are involved.

And I will stick to JSP for a start, as it is also a low entry barrier choice. One might argue that velocity & freemarker are easy to pick up too, but given a handful of java web developers, there is a higher chance that you get someone who knows JSP than the other two.

I'll see if I can have a followup entry on my test-drive experience in the coming weeks.

IBM to buy Sun, but what about Java?

Disclaimer: These are my personal opinion and is not representative of where I work

The rumors and news are spreading like fire, and there are lots of arguments on both sides. They talk about cloud strategies, hardware, existing customers, etc.

But what about Java?

First, I have to say my post here might be biased. I'm not exactly in favor of this buy over. I work in a small company which is a Sun partner, which means I do have quite a bit to lose in this buy over.

Anyway, about Java.

There are many companies with a lot of stakes in Java. There is IBM (the buyer), HP, Oracle, for the players that I can think of offhand. I'm sure I missed out many others, but I have limited exposure.

Java is open. It is run by JCR. Many would have argued. But Java is still very much associated with Sun, who is seen as a steward of Java. A protector. The hero against Microsoft and .NET (though, I do find .NET and C# to be quite appealing and nicely designed).

Sun own the trademark to Java afaik. One has to wonder what happen when IBM buys Sun.

Sun is a strong innovator. Sadly, they did not manage to capitalize on them too well. For example, for tools, IBM manage to overpower them with Eclipse back then. BEA WebLogic for Application Server.

Which is not exactly a bad thing I guess. Java is truly open. It is fair game.

Now, what happens if IBM buys over Sun. Will IBM take advantage of this and start steering Java to a direction which is advantagous to themselves? Gain an edge in the speed they push out Java Standards than the others? Will the others be playing catch up with IBM in terms of Java Standards?

These are just speculations of course.

On the other hand, what about the various Sun projects? Regardless if they are open-source or otherwise, they are still very much managed by existing Sun engineers. Glassfish, OpenESB, OpenSSO, MySQL, even OpenDS. I'm not all together familiar with IBM's portfolio, but I would think they have their own offerings of Application Server, SOA architecture, Directory Server, Single Sign On, Identity Management, a

Spring TestContext Framework

It has been an exciting, and at the same time, frustrating time for the past few days.
For better or for worse, I decided to take a dive into Spring MVC and Spring Webflow, trying and playing around with it.

I walked away knowing more about Spring TestContext Framework instead.

Before I begin, I have to conclude that Spring MVC is really very neat. Webflows was interesting, and came with its own testing framework, which kinda irritates me... (they should all stick to Spring TestContext Framework instead). However, the inability to test the resulting view properly (without firing up Selenium) was a sore point for me..

So anyway, back to Spring TestContext Framework. It really is extensible.
Almost.

I could create custom TestExecutionListener, hook it up with the test class, and have it run before/after every test method!

So my first experiment was to do a MockTestExecutionListener. What it basically do is to create mocks of fields marked with @Mock (much like MockitoAnnotations). Pretty simple and easy. Then I hook it up to my test class.

The fastest way was to use @TestExcutionListeners, but I would have to declare DependencyInjectionTestExecutionListener as well, so that it would still remember to perform Spring Injection. Whatever classes declared in @TestExecutionListeners would override the default TestExecutionListener that Spring offers in SpringJunit4ClassRunner.

So instead, I extended SpringJUnit4ClassRunner, and added my TestExecutionListener in TestContextManager#getDefaultTestExecutionListenerClasses.

And then I got abit more adventurous.

I tried to reuse my mocks, and at the same time, verify that there were no more interactions in the afterTestMethod. So I kept track of my mocks in the given TestContext, and verify them.

The painful part was reusing the mocks. There is just no clean way to remove all interactions from mocks created from Mockito cleanly. Short of duplicating code from Mockito internals, cleaning delegates, blah blah blah. I'm trying to forget the ugly part of it.

So now, we have auto-mocking for our Spring Test!

And the next step. Wouldn't it be nice if some of my mocks are autowired to my Spring objects?

There is just no way it could be done nicely, especially if mocks are recreated for every test method execution.

A custom context loader can be provided, which can customize the loading of an application context.

Singletons are created and cached. And the application context are cached as well. So if two classes are using the same set of test configurations, the application context are cached across two class executions.

So it probably is out of the question.

And then I started playing with DbUnit. It is an excellent candidate for TestExecutionListener. It can auto locate a dataset xml file, initialize it for each test method invocation, and verify that the current state of the database match an expected dataset.

Only grip I had was in order to provide a DataSource to my TestExecutionListener, I had to locate it via the current test's application context.

Spring TestContext Framework is a very interesting framework, but it would be better if many methods are not made final, and better ability to declare custom classes instead of the in-built TestContext, TestContextManager, etc.