Dedicated hosting and SLA

When you are on a project with a stipulated SLA (service level agreements), like 1000 concurrent requests per minute, it is always good to be selfish.

That is, to hog all the system resources to yourself and avoid shared hosting.

What happened recently was that we had a client who asked us to develop an application and host it for them, with a certain SLA on response time. However, the client would also like the server to host another project.

We did not think much on it. However, at regular intervals, our application would slow down, with long response time. We wondered if we had to tune our application, web server, or databases.

Turned out however, that the other project hosted by the client would, at regular intervals, eat up almost 100% CPU usage for a short while!

That would surely hurt our stress test.

So we informed the client, shut down the other project, before doing our stress test.

Lessons learnt? Always be selfish and go for dedicated hosting. You might not meet a reasonable client every time.

Beware of firewalls

I'm always weary whenever I see firewalls in deployment architecture. It almost always spell problems. Problems that one will never realize during user acceptance tests. Simply because a user acceptance test environment will never mirror production environment. Hardware firewalls are not cheap.

Luckily, most firewall problems I had seen always boil down to the same thing.

Closing of idle connections.

Most applications would use connection pools, to reuse existing opened connections, for better performance. It can be for database connections, ldap connections, or any other kind.

However, a problem occurs if a connection is allowed to lie idle over a period of time.

Especially if the pool idle connection timeout is longer than the firewall idle connection timeout.

What usually happens is this:

1. Firewall idle connection timeout is set to 30 mins.
2. Database idle connection timeout is set to 60 mins.
3. Database connection timeout is set to 10 mins.
4. Connection is opened from client to server.
5. Connection is left idle for 40 mins, and the firewall closes the idle connection.
6. Client application retrieves the closed idle connection from pool, and sends a database query.
7. Client application blocks for a 10 mins before timing out, because the connection is already closed.

There are three ways to resolve this problem.

1. Set the connection to 'ping' the server while idle from time to time. This could be a select 1 query while idle.
2. Set the database pool idle timeout to be a value lower than the firewall idle connection timeout value (if you are allowed to)
3. This is the more common approach, which is to set the firewall idle connection timeout value to a value higher than the database pool idle timeout. This sounds exactly the same as the 2nd way, but people are usually more inclined to modify firewall settings than application settings.

Creating social network accounts for corporations

It's getting very common nowadays to have an account on the social network for your corporation. It could be a need to have a Facebook or Twitter presence, or creating a Facebook application for your corporation.

But regardless of the nature of the presence and account, it is undeniable that you really want the account to be owned by the corporation, not an individual employee.

Imagine that today you asked an employee to create a Facebook application or fan page. He creates it with his personal Facebook account. His personal account is thus the owner of the Facebook application or fan page. You could have multiple admins or developers, but there can be only one owner, and as far as I know, it is non-transferable. Some year later he leaves. And there goes your owner of the Facebook application or fan page. I do not think it is possible to revoke his access to the application or fan page, because he is the owner.

That sure sounds like bad news to me.

The correct approach here should be to create a new social network account for the corporation. Have all operations and management done via that account. Make it the owner of all media and content on the social network. In fact, create a corporate email address for this too. Like social@mycorporation.com.

And then have the corporation own the account. Control access to that account, and if passwords are shared, change the passwords when an employee who knows the password leaves. But really avoid too many people knowing the password. Assign additional admins or developers on social media projects with delegated capabilities.

As social media become more and more predominant, it is critical that companies develop a social media policy, and be aware and protective of their social media presence.

Do not assume what a product officially support without first checking

Just a simple tip here. Never ever assume that just because a product has a new release, that it will have the same list of supported platforms and 3rd party products.

It is common for new releases to drop support for older platforms. Or less popular platforms.

Even if you are asked if a popular platform is supported, it is best to still look the list up. Check if there are any need to first patch the system or products. Find out all possible incompatibilities and known issues,

Always look up what is officially supported by the product company. After all, if a product is officially supported you know who to turn to if there is a problem. Otherwise, you are on your own resolving the issue. And you will really wish you had quoted a premium price for helping to support an unofficially supported product.

Meeting: pre and post activities

Ever went to a meeting with the clients/vendors with no idea what to ask, discuss, or report?

Ever left a meeting without being sure of what was achieved?

Or worse, not invited to the meeting itself, and not informed before and after the meeting at all on what was discussed?

These meeting are usually missing two important activities. The pre-meeting activity, and post-meeting activity.

Before meeting up with the clients/vendors, or any other external parties, it is always good and crucial to meet up internally first. Establish a set of understanding of where everyone's progress/standing is within the team, so that when meeting the external parties, a unified front can be shown.

In additional, the project manager (from this side of the team), could collect all the questions that needed to be asked to the external party before hand. List of issues to bring up, and any progress since the last meeting could be consolidated as well. Having this list prepared before hand is helpful. Popping the questions on 'anyone has anything to say' constantly in the meeting, or doing a round-the-table talking does not show that the project manager is in control. Probably would reduce the external parties' confidence in the team too.

Do some 'paper-play' as well before the meeting, especially on the issues to bring up. Think about how to response to the enquires, and offer solutions even before they ask for. This adds to the image of being professional. Definitely will justify the money spent on the team and project.

After the meeting, hold another internal meeting again. Go through once again what was discussed. Broadcast the decisions made and discussions out to the team if they were not in the meeting with the external parties. Their work require them to be aware of what is currently happening.

Also, always come away from the meeting with an action list. It is pretty impossible to not take any actions after any meeting. The team should own at least some actions, while the external parties would own others. Come back and start distributing the actions out too. Revisit the list from time to time, and report the status of this actions on the next meeting with the external parties.

Any meeting with the external parties without doing the pre and post activities would, in my opinion, not be effective, and one would end up with a confused, demoralized, and disgruntled project team.

Workflow solutions are never easy to do well

As I do more and more workflow related projects, a similar problem kept surfacing again and again.

Basically, the idea is this. A workflow, for example, approval for a new hire changes could go on for days. The request could route from departments to departments. It could sit in the manager's inbox for days before he acts on it. Now what if, while waiting to be processed, the hiring process changes?

In a paper world, that could be handle relatively easily. If there are more signatures needed, simply find the relevant parties and ask them to sign accordingly, on the same paper request.

However, most workflow systems cannot handle this well. In fact, most of them require a resubmission of the same copy of request! That is because these systems emphasis on the workflow, rather than the data flowing through the system. A rigid workflow system proves to be significantly harder to adapt to new changes 'in-place', applying the changes to the existing running requests.

Another problem I encountered was more interesting. Most approval workflows treat the request as pure data. There had been rare cases of items requested during submission to be no longer valid when the workflow actually has to act on the data (eg creating a login account on an in house system). This is especially possible for scheduled tasks, where you actually plan for usage in the future, either activation of new accounts or deactivation of existing ones.

Another interesting issue but non-technical has to do with the presentation of information during request submission. Let's say that you have a company going for a role based access policy. A manager role would have access to system A and system B. When a user request for a manager role, he is told that he would be given accesses to system A and system B. Now, while the approval workflow is going on, the policy changed. A manager role will instead only have access to system A. The right business implementation would do just that. Give access to system A. But the user might not be informed of the policy change. All he sees is that he is missing access to system B. Nothing wrong technically, but just some confusion. The workflow had never planned to inform the requester on the changes.

And another thing I keep seeing is the need to view the history of a request. What a user requested, when was it approved, when it was granted, etc. These are really request specific, and so each action must be audit manually. Luckily, the implementations I had been involved had mostly forsee these, and planned them in early. But there are cases where we had to modify the workflows to capture even more data beyond the requirements, as the client think up more and more of what they need.

Of course, most of these are beyond the responsiblities of the workflow engines we use. The solution provider should really plan to resolve these as part of the implementation. And they should be planned early. They are not easy to solve, but it will surely make for a much better user experience.

Application Internationalization and Localization

First off, a definition of what these are.

An application usually can and will be used by people from diverse cultural and language speakers. And so there will be demand for the application to speak to them in their own culture (date and money formats) and languages.

And so the act of creating culture and language specific versions is known as localization. But before that, you had to perform internationalization to extract all possible.

I have done two larger projects that involve internationalization of applications, with various smaller ones, but only one that actually involve localization.

And they sure are a pain to do. They use very simple concepts. Simply extract all string from the existing application and put them in a property file. And define a format string for each money and date format use, and place these in property file as well.

But if you treat each string as a unique entry, you tend to end up with alot of duplicated string. A message like "Approver name" could appear in both a form submission page, and a form status page.

It is extremely tempting at this point, to simply allow the two placeholders to use the same string entry. And in fact, that's a perfectly reasonable and valid approach.

Until the day come with you gotta change the text themselves. Imagine that, someone decided to use the message "Choose an approver" in the form submission page. Before the change to the message, he has to first evaluated the impact of change. Like, how many other parts of the UI is using the same string entry?

It is only when there are no other parts of the UI using the same string, could the developer make the change safely. But if it is actually used n many other areas, he gotta duplicate the existing entry, give it a new message id, and update that only.

So be careful of merging and reusing messages. Do them only if both are the same message, within the same context. A message string that is used during the selection of an approver, and one that is used to display the approver might have the same string, but they are in different context.

Another point in hand, is to really hire someone who is proficient in both the language translating from and to. The least worry you want to have is on the grammer and spelling related mistakes.

And lastly, avoid concatination of messages. You might have a message that goes like this: "Your account expired on XXX. Would you like to renew?"

It is tempting to have two string here, "Your account expired on ", and "Would you like to renew?". Do that, and you might wish you had never been involved in the project when there are message changes. One day the message might be changed to "XXX: Account expired. Renew now?"

It is better and cleaner, instead, to use a single string with placeholders like this: "Your account expired on {0}. Would you like to renew?". That way, you get the absolute flexibility of having the formatted value anywhere in the message, and you get to keep the message as a whole. During localization to other languages, the context of the message is clearer to the person working on it as well.

But granted, what I have described are simple principles and techniques, but doing the actual work itself is never easy. Especially when you are doing a 'extract message id' along the way you develop in a team environment. "I need a new message, does it exist? I have no idea. I'll just add it and hope someone will use it later". And then down the road the same message, of the same context, might have like five entries, all used in different part of the UI, just because they were all added by five different people.

So lastly, I proposed the addition of a internationalization owner, who owns, manages, and gives out message id. Tell the person the message you want, and the context, and the person decides if it should use an existing message id, or a new one.

I have not seen it in work, but it sounds feasible. Any comments?