Premature Generalization is the Root of All Evil

Finally! I have now finished the first reincarnation of the new logger hierarchy for Microlog. It is a very simple tree structure that keeps track of all the created Logger objects. I choose to do a specialized tree implementation for the purpose, not a general tree implementation. Since there is no tree implementation in Java ME, it could be tempting to do a tree implementation for Java ME. When finished this could be re-used in a lot of other Java ME projects. This got me thinking about those good old days when I was a junior programmer, just fresh out of school.

I was involved in a research project with the purpose of showing the opportunities with an embedded device with an ethernet connection. The embedded device contained a simple web server that was equipped with simple CGI support. My assignment was to create a web pageswith a Java applet for administration of the embedded device. This was actually a proof of concept study. If we managed to convince our product managers about our genius invention, we would of course re-use the code for the real product. So far, so good.

I made a very ambitious design before even starting to do some coding. I can tell you, my manager was really nervous when I did not start code immediately. "No worries, good design is important for fast and good implementation. You need to be prepared really good before starting your implementation!". This was how I was taught. Anyway I managed to do a real fancy design, with many really nice-to-have-classes. I ruled the world of design and my way to fame and glory was laid out for me! As part of the design I did a very general API for the communication. General design is good, I reasoned. This was a BIG mistake!

When we were closing in on delivery date for the demonstration, I had made the general network communication. However I was not finished with real implementation, the parts that was vital for the demonstration. Anyway I was finished about 30 minutes before my project manager was going to leave for the airport, to meet the people who was sponsoring our research. I was tired since I have worked all night and my project manager was not happy. For me it was embarrassing that I was not able to deliver something that I was proud of and I was to close to the deadline. My project manager did manage to do a successful demonstration. Finally the management decided to make our prototype into a real product.

After a couple of weeks it was decided that we was going to use another low level protocol for the communication. It turned out that my general design was useless, since I did not anticipate all the aspect of the problem. But what did I learn from this rather embarrassing moment of my life as developer?

First of all I realized that I do not have the powers, like Nostradamus, to see into the future. I also came to the conclusion that you have to know the domain, before making any generalization. You have to do your first implementation in that domain, before you should make any general design. When you continue to evolve your product, you can figure out what parts of the code that is reasonable to generalize.

So I invented a rule that is "Premature generalization is the root of all evil design". It is of course inspired by "Premature optimization is the root of all evil". Nonetheless I think it is a good rule. This is why I choose to do a specialized tree structure for Microlog, instead of making a more general tree implementation. I have seen the phenomena of "premature generalization" in many projects since. For some reason it tends to be junior developers who fancies these general designs. Today many more people learns design patterns, than when I finished school. There is a big risk that you use to many design patterns, without really understanding the consequences. I prefer to get the code working and to deliver it in time for the demonstration. When adopting my golden rule stated above, this is not a problem. Thanks to agile methods, like Scrum, you are forced to make your application ready for demonstration early on. But that is another story.

Microlog Group @ LinkedIn

One networking forum that I really like is LinkedIn. It is a proffesional networking forum, not focusing on the social aspect like Facebook. I have learnt to know a lot of people through LinkedIn. Many of my present and former collegues and friends are there as well. There are many interesting groups at LinkedIn, like the FOSS Proffesionals Group.

I have had the idea of creating a Microlog group lingering around in my head for some time. During the weekend I finally decided to create a Microlog grup. When the decision was made, the registration was simple. I used the new project description and added a group description. Unfortunately the existing Microlog icon did not comply to the required criteria, but I settled with the LinkedIn default icon.

Please join the Microlog group at LinkedIn.

How to do Logging

I tend to get into a lot of discussions about logging. It could have something to do with my Microlog development. Sometimes I find it hard to argue what is good practices when it comes to logging. So I searched the Internet and found these good articles on the subject:

• 7 Good Rules to Log Exceptions
• 7 More Good Tips on Logging

I hope that you read them thoroughly, because the are definitely worth reading. But until then you are maybe interesting on my comments about them?

Starting with the first article, I must say that all of the tips are really good. From my point of view, the last rule "7. Do not log with System.out or System.err" is especially important. The first sentence says it all "Always use a log framework, it can handle logging better than you." Of course I want you to use a logging framework. If you are working with Java ME or Android development, please try out Microlog!

The second article is a little more general. The first advice is a little bit tricky; "No debugging logs in production". Microlog has no built-in feature to solve this, so I usually recommend to use the pre-processor found in MTJ. If you are running in other environments, like vi, you could use the ProGuard pre-processor. The second advice "Look through your logs" is easy to do with Microlog, if you log to a server. Microlog has the capability to log to several types of servers. The easiest would be to log to a syslog daemon or to Amazon S3. Most syslog daemons have some sort of filtering feature and export functions. This should make it easy for you to filter out the interesting stuff. The 3rd rule; "Never log locally" is as you now understand fairly easy with Microlog.

I hope that these articles will give you some good advice that you will be able to use.

Microlog & Microproperties Updates

This weekend I decided to start the replacement of the old Microlog configuration with the new one; Microproperties. Ironically this work was to copy the file reading part from Microlog to Microproperties. When this was done and tested, I decided to do a new Microproperties release. This is the new V0.2.0 release. When this was done I added Microproperties V0.2.0 to Microlog.

However this weekend is approaching its end and I decided to do the first snapshot release of V2.0.0, before continuing my work. The Android extension is included in this release. My hope is that developers who prefer the Log4j API will find Microlog for Android an interesting alternative. Please download and try it out!

Using Microlog for File Logging on Android and Viewing Files on Android Emulator

My goal yesterday was to re-factor the FileConnectionAppender in Microlog. I wanted to separate the logging part from the actual file writing. The logging should be in an abstract super class, while the file writing should be in the the concrete class. The purpose of this exercise would of course to be to support file logging on both Java ME and Android platform.

It took some time to create a good structure in the new AbstractFileAppender and the FileConnectionAppender. When this was done, it was a piece of cake to create the FileAppender (for Android). I used the Eclipse class wizard to create a sub-class to AbstractFileAppender. Most of the time was spent browsing the Internet and looking in the Android documentation. Then it was a matter of implementing a handful of methods. I had never tested to create and write to a file on Android before, but I was curious and executed the code directly when most of the methods was finished. This excluded the flush() method for example. The diagram below shows the new class structure.



The next step was to figure where to find the log file. It was not long time before I found a file with a .img extension. From some documents found on the Internet, I could gather that this was a disk image. But how do I read the data found in the disk image? As it turns out there is several ways to do this:

1. Use the command line tool called adb
2. Use the file browser found in Android DDMS (Dalvik Debug Monitor System)
3. Use the file browser that is part of the Android plug-in for Eclipse.

I will show the 2nd way. If you manage this, you will almost certain be able to the 3rd way. All those shell lovers will have to take a look at the documentation for adb.

You start by typing ddms in your shell window or command prompt, depending on your operating system. This will bring up the DDMS tool. Select the device and use the Device->File Explorer... menu to access the file explorer, see picture below. You could now browse the file system in your Android emulator. In our case we browse to /data/data/net.sf.microlog.android.example/files. This is where you will find the file microlog.txt. If you wish to see the content of the file, which I gather that you do, you press the button marked yellow below. This will pull the file out of the Android file system to your file system. You get the chance to select where you want to save. Pick up your favorite text editor to view the marvelous content of your log file.




It is as easy as this to view a file on the Android file system. I hope you found this tip helpful.

Microlog4Android

I have continued with the re-factoring of Microlog. Tonight my plan was to re-arrange the package structure to separate the Microlog core from the rest of the code. One reason for this was to make it easier to make different flavors of Microlog. The Microlog core should be possible to execute on CLDC. The plan was to move the MIDP specific classes to its own package structure. One new flavor that I had in mind was the Android flavor of Microlog. Many people have asked me about Microlog on Android. My answer has always been something like "It should work on Android, at least the core.".

After the re-factoring it was time to prove my point, that it is possible to execute Microlog on Android. And it did! The setup was simple, it was Microlog with a ConsoleAppender and a SimpleFormatter. Not so very much at the moment, but I think it is a good starting point.

During the process I learnt that it was in fact possible to see the System.out.println() statements. Many people claim that it is not possible to see the System.out.println() in the Console. That is true, but it is possible to get hold of the prints. The trick is to show the "LogCat" view. This is activated this way:

1. Select "Window->Show View->Other..."
2. Open up the Android category and select "LogCat"

You can now see the output from System.out.println() and System.err.println(). A nice side effect is that you now can see your Microlog statements as well!

One might ask why should I use Microlog on Android, since there already is a logging API available. This is a perfectly legitimate question. The reason would be that Microlog brings a Log4j like API to the Android platform, but the supplied API is the java.util.logging package. I have always preferred the Log4j API, which was the reason why I choose it for Microlog in the first place. But I guess that you could use Log4j on Android, but Microlog is designed for constrained devices, which Log4j was not.

Microlog Work During the Weekend

During this weekend I finally started my work on the new Microlog V2.0.0. The first thing I did was to implement support for client identification. I did it by adding a property to the Logger that is called clientID. The clientID must be set manually. Since I plan to change the configuration classes, I did not put any effort in adding support for this in existing configuration classes. This will be implemented when I change to to Microproperties.

I have added support for showing the clientID. The SimpleFormatter and ConfigurableFormatter always prints the client ID, if not null. It is possible to use the PatternFormatter as well. You use the %i to print the clientID. It is as simple as that.

The developer have the option of setting the clientID manually or letting Microlog create a unique clientID. I still need to figure out an automatic way of creating a unique clientID. I will have to get back on that.