Mar 10, 2009

How to do Logging

I tend to get into a lot of discussions about logging. It could have something to do with my Microlog development. Sometimes I find it hard to argue what is good practices when it comes to logging. So I searched the Internet and found these good articles on the subject:
I hope that you read them thoroughly, because the are definitely worth reading. But until then you are maybe interesting on my comments about them?

Starting with the first article, I must say that all of the tips are really good. From my point of view, the last rule "7. Do not log with System.out or System.err" is especially important. The first sentence says it all "Always use a log framework, it can handle logging better than you." Of course I want you to use a logging framework. If you are working with Java ME or Android development, please try out Microlog!

The second article is a little more general. The first advice is a little bit tricky; "No debugging logs in production". Microlog has no built-in feature to solve this, so I usually recommend to use the pre-processor found in MTJ. If you are running in other environments, like vi, you could use the ProGuard pre-processor. The second advice "Look through your logs" is easy to do with Microlog, if you log to a server. Microlog has the capability to log to several types of servers. The easiest would be to log to a syslog daemon or to Amazon S3. Most syslog daemons have some sort of filtering feature and export functions. This should make it easy for you to filter out the interesting stuff. The 3rd rule; "Never log locally" is as you now understand fairly easy with Microlog.

I hope that these articles will give you some good advice that you will be able to use.

31 comments:

Fiora said...

Hi,

nice blog!!

I'm whith some doubts about microlog can do or not.

I have to log the output of a third program, but I dont have access to its code. So, there is no way I can input some code direct to the program.

I want to know if microlog can log the output of a third program whithout access to its source. Something like redirecting its output.

Thanks.

My Open Source Software Development Blog said...

Hi,

first of all I thank you very much for your kinf words. It is always nice to know what people think about my work.

Second I think that Microlog is not what you are seeking for. The ConsoleAppender found in the Java ME version of Microlog, uses System.out for logging. But the System.out is not possible to re-direct on Java ME platforms. Compared to Java SE, this is a clear disadvantage. But it could be possible to re-direct on the Android platform.

On some mobile platforms, such as Symbian,it is possible to re-direct your output from an application you do not have access to the source code. I have written about this in the FAQ section for Microlog:
http://microlog.sourceforge.net/snapshot/microlog-logger/faq.html

If you give me some more details on your platform, I could answer your question with a little bit easier and more concrete.

Regards
Johan

Fiora said...

Wow,

thanks for the quick answer Johan.

I work as a tester and I'm trying to develop an automated test tool for mobile app. The main purpose is to test J2ME app, but we want test native app too(for Symbian, Windows Mobile and Android).

The biggest problem I'm having now is to do the asserts, because I dont know how to access the output of a program. So I woul like how to log its output whitout modifying its source code.

About the redirect of System.out and System.err, see this article(it really works):
http://wiki.forum.nokia.com/index.php/How_to_get_System.out_output_from_a_MIDlet_and_save_it_to_a_file_in_S60_devices

Thanks,
Fiora

My Open Source Software Development Blog said...

Hi,

it is maybe something like Motorolas Gatling test framework you are looking for? It actually uses Microlog for logging on the devices :)

Read more about it here:
https://opensource.motorola.com/sf/projects/gatling

Regards
Johan

My Open Source Software Development Blog said...

BTW I have not used the Gatling framework, but the test organisation at my current work uses it with great success.

Regards
Johan

Fiora said...

Ummm..... I already read gatling and I had the impression that it test APIs.... but no problem, I'll ready it again. =)

Well, what do you test at your work? APIs or (GUI, Usability, Performance, Compatibility,...) testing?

Thanks,
Fiora

My Open Source Software Development Blog said...

Hmmm... I guess you are right, the Gatling project is not suitable for GUI testing. My mistake...

They use it for API and performance testing at my work.

Sorry, but I am out of ideas :(

Regards
Johan

Fiora said...

Dont worry, and thanks for all the explanation and help.

And continue with microlog! It's a usefull toll!!

Fiora.

My Open Source Software Development Blog said...

Thank you very much!

I hope that you find what you are looking for.

Regards
Johan

Chethan said...

Hi,
I am using Micro Log.Every time i try logging it overwrites the existing logs. How do i avoid the overwriting?

Thanking you,
Chethan

My Open Source Software Development Blog said...

Which appender do you use? I have changed the latest FileAppender to append at the end, without overwriting existing data. This will be available in the next release.

Chethan said...

Hi,
Thank you. I am using the file appender.This is what i am doing
this.log = LoggerFactory.getLogger(SimpleLoggingMidlet.class);
FileAppender appender = new FileAppender();
PatternFormatter formatter = new PatternFormatter(); //Creates a PatternFormatter
formatter.setPattern("%t %d [%P] %m %T");
appender.setFileName(SimpleLoggingMidlet.getApplicationProperty(FILE_URL));
appender.setFormatter(formatter);
log.addAppender(this.appender);

When will the next release be available?

Thanking you,
Chethan

My Open Source Software Development Blog said...

Hi Chethan,

the code is already available as source code via Subversion. I will make the release within a couple of days.

I will get back when the release is available.

Regards
Johan

Chethan said...

Hi,
Thank you. From the svn what are the projects i need to download. Can you kindly help me with this.

Thanking you,
Chethan

sam said...

Hi,
I am trying to log using the file appender,i use the following pattern "%d [%P] %m", but the time that it displays on the logs is not the device time. Am i doing something wrong?


Thanking you,
Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

There is a new release available for you to download. I hope that you enjoy it.

My Open Source Software Development Blog said...

BTW I think that %d should print the date. What happens? Is it skipped totally? Do you get any exceptions?

sam said...

Hi,
It logs the time, but the time that gets logged is not as per the device time. The device time is 11:34 am but wot i see on the log is as below:
06:06:18,756 [DEBUG] Started application
06:06:26,526 [DEBUG] Processed Randomn Number
06:06:28,255 [DEBUG] Processed Randomn Number
06:06:28,599 [DEBUG] Refresh
06:06:29,826 [DEBUG] Processed Randomn Number
06:06:32,75 [DEBUG] Notify
06:06:36,742 [DEBUG] Processed Randomn Number

Also i dont see the date too. The pattern i specify is "%d [%P] %m" Am i doing something wrong?

Thanking you,
Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

Hi,

I looked through the code and I get a little confused on how I once did it. My first feeling is that something is wrong with the timezones. I use System.currentTimeMillis() to get the time. This time is used to create a Date object. The Date object is then feed into a Calendar. The Calendar object is set to GMT.

So this is a timezone problem. The question is if this is a bug or is a problem in the platform?

What timezone is your device set to?

Regards
Johan

sam said...

Hi,
Thank you. The time zone is set to GMT. As you rightly said i guess this is platform specific, because when i tried the same on a blackberry device the time seems to be fine, but on a Nokia E71 it fails to give the correct time. Another thing is that i do not get the date on the logs. Am i missing any pattern?

Thanking you,
Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

If you want to log the date format you could use this pattern:

%d{DATE}

One solution would be to skip using the Calendar and implement a custom conversion. But I guess that this is rather complicated. It would also increase the size of Microlog.

I wonder if there is any workaround for this problem? Any ideas?

sam said...

Hi,
I am using the %d pattern but as seen in my above post i fail to get the date..:-(

My Open Source Software Development Blog said...

Ok, sorry that I am not expressing me clearly.

1. There is a bug, that on some platforms that make the time wrong.

Q: Is there a workaround for this? Could the code in Microlog be changed to fix this?

2. If you want to show the date instead use the following pattern:

%d{DATE}

===============

Is it clear now? :)

sam said...

Hi,

Thank you. Now i see the date..:-)



Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

Nice that I could help you out.

sam said...

Hi,

Is it possible to set a size limit on the Log file and once the size is exceeded to continue logging on a different file?

Thanking you,
Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

I am sorry to say no, but this is on the to-do list.

Regards
Johan

sam said...

Hi Johan,

Thank you. I am trying to get the size of the log file by using the following :

this.appender.setFileName("C:/data/images/abc.txt");
System.out.println("File Size:" + appender.usedSize());
System.out.println("File Size:" + appender.totalSize());

Both usedSize() ans totalSize() return -1. Am i missing out on something?

Thanking you,
Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

Hi Sam,

there is not guarantee that the underlying OS is returning anything valid. I have tested it on Sony Ericsson and some Nokia phones.

Regards
Johan

sam said...

Hi Johan,

I am trying to send the logs to the server. To do this i first close the appender then read the contents of the log files and send it to the server. Now when i try to open the appender it fails. How do i acheive this. Kindly help me with this.

Thanking you,
Regards,
S.A.Norton Stanley

My Open Source Software Development Blog said...

What kind of server do you wish to log to? There are several appenders for different kinds of servers. For example there is the HTTPAppender that logs to a HTTP server.

Anyway there is no need to do any manual copying for server logging. Of course you could log to a file and then manually copy the file, but I believe that is not what you are looking for?