Using Amazon S3 in a Different Way

You might have heard about Amazon S3, which stands for Amazon Simple Storage Service. Most people use it for backup. I will tell you a story about how I found a different way to use Amazon S3.

During the summer I felt a deep urge to implement a JDBCAppender for Microlog, like the JDBCAppender found in Log4j. In situations when the on-device logging is insufficient, this would give the developer a chance to log into a database. When searching the Internet I found less than a handful of JDBC implementations for Java ME with CLDC. Most implementation for Java ME devices was CDC compliant. Another problem was that I was to lazy to setup a database. Last but not least, my knowledge of JDBC was a little bit weak since I had not done any JDBC programming for many years. Soo my initial urge was gone like the wind.

But still I felt that there was a need for off-device logging in Microlog. How about Amazon S3? Could it work? Since Amazon S3 could be accessed with REST and/or SOAP, it should be possible to access it from Java ME. The developer resources for Amazon S3 are rather good, and it came as no suprise to find a J2ME Toolkit for Amazon S3. This was perfect for me since it was open source, and is using the Apache License 2.0. This is the same as we use for Microlog.

I coded two appenders for Amazon S3; the S3FileAppender and the S3BufferAppender. The S3FileAppender logs to file. When the file reaches a certain size and/or you log at a specified level, the file is copied to your S3 account. The S3BufferAppender logs to memory instead of a file. This is useful for applications that do not have access to JSR-75, i.e. file access.

But how do you use the S3 logging capabilities of Microlog? It is dead simple. The code look like this:

public S3FileLogMidlet() {
   S3FileAppender appender = new S3FileAppender();
   // You must set the accessKeyID & secretAccessKey in order for this to work.
   // For more information, see your Amazon account.
   appender.setAccesKeyID("accessKeyID");
   appender.setSecretAccessKey("secretAccessKey");
   log.addAppender(appender);
   log.setLogLevel(Level.DEBUG);
   log.info("Setup of log finished");

   log.error("This error message shall trigger file => S3.");
}

As you can see, you need to insert an access key id and a secret access key. These are obtained from the Amazon AWS pages. If you have ever bought something on Amazon, you already have an account. The only thing that you have to do is to active the AWS service. When you do this you could access these auto-generated codes. Copy and paste them into your code. A word of caution. Be aware that if you somehow loose your mobile device or your account details, you have to change your codes at Amazon AWS. Another solution would be that you enter these codes when starting your MIDlet or that you store them encrypted in a file. But do not say I did not warn you!

The rest of the code is simpe. The appender is added to the logger and the log level is set to DEBUG. The S3FileAppender is by default set to send the file to the S3 server when an ERROR or FATAL message is logged, or when the file is to big. But if you like to send the file only when the file is big this is possible as well. It should be noted that this setup could also be done in a properties file. Please take a look at the Microlog documentation for more details on the different setup options that are available.

When you have compiled and transferred your MIDlet to your device, the logs will be saved on your Amazon account. I use the Jungle Disk application to mount a virtual drive. The logging file will be accessible in your Microlog folder. It is as simple as that. The Jungle Disk application is available for $20 for the version that I use. There are nearly a zillion solutions for Amazon S3, and one of those will fit your purposes.

The benefits of using Amazon S3 for Microlog are many;

• No need to spend time on setup and maintenance of a database. This will save many $ for you.
• It is cheap to store the logs at Amazon S3. Even if you log huge amount of Microlog data, it will be very cheap. For example; let's say you store 1 GB of Microlog data. That would cost you $0.20 for the transfer and $ 0.15 if you save it on the server for a month. I believe that under normal circumstances you do not store it for that long.
• No need for JDBC knowledge. You only need to learn how to use Microlog.
• Focus on the development.

The only drawback is that the security could be an issue. However as explained, there are workarounds for this problem.

I would like so say some words about the future of S3 logging in Microlog. There is room for improvements. For example, I think it would be useful to log several users log files. As it is now, the S3 appenders are only for a single user. Of course there are other improvements, some of which I am unaware of. Maybe YOU are the one that could help me improve Microlog? Please use the Microlog forums if you have any suggestions.