Visualize Apache Log on Google Map

I'm lately playing by mapping Apache log on Google Map. This is it, named Access Visualizer. (I recommend you see it with Google Chrome. Some browsers cannot handle some features in the app.)

If you put your mouse cursor on a marker, you can see the next information:

• ISP
• Organization
• Country
• Region
• Coordinate (latitude, longitude)

I used a witty service that can convert IP addresses into location information. And this service allows you to use their functionality by API (XML-based REST).

What I did to create the app is:

1. consider the DB structures and create DB tables.
2. write a shell script and some PHP scripts to parse Apache log, run queries, and store the data into MySQL database.
3. write a PHP script that automatically generate Javascript code which utilize Google Map API.

Not so difficult tasks. Let's go deeper into each section.

1. is an interesting part. Since the query number is limited to 100 times a day, I think it's good to store IP-location data into my own database so that I can re-use the same queries previously used. In the current IP address scheme, there's 2^32 IP addresses, which is about 4*10^9 in decimal. But an organization has a several, or hundreds of global IP addresses. So considering an efficient database table structure is interesting!

2. is kind of tedious. Just wrote a shell script and PHP scripts.

3. is the first time for me to use Google Map API. I actually tried to use the API before, but I gave up it since the use was restricted and some registration was needed at the time. But now its version is updated, and now, you can utilize their APIs without any registration and restriction!! Yikes, thanks Google!!

The future trials:

With lots of markers, there are some parts you cannot see. Japan is hidden and unseen due to the markers now lol So I'm considering to create some layers that control whether show a marker or not. Some markers are to be unseen when zoom-out, but be seen when zoom-in.

Plus, public location information about IP is not so accurate unfortunately . For example, KDDI, University of Tokyo and other organizations are located in the exactly same point according to utrace. Due to this, you cannot see the marker for University of Tokyo and KDDI. I have to scatter the points at the exactly same point in some manner.