Javier Thoughts

Well, I don’t know if technically you can call it crash when you don’t move an a car tries to run over you. But for my poor beamer it was. Thanks god I got frame sliders last week.

The day didn’t start really well. Some technical problems in the office while I was away early in the morning. I decided to pass by to check if everything was ok. Looking for a spot to park I passed one and I decided to turn and park the motorbike. Bad idea.

The parking spot was close to a garage door. I turned the motorbike almost 90 degrees to park it and I saw the garage door opening. I stopped till I could see what the other driver wanted to do. Well, I was in the road and the driver didn’t had priority there there, but in Greece you never know (and unfortunately that turned true ).

The driver started to leave the garage as I wasn’t there. It was quite fast. I just remember me stopped watching the door opening. The rounded lights of the car. An old golf I said to myself. I saw the driver starting to leave. And one second later the car was hitting my front wheel and trying to pass over me.

Of course the other driver has another story. That she stopped when she saw me coming and I runed over the car. Plus when she saw me she expected me to go reverse. Also that first she saw me balancing my motorbike and then I fell (yea, first you hit me and then the front of you car goes over my wheel). And that I shouldn’t try park near his parking (nice excuse to pass over anything that moves, I like it). There were other storied but they were so unconnected that they’re difficult to remember.

Bikes with reverse, I only know the Yamaha goldwing and the new Piaggio mp3 hybrid. Any other motorbike or you step down and you push it. Or you slowly with small steps go backwards.

In the pictures you can see that the frame slider didn’t draw any line in the road. The motorbike just fell while it was stationary. And how the car was trying to pass over me (literally).

I believe the driver didn’t see me or she thought there was enough space for her to pass. The fact is that is the second time a car hit one of my bikes from a stationary state because…. and prepare yourselves…. Most motorbikes don’t have reverse!. Repeat it 1000 times.

An interesting thing: two motorbike drivers came to help me. They asked me if I was ok. They weren’t interested in anything else: are you ok? do you need any help? Thank you both.

A bad point goes for some neighbors that weren’t there but were saying that the car coming out the parking had preference and that I should had avoided that somehow. Sorry guys, I’ll get a Goldwing next time with reverse or an helicopter.

A grey point goes to the policeman there. He tried to be professional. But there are some things to improve. While he helped me to get the motorbike up (Thanks). He wanted to do it with the car over it. So we would have hit the car again and for sure break something in the motorbike forks (Uh!). Or perhaps he wanted to “drag” it. At least he realized after some shouting and ordered the driver to put her car back some meters. (I’m sorry for the shouting though).

He also wanted to make a point saying that I should maneuver only when I’m sure I can do the entire maneuver. I was sorry to tell him that the reason I stopped was I saw the door opening. BTW: the policeman came in the street driving in the wrong direction. Just imagine a car leaving a parking spot. A motorbike moving in the wrong direction. The perfect ingredients for an accident

Anyway, sorry for posting my rants, just wanted to make it public. I wasn’t annoyed about the accident. The motorbike is just a piece of metal. What broke my nerves was that someone without any priority runed over me and I had to defend myself!. Next time it would be a moving motorbike, a bicycle or a baby stroller. Too bad and quite sad.

We use Zabbix were I work. It merges Cacti and Nagios also it makes management a breeze. You can build an architecture based on proxy services collecting measures from near machines and send them back to a Zabbix server.

We had a small problem in the proxy system. It can store measurements for a while without any data transfer to the server. But after a while the proxy database fills and the proxy starts slowing down. We blamed the virtual machine the proxy resided for the poor performance. But the performance was soo poor that perhaps there was a problem with Zabbix or our network connection between the proxy and the server.

I wan’t my measures back!

After a weekend full of problems. Monday was waiting us with 2 days missing on Zabbix, 3 million measurements pending and a zabbix proxy killing the poor mysql process inside a small virtual machine. The Zabbix proxy wasn’t able to catch up with the current measurements and mysql was agonizing due to iowait.

I got the Zabbix source code and stripped everything but the server sending routines. I wanted a tool to do only the job of sending the measurements from the proxy to the server. It was a butcher’s job, but it worked quite well.

The culprit

After the first run the zabbix proxy logged this query:

mysql> explain select p.id,h.host,i.key_,p.clock,p.timestamp,
p.source,p.severity,p.value,p.logeventid 
from hosts h,items i,proxy_history p where h.hostid=i.hostid 
and i.itemid=p.itemid and p.id>21824754 order by p.id limit 1000\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: p
         type: range
possible_keys: PRIMARY,id
          key: PRIMARY
      key_len: 8
          ref: NULL
         rows: 1446768
        Extra: Using where; Using temporary; Using filesort
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: h
         type: index
possible_keys: PRIMARY
          key: hosts_1
      key_len: 66
          ref: NULL
         rows: 5
        Extra: Using index; Using join buffer
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: i
         type: eq_ref
possible_keys: PRIMARY,items_1
          key: PRIMARY
      key_len: 8
          ref: zabbix_proxy.p.itemid
         rows: 1
        Extra: Using where
3 rows in set (0.05 sec)

The temporary and filesort messages explained everything: When you don’t have much data or your mysql server is quite powerful you won’t have any problem but if you use a small VM with poor IO. It will take… last time it took 3 minutes

The solution

Reading about mysql and filesort I got to a nice site (http://venublog.com/2007/11/29/mysql-how-to-avoid-filesort/) about how to avoid filesort. And there was a very nice example where you put the problematic table as a right outer join and you get mysql to use the index for the order by.

And from 3 minutes to a tenth of a second:

mysql> explain select p.id,h.host,i.key_,p.clock,p.timestamp,
p.source,p.severity,p.value,p.logeventid 
from hosts h inner join items i on h.hostid=i.hostid 
right outer join proxy_history p on i.itemid=p.itemid 
where p.id>21824754 order by p.id limit 10\G
*************************** 1. row ***************************
           id: 1
  select_type: SIMPLE
        table: p
         type: range
possible_keys: PRIMARY,id
          key: PRIMARY
      key_len: 8
          ref: NULL
         rows: 1011520
        Extra: Using where
*************************** 2. row ***************************
           id: 1
  select_type: SIMPLE
        table: i
         type: eq_ref
possible_keys: PRIMARY,items_1
          key: PRIMARY
      key_len: 8
          ref: zabbix_proxy.p.itemid
         rows: 1
        Extra: 
*************************** 3. row ***************************
           id: 1
  select_type: SIMPLE
        table: h
         type: eq_ref
possible_keys: PRIMARY
          key: PRIMARY
      key_len: 8
          ref: zabbix_proxy.i.hostid
         rows: 1
        Extra: 

Final words

Without open source I couldn’t have tracked this so easily. So next time you choose a tool, check if you get the code with it or a black box.

Tomorrow time to send the patch to the Zabbix devs. I love to help to make nice tools better.

Also a big “thank you” to my wife and our sysadmin. He stayed till I made this work (good company really makes a difference). She didn’t complain after 12 hours in the office.

Waking you up for a rafting day.

25th of March. Greece independence day. A call from Mr. Fazer the day before and the weather forecast telling of a sunny day made the decision easy.

The route included the following landmarks:

• Mount Kitharon (1400m over the sea level)
• Porto Germeno
• Psatha
• Alepochori
• Megara

The road form Mandra to Porto Germeno welcomes you with nice turns over a better-than-expected tarmac. Still there are some pending road works (normal in Greece). It’s not a fast road, but it has all kinds of turns to learn to handle your motorbike.

There is a special mention to the one lane road to Mt. Kitharon. Do it for the views: it deserves the 20 min uphill. The tarmac isn’t good and it’s has many places with small stones. We even faced a “piece of the mountain” in the middle of the road. Not for cars, that’s for sure.

The coffee was in Psatha. Beautiful beach. I had my first cold coffee in this year. Summer is close

I leave you here some pics of the trip:

I’ve upgraded my MacBook recently. This included a clean Mac OS 10.6 installation. Before making everything “dirty” with “eggs” and “gems”, I decided to organize the development environments from the very beginning.

Python

Let’s start with Python. Mac OS 10.6 comes with python 2.6 preloaded. I wanted to install Python 2.7 and 3.2. This was the easiest part:

• Go to http://www.python.org/download/
• Grab the version/s you want to install. They’re packaged as dmg files
• I used the 64 bit version. But this depend on your Mac OS installation.

These versions will be installed in the /Library/Frameworks/Python.framework/Versions folder. With a link to the “Current version” and a bunch of executable links at /usr/local/bin.

The installations will modify your $HOME/.bash_profile. They’ll add their paths, so after installing 2.7 and 3.2. The first python executable will be the 2.7 version. My personal preference was to disable them and wait till I setup virtualenv.

Setting up virtualenv

In order to use clean environments it’s time to install: virtualenv and virtualenvwrapper. This will allow you to build virtual python environments without installing packages on the system/root python.

My personal preference was to install pip, and then virtualenv & virtualenvwrapper in the pre-installed python 2.6. I used sudo to have them available in all environments, but I believe you can install them on your $HOME.

To show a bit how this works, let’s run the setup directly on the shell.

Initialize virtualenv

plakaki:~ alff$ export WORKON_HOME=~/PythonEnvs
plakaki:~ alff$ mkdir -p $WORKON_HOME
plakaki:~ alff$ source /usr/local/bin/virtualenvwrapper.sh

Create an environment for daily work

plakaki:bin alff$ mkvirtualenv -p python2.7 daily
Running virtualenv with interpreter /usr/local/bin/python2.7
New python executable in daily/bin/python
Installing setuptools............................done.
virtualenvwrapper.user_scripts creating /Users/alff/PythonEnvs/daily/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/alff/PythonEnvs/daily/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/alff/PythonEnvs/daily/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/alff/PythonEnvs/daily/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/alff/PythonEnvs/daily/bin/get_env_details
(daily)plakaki:bin alff$ python -V
Python 2.7.1

Working and active

Changing environments

(daily)plakaki:~ alff$ deactivate
plakaki:~ alff$ workon daily
(daily)plakaki:~ alff$ python -V
Python 2.7.1

Run it in every terminal

I added the following extra lines in my .bash_profile:

WORKON_HOME=~/PythonEnvs
export WORKON_HOME
source /usr/local/bin/virtualenvwrapper.sh

Nothing else to add. Now it’s time to keep the code flowing

A really nice route starting from Athens, coffee in Porto Rafti and ending in Kaki Thalassa.

The “team” was composed by:

• Mr. Yamaha Fazer.
• Ms. Honda CBF.
• Mr. BMW F800 ST (me).
• Missing was Mr. Ducatti Multistrada.

It was my first ride with the F800. We got to know each other: gears, handling, steering. Still we need to talk more about which gear does she likes when steering.

The week was a bit cloudy, but the bad weather was waiting for the weekend.

Before selling my Piaggio MP3, I had to repair one plastic part. The stickers needed came in pairs. This is what I did with one of them

Well, these are my wife’s words.

I got this second hand beauty two days ago. The owner had to put it in the road for me. Almost two years without changing gears. The engine stalled 3 times in the first Km, and my wife asked me “why are you smiling?”. Well it’s something difficult to explain, only my job mate Niko could understand it (he had such a smile you could see it through the helmet).

Next step is to take it to the repair shop to fix some small details and ask a bunch of questions. Then find a good teacher because I’m a n00b driver.

Bye bye MP3 :’(

Two days ago I was reading this:”NoSQL took away the relational model and gave nothing back“. In that post I found a really interesting comment: “ACID transactions are clean and understandable and that’s why people like them.” Really?!.

What ACID is you can find it here or here. But I really believe that not many people understands ACID because:

• They don’t know they’re using a system implementing it.
• They don’t need it.

So you’re using it and you don’t know it

How many people ask in forums about the solving transaction deadlocks. Or write about it. This is clearly because everybody understands ACID and loves to use it.

Three weeks ago I had to wear the DBA cap in my job because we all forgot that our database was in ACID mode. I’m quite sure that if I ask one by one each member of my team about what ACID is they will answer correctly. This made me think about the second point:

You don’t need ACID

There you have your shinny web application. You don’t know why but someone had the great idea of put all the data in that expensive relational database. And that system, by default, works in ACID mode. But you didn’t care because… you don’t need it.

You store in your database:

• A huge amount of static information. Some times you even think to load some of it in memory.
• Your sales/actions/whatever there, but you fire and forget. If someone loads your site and you clicked save, you don’t care if the site doesn’t show the new sale/action/whatever.
• Messages or actions like in a queue system. If the destination doesn’t read it now, it will read it later
• In some special situations you really open a transaction, and do the typical inventory transaction: if it has products then, products– and add products to the sale. There you really don’t want to sell the same item to two customers.

Can you see in how many of the previous points you need an ACID system? Even the last one, you can take a look here to see the problem form a different point of view.

Conclusion.
There are many reasons not to use a relational database in your project. ACID can be one of those. Do not forget to ask yourself about it in your next gig.