I saw something interesting pop up on twitter:
http://code.google.com/p/nagios-was/
that made me think of my websphere monitoring work:
http://bit.ly/I6YUw
Admittedly, mine isn't actually a Nagios plugin but I could pretty easily convert it to one. The logic is all there. Maybe I'll download a trial version of WAS6 (mine was written again V5) and see if I can make it an actual plugin.
Thursday, June 25, 2009
More Adobe AIR stuff
So I recently wiped my Dell XPS M1710 laptop. I had purchased another 2GB of memory (bringing it to 4GB) and a 500GB 2.5" sata drive to replace the shitty 100GB that came with it.
I decided to go 64-bit Jaunty to really take full advantage of the memory. I haven't run a 64-bit desktop Linux since I ran Gentoo. I remember a whole host of problems then with chroot and such but figured things were better now.
Indeed they are except for a few closed source programs. One of those happens to be Adobe AIR.
Adobe provides some nice tips on their website for 64-bit linux installs but most of those were deprecated for Jaunty. TweetDeck would fire up but not let me click anything similar to the problem I had before with gnome-keyring vs. kwallet.
A little research later lead me to this post:
http://joefleming.net/2009/06/14/ubuntu-64-bit-and-adobe-air/
Essentially, getlibs makes all the problems I had before a non-starter. I was able to grab the 32-bit version of gnome-keyring and have it be a part of the package system (so no stray files) and start TweetDeck.
I'm still getting some cocked-up error about libcanberra but that appears to be a known issue:
https://bugs.launchpad.net/ubuntu/+source/ia32-libs/+bug/369498
I decided to go 64-bit Jaunty to really take full advantage of the memory. I haven't run a 64-bit desktop Linux since I ran Gentoo. I remember a whole host of problems then with chroot and such but figured things were better now.
Indeed they are except for a few closed source programs. One of those happens to be Adobe AIR.
Adobe provides some nice tips on their website for 64-bit linux installs but most of those were deprecated for Jaunty. TweetDeck would fire up but not let me click anything similar to the problem I had before with gnome-keyring vs. kwallet.
A little research later lead me to this post:
http://joefleming.net/2009/06/14/ubuntu-64-bit-and-adobe-air/
Essentially, getlibs makes all the problems I had before a non-starter. I was able to grab the 32-bit version of gnome-keyring and have it be a part of the package system (so no stray files) and start TweetDeck.
I'm still getting some cocked-up error about libcanberra but that appears to be a known issue:
https://bugs.launchpad.net/ubuntu/+source/ia32-libs/+bug/369498
Friday, June 12, 2009
Updates to mhs
So I did some updates to MHS (the mysql health daemon I wrote for work). I was implementing our scripting framework from bash in perl to provide the same wrapper functionality we have in our shell scripts.
It's not as transparent but it logs script stop and start. This one is pretty useless outside of our shop because it's now implementing our own perl modules:
HSWI::Base
HSWI::Log
HSWI::DB
I also stopped using Proc::Daemon because of some logging issues I was seeing. No matter which order I did Proc::Daemon::Init(), logging would stop. I even changed Log::Log4perl to use init_and_watch but nothing worked. I ended up having to not only change to App::Daemon just to get the tty detach (we start mhs from an init script) but also stop using File::Dispatch::LogRotate because it doesn't honor setting ownership of log files.
As soon as I have time to sanitize it, I'll post a new copy on dev.lusis.org.
It's not as transparent but it logs script stop and start. This one is pretty useless outside of our shop because it's now implementing our own perl modules:
HSWI::Base
HSWI::Log
HSWI::DB
I also stopped using Proc::Daemon because of some logging issues I was seeing. No matter which order I did Proc::Daemon::Init(), logging would stop. I even changed Log::Log4perl to use init_and_watch but nothing worked. I ended up having to not only change to App::Daemon just to get the tty detach (we start mhs from an init script) but also stop using File::Dispatch::LogRotate because it doesn't honor setting ownership of log files.
As soon as I have time to sanitize it, I'll post a new copy on dev.lusis.org.
Monday, June 8, 2009
Tinis for Tatas
I wanted to remind everyone who might possibly read this and lives in Atlanta that Crepe Revolution is hosting Tinis for Tatas tomorrow night:
http://creperevolution.wordpress.com/2009/05/03/tinis-for-tatas/
This is a pretty big deal as one of our dinner club friends is a survivor as well as one of my guild mates in World of Warcraft. If you're in the Atlanta area, please try and come.
http://creperevolution.wordpress.com/2009/05/03/tinis-for-tatas/
This is a pretty big deal as one of our dinner club friends is a survivor as well as one of my guild mates in World of Warcraft. If you're in the Atlanta area, please try and come.
Thursday, June 4, 2009
MyISAM vs. InnoDB via Twitter
So I saw the following tweet come across TweetDeck (using the search functionality as a custom feed no less):
140 characters is nowhere near enough space to answer that question. I'm going to put my thoughts here from the perspective of someone who's had to deal with large databases (500+ GB) in both MySQL and PostgreSQL doing both OLTP and OLAP work. Here's probably the best summation:
MyISAM
- Fast
InnoDB
- Reliable
That's not to say that MyISAM can't be reliable or that InnoDB can't be fast but that's the best way to look at it. But you can't balance which table type you choose based on those two criteria. There are reasons you might not need full ACID compliance that would make you still want to use InnoDB over MyISAM. Hell, MEMORY is faster than MyISAM but you don't want to use it for all your tables. Often times, the right answer is somewhere in the middle.
I tend to default to using InnoDB across the board. It's always defined as my default table type in my.cnf. I do this because often times the developers are coding around the assumption that the database supports things like foreign keys, transactions and the like. Admittedly, this is often hidden behind the ORM they use such as Hibernate but it's still important.
But what about speed? Why is InnoDB "slower" than MyISAM. It's basically because it's doing "more" than MyISAM. It's managing key constraints, logging transactions and all the other things that make it ACID compliant. Much of that "slowness", however, can be mitigated by getting away from the default InnoDB configurations and profiling the system over time to size bufferpools and the like:
One area where InnoDB is faster than MyISAM natively is in concurrent CRUD operations. That's because InnoDB uses row-level locking. I'm not as clear on the specifics of the locking as I am with say DB2 (DB2 will actually lock the row before and after the one you're modifying) but it's better on multiple concurrent operations than table level locking.
So when would you want to use MyISAM then? One area we really found that using MyISAM made sense was on non-relational tables within a schema that normally had InnoDB tables. In one case, we had a table that had two columns - an id and a blob. That table was MyISAM. Conceivably anywhere you are denormalizing the data quite a bit, it can make sense to use a MyISAM table especially if it's a frequently rebuilt table (like a summary table updated from a cron job). Of course we've also used MEMORY tables for that same purpose. Just be careful how you intermix these in the code (the aforementioned Java transactions for instance).
So here's my recommendation:
OLTP tables - InnoDB with a few caveats
OLAP tables - MyISAM with a few caveats
ok #mysql / #database geeks on twitter (twittabase tweeks?) .. which is better: myisam or innodb, and where and why.from @narendranag
140 characters is nowhere near enough space to answer that question. I'm going to put my thoughts here from the perspective of someone who's had to deal with large databases (500+ GB) in both MySQL and PostgreSQL doing both OLTP and OLAP work. Here's probably the best summation:
MyISAM
- Fast
InnoDB
- Reliable
That's not to say that MyISAM can't be reliable or that InnoDB can't be fast but that's the best way to look at it. But you can't balance which table type you choose based on those two criteria. There are reasons you might not need full ACID compliance that would make you still want to use InnoDB over MyISAM. Hell, MEMORY is faster than MyISAM but you don't want to use it for all your tables. Often times, the right answer is somewhere in the middle.
I tend to default to using InnoDB across the board. It's always defined as my default table type in my.cnf. I do this because often times the developers are coding around the assumption that the database supports things like foreign keys, transactions and the like. Admittedly, this is often hidden behind the ORM they use such as Hibernate but it's still important.
But what about speed? Why is InnoDB "slower" than MyISAM. It's basically because it's doing "more" than MyISAM. It's managing key constraints, logging transactions and all the other things that make it ACID compliant. Much of that "slowness", however, can be mitigated by getting away from the default InnoDB configurations and profiling the system over time to size bufferpools and the like:
- Don't use a single tablespace file for InnoDB (innodb_file_per_table). The global tablespace will still need to be used but it's much slower than if you were to use a tablespace per table. This also gets you the benefit of being able to recover disk space after dropping an InnoDB table. If InnoDB is using the global tablespace file, the ONLY way to recover that space from dropping an InnoDB table or whole schema of InnoDB tables is to fully backup ALL schemas, remove the ibdata/log files, restart the database and then reload from backup. Not pretty.
- Use the smallest possible primary key where you can. At one company, we were using GUIDs for the primary key. InnoDB prefaces every index with a subset of the first bytes of the Primary Key for that record. I can't remember the exact number off-hand but it was only a subset. Considering the first X bytes it was using could potentially be the same across multiple records, it took more work than if we had used ints. Additionally, not only were our primary key indexes unneccesarilly large, every subsequent index was as well. This wasn't as much a big deal on columns with smaller datatypes but indexes on columns of datatypes like varchar and lob were pretty ugly
- Consider using a larger log file size. This has a trade-off in recovery time though. Your call there.
- Get fancy with disks. If you have multiple volumes of different raid types, you can not only tell InnoDB to put the global tablespace and log files on a different path but you can also "move" the database to a different volume as well. This involves creating the database, shutting mysql down, moving the database directory from the mysql data directory to a new location and then symlinking it. Until MySQL or InnoDB gets me the ability to define where I want a given tablespace, this is the next best thing.
One area where InnoDB is faster than MyISAM natively is in concurrent CRUD operations. That's because InnoDB uses row-level locking. I'm not as clear on the specifics of the locking as I am with say DB2 (DB2 will actually lock the row before and after the one you're modifying) but it's better on multiple concurrent operations than table level locking.
So when would you want to use MyISAM then? One area we really found that using MyISAM made sense was on non-relational tables within a schema that normally had InnoDB tables. In one case, we had a table that had two columns - an id and a blob. That table was MyISAM. Conceivably anywhere you are denormalizing the data quite a bit, it can make sense to use a MyISAM table especially if it's a frequently rebuilt table (like a summary table updated from a cron job). Of course we've also used MEMORY tables for that same purpose. Just be careful how you intermix these in the code (the aforementioned Java transactions for instance).
So here's my recommendation:
OLTP tables - InnoDB with a few caveats
OLAP tables - MyISAM with a few caveats
Wednesday, June 3, 2009
Adobe AIR apps and Linux - Tweetdeck particularly
So I ran into an interesting issue yesterday. I decided to give TweetDeck a shot. I wanted to get hashtagged search results as a feed. TweetDeck can do it. Thwirl cant.
So I make sure I have the latest TweetDeck and fire it up.
Er...what's this black window I see? I can't click on anything. The main canvas is grey and the only thing I get is tooltips on mouseover of the buttons. Nothing works.
So I do some research and find out that this appears to be a known problem. No one has been able to down exactly what's going on. Some people were mentioning that starting kwallet solved the problems while other's said that it was gnome-keyring. My brain started churning.
I started to think back to when I installed AIR on my desktop and when I installed TweetDeck. It was right after I left MediaOcean. I was setting up my desktop at home to be more like the setup I had at work so I could stay in the same mindset while looking for a job. That's when it clicked.
I installed the AIR runtime while I was running KDE. One thing AIR boasts under Linux is integration with either kwallet or gnome-keyring. I wonder if maybe it "locks" that choice in place during install. Well I run through a few quick tests which involve installing and uninstalling AIR , removing some dotfiles and directories where settings are stored. Nothing seems to work.
Here's what finally worked:
After that, I reinstalled the AIR runtime and TweetDeck. TweetDeck was working!
So what's the dealie yo? Well it appears there are two issues:
I have yet to reinstall Amarok again so I don't know what will happen once kwallet is available but it appears to me that Adobe needs to fix this behaviour. Maybe give the user an option to choose which vault will be used at install time or possibly someone can create an application that can switch the vault from kde to gnome or vice versa.
So I make sure I have the latest TweetDeck and fire it up.
Er...what's this black window I see? I can't click on anything. The main canvas is grey and the only thing I get is tooltips on mouseover of the buttons. Nothing works.
So I do some research and find out that this appears to be a known problem. No one has been able to down exactly what's going on. Some people were mentioning that starting kwallet solved the problems while other's said that it was gnome-keyring. My brain started churning.
I started to think back to when I installed AIR on my desktop and when I installed TweetDeck. It was right after I left MediaOcean. I was setting up my desktop at home to be more like the setup I had at work so I could stay in the same mindset while looking for a job. That's when it clicked.
I installed the AIR runtime while I was running KDE. One thing AIR boasts under Linux is integration with either kwallet or gnome-keyring. I wonder if maybe it "locks" that choice in place during install. Well I run through a few quick tests which involve installing and uninstalling AIR , removing some dotfiles and directories where settings are stored. Nothing seems to work.
Here's what finally worked:
- Drop to a shell.
- Remove the package that provides kwalletd
jvincent@jvxps:~$ dpkg-query -S /usr/bin/kwalletd
kdebase-runtime-bin-kde4: /usr/bin/kwalletd
jvincent@jvxps:~$ sudo apt-get remove kdebase-runtime-bin-kde4
[sudo] password for jvincent:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libclucene0ldbl apache2-utils libxine1-x librdf0 kdebase-runtime-data-common php5 libxine1-misc-plugins kdelibs5 libxcb-xv0 kde-icons-oxygen libxine1-bin libexiv2-5 librasqal1 apache2-mpm-prefork libsoprano4 redland-utils
apache2.2-common kdelibs5-data cvs libxcb-shape0 phonon-backend-gstreamer libstreamanalyzer0 libphonon4 libsvnqt5 kdelibs-bin libstreams0 exiv2 php5-mysql libapache2-mod-php5 libxcb-shm0 soprano-daemon kdebase-runtime-data
php5-common libxine1-console libxine1
Use 'apt-get autoremove' to remove them.
The following packages will be REMOVED:
cervisia cvsservice kdebase-runtime kdebase-runtime-bin-kde4 kdesvn kdesvn-kio-plugins khelpcenter4 kompare
0 upgraded, 0 newly installed, 8 to remove and 30 not upgraded.
After this operation, 17.8MB disk space will be freed.
Do you want to continue [Y/n]?
- One thing to watch carefully is the list of packages that will be removed. I did this on my desktop and it wanted to remove Amarok. I figured I could reinstall Amarok if I needed to after the test.
- Once that was done, I uninstalled AIR and all the apps I had installed (in this case, TweetDeck and Thwirl).
- Backup and remove the .appdata directory from your home directory. I'm not sure if this step is absolutely required but I did it anyway.
After that, I reinstalled the AIR runtime and TweetDeck. TweetDeck was working!
So what's the dealie yo? Well it appears there are two issues:
- Which ever keychain you are using when you install AIR (gnome-keyring for gnome and kwallet for kde) becomes the default keyring for the life of the AIR installation. I realized later that when I switched back to Gnome, Thwirl was always asking for my password even when I told it to save it. Now I know why.
- It appears that Kwallet is the DEFAULT keychain used if you have it installed. That's why I had to fully uninstall KDE just to install AIR. I run under Gnome again. I don't use any KDE apps other than Amarok. Kwallet may not always be running.
I have yet to reinstall Amarok again so I don't know what will happen once kwallet is available but it appears to me that Adobe needs to fix this behaviour. Maybe give the user an option to choose which vault will be used at install time or possibly someone can create an application that can switch the vault from kde to gnome or vice versa.
New Nagios Exchange online
Got a tweet from@nagioscommunity yesterday:
New blog post: Launch of the new Nagios Exchange ! http://bit.ly/1aS44UWent ahead and moved the stuff that was hosted at monitoringexchange (nee nagiosexchange) over to the new spot.
Subscribe to:
Posts (Atom)
