Linux Living

acpitool as an Alternative Suspend Method for Ubuntu 10.04 (Lucid Lynx)

2010-05-03T18:24:00.002-04:00

I've noticed that people still seem to be having difficulties suspending their laptops after upgrading to Ubuntu 10.04 LTS (Lucid Lynx). I had mentioned previously that there is a command-line tool called powersaved that allowed me to put my Toshiba Satellite A200 laptop in suspend mode when the main suspend function in Ubuntu 9.10 could not. The good news, after upgrading to Lucid Lynx, is that the main suspend function (accessed through the session management menu in Gnome-Panel) actually suspends my laptop! The bad news is that, for those of you who are still having suspend issues after upgrading to Lucid, powersaved is no longer available.

To help, I set my mind to searching Synaptic Package Manager for a command-line program that works like powersaved but is actually available to Lucid Lynx users. I have thus far found a very helpful little command-line program called acpitool. It's so simple to use! Once you install it, just type sudo acpitool -s and your computer will go into suspend! If you follow my directions on how to make a launcher script from my previous post regarding powersaved then you'll also have a button to make it even easier to use this command-line tool.

Hope this helps people!

My Experience Upgrading to Ubuntu 10.04 LTS (Lucid Lynx)

2010-05-02T12:30:00.003-04:00

I recently upgraded my Ubuntu 9.10 installation to Ubuntu 10.04 LTS. At the outset of the process I was really hoping that I could just start the upgrade, leave my computer running over night, and come back to find that the upgrade all finished. Unfortunately, I was wrong. Throughout the upgrade process there were several points where I needed to decide whether I wanted to keep some old configuration file or overwrite it with something completely new. Seeing as how I kept leaving my computer as it was upgrading (in the hope that it would finish automatically) only to come back and find that it wanted me to tell it what to do at these decision points, the whole process took about a day!

While it was certainly annoying to get through the upgrade, I'm very happy to report that absolutely nothing cataclysmic or annoying happened to the programs I had installed in my 9.10 installation or the data that I had accumulated. Everything's running very smoothly: My OpenSSH server, Alpine, PostgreSQL, R, Python, Firefox and Thunderbird, and so forth.

The one thing that kind of annoyed me was the messaging menu. It's a little envelope icon that sits next to the volume icon in the Indicator Applet. I'm not really that into microblogging, so I don't have a use for a panel applet that facilitates such functionality. Thankfully, I learned a way of getting rid of it. Just go into your terminal and type in sudo apt-get remove indicator-messages. Once the process completes, you right click on the indicator applet (the volume icon and the envelope icon together), click on "Remove From Panel", then right click where the two icons previously were, click on "Add to Panel...", highlight the Indicator Applet and then click on "Add". Voila! The messaging menu is gone :)

I've read numerous times of people having problems upgrading Ubuntu, versus doing a fresh install of a new version. Thankfully in the case of upgrading to Lucid Lynx from Karmic Koala, I have seen no issues! So, I heartily reccomend upgrading to the latest and greatest version of Ubuntu.

----------------- Addendum:

I mentioned previously that Ubuntu 9.10 tended to have problems trying to suspend my Toshiba Satellite A200 laptop. I'm happy to report that Ubuntu 10.04 does not have such problems.

My Review of SimplyMEPIS 8.5 and Xubuntu 10.04 RC

2010-04-26T20:14:00.002-04:00

I am soon going to be getting a perfectly functional laptop from my older brother that has a broken screen. I want to use this laptop strictly as an easy-to-use home server computer, and so I will mostly be using it by logging into it from other computers (hence why I don't care that the screen is broken). After the laptop was promised to me, I began looking for a debian-based distribution that would perhaps be a bit lighter on resources (and still easy to use) than Ubuntu. I've so far tried out SimplyMEPIS 8.5 and Xubuntu 10.04 RC. In both cases I downloaded the ISO archive containing the setup files and mounted each on my USB stick as a live usb using the wonderful UNetbootin.

SimplyMEPIS was pretty easy/harmless to install. Once installed, I set about the task of learning the basics of getting around in its KDE 4.3.4 interface. Having used Ubuntu for almost a year now, I am very familiar with where things are when GNOME is the main graphical interface. Needless to say, KDE was very different for me. One thing that really impressed me was how very easy it was to set up samba network shares between my computer and my wife's computer. This was a difficult issue for me to get around in the past using Ubuntu, which I wrote about earlier.

A few things that bothered me about SimplyMEPIS: 1) I don't know why but try as I might I could not get my HP Laserjet P1505 printer to work properly!! Whenever I tried to print something from any application, my printer would feed the paper through and not print anything on it. It was very weird; almost like my computer had established a connection to the printer but was not sending any print data to it. 2) The distro repositories that I was using weren't so fast. I looked for faster repos to use, but didn't find anything that impressed me like what I get in Ubuntu. In Ubuntu you always have the option in the Software Sources menu applet to select the best repository so that you get maximum download speed. Couple with the fact that there are many repositories that offer Ubuntu updates and you get a powerful combination. 3) Okay this is a very superficial reason, but KDE makes linux too Windows-y! No more comments on this point, hehe.

Like SimplyMEPIS, Xubuntu was pretty easy/harmless to install. Once installed, I noticed that the Xfce interface is laid out very similarly to the GNOME interface. It's definitely a biasing factor, as it means there is less to learn. Once I set up my /etc/fstab and my /etc/samba/smb.conf files to be the equivalent of the same file on my Ubuntu installation, I learned that all I had to do to set up shares from my wife's computer on my own is a quick terminal command, smbmount //servername/sharename /mountdirectory. When using Ubuntu, this would normally be done through GNOME's Nautilus file browser.

In Xfce, there doesn't seem to be native network share browsing, necessitating a move to the command line for setting up your desired network share mounts. According to a post in Ubuntu Forums, you can modify your /etc/fstab file so that the network share you want access to is automatically initialized when your computer starts up. Just type in the following line (making the appropriate replacements for your situation): //servername/sharename /mountdirectory smbfs username=windowsuserename,password=windowspassword 0 0.

Finding out that Xubuntu doesn't allow easy GUI driven network browsing was a bit of a disappointment, but there are still a few things that make me believe it's the best choice of the two: 1) The Xfce interface is really nice, clean, and simple looking. Navigating through GUI elements was remarkably simple. 2) It comes with the ease of updating using the fast, expansive Ubuntu repositories. 3) Functionality in general using Xubuntu was indeed pretty fast and lightweight.

So, it looks like I'll be installing Xubuntu on my brother's laptop with the broken screen. It will be a good use of his laptop, which will as a consequence see more years than my brother ever thought.

If you think I should try out any other (light-weight and easy-to-use) operating system before I get this new-old laptop, tell me your opinion!

Get your Android Phone and Thunderbird in Ubuntu to work with MS Exchange

2010-03-17T21:20:00.002-04:00

I recently got full time employment in an organization that depends on Microsoft Exchange for all of its email and scheduling needs. So, something that I investigated as soon as I could was how I could get my Rogers HTC Dream (a.k.a. the T-Mobile G1) and my Ubuntu Linux Laptop to connect to my new employer's MS Exchange server to receive and send emails, and to do my scheduling.

I was delightfully impressed with how incredibly easy it was to set up my new MS Exchange-based email account on my HTC dream. I just navigated to the Data Synchronization submenu in my phone's settings menu, clicked on "Microsoft Exchange", entered in my new email address and my password, waited while it communicated with the server, and then voila it finished!

Unfortunately it was not that easy on Ubuntu! It took me a week before I found any answers on the internet, but I finally got my MS Exchange email and calendar account set-up in Thunderbird. I first tried to use Ubuntu's pre-installed Evolution Mail to connect to MS Exchange, but alas it didn't work.

The solution came in the form of a Gateway program called DavMail. The principle of this program is that it mediates a (POP3/IMAP/Caldav) connection between your email client and the MS Exchange server in a way that makes both parties happy. Although I haven't tried this with a program other than Thunderbird, it should in theory work alright.

Once you've downloaded and installed DavMail using the link above, make sure that you have Sun Java version 6 installed. To do so, type in sudo apt-get install sun-java6-bin. After you're finished, start up DavMail by navigating through the following Gnome menu items: Applications > Internet > Davmail. A yellow circular icon will appear in the Gnome system tray indicating that DavMail has started up. Right-click on that icon, then click on settings, and then input your organizations Outlook Web Access (OWA) address. It may be something like https://mail.domain.org/owa. Take note of the local ports that it's opening up for various services (e.g. POP3/IMAP/Caldav) to see what ports your programs will need to use to access those services.

The next step for me was enabling IMAP access to my MS Exchange email. You can choose POP3 if you like, but IMAP will ensure that you have access to all the email folders that you regularly access at work. The DavMail folks already set up a nice tutorial on how to set-up Thunderbird with IMAP, so go through the steps they've already posted (warning, their screenshots are in French. This shouldn't be a problem though, as their instructions are in English!). Once you've completed those steps, go into your Thunderbird Account Settings for your new account and make sure that the port number it uses for accessing your email through IMAP is the same as the DavMail setting (the default is 1143).

Next item of business is to set up your MS Exchange calendar through Thunderbird's Lightning extension. Again, the DavMail folks have covered the steps to do these so do read what they've posted. Now you should have access to your email and calendar from Thunderbird at home!

The caveat to all of this is that, as a Gateway program, DavMail has to be running for you to communicate with your employer's MS Exchange server. So, be sure to start up DavMail if you want to check your MS Exchange email/calendar. It would be nice if this functionality was native in Thunderbird, but oh well. At least these steps allow you to use Thunderbird at all!

Workaround for broken Ubuntu Suspend functionality in your Toshiba Satellite

2010-03-06T20:03:00.002-05:00

Lately I've been noticing that my Toshiba Satellite A200 Laptop, on which I'm running Ubuntu 9.10, simply won't go into Suspend-mode properly. I did some googling and found out that there's an alternative program you can download to put your computer in suspend mode called Powersave. Its use is remarkably easy and I'm happy to report that it is managing to put my laptop in suspend when the regular suspend in Ubuntu can not manage.

Usage

Go into your terminal and type in: powersave -u. That's it!! It is also technically supposed to be able to put your computer in hibernate by typing in powersave -U, but it simply doesn't work on my computer for some reason.

Installation

If you are having similar problems as I am and want to install this program, either go to Synaptic Package Manager and look for and install powersaved, or go to your terminal and type in sudo apt-get install powersaved. You can make it even easier on yourself by creating a shell script containing the powersave command with the suspend argument. Simply navigate in Nautilus to the directory where you want to create your shell script, right click on the background of that directory's window, click on Create Document > Empty File, and then add the following into the empty document:

#! /bin/bash

powersave -u

When you're done, save it with a filename such as suspend.sh, and then create a launcher that activates your script. I right-clicked on my upper Gnome-Panel and clicked on Add To Panel... > Custom Application Launcher, entered in "Suspend" in the name field, sh /home/inkhorn/Scripts/suspend.sh in the command field, and chose a custom icon to represent the launcher on my panel.

Now all I have to do is click on my newly created launcher button on my Gnome-Panel and my computer goes into Suspend mode like it should! You of course don't need to go through the trouble to make a button for this on your Gnome-Panel like I have, but it certainly saves typing in the long run.

Block image and flash-based advertisements in Firefox

2010-02-25T22:22:00.003-05:00

One of my biggest pet peeves about browsing through my daily regimen of websites is the advertisements that I come across. Usually they are something about "getting ripped" in 3 weeks or whitening your teeth. I'm not the kind of person who can just ignore things like that so easily, so anything that takes away those annoying advertisements is like candy to me. Enter in Firefox's native image-blocking abilities, and Flashblock. Behold the following screenshot of a web-site and look at all the wonderful, advertisement-free white-space:

The lack of stuff to fill in the white-space may scream out at you, but it's a huge relief to me. With Firefox and Flashblock, locking image-based or flash-based advertisements is a cinch! For image-based advertisements, all you have to do is right click on the image, and then click on Block images from ... and you're set. For flash-based advertisements, Flashblock basically turns all instances of Flash on websites into buttons that you can click on if the Flash is a functional part of the website. If you want to get rid of the button altogether, all you have to do is right click on it and then click on Remove Flash.

That's all! Enjoy surfing with no advertisements :)

Wireless just works better with WICD than Gnome Network Manager

2010-02-23T20:49:00.002-05:00

As I mentioned in my post about getting Ubuntu to start your wireless connection before login, Gnome Network Manager has to be uninstalled so that Ubuntu will rely on your /etc/network/interfaces file to start up your wireless connection. This worked great for me and my Ubuntu laptop until I started having to take my laptop away from home and use other wireless networks. In order to connect to other wireless networks, I would have to set my /etc/network/interfaces file so that my wlan0 interface relied on dhcp instead of static settings. I would then have to re-initialize my networking settings and then use the command-line to scan for dhcp requests on my wireless interface. This was no good!

To remedy the situation, I discovered another graphical network manager called WICD. The version in the Ubuntu repositories was actually a bit buggy, so I went directly to the WICD launchpad website and downloaded the latest release (version 1.7.0). I've tested it out at home and away and I can now say that it works quite nicely. WICD allows Ubuntu to start up my wireless connection before the Login screen. It also allows me to connect to other wireless networks despite the settings in my /etc/network/interfaces file.

As you can see above, WICD puts an icon in your system tray with a green indicator bar that tells you how good your signal is. If you hover your mouse over the icon it tells you what network you're connected to, what percentage signal strength you have, and what your IP address is.

Above is the window that pops up when you left click on the WICD icon in the system tray. As you can see, the window contains information about all available wireless network SSIDs in your vicinity, how strong the signal is to those networks, and what type of encryption (if any) protects each network. You can click on the properties button for any network that you want to connect to if you have to put in a wireless encryption key. The screen that pops up follows

Once you're ready, you can click on the Connect button and you're on your way!

Setting Up WICD

As I mentioned above, I first installed WICD from the Ubuntu Repositories. I think this forced my computer to install all the appropriate dependencies so that the newer version could work. When you download the newer version, extract it to whatever directory you want. Then, open up your terminal, navigate to the directory with the WICD setup files that you just extracted, and type in sudo python setup.py install. Now, when you restart your computer, you should see WICD in your system tray ready to do your bidding.

Install OpenOffice 3.2 in Ubuntu!

2010-02-11T20:11:00.002-05:00

Today I checked the openoffice.org website to see if OO 3.2 was released yet. I discovered it was, and decided to download and try to install it. I think this is the first time that I've tried to completely remove one installation of OpenOffice in Ubuntu and install another. If you'd like to do the same and check out the new OpenOffice, here are the steps I used:

Choose a mirror site to download an Archive containing the appropriate OpenOffice.org installation for your system
Once you've downloaded the archive, extract it to your directory of choice
Navigate to that directory in your terminal
Now, remove all packages in your system associated with OpenOffice by typing in sudo apt-get remove openoffice*. For some reason this removed alpine from my system. It was easy enough to reinstall alpine and I discovered that none of my settings were lost. Do take note of what your system removes using the previous command
Now, navigate to the DEBS subfolder of your openoffice setup directory
Install all openoffice packages by typing in sudo dpkg -i --force-overwrite *deb
Finally, navigate to the dekstop-integration subfolder of the DEBS directory and type in sudo dpkg -i --force-overwrite *deb

Now you will have the new stable OpenOffice Suite version 3.2 installed on your Ubuntu system!

Start your wireless connection before Ubuntu login

2010-02-09T09:02:00.003-05:00

Recently I started asking myself how I could get Ubuntu to forge its wireless connection (with WEP encryption) to my router without having to log-in through the GUI. I noticed that the default way of logging in through the GUI and then having a wireless connection available was limiting for two reasons: (1) If I wanted to be able to connect to my openSSH server then I needed to be logged in, and (2) If there was something wrong with the GUI and I had to log-in through the command line then I wouldn't have internet access.

So I started looking around on the net. Most pages that I found showed messageboard threads suggesting modifications to the /etc/network/interfaces file. This is the file that is supposed to tell your Linux system what to do with your network interfaces at boot-up. I tried to modify it so that my wireless card would start-up with a static IP at boot-up, but no matter how I changed this file, nothing worked.

Finally, I decided to ask a question on ubuntuforums. I found out that Gnome Network Manager, the program in Ubuntu which manages all your wired and wireless network connections, works terribly with the settings in the /etc/network/interfaces file. In other words, if you want Ubuntu to start your wireless connection at bootup instead of after login, you have to get rid of Gnome Network Manager!

So, on the advice of chili555 from ubuntuforums, I got rid of Gnome Network Manager by typing in sudo apt-get remove --purge network-manager and modified my /etc/network/interfaces file to look like the following (SSID and WEP key changed for privacy):

auto lo
iface lo inet loopback

auto wlan0
iface wlan0 inet static
address 192.168.2.11
netmask 255.255.255.0
gateway 192.168.2.1
wireless-essid MYSSID
wireless-key 12345678912345678912345678

iface eth0 inet dhcp

Now, every time I start my computer, my wireless connection is available before login so that I can login to my openSSH server or go to the command-line without logging in to the GUI!

Alpine, Gmail, and SSH. How to make your computer send emails when you're not there!

2010-02-04T11:03:00.003-05:00

Out of the many things I've learned in the past half year about using Linux, I find the concept of remote access of my computer through SSH to be the coolest. Periodically I will be away from home, and I'll realize that I need access to some file on my computer so that I can send it to someone. I recently learned how to set up a Secure Shell Server on my Ubuntu Linux laptop that I can access from my Android cell phone using an Android app called ConnectBot. Once I get into my computer, I can then open up a fantastic text-mode email program called Alpine and compose emails with files attached right from my computer's hard drive and then send them through my Gmail account. Nothing makes me feel more like a huge nerd than this!

Here's how I set up an SSH server and then install Alpine with access to my Gmail account (my Gmail is setup to check and respond from 2 additional email addresses, so the instructions are tailored to that scenario):

Setting up openSSH

First you need to install the openSSH server. Easily enough you just have to type in sudo apt-get install openssh-server into your terminal to get the server. Amazingly, it's ready right away to be used! I wanted the SSH server to use a different port than the default (port 22), so I opened up /etc/ssh/sshd_config and changed the port number on the 5th line to something that was wacky enough not to be predictable.

Next came two considerations: (1) I'm behind a router, and therefore needed to set-up port forwarding so that I'd be able to access my openSSH server remotely. If you have a router and forget this step then prepare for lots of frustration! (2) Even having port-forwarding set-up, who wants to remember and type in an IP address so that you can connect to your server? So I needed to sign up for a free dynamic hosting service that would give me a constant host name regardless of my IP address.

Setting up Dynamic DNS hosting

I found a great tutorial on the Ubuntu Help Site for setting up Dynamic DNS hosting for your computer. Basically, I went to DynDNS, set-up an account for my computer, and then followed the instructions on the above-mentioned Ubuntu Help Site to install and configure ddclient. ddclient will periodically send updates to DynDNS so that the hostname they provide you with always points to your IP address. When you're done configuring all of this, you will then have a host-name pointing to your computer such as tommy.gotdns.com. That's a lot easier to remember than 40.831.391.33 right?

Initial installation and set-up of Alpine

Next I set-up Alpine. Installing it was easy enough: sudo apt-get install alpine. Once it's installed, you have to set it up to work with your Gmail account. Refer to this tutorial from the University of Virginia for a tutorial on how to do that. Be sure that when you're in the Config screen of Alpine, look for the field named Inbox Path and enter in the following: imap.gmail.com/ssl/user=username@gmail.com . Be sure to replace 'username' with your own Gmail account username. This will make sure that when you start-up Alpine and go into your Message Index, you will see the emails in your Gmail Inbox right away, instead of having to navigate to that inbox.

Configuring Alpine to send emails using different 'From' email addresses

Now, when I reply to emails that I've picked up using my Gmail account, I usually want to reply using different email accounts. When you're only using the web-based Gmail checker, it allows you to respond using the account that the email was downloaded from. I wanted to have that same functionality in Alpine. This is where the Alpine concept of a Role comes in handy. From the main screen of alpine, type the following keys (don't type the triangular brackets): S > R > R > A . This will bring you to a screen that allows you to set-up the conditions under which Alpine will let you send an email using a different From email address.

First, set a nickname up for your role; maybe the name of your email provider (i.e. Sympatico). Next, under Current Message Conditions and beside To Pattern put one of the (or the only) email address(es) that Gmail is checking for you (e.g. bill@sympatico.ca). This creates an expectation that whenever this email address is in the the To field of an email, Alpine will do whatever you tell it in your role.

Then, under Actions Begin Here and beside Set From put your name and then email address that Gmail is checking for you (e.g. Bill Nye <bill@sympatico.ca>).

Finally, under Uses Begin Here you need to make the following changes (if necessary): (1) Under Reply Use = make sure to highlight and press enter on Without Confirmation and (2) Under Compose Use = make sure to highlight and press enter on With Confirmation. This will make it so that if someone emails you at bill@sympatico.ca, when you reply to that email, Alpine will automatically put bill@sypmpatico.ca in the From email address and not bill@gmail.com.

When you're done, you can press E to exit the Role set-up. If you check two email accounts through Gmail like me, press A once more when you're in the Setup Role Rules screen in alpine and repeat the above instructions to set-up a role for your other email account.

Saving your Gmail password in Alpine so that it doesn't prompt you all the time

Finally, you'll probably be annoyed with entering in your Gmail password every time you start up Alpine. Thanks to a helpful user on Ubuntu Forums, I found out a simple way of making Alpine save your password. Open up your terminal, change directories to your home directory, cd ~/, and then type in touch .pine-passfile. After that, run Alpine. It will ask you for your password on start-up, and on sending an email, only to save your password indefinitely so that you no longer have to type in your password all the time.

Conclusion

Wow, those were a lot of very little steps which, put together, seem mountainous. However, now I can punch in my dynamic DNS hostname into ConnectBot on my Android cell phone, access Alpine and send emails with files attached right from my computer's hard drive! I also like that I can update my databases on the fly with important personal info, manage, and even turn my computer off from afar. It's quite neat and if you're into feeling nerdy, I highly recommend getting your own openssh server configuration.

Making Numeric Row Identifiers in PostgreSQL

2010-01-28T21:43:00.002-05:00

Around a month ago I started to learn how to use PostgreSQL to create, maintain, and query databases. It wasn't hard to find things in my personal life that were appropriate for databasing: Job applications, Hydroelectricity usage/charges, usernames and passwords. Putting data from these aspects of my life into databases has given me some great practice using a query language.

The thing about databasing that really trips me up is the concept of indexing. Depending on how you query (search) your database, any one record (row) in your database could show up in the 11th, 27th, 1st or 99th row of your query results. Hence, trying to access the record in which you stored your facebook username and password by its absolute row number just won't work. That's a concept for Python and not PostgreSQL (please correct me if I'm wrong!). What you can do however is assign each of your database records a unique numerical identifier and so any time you need to modify a single row, you can refer to it by its identifier no matter what the content in the other fields of that record.

What I wish I knew when I made my database is that it is REALLY easy to create a type of column in your table that automatically assigns unique numerical identifiers to your records when you insert new values into the table. When you're making the table in your database, declare your id column as type SERIAL. Then when inserting new values into your table, all you have to do is enter the word DEFAULT in place of a number that you would come up with yourself. The number id in the SERIAL column type seems to start at 1 by default, obviously incrementing by 1. Nice and simple. Following is an example:

CREATE TABLE test (
blah SERIAL
yadda varchar(50)
);
INSERT INTO test VALUES(DEFAULT, 'hello');
INSERT INTO test VALUES(DEFAULT, 'goodbye');

SELECT * FROM test;
blah | yadda
------+---------
1 | hello
2 | goodbye
(2 rows)

In my situation, I didn't seem to be able to make a SERIAL column for my existing table, or convert a column I already had. So, I needed a way of generating unique numerical identifiers after 100 records had already been inputted. I found two solutions (if there are others, please tell) to work for me. First the easy solution:

ALTER TABLE tablename SET WITH OIDS;

OIDS stands for Object Identifiers. When you use the above statement, PostgreSQL assigns a unique numerical identifier to each row in your table. This solution didn't look as clean to me as the scenario where the SERIAL type column was made when the table was made. The raw value that the OIDS in my table start at 16522, which is a little weird. I think this might be due to data that I have in other databases on my computer, but I can't be sure right now. The bonus is that the OIDS seem to have been assigned roughly according to the order in which I inputted these records into the table in my database. Thus it's a simple solution and allows me to refer to any one record by a simple unique identifier. I've read that the raw value of these OIDS will wrap around to something very small after you reach about 4 billion records. I don't feel in danger of that using PostgreSQL for personal matters though!

The next solution was more difficult, but gave me nicer results. First you need to figure out how you want to order the records in your database. I wanted to order them by date, so let's call my record dating column rdate. Unfortunately, many records were inputted on the same date, which means if I only relied on the date for ordering, I would get many records having the same numerical index. So, you have to take advantage of as many additional fields in your table that will make your record unique, and ensure that the following process results in as many different numerical identifiers as there are records. Let's say that in addition to specifying rdate as your ordering column, you also specify columns yadda, and blah. Also, your id variable is simply named id. You would then define the following function in PostgreSQL (my reference for this function comes from a pgsql mailing list posting):

CREATE OR REPLACE FUNCTION reorder_table() RETURNS INTEGER AS '

DECLARE
newcode INTEGER ;
table_record RECORD ;

BEGIN
newcode := 1 ;

FOR tablerecord IN SELECT * FROM tablename ORDER BY rdate LOOP

UPDATE tablename SET id = newcode
WHERE rdate = table_record.rdate AND yadda = table_record.yadda AND blah = table_record.blah;

newcode := newcode + 1 ;

END LOOP ;

RETURN 1;
END;
' LANGUAGE plpgsql;

Once you've entered this in, all you have to do is call your function through a simple statement: SELECT reorder_table();. If your inclusion of yadda and blah made each record in your table unique enough, then you will now have numerical identifiers ranging from 1 to the total number of records in your table! Of course if you want your numerical identifiers to be assigned according to the order of a different column in your table, replace rdate in the SELECT statement in the above function with the column name that you want as your ordering column.

What I like about using the OIDS in my situation is that the identifers are automatically generated, whereas I would need to run the above function every time I update the table in my database in order that every record has an identifier. Suffice it to say that in the future, I will create a SERIAL type column next time I need identifiers like this!

Update your Rogers HTC Dream from Ubuntu

2010-01-27T09:56:00.002-05:00

As with everybody else who owns an HTC dream/magic from Rogers, I recently started to get pestered to apply a software update to my phone so that I could safely call 911 and still be able to use my data plan. Most times I'm a pretty lazy person, and so I only got around to doing this AFTER Rogers took my data connection away from my phone. When I looked into it at first, it looked like the phone update could only be done using Windows software. Well that's a big problem for me because, as you know, I only use Ubuntu Linux on my computer!

I ended up speaking with a Rogers representative on the phone and asked her what Mac users are doing about this software update, thinking that the solution for them might be good for me too. It turns out it was, and now my phone is updated and back to full functionality. Of course all the programs I previously downloaded to the phone from the Market are gone, but I don't care much. Here is what you need to do to update your phone without HTC's annoying Windows software:

1. First you have to download a zip file containing the update (just a single file really, click here for the file).

2. Connect the HTC dream to your linux machine via USB, pull down the notification area, press your finger on the USB connected notification, and press Mount.

3. Now you'll see a notification that you've connected a media device on Ubuntu (or whatever other graphical form of linux you use). Tell your computer to open up the folder of the connected device.

4. Extract the contents of the zip file you downloaded in step 1 to the root of your HTC dream's microSD card.

5. Now turn off your HTC dream.

6. Once your HTC dream is off, press and hold 2 buttons together: the Camera button and the Power button.

7. After a few moments your HTC dream's screen should show something similar to the picture below. When you see that on your screen, press the trackball (Action) button.

8. When the HTC dream tells you that it's finished, it will ask you to press the Action button to reboot. Press the trackball when that happens and you're done!

Pylab, R, and QtiPlot Plotting Compared

2010-01-24T19:15:00.009-05:00

Today I want to compare and contrast the plotting of statistical graphics in three very neat software packages freely available:

Pylab via Ipython
The R Statistical Computing Environment and
QtiPlot

First we'll take a look at the raw table of numbers to be plotted, I'll show you the resultant plots, and finally I'll show you the code or steps necessary to get to those plots.

In the sample dataset above, I've included an "X Axis" column composed of 7 integers simply called X. I've also included two Y columns of means and two columns containing the Standard Errors of the datasets from which those means came. My plotting aim was to create a plot containing two lines describing the two Y columns and error bars matching the values from the two Standard Error columns.

Pylab plot

R plot

QtiPlot

As you can see from the above plots, no one of these three software packages produces a bad looking plot. Some of the graphical parameters (such as font size, type of major axis ticks, whether or not a full box is drawn around the plotting area, and how far the plot title is from the top of the plotting area) are different from program to program, but that's more a matter of my unwillingness to get the programs to output exactly similar graphs than an inability in the programs themselves.

How I got the plots using Pylab and R
First and foremost is the fact that Pylab and R require you to type in some code to do your plotting whereas QtiPlot gives you a point-and-click GUI interface to complete the task. Pylab and R have their own idiosyncratic syntax for plotting, but thankfully neither requires much more code than the other. If you didn't know already, Pylab is a module of python and therefore allows you to seamlessly weave plotting commands into pure python code. It will therefore be advantageous for anyone who already has a Python background to use Pylab. Below I will show you the code I used to make the plots.

Pylab via IPython

infile = open('/home/inkhorn/Documents/data.csv','rb')
data = loadtxt(infile, delimiter=',')
errorbar(data[:,0], data[:,1], yerr=data[:,2],color='b',ecolor='k',elinewidth=1,linewidth=3);
errorbar(data[:,0], data[:,3], yerr=data[:,4],color='r',ecolor='k',elinewidth=1,linewidth=3);
axis([-0.2, 6.2, .3, 1.0]);
xlabel('X Axis Label', fontsize=14);
ylabel('Y Axis Label',fontsize=14);
title('Line Plot with Error Bars',fontsize=16);

error.bar <- function(x, y, upper, lower=upper, length=0.1,...){ if(length(x) != length(y) | length(y) !=length(lower) | length(lower) != length(upper)) { stop("vectors are not the same length")} else { arrows(x,y+upper, x, y-lower, angle=90, code=3, length=length, ...)} }
data = read.csv('/home/inkhorn/Documents/data.csv')
png('/home/inkhorn/Desktop/test.png', height=1033, width=813, type=c("cairo"))
plot(data$y2 ~ data$x, type="l",col='red',lwd=4,ylim=c(.3,1),main='Line Plot with Error Bars', xlab="X Axis Label", ylab="Y Axis Label")
par(new=TRUE)
plot(data$y1 ~ data$x, type='l', col='blue', lwd=4, axes=FALSE,ylim=c(.3,1),ann=FALSE)
error.bar(data$x, data$y1, data$y1err)
error.bar(data$x, data$y2, data$y2err)
dev.off()

R doesn't seem to come with installed with readymade functions that allow you to easily plot error bars in your statistical graphics, which is the reason for the function definition in the entry under the R code column above. Thanks for the coding of the R error bar function goes to the maintainer of a blog called monkey's uncle. Some people complain about the strange syntax required when using R, but you can see that you really don't need that much more typing in R than you do when you're using Pylab via Ipython. Still, Pylab gets extra points for coming installed with a readymade errorbar function!

How I got the plot in QtiPlot QtiPlot follows a very similar concept as Excel. Namely, it provides table-space to enter in your data, allows you to make plots from your table data, gives you easy point-and-click access to manipulate each component of your graph, and lets you save data and plots together in one project file. To get to the plotting, first you have to click File > Import ASCII ..., which brings you to the screen shown below:

You then choose your data to import, specify the separator, whether or not you want to ignore lines at the top, then press OK.

You are then shown your data in a Table view and must now right click on the columns and set their roles as shown above. As you can see, your columns can represent X variables, Y variables, Y error variables, even 3rd dimension, or Z variables. When you're done setting your column roles, navigate to the Plot menu and click on Line, as shown below.

A line plot will then be generated, using default values that you can change to your heart's content. The plot title and axis titles are very easy to change; all you have to do is double click on them and edit the default text that is already there. If you want to change any other aspect of the graph, it suffices to right-click on that part of the graph, and then click on Properties, such as what I did below with one of the lines on the graph.

You can also modify how you want each of your axes to look by right clicking on the numbers of that axis and again clicking Properties. You can then change some general graphical properties of each axis, or change the way that the axis is scaled.

Axis options Scale options Once you're finished specifying your graph's visual parameters, it's then easy as pie to save it. Click File > Export Graph > Current, then choose a folder to save your graph in, name it, then press Save and you're done!

Conclusion The truth of the matter is that you need to choose the right tool for the right job. I have often found that it was necessary for me to load data to be plotted into Ipython that I wouldn't have been able to read into R. IPython provides the opportunity for easy interactive plotting for simple one-graph projects, but can scale up to more complex programmatic plotting in larger projects. It hasn't been often that I've had to do larger scale projects where many plots are outputted programmatically, but IPython would certainly be the environment of choice for me.

R has amazingly expansive plotting capabilities and certainly does not lose points on graphical quality. As you can see however, its syntax can be difficult to manage. I've used R for making summary plots of data that I also had to statistically analyze. Therein lies the ultimate use of R; it provides a single integrated environment for the plotting and analyzing of many different types of data.

When it comes down to it, however, I am quite lazy. I only recently discovered QtiPlot, and I think it's great! According to the QtiPlot website, it even provides an interface that allows you to script QtiPlot operations using Python. I don't know anything about that interface just yet, but it makes me very impressed with the program overall. Given my laziness, the quality of the plots that come out of QtiPlot, and the ease with which you can manipulate them, QtiPlot rates very highly in my books. I will surely be using it more in the future for plotting where the data is easily accessible and will highly recommend it to others.

How to screw up then fix Gnome in Ubuntu!

2010-01-19T17:46:00.002-05:00

Yesterday I read about a supposedly nice-looking patched version of Nautilus and decided to try it out. I installed it, used it for a few hours before bed, and realized that I really hadn't gained anything special by installing it.

When I uninstalled it this morning, I really screwed up my installation of gnome. So much so that I simply couldn't log into ubuntu using the Gnome Display Manager (the meat and bones of the Ubuntu Grahpical User Interface). Thankfully, I still had access to the Virtual Console by pressing Control+Alt+F2 at the login screen.

One thing I learned is that I still don't know how to initialize an ethernet connection to my router/DSL modem to give me an internet connection from the command line. I know that you can set your ethernet interface's IP address by typing in sudo ifconfig eth0 address and you can substitute address with netmask address to change the netmask (I think it's usually 255.255.255.0 on most consumer routers). That didn't work for me however. I also tried toying with the route command, but I really didn't understand how to work with that.

What ended up saving me was a suggestion that I read on ubuntuforums.org, in response to someone in a similar situation, that they should download an alternative ubuntu installation iso, burn it to CD and use it as a source for package updates. I didn't end up burning it to CD. Rather, I did the following:

downloaded it to my USB stick
mounted the ISO in a newly made separate directory directory (using the command sudo mount -o loop ubuntu-9.10-alternate-i386.iso /media/testing)
navigated to that directory (cd /media/testing)
switched bash into root mode (sudo bash -i)
Finally I initialized an upgrade from the mounted Alternative Install ISO (./cdromupgrade)

The alternative install ISO fixed my broken GNOME/Nautilus packages, restarted my computer and .... VOILA! Everything was back to normal! Nothing was lost and I'm now a happy camper.

I'm definitely going to keep an alternative install ISO on me and probably also a regular install ISO too. After-all, they are free tools that don't even need to be burned to CD.

pyBloggerU now has a GUI

2010-01-18T10:09:00.003-05:00

Yesterday I sat myself in front of both Qt 4 Designer and Wing IDE and didn't tear myself away from my computer until I finished a GUI for pyBloggerU.

For those who didn't read my original post on the matter, pyBloggerU is a script I made that will upload an html file containing your Blogger post and images to your Blogger account for online publication. The script deals with the weirdnesses inherent in how Blogger mangles the HTML code so that what you see in your HTML editor is not what you get on Blogger.

Unfortunately it still is not programmed to handle HTML files generated from WYSIWYG editors, as they create too many complications for me to be able to handle with this script. But I've found that creating blog posts in an HTML editor called Quanta Plus to be easy enough. Quanta Plus has lots of buttons that shoot out HTML code for you, code and tag completion, and even a Visual preview mode if you're interested.

Once you've created your blog post in an HTML editor, like Quanta Plus, you just double click on the pyBloggerU_GUI.py file, press Run at the next window and enter in all the details shown in the picture below:

Be sure that any images referred to in the HTML contain their full file paths (the one above is "/home/inkhorn/Pictures/pyBloggerU.png") so that pyBloggerU will be able to upload them from your computer to your Blog's picasa web album. When everything is set, you can press the Upload button and your HTML file will become your blog post! When pyBloggerU succeeds at sending a blog post, you'll see a "Success" information window pop up a few moments after pressing the Upload button. Also, you'll be able to save your Blogger account info by pressing Save after filling out all the fields. When you have a new post to upload to blogger, you can then double click on the entry in the list to the left and your email, password, and blog title will appear in the fields to the right.

If you would like to download pyBloggerU, it's easily acccessible as a Bazaar branch on Launchpad. Even if you don't know what a Bazaar branch on Launchpad is, go to your terminal, and type in bzr branch lp:pybloggeru. This will create a directory called pybloggeru in your home directory and will store the python files for the program and all of its dependencies therein.

Of course if you would like to report a bug, ask a question, or contribute to the program, use the web utilities on the official pyBloggerU launchpad page.

Enjoy your WD My Book 1TB Drive: No more WD SmartWare icon in Ubuntu!

2010-01-14T21:14:00.002-05:00

As I mentioned in my last post, I recently picked up a Western Digital My Book Essential 1 TB external hard drive. Although it doesn't as yet display the same problems that my Simpletech hard drive was having, it does come with an annoying Virtual CD installed, literally in the unit's firmware, that contains Western Digital's SmartWare backup software. Of course the software is not linux compatible and wouldn't be something I would want to use anyway. The big question I had was: how do I get rid of it?

Well the problem has been well documented on the internet, with many people complaining about it. I found a helpful comment on a blog maintained by a mac admin. User 'yakkoj' suggests in his comment to make an entry in the /etc/fstab file on your mac that basically tells Mac OSX to ignore the Virtual CD so that it doesn't load and show up on your desktop. As Mac OSX is a relative of Linux, his comment was easily adaptable as a solution for my Ubuntu system too! Following is the line that I put in my /etc/fstab file and I can vouch that I now no longer see an icon for the WD Smartware Virtual CD on my desktop:

/dev/sr1 none udf rw,noauto 0 0

The Virtual CD always shows up as /dev/sr1, and I don't think there's any danger of other devices taking up that partition label. So, put the above line into your /etc/fstab file, and it will basically tell Linux that any time a device pops up and wants to inhabit the /dev/sr1 partition then it shouldn't be mounted at all. Then you will no longer have to stare at the dumb WD SmartWare icon on your linux desktop again! However, please note that this does not in fact rid the hard drive of the software, it just tells your computer to ignore it.

Bye bye Simpletech and Windows, Hello more space for Ubuntu!

2010-01-13T22:28:00.002-05:00

Today I exchanged my terrible terrible Simpletech 500GB [re]Drive for a sleek looking Western Digital 1TB My Book Essential. After filling the new backup drive with the files I previously had on the Simpletech, I had a mind to doing some spring cleaning on my Windows Vista partition.

Yes, I still had Windows Vista after approximately 5 months of using Ubuntu as my primary operating system. I moved some files and folders that I wanted to keep from my Vista partition to the backup drive and then thought it was about time to get rid of Windows and its partition. I never use it anymore! I only had 45 GB allocated to my active Linux partition (ext4 filesystem) and space was beginning to look a little scarce.

So I downloaded a Knoppix iso (little did I know it was German!), burned it to CD, booted off of it, used GParted to get rid of my Windows partition and resized my active Linux partition. It took about 40 minutes to complete but now my computer is rid of Windows! Now Ubuntu has a 148 GB partition to run off of with 120 GB of free space :)

Bye bye Windows, nice knowing ya!

Search Workopolis Easily with Python

2010-01-12T13:47:00.002-05:00

I've been looking for jobs lately and thought how nice it would be if I could skip the rigamarole of opening up my browser, going to a job search website, typing in the search arguments, and sifting through the results. In view of making job searching a little easier, I've made a Python script that for now will search Workopolis.com using keywords and a city location that you, the user, specify. It will then output a csv (comma separated value) file containing the job search results in the directory where you executed the script. You can then open the search results at your leisure, sift through them without the annoyances of advertisements, and possibly add them to your own database.

Just like pyBloggerU, I've uploaded the files for this Python script to launchpad for others to see and modify at their leisure. Go here if you'd like to download the program files, and here if you'd like to read more and possibly contribute to the project!

Once you put the program files in a particular directory, go to your command line, type python PyJobFinder.py, and then answer the questions that it asks you. Quickly after putting in your search terms, the program tells you the file name of the job results file and then you're free to open it up!

View a Windows desktop remotely with Ubuntu and xvnc4viewer

2010-01-08T16:06:00.005-05:00

Today I needed to give on-the-phone support to a colleague concerning a MATLAB program that I contributed to developing. If you've ever tried to tell someone what to do on their computer without having a similar screen in front of you on the computer, you'll know that it's not easy. To remedy the situation, I looked on the net for a program/protocol to use to view my colleague's desktop remotely. I found out that there is a protocol called VNC (Virtual Network Computing) for which there are client/server applications available on Windows and Ubuntu.

I got my colleague to install a VNC server for Windows called TightVNC, while I installed a VNC viewer for Ubuntu called xvnc4viewer. It was pretty easy for us to get the connection running. She only had to enter in a password for accessing her VNC server, and needed to tell me her public IP address. TightVNC tells you the IP address(es) that your server is broadcasting on when you hover your mouse cursor over the V icon that shows up in your system tray.

Above I've shown you the Connect Options window for xvnc4viewer so that you can see what can be changed. However, the only options I changed were Colour Level (to Full Colour) and I enabled Full Screen Mode. Once she told me her public IP address, I just had to enter it into the textbox in the VNC Viewer: Connection Details main window shown above and press okay. Upon connection, the VNC viewer expands as much as it can to accomodate the resolution of the VNC server's screen and then anything they do on their computer can be seen on your computer! It's pretty cool except I'm not sure it allows the person with the viewer to control anything on the server's computer. That was acceptable for my over-the-phone support, as I only needed to see what she was seeing!

If you want to install xvnc4viewer for ubuntu, simply open the terminal and type sudo apt-get install xvnc4viewer. Easy peazy right? Once you have it installed, to start it up, go to the terminal and type in xvnc4viewer. That's all!

Ubuntu Rescue Remix helped factory reset my Acer Netbook!

2010-01-07T22:28:00.005-05:00

I've had an Acer Aspire One Netbook for a bit longer than a year now and I finally have no more use for it. It's been a fantastic little computer, serving its purpose in letting me take notes in grad classes when I've needed it. It's incredibly light (2.2 lbs), runs Windows XP, has a big enough hard drive (120 GB), 3 USB ports, digital camera memory readers and a wifi Anyway, it's not my intention to sell my netbook over my blog. I am however intending on selling it to people over craigslist. In order to give it to someone, I needed to restore the netbook to its manufacturer intended state. Conveniently, Acer shipped these things with windows install files on a separate partition. Not so conveniently, their "erecovery" program they make available to download for this purpose doesn't do a thing! I was very frustrated and wanted a way of getting to those install files on the separate partition.

Along the way I thought of using a Live USB version of linux to somehow get to those files. At first I tried Ubuntu Rescue Remix version 9.10. The website provides you with an ISO, which you then use in conjunction with USB Startup Disk Creator (accessible by clicking System > Administration > USB Startup Disk Creator).

You press the Other button above in order to load up the ISO file that you downloaded. Then you look for your USB stick partition at the bottom (you probably have a pet name you assigned to it) and click Make Startup Disk. This organizes your USB stick accordingly and allows you to boot using Ubuntu Rescue Remix. Weirdly, version 9.10 didn't work for me, but 9.04 worked just fine. Be warned, it does boot you into a command line interface, but you don't have to know very much to proceed. All I had to do was:

Press the Enter key when the login screen showed up
Type in sudo parted to load the Partition Editor
Type in print all in order to list my partitions
Type in set 1 boot on to make the windows installation partition bootable (I think it was partition 1, you'll probably have to play around to be sure which is the windows installation files partition)
Type in quit
Then I shutdown my computer, made sure the USB stick was out, turned it back on and voila, the Acer windows restore utility started running!

Once it's finished installing, you actually have to boot back into Ubuntu Rescue Remix and make it so that your main windows partition is bootable again. Go through the above list of instructions and you'll have your Acer Aspire One Netbook restored to it's manufacturer's intended state!

Ubuntu saved Windows, how about that?!!?

Matlab's free cousin Octave

2010-01-04T19:25:00.004-05:00

My educational/work background is in experimental science. Therefore, much of what I've done for work purposes involves processing data, plotting data, and analyzing data. I did most of the data processing and plotting in Matlab. Matlab is a fantastic language for scientific computation which is even available for Linux. Of course, the downside to Matlab is that liscences for it cost big bucks outside of academia!

Recently, I discovered another language for scientific computation called Octave. The amazing thing about it is that it is a free-to-download, extensively Matlab-like language! In other words, if you already know Matlab, you barely have to learn anything new in order to use Octave. The functionality is VERY similar, with Octave able to do things like Matrix operations, Signal Processing, Statistics, Scripting and Plotting. I installed Octave in Ubuntu by typing
sudo apt-get install octave into the terminal. Unexpectedly, Octave came packaged in the Ubuntu repository with QtOctave, a complete graphical development interface for working with scientific data. Check out the screenshot below:

As you can see, the GUI includes the most important elements found in the Matlab GUI:

The command prompt
A command history
An area to access all the variables/data loaded into memory
A filesystem navigator
Finally, a script editor window (accessible via the fat red pen icon

I loaded the download speed data that I sampled last week into Octave and did some modelling to extract the general trend that the download speeds followed. I made a graph of the data and modelling in Octave to see how it looked.

If you've ever done any plotting in Matlab, you'll recognize the same graphics quality in the graph above. Just like Matlab, the default plotting parameters aren't the prettiest, but that's okay. The important thing is that Octave is free, it's sophisticated, and very easy to use for Matlab veterens!

Tis a Gift to be Simple

2010-01-03T18:44:00.002-05:00

Today I got interested in trying out a few different blogging sites (wordpress.com and tumblr.com) to see if either one of them provided a better/easier interface to work with than blogger. Going on Wordpress, I saw an interface that was both sophisticated and nice looking. Wordpress seemed to have everything.

Unfortunately there were a few drawbacks:

Wordpress seems to intentionally set the width of your "Content Column" (the column within which your blog post appears) to be too narrow for my liking. Resultantly, the bigger pictures that I post appear to be cut off :(
After investigating possible solutions to my first problem, I found out that I would need to pay a fee to customize the width (along with other features) of my blog to my liking. I don't like that at all.
Even if I was to live with the narrow width thing, I am surprised that Wordpress doesn't automatically size your pictures according to the width of your content column.

Oh well, I probably won't be switching to wordpress! As for tumblr, I found their registration to be super easy, but the exact method to customizing your blog using CSS eludes me. I admit I know nothing about CSS, so tumblr may still be an option to investigate further.

Until then, "tis a gift to be simple". Thanks Blogger!

Blog from your HTML Editor and Python

2010-01-01T12:56:00.004-05:00

So this is a test of the Python script I made to upload a custom made HTML file to my blog. As I mentioned in my last post, Google has provided its own Python Client Library to the public for connecting to Google's various services. What really confused me at first was the question of where exactly Google stores the pictures you want on your blog when you use the regular Blogger New Post interface. I soon discovered that if you're not linking to a picture that's on another website, any picture you put on your Blogger Blog is actually stored under a Picasa Web Album named after your blog and accessible by your Blogger account username and password. Knowing this, I just had to follow a few simple steps for using the Python Client Library to automatically upload pictures to the user's Picasa Web album (see here).

What I figured out is that if you want to put pictures on your blog post, you don't need to know where they'll be uploaded to. Just link to the pictures on your local hard drive and submit your HTML blog post file to my script. My script then:

looks for any local image links
uploads those files to your Picasa Web Album
gets the URLs of your newly uploaded images
Uses those URLs to replace the local path links in your HTML file, and finally
Uploads your newly modified HTML file to Blogger!

Also, I noticed that Blogger ignores the header/footer html tags found in all web pages like <html>, </html>, <head>, </head>, <body>, and </body> but retains the newline codes that follow them all. Finally, when Blogger processes <br> tags that are followed by a newline code, it creates two new lines. To fix this using my script, I made it so that the inputted HTML file gets sliced from the end of the first <body> tag to just before the beginning of the </body> tag. As well, it gets rid of all <br> tags, leaving only newline codes in their places. At the moment it's a command line script that I haven't yet tested on Windows. If you'd like to take a stab at using it, download it here. To make life easy, download it to the directory where you're saving your HTML blog post file and just call it from the command line by running python pyBloggerU.py. The program will ask you for the following pieces of information:

The email address associated with your Blogger account
Your Blogger account password
Your Blog title (case insensitive), and finally
Your Blog post title

If you'd like to help me out, I've registered my project in launchpad and you can submit a branch of my code. Go here
if you want to donate your time to help me out :).

And now finally, it's time for me to test out media uploading:

Linux Blogging Solution?

2009-12-30T16:09:00.001-05:00

I don't much like writing to this blog from the New Post interface that blogger provides. It's too small and restricting. So, I've done a lot of googling to find out the best way of writing to a blog from Linux. One option I found was using Google Docs to create a text document (replete with rich text and images) and publish it to blogger.

I like the format of Google Docs and would certainly continue using it if it weren't for weird formatting incompatibilities between Google Docs and Blogger. In other words, when writing a blog post from Google Docs and submitting it to Blogger, what you see is not exactly what you get. I find that the text spacing and alignment become perverted when you submit a Google Docs text document to Blogger.

So, that's where the Google Data Python Library will come in handy! Using this set of Python modules, it's possible to upload photo media to the Picasa Web Album associated with your blog, get the associated URLs, and upload new blog posts containing your newly uploaded photos.

Using this functionality, I should be able to make a script that will take an HTML file that you create with any old web page editor, replace the links to images on your local hard drive with links to images on your picasa web album, and post your HTML file to Blogger!

Stay tuned. I'm hoping this ends up being better than Google Docs or Scribefire (I have issues with both platforms).

Your download activity in one file!

2009-12-28T17:43:00.005-05:00

I started downloading a very big set of files today on bittorrent (using the simple and elegant Transmission Bittorrent Client that comes with Ubuntu). As I was watching the download speed indicator fluctuate up and down, the data analyst in me started stirring. I thought it would be neat if I could find a program that would output my download speed over time to a dataset that I could then analyze for trends, averages, highs and lows.

I inputted the term 'bandwidth' in Synaptic Package Manager to see what kinds of programs were available for this purpose. I found some simple command line programs such as nload, bmon, and iftop. All of them provide text-mode screens that help you monitor your bandwidth, but none of them allow you to output download speed over time to a plain text file.

After examining the man (manual) pages for each of these programs, I noticed a commonality: each of these programs sample bandwidth information from a file named dev in the /proc/net/ directory on my computer. Curious for more info, I navigated to that directory and opened up the file. Here's an example of what it looks like:

If you've spent any time messing around with your network connection settings in Ubuntu (or most other operating systems) then you'll recognize most of the row titles in the above image. Now, my computer is only connected to the net through a wireless connection to my Router/ADSL Modem. Therefore, I don't expect any significant data in rows other than the one titled wlan0.

Next is the neat step: Linux repeatedly updates the dev file with the total number of bytes and packets received since boot-up. All you need to do to find out the download speed in bytes/second is:

Open the file
Copy the numeric value just to the right of wlan0, under the bytes column
Wait a second
Do the same thing with the next value
Subtract the two!

Of course nobody in their right minds would want to do this manually, but the implementation of the above algorithm is very simple in Python:

def get_bytes():
   dev = open('/proc/net/dev','rb').read()
   import re
   pat = re.compile('wlan0:\s*([0-9]+)\s*')
   return int(pat.search(dev).group(1))

def get_kbps():
   import time
   bytes1 = get_bytes()
   time.sleep(1)
   result = round(((get_bytes() - bytes1)/1000.0),1)
   return '%3.1f' % result

The first function above simply opens the dev file and returns the numeric value beside wlan0. The second function uses the first function to get the bytes value two times and calculates Kilobytes/second (it's a smaller number than bytes/second).

I'm now using these functions to calculate Kilobytes/second every 3 seconds for a total of 4 hours to get my dataset! If you'd like to use these functions just remember that if you're using a wired ethernet connection, change wlan0 to eth0 and you will be able get your data.

Permissions, Samba Sharing an External NTFS drive, and fstab

2009-12-26T22:25:00.004-05:00

For my first post on this blog I wrote about setting up samba shares for my printer and external hard drive. When I wrote that entry, I thought I finally succeeded in sharing both devices. I was disappointed to learn that my solution was temporary, and my external USB hard drive was still unable to be shared with my wife's Windows XP machine.

After hours and hours of pursuing the solution to this problem like Captain Ahab come to life, I started to realize that my problem wasn't anything to do with Samba itself, but the permissions that were automatically assigned to my external hard drive upon its mounting. Every time I looked at the extended information given by the ls -l command in my /media directory, I saw the following:

drwx------ 1 inkhorn inkhorn 4096 2009-12-06 18:24 backup

In other words, my external hard drive was mounted so that I personally would have read, write and execute access to the drive, that I could mount and unmount it, but nobody else could access my drive in any way at all.

From googling my problem, I learned that if you modify the fstab (file system table) file in the /etc directory, you can change the permissions linux assigns to your hard drive when mounting it. I found numerous message board postings with examples of how to set up an entry for your hard drive in the fstab file, but none of them helped for very long. Finally I found a web page that described a program available to Ubuntu users called pysdm (Python Storage Device Manager) that provides a GUI allowing the user to set permissions for any attached storage devices.

As you can see above, it displays all connected partitions and provides an Assistant button that allows you to choose your mounting options intelligently. In my case, without having a USB stick connected first, my external hard drive shows up as partition sdb1, or /dev/sdb1.

When you click on the Assistant button, you're brought to the above Select options window. Most of these options were already set as enabled when this window popped up. I think the only option here that I changed was to enable Allow a user to mount and unmount the file system.

I then found an option I thought might be important in the Miscellaneous tab. I enabled the option above labelled This file system requires network. So, I pressed OK, then at the main window I pressed the Apply button. There was one catch however: the line that it automatically entered into my /etc/fstab file linked the partition to the mounting point, not the device itself. See the fstab entry below:

/dev/sdb1 /media/backup ntfs nls=iso8859-1,_netdev,umask=000,user,owner 0 0

In other words, what this entry in my fstab file says is that any time partition /dev/sdb1 comes online, mount it to /media/backup; even if /dev/sdb1 is my USB stick, and not my external hard drive! You see what I mean?

This is where what's called the UUID comes in handy. The UUID is an alphanumeric sequence that uniquely identifies each connected storage device independently of the partition assigned to that device. After all, if I connect my USB stick and then my external hard drive, it's my USB stick that would show up as partition sdb1, while my external hard drive would show up as partition sdc1. In order to make sure that it's my external hard drive that gets mounted as /media/backup and not my USB stick, I had to replace /dev/sdb1 with the UUID. To find out the UUID of a connected device, go to the terminal and type in sudo blkid. Here's the output on my terminal:

inkhorn@inkhorn-laptop:/media$ sudo blkid
/dev/sda1: UUID="2fb4ca03-eca7-41f8-9767-bbfaf88a1890" TYPE="swap"
/dev/sda2: UUID="34B836C6B8368700" LABEL="S3A6134D002" TYPE="ntfs"
/dev/sda5: UUID="ece79ff2-c123-4a72-80a9-c187b2cfd785" TYPE="ext4"
/dev/sdb1: UUID="010C7CC23301CA6C" LABEL="backup" TYPE="ntfs"

So, I opened up the fstab file again and replaced /dev/sdb1 with UUID=010C7CC23301CA6C. I then saved the fstab file, restarted, and VOILA!!! My Samba share for my external hard drive is finally accessible from my Wife's Windows XP machine!

If you want to install pysdm on your Ubuntu system, type the following in your terminal:

sudo apt-get install pysdm

Then you can access it through the following menu items: System > Administration > Storage Device Manager.

Schedule a delayed shutdown

2009-12-25T10:25:00.002-05:00

Several times I've initiated a download of a big movie file late at night before I go to bed. Without knowing anything else, I'd normally have to leave my computer on all night just so that I could download that movie file that will probably only take up 3 hours. Wouldn't it be nice if you could tell your computer to turn off after the time it takes to download something?

That's where the shutdown command in bash comes in handy! Here's an example of the way I usually use it:

sudo shutdown -P 300

Firstly, you need to have root access to use shutdown, hence my use of the sudo command. I used -P to indicate that I want to power off after shutdown (as opposed to rebooting, or 'halting'). Finally, I typed 300 to indicate that I want the computer to shutdown in 3.5 hours (300 minutes).

Another way you can use the command is by specifying the clock time at which you want your computer to shutdown in 24 hour format. So let's say you want your computer to shut down at 4:15pm:

sudo shutdown -P 16:15

Type that in and your computer will shut down at 4:15pm. That's it!

Split files into parts easily in bash

2009-12-22T22:44:00.002-05:00

These days, USB sticks have obliterated the worry of not being be able to carry computer files from one computer to another. However, there will be instances when you'll be tempted to email a file to someone and that file will be too big to attach to your email. Whatever do you do!?!

This is where the split command in your Linux terminal comes in handy. Let's say you have an .mpg file that runs for roughly 25 minutes and happens to take up 40 megabytes (e.g. 'speeches from wedding video.mpg'). Let's also say that your email server will accept attachments of maximum 10 megabytes. So, we know that we need to split this .mpg file into 4 parts of 10 megabytes each.

Here's the command to use for the example file:

split -b 10000k 'speeches from wedding video.mpg' speeches.

Here's what the arguments mean:

-b tells the split command that you want to specify the 'byte' size
10000k, or 10,000 kilobytes is the size of the parts we wanted (10,000 kilobytes = 10 megabytes)
'speeches from wedding video.mpg' is the file we want to split into parts
speeches. tells the split command that all part files should begin with 'speeches.'

This will result in the creation of 4 new 10 meg files named:

speeches.aa, speeches.ab, speeches.ac, speeches.ad

Putting them back together again

Admittedly, this is a solution where the other person has to be a Linux user. The person on the other end has to navigate to the directory where the part files are stored and enter in the following command:

cat speeches.?? > 'speeches from wedding video.mpg'

Voila! Humpty dumpty has been put back together again :)

This neat Linux trick has been brought to you by an awesome book I'm currently reading called Linux All-in-One Desk Reference For Dummies. I heartily recommend it to Linux newbies!

Finding Files in Google Desktop and bash

2009-12-21T15:00:00.006-05:00

Sometimes I want to find a file and I have no clue where it is. When I was using Windows Vista, this was a simple matter of using the native file searching utility, which dutifully searched for my lost file using the index it created of my hard drive. File indexing is an amazing thing to take advantage of. When your computer searches a file index, it gets results FAST!

If you don't have a file index (remember, it's literally a simple listing of every single file you've put on your computer in one relatively small database file) then your computer will have to loop through every single file on your computer, trying to match your search argument (e.g. 'Resume*.pdf') to the file names that it processes.

Today I'll talk about three linux programs for indexing and/or finding your files:

Google Desktop (for easy indexing and finding using a GUI)
locate (for finding indexed files in bash/terminal)
find (for finding files in bash/terminal)

Google Desktop

Even if you are using windows, click on the link above and get this program. It's VERY convenient to use! Google Desktop will not only allow you to search for your files, but emails that you've sent and received as well! Once you download it, you will want to make sure that it builds an index of the directories in which your important files will be stored. As shown below, right click on the Google Desktop icon and click on Preferences. On my computer I have it indexing my home user directory and my USB drive root directory. You can of course add any extra folders you like by click on Add folder to search.

Preferences are opened in your default browser

You may want to modify settings in the other tabs shown above. Go to the Gmail tab and follow the instructions there if you would like Google Desktop to index your gmail account. Also, go to Display and click on the Change Hotkey button lower down on the page in order to set up a key combination that will bring up a quick search box (shown below).

My key combo: Shift+Control+?

When you're sure that everything is set, click on Save Preferences and Google Desktop will start indexing your files! Bear in mind that the process is slow, so you'll have to give it several hours until it's done indexing everything. You can check on Google Desktop's progress in indexing by right-clicking on the Google Desktop icon, then on Index, then on Index Status. You'll then see its progress in indexing all of your files and emails.

What I really like about Google Desktop is the simplicity in finding and opening anything on your computer. Type your key combo, quick search box comes up, type in your search argument, click on the result matching the file you were looking for, then your file opens!

locate

As I understand it, Ubuntu 9.10 (and i'm sure basically every other distro of Linux) indexes all my files for me on some periodic basis. This is what enables the locate program to work in the terminal. locate searches through the generated file index database for a term of your choosing. So let's say I want to search for the manual I recently downloaded for my Hammer Drill, but I have no clue where it went! Here's how I would use the command to find my file:

inkhorn@inkhorn-laptop:/$ locate -i hammer

Here's what the arguments I used mean:

-i tells the locate program to ignore the case of your search term
hammer is simply the search term

Here is the output from the locate command:

/home/inkhorn/Documents/Hammer Drill Manual.pdf
/usr/share/pixmaps/pidgin/emotes/default/hammer.png
/usr/src/linux-headers-2.6.31-15/arch/avr32/boards/hammerhead
/usr/src/linux-headers-2.6.31-15/arch/avr32/boards/hammerhead/Kconfig
/usr/src/linux-headers-2.6.31-15/arch/avr32/boards/hammerhead/Makefile
/usr/src/linux-headers-2.6.31-16/arch/avr32/boards/hammerhead
/usr/src/linux-headers-2.6.31-16/arch/avr32/boards/hammerhead/Kconfig
/usr/src/linux-headers-2.6.31-16/arch/avr32/boards/hammerhead/Makefile

The first result is the one I'm looking for! Now I know where to look to open my hammer drill manual.

find

The find command has to loop through each and every file in the directory you specificy in your search argument. If you told it to search the root directory '/' for a file with the word 'Hammer' in it, your computer would probably take an intolerably long time finding it. Happily enough, you don't need to do that and probably have an idea generally where it is. To use my hammer drill example:

inkhorn@inkhorn-laptop:/$ find /home/inkhorn -iname 'Hammer*.*'

Here's what the arguments I used mean:

-iname tells find to ignore case when comparing your search argument string to the file names that it loops through
'Hammer*.*' tells it to look for a file of any extension (.*) starting with the word 'Hammer' and continuing with any other words after that (*)

Here is the output from the find command:

/home/inkhorn/Documents/Hammer Drill Manual.pdf

Cool thing is that the find command only outputs file results while the locate command outputs directory results! In this instance, the results came on screen really quickly.

Now for something even cooler: I can enter in a command that looks for a file that I'm interested in and opens it in a program that reads it! Here is what I would enter in if I wanted to look for the hammer drill menu in terminal and automatically open it up in an excellent PDF reader named Okular:

inkhorn@inkhorn-laptop:/$ find /home/inkhorn -iname 'Hammer*.*' -type f -exec okular '{}' \;

You've already seen the first part of the find command in action. The second part tells the find command to execute a program called 'okular' that should then take the find results as an argument. In other words, okular opens up the file listed in the find output.

You can use this to open up other file types that you want to find. If you're looking to open a word document, replace 'okular' with 'oowriter', a spreadsheet, replace 'okular' with 'oocalc'.

For more info on using find and locate, click here.

Backing up your computer files with rsync

2009-12-19T18:49:00.000-05:00

If you're paranoid about losing ages and ages of work you need to find a way of backing up your computer files! In Windows I would use a program called Save-n-Sync to make sure that a synchronized copy of all my work files could be saved on a backup source. In Linux this can be done with one command in the terminal using a shell program called rsync:

rsync -r -u --log-file=[path for your new log file] --stats [path for your source directory] [path for your target directory]

I'll explain the arguments:

-r tells it to recurse through the subdirectories of your specified source directory, meaning that it will synchronize files found directly in the source directory, but also files found within subdirectories inside your source directory.

-u tells it that if a file is in both the source and target directory, it should only update the target directory file if it is an older version than the one in the source.

--log-file=[path for your new log file] is the command you should use if you want the results of your file sync to be saved in a log file named however you like. For example: --log-file='/home/user/Desktop/Backup Log.txt'

--stats tells it to output some useful file sync statistics for you such as how many files were scanned through, transferred, size of the transfer, etc.

[path for your source directory] should be replaced with your source directory such as '/media/USB Stick'

[path for your target directory] should be replaced with your target directory such as '/media/Backup Drive'

You can add or take away whatever of these options you like and many many more. Type in rsync --help in the terminal to learn more. Now, if you don't feel like typing this command in every time you want to back your stuff up then you can create the equivalent of a shortcut, in Ubuntuspeak a 'Launcher', on the desktop. Do the following and all you'll have to do is double click on your launcher icon to back your files up:

Creating launchers is simple!

Right-click (or if you're a lefty, left-click) on your desktop and select Create Launcher

The selection next to Type: should be left as Application

Beside Name: you can put whatever you like to indicate that it's a Launcher for your backup command

Now next to Command: put in your full rsync command with all the arguments discussed (or not) above

Feel free to click on the icon box at the top left to select a more interesting icon for your backup launcher (i.e. select an icon, and press OK)

Press OK when you're done and now you have a Launcher that you can double click whenever you need to back your stuff up!

Happy file-synchronizing! This program does a blazing fast job :)

Red-Eye Reduction in GIMP and Gwenview

2009-12-19T17:50:00.003-05:00

I have many photos on my computer and a good portion of them tend to be of people whose eyes become red on account of the camera flash (despite the orange light my camera emits as it's taking pictures). In windows it's fairly easy to open up your red-eye affected photo and fix it up (practically in a few clicks of the mouse). How easily can we accomplish the same task in Ubuntu?

Today I'm going to load a cropped picture of two people with red-eye into two well-known programs in Ubuntu, GIMP and Gwenview.

The Offending Picture!

Red-Eye Reduction in the GIMP

I'll trust that you know how to open a picture in any program and cut to the specifics. From here we have two ways to proceed:

(1) If there are other red colours in your picture that you want to preserve:

Select one of the red-eyes in your picture with the ellipse-select tool
Click on the 'Filters' menu
Click on the 'Enhance' submenu
Click on 'Red-Eye Removal' (big surprise!)

Not so intuitive yet not too much to remember right?

Hope your fine motor control is effective here!

Next you will see the Red-Eye Removal configuration window where it will ask you to choose a red-eye threshold. Above you can see that it gives you a preview of the eye that you're modifying. The default threshold value seems to work without a problem. So, press okay and move on to the other eyes in the picture.

The Result!

(2) If the only intensely red colours in your picture are in the eyes:

Don't select anything in the picture
Navigate to the 'Red-Eye Removal' filter as before
Press okay at the threshold that looks best (probably the default)

Easy for this picture, but beware of pictures with other reds in them!

Red-Eye Reduction in Gwenview

There's only one way of reducing red-eye in Gwenview and it is not very difficult:

Open up your photo in Gwenview
Under the 'Image Operations' pane, select 'Red Eye Reduction'
When your mouse cursor turns into a cross-hair, select the middle of any one of the red-eyes that you want to fix
A small circle will now appear on the spot where you clicked to 'de-red' the red-eye. You can change its size using the dark grey panel that appears below the picture
When you're done, click apply and move on

Still need to have some good fine motor control skills here!

The Result!

Conclusion

The GIMP is a fantastic image manipulation program with so many features. However, I find that Gwenview is simpler to use for this purpose. My feeling is that many users wanting to edit photos will only want to do some of the simplest tasks: cropping, rotating, flipping and red-eye reduction, to name a few. As I've demonstrated here with red-eye reduction, Gwenview is a very simple program to use. To the user with such simple needs (I'm including myself here) I would recommend Gwenview. However I certainly plan on keeping GIMP on my computer as there are times when I need something above and beyond the simple stuff.

Thunderbird 3 - Nice and Polished!

2009-12-14T22:08:00.003-05:00

Many years ago I used MS Outlook to organize my many emails, my calendar and my contacts. I liked Outlook alright until I finally got sick of how resource intensive it felt after years of use. I'm talking about the feeling you get when you hear the frantic clicking of your computer accessing stuff on your hard drive and executing processes like a type A personality on cocaine. So I felt the need for a change!

I discovered Mozilla Thunderbird and I never looked back. Thunderbird had (and still has) about all I can ask for. It loads fast, handles all sorts of email accounts (POP, IMAP, Gmail, Hotmail, etc), has a calendar plugin you can use called Lightning, and has BLAZING fast email search functionality!

The email search functionality of Thunderbird is perhaps my favourite feature. I actively use 3 email accounts and so accumulate A LOT of email from day to day! This is no matter for thunderbird, as it quietly indexes each and every email as they come in (or after you import your emails from another source). I most often use the 'Subject or Sender' search when looking for emails. Want to find an email? Type in a word or more from the subject, or type in the name of the person whosent it and WHAM, it shows you all the search results lickety split!

So, when I read recently that Mozilla came out with Thunderbird 3, I absolutely had to check it out!

Thunderbird 3 Screenshot

For those of you who have seen Thunderbird before, you'll see that apart from some icon style changes, Thunderbird 3 introduces tabs to its email interface. I wasn't exactly waiting with baited breath for this feature, but you have to admit that Email clients might as well have tabbed interfaces now that all browsers have them.

New email opened in a tab

Want to see your email in a new window? Just click on 'other actions' then 'open in new window'. Another really cool feature you'll spot in the 'other actions' menu is 'show in conversation'. This threads all emails bearing the same subject line as part of one tree structure, where each reply to an email in that conversation creates its own branch.

'show in conversation' view. Google GMail meets Mozilla Thunderbird?

I think that the 'conversation' view is a much needed addition to Thunderbird's complement of tools to organize your email.

Finally, I'm happy to report that searching for emails in Thunderbird 3 is just as good as it has ever been. There is at least one search type that I do not understand ('Recipient filter'.... How is that different from a 'To or CC filter'?) but that's okay because I rarely use anything other the 'Subject or From filter' search anyway.

So, if you're using Windows or Mac, go here, or if you're using Ubuntu, paste the following 4 lines (taken from the Ubuntuzilla project site) into your terminal:

(1) echo "deb http://switch.dl.sourceforge.net/project/ubuntuzilla/apt all main" | sudo tee -a /etc/apt/sources.list > /dev/null

(2) sudo apt-get update

(3) sudo apt-get install ubuntuzilla

(4) ubuntuzilla.py -a install -p thunderbird

Most of all, enjoy!

Setting up file sharing in Ubuntu Linux

2009-12-06T15:05:00.003-05:00

I made a BIG switch from Windows Vista to Ubuntu Linux a while ago. I have been able to figure out how to do nearly all of my computing tasks in Ubuntu that I need on a regular basis. The one thing I had a lot of difficulty with for some reason was file sharing.

My linux machine is wirelessly networked with my wife's Windows XP desktop and my own Windows XP netbook. I wanted to share my external backup hard drive over the network so that she could just copy-paste stuff right into the drive over the network.

For some reason I found that the simplest method just didn't work:

Right-click on the icon for my backup drive in Nautilus
Click on 'Sharing Options'
Click on 'Share this folder'
Click on 'Allow others to create and delete files in this folder'

It kept giving me error messages! Argh! I kept finding a bunch of how-to's telling me how to modify my samba configuration file. That was promising but I kept messing stuff up:

My printer share stopped working
All my shares stopped working
I could only view shares from her computer

Finally I found an answer. Today I looked up "easy samba configuration" on google and found this page that gives a simple samba configuration file that only needs a few tweaks to work. First I modified the [global] section:

I put my computer name, "inkhorn-laptop", next to "netbios name = "
I put my workgroup name, "MSHOME", next to "workgroup = "

Then I modified the [network-share] section:

I changed [network-share] to read [backup-drive]
Put the path to my backup drive, "/media/backup" next to "path = "
I deleted the "force group" line, as that didn't seem to help
I put the username I set up for my wife next to "force user = "

Just make sure to follow the directions in the page I linked to above to create a samba username and password for the person you want to access your share.

Finally file sharing and printer sharing are working at the same time :)

--------------- EDIT: Dec. 27, 2009----------------------

The above seems to work just fine for sharing folders on my main hard drive. To share my external drive it turns out I needed to jump through some hoops to change its permissions so that other people can read, write and execute files on it. See my latest blog post on the topic here.