The Old Monk

I use remind(short howto) to keep my calendar of events and reminders. Its a wonderful utility - and all commandline. I split up the birthdays, anniversaries, personally important days, and the usual stuff into different files, and include them in my ~/.reminders file. Here's what my .reminders looks like :

gera@gera-laptop:~$ cat .reminders 
include /home/gera/reminders/birthdays
include /home/gera/reminders/anniversaries
include /home/gera/reminders/impdates
include /home/gera/reminders/stuff

All this works well, except for a small issue. It takes too much time for me to add reminders for my day-to-day tasks. Firing up an editor and typing all that remind syntax is something that can be avoided. Also, it really hurts to look at the calendar to figure out fuzzy dates like "tomorrow", or "sat". I was planning to write this big Perl script which would do the heavy lifting for me. Instead, I found Date::Manip which already does the heavy lifting. It understands stuff like "tomorrow" and "sun". I just needed a small 'remindme' script then :

#!/usr/bin/perl -w
use strict;

use Date::Manip;
use Fcntl qw(:flock);

my $reminders = "$ENV{HOME}/reminders/stuff";

my $argstring = join (" ", @ARGV);
my ($when, $what);

(undef, $when, $what) = ($argstring =~ /^(on |)(.*?) to (.*)$/);
quit("what?") unless $what;
quit("when?") unless $when;

# form the line to be appended
my $date = ParseDate($when);
quit("cannot parse date") unless $date;

my $line = UnixDate($date, "REM %b %e %Y +1 MSG");
$line .= " $what %b.%\n";

# write in file

# open lock file
open LOCK, ">$reminders.lock" or quit("cannot open $reminders.lock : $!");
flock LOCK, LOCK_EX or quit("cannot obtain lock : $!");

# open data file, write and close data file
open REMINDERS, ">>$reminders" or quit("cannot open $reminders : $!");
print REMINDERS $line;
close REMINDERS;

# release locks
flock LOCK, LOCK_UN;
close LOCK;

exit (0);

sub quit
{
        my $msg = shift;
        print STDERR $msg, "\n";
        exit (-1);
}

Now, I can use it like this :

$ remindme on sat to eat everything I can
$ remindme tomorrow to pay the phone bill
$ remindme next week to think up of something useful to say
$ remindme next thursday to find something better to do

nifty, eh?

update: bugfixed the script. An extra space creeped up between the '+' and the '1'. Thanks to AmitU for pointing that out.
update 2: added the capability to get reminded n days in advance - a feature suggested by AmitU in the comments, but with a different syntax ("remindme next week to do this for 2 days"). Also, remindme's git repository and home page. The repository contains the bash-completion script as well.

I was introduced to GTD around a year ago, but wanted to manage mine using a commandline. I discovered todotxt.com and modified a todo.py to suit my needs (removed priorities, added some commands, bash-completion etc).

But I stopped doing it.

Because I had nothing to manage my "projects" (tasks with multiple action items). I maintained a list of projects out-of-band, as a directory structure, which required me to identify the next task for a project and move it to my todo list. Of course it didn't work!

So I wrote gtdo. gtdo has a similar interface - todo.py was the inspiration - but in addition to the contexts (marked by a '@'), it supports groups (or "projects" - marked by a '/'). So, you might have a project about starting using GNUcash. Of course, the first step is to install/set-up GNUcash. Adding your portfolio comes later. If you have both these items in your todo list, the second one does nothing but adds noise and makes the list longer. The answer is to add them in a group /gnucash. If there are multiple tasks in a group, only the first one is displayed - till its marked done. By default, new tasks are added at the end of the queue in a group, but may be added at a specific position.

An example would be better than the ramble above (gtdo being aliased to 't'):

$ t add /gnucash @online install gnucash
$ t add /frames @errands buy wood
$ t add /email @online set up spamassassin
$ t add /gnucash @online add portfolio to gnucash



# only the immediate next task is displayed when you 'ls'
$ t ls
1 : /gnucash @online install gnucash
2 : /frames @errands buy wood
3 : /email @online set up spamassassin



$ t ls @online
1 : /gnucash @online install gnucash
3 : /email @online set up spamassassin



# except when you specifically ask for a group
$ t ls /gnucash
1 : /gnucash @online install gnucash
4 : /gnucash @online add portfolio to gnucash



# well, we need to fix F::Q::IndiaMutual before we add our portfolio
# we'll try adding that as the second step in the /gnucash group
$ t add /gnucash.2 fix F::Q::IndiaMutual



$ t ls /gnucash
1 : /gnucash @online install gnucash
4 : /gnucash fix F::Q::IndiaMutual
5 : /gnucash @online add portfolio to gnucash

Comes with standard todo.sh style bash-completion. The Git repository is available at http://repo.or.cz/w/gtdo.git. You can obtain a tarball by clicking here.

Reading configuration items is a task that almost every application/utility does. Writing code for all those is stupid. Even writing code once and linking it in leaves the language binding issues open. What encouraged me to write this is a frequently invoked piece of (network) code which ideally would check its configuration for each invocation. The disk I/O seemed unnecessary.

Enter configd - the configuration daemon. Reads configuration items (key=value pairs) for "domains" (applications), and gives them out to anyone who cares to ask over HTTP. Supports HTTP GET/HEAD, with the "domain" and "key" headers, handing out the value as the "value" header in the response. Returns a 404 Not Found if it can't find a value, or a 400 Bad Request if the request is malformed. A SIGHUP to configd makes it re-read the (possibly changed) values from the configuration.

configd is consistent with the Unix Philosophy. A simple tool that does only one job, and does it well.

Although configd advertises configurations, it doesn't stop anyone from using it as a trivial key=value "database" (for a stretched definition of the term). It's HTTP - something that's very well understood and supported, and can scale extremely well behind a load balancer, offering caching and remote configuration.

PUT requests for storing values, and some form of authentication are something that might be added some day, although I don't have the need for them right now.

The Git repository will shortly be online.

Edit: The configd repository is now online.

Something that I thought just now: On the trivial database thread, configd can present a consistent interface, once it implements PUT. Reads are cached, load balanced, yadda yadda. And writes are PUT requests. The backend doesn't matter to the application. One can take out the flat files, and replace it with MySQL (with some type checking at the configd/application end), and the application wouldn't know better. A connection can be equated to a transaction. All PUT requests over a connection are rolled as a transaction. How cool is that?

Also: If you use it as a "database", you cleanly separate the scale layer from the business logic layer. Right now, even prototypes use an SQL backend because people want to scale in the future. There's a lot of DBI/SQL where it doesn't belong. Engineers who write prototypes also write SQL instead of a DBA ("its just a prototype"), which they have to maintain as legacy code/design. All these problems vanish away when you use the simple approach. Engineers don't have to worry about SQL or scale. They can be up and running with a flat file backend which can be changed to SQL without the app noticing. If and when you swap in a real database behind it, SQL queries can be written for the engine by real DBAs (you're scaling this up - its no longer a prototype - it makes sense to assign a DBA for this). To top it all, your *hardware* can do sharding. HTTP load balancers can load balance based on headers - and domains/keys are headers here. The applications need not worry about scaling the database _at_all_.

Single purpose ssh keys are awesome, but there's one gotcha - they'll work if you enclose the command in double quotes, but not in single quotes.

Don't ask me why.

I have a tiny helper script to collect addresses from my outgoing emails, and add them to my .aliases file. I only care about people who I send out email to, and I've noticed that collecting addresses from incoming emails leads to too much noise. Although I could set it up to *not* pick up addresses from spam and mails which have my firstname (ecommerce stuff which is NOT spam). But that's for later.

Here's the helper (I use msmtp for sending out email as evindent in my mutt config) :

#!/usr/bin/perl -w
use strict;

my $home = "/home/gera";
my $msmtp = "/usr/bin/msmtp";
my $aliasfile = "/home/gera/.aliases";

open ALIASES, "<$aliasfile" or quit("cannot open $aliasfile : $!");
my %aliases = ();
my %collected = ();
while() {
	s/#.*$//;
	chomp;
	next unless $_;
	my ($nick, $name, $email) = ($_ =~ /^alias\t+(\S+)\t+(.+?)\t+<(\S+)>/);
	quit("cannot parse $_") unless ($nick && $name && $email);
	$aliases{$email} = {
		"nick"	=> $nick,
		"name"	=> $name,
	};
}
close ALIASES;

# now read the email contents and try to figure out the addresses
local $/ = undef;	# slurp mode

my $email = ;
my ($to) = ($email =~ /^To:(.*)$/m);
my ($cc) = ($email =~ /^Cc:(.*)$/m);
my @addresses = split( /,/, "$to,$cc");
foreach my $address (@addresses) {
	chomp;
	next unless $address;
	# the address is of the form of "Firstname Lastname "
	# or "" or "email@address"
	my ($name, undef, $email) = ($address =~ /^(.*?)\s*(<|)([^<>]*?)(>|)$/);
	quit("cannot parse $address") unless $email;
	next if exists $aliases{$email};
	# else, add it to the aliases.
	# See if everything is populated. We'll have to invent a nickname here
	if(! $name) {
		($name = $email) =~ s/\@.*//;
	}
	my $nick;
	($nick = lc $name) =~ tr/ //d;
	$collected{$email} = {
		"nick"	=> $nick,
		"name"	=> $name,
	};
}

if(scalar(keys %collected) != 0) {
	# save to tmp file
	open ALIASES, ">$aliasfile.new.$$" or quit("cannot open $aliasfile.new.$$ : $!");
	foreach my $email (sort keys %aliases) {
		my $nick = $aliases{$email}->{nick};
		my $name = $aliases{$email}->{name};
		print ALIASES "alias	$nick	$name	<$email>\n";
	}
	print ALIASES "\n\n#collected email addresses\n\n";
	foreach my $email (sort keys %collected) {
		my $nick = $collected{$email}->{nick};
		my $name = $collected{$email}->{name};
		print ALIASES "alias	$nick	$name	<$email>\n";
	}
	close ALIASES;

	# move tmp file to $aliasfile
	rename("$aliasfile.new.$$", $aliasfile);
}

# send mail onward
open(MSMTP, "|$msmtp @ARGV") or quit("cannot open pipe to $msmtp @ARGV : $!");
print MSMTP $email;
close(MSMTP) or quit("cannot close pipe to MSMTP : $!");

exit(0);


sub quit
{
	my $msg = shift;
	open LOGFILE, ">$home/.msmtp_helper.log" || die "cannot open log file : $!";
	print LOGFILE "$msg\n";
	close LOGFILE;
	exit(127);
}

Edit : fixed formatting.

I used GnuCash for a while and maybe I'll start using it again. A neat thing that one can do with it is to track one's portfolio. Stocks are easy (GnuCash can use Finance::Quote::Yahoo where NSE stocks have symbols ending in '.NS', and BSE stocks are numbers). Mutual funds in India do not have ticker symbols, but prices/NAV etc. can be obtained via the Finance::Quote::IndiaMutual module, which fetches them from the AMFI website.

To use the module, one can grab the code corresponding to the fund from the table at http://amfiindia.com/downloadnavopen.asp and supply that to the module.

All that is fine, but there's a tiny regex bug in the module which chokes on many rows. The fix is trivially simple and the diff totals to

74c74
<       my ($symbol, @data) = split /\;/;
---
>       my ($symbol, @data) = split /\s*\;\s*/;

I hope that helps someone.

I recently switched from using Thunderbird to mutt as my mail client. While making that jump, I also made the switch from mbox to maildir as my email storage format (Thunderbird only supports mbox, mutt supports both), with the help of mb2md.

Here's my .muttrc :

# -*-muttrc-*-
#



set realname="Devendra Gera"
set from=gera@example.com	# my real email goes here



set mark_old=no			# unread messages remain as "new"
set mbox_type=Maildir		# Mbox kinda sucks
set folder="/home/gera/Mail"	# the mail system resides here
set mask="!^\\.[^.]"		# I don't want to see the current dir alias '.' I do want to see '..'
set mbox="/home/gera/Mail/"	# my mailbox
set record="+.Sent"		# a copy of sent messages is kept here
set postponed="+.Drafts"	
set spoolfile="=.Inbox"		# mutt starts in this folder
set tmpdir="/home/gera/.mutt-tmp"	# my $HOME is encrypted. I want mutt to remain in that



# set up mailboxes by scanning all subfolders in mail dir
mailboxes `echo -n "+ "; find /home/gera/Mail -type d -name ".*" -printf "+'%f' "`



# some useful macros
macro index c "?" "open a different folder"
macro pager c "?" "open a different folder"
macro index C "?" "copy a message to a mailbox"
macro index M "?" "move a message to a mailbox"
macro compose A "?" "attach message(s) to this message"



set delete=yes			# delete at exit
set strict_threads		# only a subject match not sufficient for including in a thread
set index_format="%2C %Z %{%b %d} %-15.15n  %-.58s %> %c"	# status line for index views



# How to process file completion using ^T
set query_command="lbdbq %s"	# I rerely if ever use this. Instead, my alias file is auto updated



# editing outgoing mail
set editor="vim +':set spell' +/^Subject:  ++0"	# use vim with spellcheck and position at the subject line
set autoedit					# jump to the editor directly
set edit_headers				# allows me to change/see headers
set reply_to					# uses reply-to: header
set copy=yes					# preserve a copy of outgoing mail
set attribution='On %{%a, %d %b %Y}, %n wrote:\n'	# attribution for replies
set forward_format="Fw: [%s]"			# for forwards
unset reply_self				# do not include myself in group replies



# headers - choose the ones I like to see
ignore *
unignore from date subject to cc reply-to X-Mailer User-Agent
unignore x-mailing-list:
hdr_order From: Reply-To: To: CC: Date: Subject:



# outgoing mail
set sendmail_wait=0				# wait indefinitely for 'sendmail'
set sendmail="/home/gera/bin/msmtp_helper"	# I use msmtp, but this wrapper auto edits my .aliases
set realname="Devendra Gera"
set use_from=yes				# this is used to select the smtp server/account
set envelope_from=yes



# the real domains and CompanyName are masked in the following
macro generic "1" ":set from=gera@example.com"
macro generic "2" ":set from=gera@example2.com"
macro generic "3" ":set from=gera@example3.net"
folder-hook =.Inbox	set from=gera@example.com
folder-hook =.Inbox	set pgp_sign_as="0x0F4D93D7"
folder-hook =.SiteEmail	set from=gera@example3.net
folder-hook =.SiteEmail	set pgp_sign_as="0xF258D82F"
folder-hook =.CompanyName*	set from=gera@example2.com
folder-hook =.CompanyName*	set pgp_sign_as="0x85377D1B"
folder-hook =.work*	set from=gera@example2.com
folder-hook =.work*	set pgp_sign_as="0x85377D1B"



# misc
set check_new			# check new msgs
set fast_reply			# yeah baby!
set tilde			# I like the vim-ish view
set include			# include msg in replies
set noconfirmappend		# I'm not sure if it does anything for maildir
set menu_scroll			# scroll menus instead of paginate 
set mime_forward=ask-no		# inline forwarding by default, unless I say no
set move=no			# I want to move messages myself
set beep_new=no
set help=yes
set print_command="muttprint"
set ispell="aspell -e -c"	# aspell knows how to check emails
set pager_context=1		# one line of context works well for me
set pager_index_lines=9		# view a "folder pane" when reading messages
set sort=threads		# threaded sort
auto_view text/html		# see mailcap for html mails. html mails are dumped by 
				# lynx into the tmp dir which I specify to be in my $HOME
alternative_order text/plain text text/enriched		# try to display plain text if possible
set alias_file=~/.aliases	# this is auto controlled by my msmtp helper
set sort_alias=alias		# sort aliases in alias order
set reverse_alias=yes		# display aliases on incoming email
source $alias_file
set auto_tag=yes		# any operations are apllied to tagged msgs, if any.



# status line formats
set status_format="-%r-Mutt: %f [Msgs:%?M?%M/?%m%?n? New:%n?%?d? Del:%d?%?F? Flag:%F?%?t? Tag:%t?%?p? Post:%p?%?b? Inc:%b?  %l]---(%s)-%>-(%P)---"
set pager_format="%-10.10i %[!%a %b %d %R]"
set date_format="!%H:%M %a %d %b     "
set index_format="%4C %Z %X %[%b%d] %-15.15F %M %s"
set folder_format="%2C %t %8s %d %N %f"



set crypt_autopgp=yes				# enable pgp/signing
set crypt_autosign=yes				# always sign
set crypt_replyencrypt=yes			# reply to sectrets in secrets
set crypt_replysignencrypted=yes		# always sign
set pgp_show_unusable=no			# don't show unusable keys



#### The following comes straight out of the default.



# Note that we explicitly set the comment armor header since GnuPG, when used
# in some localiaztion environments, generates 8bit data in that header, thereby
# breaking PGP/MIME.



# decode application/pgp
set pgp_decode_command="/usr/bin/gpg --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet --batch --output - %f"



# verify a pgp/mime signature
set pgp_verify_command="/usr/bin/gpg   --status-fd=2 --no-verbose --quiet  --batch  --output - --verify %s %f"



# decrypt a pgp/mime attachment
set pgp_decrypt_command="/usr/bin/gpg   --status-fd=2 %?p?--passphrase-fd 0? --no-verbose --quiet  --batch  --output - %f"



# create a pgp/mime signed attachment
# set pgp_sign_command="/usr/bin/gpg-2comp --comment '' --no-verbose --batch  --output - --passphrase-fd 0 --armor --detach-sign --textmode %?a?-u %a? %f"
set pgp_sign_command="/usr/bin/gpg  --use-agent  --no-verbose --batch --quiet   --output - %?p?--passphrase-fd 0? --armor --detach-sign --textmode %?a?-u %a? %f"



# create a application/pgp signed (old-style) message
# set pgp_clearsign_command="/usr/bin/gpg-2comp --comment ''  --no-verbose --batch  --output - --passphrase-fd 0 --armor --textmode --clearsign %?a?-u %a? %f"
set pgp_clearsign_command="/usr/bin/gpg --use-agent --no-verbose --batch --quiet --output - %?p?--passphrase-fd 0? --armor --textmode --clearsign %?a?-u %a? %f"



# create a pgp/mime encrypted attachment
# set pgp_encrypt_only_command="pgpewrap gpg-2comp  -v --batch  --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"
set pgp_encrypt_only_command="pgpewrap /usr/bin/gpg --batch --quiet --no-verbose --output - --encrypt --textmode --armor --always-trust -- -r %r -- %f"



# create a pgp/mime encrypted and signed attachment
# set pgp_encrypt_sign_command="pgpewrap gpg-2comp  --passphrase-fd 0 -v --batch  --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"
set pgp_encrypt_sign_command="pgpewrap /usr/bin/gpg %?p?--passphrase-fd 0? --batch --quiet --no-verbose --textmode --output - --encrypt --sign %?a?-u %a? --armor --always-trust -- -r %r -- %f"



# import a key into the public key ring
set pgp_import_command="/usr/bin/gpg  --no-verbose --import %f"



# export a key from the public key ring
set pgp_export_command="/usr/bin/gpg   --no-verbose --export --armor %r"



# verify a key
set pgp_verify_key_command="/usr/bin/gpg   --verbose --batch  --fingerprint --check-sigs %r"



# read in the public key ring
set pgp_list_pubring_command="/usr/bin/gpg   --no-verbose --batch --quiet   --with-colons --list-keys %r" 



# read in the secret key ring
set pgp_list_secring_command="/usr/bin/gpg   --no-verbose --batch --quiet   --with-colons --list-secret-keys %r" 



# fetch keys
# set pgp_getkeys_command="pkspxycwrap %r"
# This will work when #172960 will be fixed upstream
# set pgp_getkeys_command="/usr/bin/gpg --recv-keys %r"



# pattern for good signature - may need to be adapted to locale!



# set pgp_good_sign="^gpg: Good signature from"



# OK, here's a version which uses gnupg's message catalog:
# set pgp_good_sign="`gettext -d gnupg -s 'Good signature from "' | tr -d '"'`"



# This version uses --status-fd messages
set pgp_good_sign="^\\[GNUPG:\\] GOODSIG"

I use fetchmail for getting my email, and msmtp to send it out.

I wrote a simple wrapper around msmtp, which adds addresses from the To: and Cc: fields to my .aliases, if they're already not there. I'll post that subsequently.

The Mumbai Airport has a really stupid security policy.

• If you're at the airport to see someone off, you have to do it from the road/pavement.
• One can enter the airport lounges only on producing a ticket for flying that day.
• The ticket holder cannot come out of the lounge after collecting the boarding pass.
• There is nothing which announces this. You get to know it only when the ticketholder is inside and you're not.

Why this is stupid :

• There is no security involved in the check-in process, or at entry into the lounge. A person does not cross a trust boundary in any of these procedures. A checked-in passenger is as untrusted as a non checked-in person. Yet, non checked-in persons can come out and checked-in ones cannot.
• There's nothing special about boarding passes. I hope their reasons are better than "someone with a fake boarding pass can enter the lounge". It's easier to print a fake ticket. E-tickets are just printouts and you can make your own. Or you can just buy a valid ticket with that nice bomb. Maybe there's a free-ticket-with-a-bomb offer as well.
• It gives a false sense of security. When only "valid" passengers are allowed inside, securitymen lower their guards. So do the people. There's this sense that the are is secure and the bad elements are out, which might lead to suspicious behaviour being overlooked or go unreported. The reality of course is that this setup doesn't *make* anything secure. Its worse than having no security at all.
• Of course, the airport loses a small stream of revenue. Visitors have, and will pay money to be longer with the passengers.

The CSIA website is another piece of turd. Weird javascript and generally borked.