<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<?xml-stylesheet type="text/css" href="http://www.theoldmonk.net/blog/styles/feed.css"?>
<title type="html">technology</title>
<link rel="alternate" type="text/html" href="http://www.theoldmonk.net/blog" />
<link rel="self" type="application/atom+xml" href="http://www.theoldmonk.net/blog/archives/technology/index-atom.xml" />
<updated>2009-08-17T18:00:57+05:30</updated>
<author>
<name>gera</name>
<uri>http://www.theoldmonk.net/blog</uri>
</author>
<id>http://www.theoldmonk.net/blog/</id>
<generator uri="http://nanoblogger.sourceforge.net" version="3.3">NanoBlogger</generator>
<entry>
<title type="html">Building a mysql-server with SphinxSE patches</title>
<author>
<name>gera</name>
</author>
<link rel="alternate" type="text/html" href="http://www.theoldmonk.net/blog/archives/2009/08/05/building_a_mysql-server_with_sphinxse_patches/" />
<id>http://www.theoldmonk.net/blog/archives/2009/08/05/building_a_mysql-server_with_sphinxse_patches/</id>
<published>2009-08-05T09:14:04+05:30</published>
<updated>2009-08-05T09:14:04+05:30</updated>
<category term="technology" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
<![CDATA[Building the mysql-server Ubuntu package (mysql-dfsg-5.0_5.0.51a-3ubuntu5) on hardy has some issues, even without the <a href="http://www.sphinxsearch.com">Sphinx</a> patches.
<ul>
  <li> The SSL cert used in the tests is expired, failing a bunch of tests with a SSL error (<a href="https://bugs.launchpad.net/ubuntu/+source/mysql-dfsg-5.0/+bug/323755">LP: #323755</a>).</li>
  <li> <strike>Some MySQL status and error messages have changed, making some tests fail.</strike> (This is not true - see update for details).</li>
  <li> Upstream has a debian directory which isn't integrated and causes build failures. (There's a dpatch for this already, but it doesn't go all the way).</li>
  <li> Sphinx provides a SphinxSE patch for MySQL 0.37</li>
  <li> The debian/rules script skips out on the pre-build automake invocation, and the sphinx patches never make it to the build, or the compilation fails midway.</li>
</ul>

<br />
I managed to get these issues sorted out via a series of dpatches. You can download the <a href="http://www.theoldmonk.net/mysql-dfsg-5.0_5.0.51a-3ubuntu5-sphinx1.debdiff">debdiff</a> and build the package yourself:
<pre>
$ apt-get source mysql-server
 .... [will download the source and apply the diffgz etc] ....

$ cd mysql-dfsg-5.0-5.0.1a
$ patch -p1 < /path/to/mysql-dfsg-5.0_5.0.51a-3ubuntu5-sphinx1.debdiff
.... [messages about files patched] ....

$ dpkg-buildpackage -rfakeroot -us -uc
.... [builds and unsigned package. Takes *really* long] ....
</pre>
<em>update</em>: It turns out that the MySQL error messages didn't change after all. I should have noticed earlier, but the difference in error messages is because the build machine was using OpenDNS, which wasn't returning a NXDOMAIN for invalid/nonexistent hosts. The debdiff is now fixed and the unnecessary dpatch has been removed. But the thing to learn from this is that if your build machine is using OpenDNS in its default configuration (it's possible to fix OpenDNS' behaviour) - the MySQL test suite *WILL FAIL*.]]>
</div>
</content>
</entry>
<entry>
<title type="html">CouchDB document update notifications</title>
<author>
<name>gera</name>
</author>
<link rel="alternate" type="text/html" href="http://www.theoldmonk.net/blog/archives/2009/04/10/couchdb_document_update_notifications/" />
<id>http://www.theoldmonk.net/blog/archives/2009/04/10/couchdb_document_update_notifications/</id>
<published>2009-04-10T11:02:00+05:30</published>
<updated>2009-04-10T11:02:00+05:30</updated>
<category term="technology" />
<category term="code" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
<![CDATA[<a href="http://couchdb.apache.org/">CouchDB</a> is a very cool document store which uses JavaScript map/reduce views. It is schema free and the docs are JSON. It also has an HTTP API which makes it possible to write purely JavaScript apps with persistence via CouchDB.
<br /><br />
Couch has a concept of DB update notifications. A process is launched (and relaunched if it crashes, via an Erlang/OTP supervisor) and is notified of any updates (adds, deletes and changes) to the DBs. That's nice, but not sufficiently useful in all cases.
<br /><br />
So I added a "document update notification" mechanism on the lines of db update notifications, which not only tells which document changed, but also what changed in the document. This makes it possible to write various kinds of "triggers" (or even "stored procedures" if you squint just right). The document update notification process gets the type of change (add/delete/update), the db name, the doc id and the bodies of the old and the new doc on each update. It can then use these to determine if something interesting changed and decide whether to take action or not.
<br /><br />
The patch was offered upstream (<a href="http://mail-archives.apache.org/mod_mbox/couchdb-dev/200904.mbox/%3C20090408110546.GA26996@gera-laptop%3E">annoucement</a>) but wouldn't land up in the trunk because a feature superseding this functionality is already planned (comet notifications). However, I will maintain the patches out-of-tree till the superseding functionality lands in the trunk. The patches are fairly simple and non intrusive, and are available for both 0.8.1 and 0.9.
<br /><br />
<a href="http://www.theoldmonk.net/couchdb_docnotifications.0.9.patch">DocUpdateNotifications patch for CouchDB 0.9</a>
<br /><br />
<a href="http://www.theoldmonk.net/couchdb_docnotifications.0.8.1.patch">DocUpdateNotifications patch for CouchDB 0.8.1</a>
<br /><br />
These patches were developed as part of my work for <a href="http://www.basecase.com">BaseCase</a>. BaseCase thinks open source is cool and has gladly agreed to the release of these patches.]]>
</div>
</content>
</entry>
<entry>
<title type="html">ssh authentication via mysql (pam_mysql)</title>
<author>
<name>gera</name>
</author>
<link rel="alternate" type="text/html" href="http://www.theoldmonk.net/blog/archives/2009/02/24/ssh_authentication_via_mysql_pam_mysql/" />
<id>http://www.theoldmonk.net/blog/archives/2009/02/24/ssh_authentication_via_mysql_pam_mysql/</id>
<published>2009-02-24T13:39:30+05:30</published>
<updated>2009-02-24T13:39:30+05:30</updated>
<category term="tricks" />
<category term="technology" />
<category term="hacks" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
<![CDATA[<p>
There's not much info on the interwebs on why ssh authentication via pam_mysql fails, and there are definitely some misleading answers like
<pre>
when logging in via ssh, the ssh daemon checks some files in ~/,
so it can be done if the user already have a valid home directory.
</pre>
from <a href="https://www.linuxquestions.org/questions/linux-security-4/authentication-to-ssh-via-mysql-169733/">this thread</a>.
</p>

<p>
The real reason why it doesn't successfully authenticate is because ssh does a getpwent() call to check if the user exists on the system. If you're using just a PAM solution (which doesn't provide accounts), that call fails if there isn't any such local user. sshd then sets the password to '^H ^M INCORRECT' before passing it to PAM, which obviously thinks it's the wrong password. The simple (and probably unscalable - but then you should be using something like nss_mysql) solution is to add local accounts to the machines for these users. That will make the getpwent() call succeed, and sshd will authenticate successfully via PAM.
</p>

<p>
I've mentioned this <a href="http://www.theoldmonk.net/blog/archives/2008/09/27/trac_mysql_and_authentication/">earlier</a>.
</p>]]>
</div>
</content>
</entry>
<entry>
<title type="html">Trac, MySQL and authentication</title>
<author>
<name>gera</name>
</author>
<link rel="alternate" type="text/html" href="http://www.theoldmonk.net/blog/archives/2008/09/27/trac_mysql_and_authentication/" />
<id>http://www.theoldmonk.net/blog/archives/2008/09/27/trac_mysql_and_authentication/</id>
<published>2008-09-27T19:06:10+05:30</published>
<updated>2008-09-27T19:06:10+05:30</updated>
<category term="tricks" />
<category term="technology" />
<category term="hacks" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
<![CDATA[What we needed at <a href="http://www.chaupaati.in">work</a> :
<ul>
 <li> A wiki</li>
 <li> VPN</li>
 <li> A bug/issue tracker</li>
</ul>
<a href="http://trac.edgewall.org/">Trac</a> gave us the first and the last, and <a href="http://openvpn.net/">OpenVPN</a> gave us the middle. The newer versions of Trac can use MySQL as a database instead of SQLite and have a nice <a href="http://trac-hacks.org/wiki/AccountManagerPlugin">AccountManager</a> plugin.
<br /><br />
And it all comes together with <a href="http://pam-mysql.sourceforge.net/">pam_mysql</a>. The only problem was that Trac likes to store passwords hashed as HTTP-Digest style hashes (not the most secure, I know) with an empty realm, and pam_mysql doesn't work for that. The solution was a simple patch to pam_mysql which adds support for such hashes. I'll publish that soon - I'm making it more general by adding support for realms rather than assuming that realm would be empty.
<br /><br />
OpenVPN (and apache - via mod_auth_pam) etc. can authenticate via <a href="http://www.kernel.org/pub/linux/libs/pam/">PAM</a> and adding authentication to anything is a simple matter of placing the right .htaccess file. And this is a *common* password across all services, which users can change on the wiki, in their browsers, with a nice polished interface. No unix shells for people who can't deal with them.
<br /><br />
This is no <a href="http://en.wikipedia.org/wiki/Kerberos_(protocol)">Kerberos</a>, but is infinitely simpler - which means a lot for a small startup.
<br /><br />
There are a few gotchas though :
<ul>
<li> Apache doesn't like empty realms. This can be worked around by switching to Basic authentication where passwords are passed on to PAM and we ignore realm. Yes, Basic is worse, but there's always SSL. Thus, one can supply _any_ value of realm (the AuthName in .htaccess). The situation might improve if my pam_sql patch accepts realms and we manage to modify the Trac AuthManager plugin to use the same realm.</li>
<li> SSH doesn't like to authenticate users which don't have an account on the machine. It does a getpwent() call, and if it fails, it sets the password to '^H ^M INCORRECT' (the ^H and ^M represent the control chars here, but this is what you see in your logs if you debug PAM). One (ugly, in my opinion) way out is to LD_PRELOAD your own getpwent() method for sshd. The other is to grant ssh access to people on a machine by machine basis. This doesn't scale beyond a point but is good enough for small setups and the most secure. We use this. The cleaner solution of course, is to use something like <a href="http://savannah.nongnu.org/projects/nss-mysql">nss_mysql</a>. This might be our next step, but not in the immediate future.</li>
</ul>
Patch to come Real Soon Now.]]>
</div>
</content>
</entry>
<entry>
<title type="html">Network profiles in Ubuntu</title>
<author>
<name>gera</name>
</author>
<link rel="alternate" type="text/html" href="http://www.theoldmonk.net/blog/archives/2008/03/04/network_profiles_in_ubuntu/" />
<id>http://www.theoldmonk.net/blog/archives/2008/03/04/network_profiles_in_ubuntu/</id>
<published>2008-03-04T22:03:31+05:30</published>
<updated>2008-03-04T22:03:31+05:30</updated>
<category term="tricks" />
<category term="technology" />
<category term="perl" />
<category term="hacks" />
<content type="xhtml">
<div xmlns="http://www.w3.org/1999/xhtml">
<![CDATA[<p>There are various ways of managing multiple network profiles in Ubuntu, but I've never been a fan of NetworkManager. Commandlines work for me very well, and even there - multiple solutions exist with the help of packages like resolvconf etc. Here's my setup which is very Debian-ish and depends on this nice package called <a href="http://packages.ubuntu.com/gutsy/ifupdown">ifupdown</a>.</p>

<p>First, there's the /etc/network/interfaces file :</p>
<pre>
# we always want the loopback
auto lo
iface lo inet loopback

# mappings
mapping eth0
  script /etc/network/map-scheme
  map dhcp eth0-dhcp
  map emergency eth0-emergency

mapping ath0
  script /etc/network/map-scheme
  map office ath0-office
  map home ath0-home

iface eth0-dhcp inet dhcp
  up iptables -F
  up lokkit -n -q --high --dhcp
  up /etc/init.d/lokkit restart

iface ath0-office inet dhcp
  wpa-driver madwifi
  wpa-conf /etc/wpa_supplicant/office.conf
  up iptables -F
  up lokkit -n -q --high --dhcp
  up /etc/init.d/lokkit restart

iface eth0-emergency inet static
  address 10.9.5.201
  gateway 10.9.4.1
  netmask 255.255.254.0
  up iptables -F
  up lokkit -q --high
  up echo nameserver 172.31.6.5 > /etc/resolv.conf
  up echo nameserver 203.197.12.30 >> /etc/resolv.conf

iface ath0-home inet dhcp
  wpa-driver madwifi
  wpa-conf /etc/wpa_supplicant/home.conf
  up iptables -F
  up lokkit -n -q --high --dhcp
  up /etc/init.d/lokkit restart
</pre>

<p>Notice the mappings section (and see 'man interfaces') - that allows me to say :</p>
<pre>
NETSCHEME="home" sudo ifup ath0
</pre>

<p>or</p>
<pre>
NETSCHEME="office" sudo ifup ath0
</pre>

<p>because the specified script (/etc/network/map-scheme) just looks up the NETSCHEME environment variable and spit out the correct mapping to go to. This thing, by the way, could be rigged to do arbitrarily complex tasks (look in /usr/share/doc/ifupdown/examples/ for sample scripts, including one which tries to ping some known IPs, and decides its location/profile based on successful pings - you could write one which looks for all known wireless SSIDs and then decide which profile to switch to). Here's my trivial script :</p>
<pre>
#!/usr/bin/perl -w
use strict;

my $scheme = $ENV{NETSCHEME} || "home";

while(<>) {
        if ( s/$scheme\s+// ) {
                print;
        }
}
</pre>

<p>The conf files in /etc/wpa_supplicant/* are of course wpa_supplicant configuration files. See 'man wpa_supplicant.conf' for details.</p>]]>
</div>
</content>
</entry>
</feed>
