The Old Monk

More Git-Bugzilla fun - GitZilla 1.9

gera — 2010-03-14T03:58:15+05:30

GitZilla version 1.9 is out. This is a RC for version 2.0, which comes after this gets some wider testing. If you're using it and have a problem, use the issues page at github.

New features and changes:

User specific auth can now be mandatory, optional or denied altogether
User specific auth can now work with a cookie file (no need to store passwords - a problem when you have single-sign-on or common passwords)
Utility scripts to generate cookie files
More auth options for custom hooks. You can now pass a callback for pybugz initialization and go wild with it.
The update hook now checks for valid Bugzilla auth even if bug status checking is not enabled.

GitZilla - Git-Bugzilla integration done right

gera — 2010-03-08T04:34:49+05:30

My old git-bugzilla integration scripts were more of a one-off attempt at some glue code and was very hackish. No wonder it accumulated bitrot.

So now, there's GitZilla! It's hosted on GitHub and there's a .deb available. Note that (for now at least), you would have to install the pybugz module separately, even if you install the GitZilla .deb.

Using GitZilla is extremely easy. You can get started by symlinking (or copying) a couple of files and creating a 4 line configuration file. (See the 'simple ready scripts' usage mode).

Debugging python Windows services

gera — 2010-02-11T13:08:49+05:30

Python makes it easy to write Windows services (via pywin32 extensions). That part is easy. The problems sometimes happen when you try to run them, with a commonly reported error - "The service did not respond to the control...", which is error 1503.

If one runs the service compiled as an exe (via py2exe for example), there are a bunch of issues like the module import paths changing. I haven't had the time to dig really deep into that. But this might strike even if the service is being run uncompiled (from a .py file).

One of the first things to check is whether pythonservice.exe (from the win32 module) executes cleanly - this is the binary responsible for actually loading your service script/module. If pywin32 was installed in the usual way, it should be located at C:\Python26\Lib\site-packages\win32\pythonservice.exe

Running this should give an immediate idea of what's going wrong. It usually turns out to be a "msvcr90.dll" or a "python26.dll" error. (Of course, all this is Python 2.6). When that is running alright, then it's time for some debug fun.

To debug the python service, just run pythonservice.exe with the '-debug' option and the service name as the argument. This attaches the stdout/stderr to the terminal. For example, if the service uses BaseHTTPServer, all the request/response logs will show up.

Windows XP file ownership

gera — 2009-08-17T17:01:16+05:30

As part of a cleanup of someone's notebook, I created a non-admin account and copied the docs/shortcuts. Thankfully, the data was already in a separate drive. Trouble is, the non-admin user would see all of it as read-only.

I tried changing the perms on the content (via the gui as well as via attrib.exe), but the non-admin user still wasn't able to write.

It turned out that the issue was file ownership. I'm not too sure what the patchlevel of the XP system was, but a right click -> properties menu didn't bring up any tab where I could see or change file ownership.

The solution: download and install Cygwin, and then a simple "chown -R /cygdrive//*".

Reinforces my opinion that one really *needs* cygwin to make Windows usable!

Building a mysql-server with SphinxSE patches

gera — 2009-08-05T09:14:04+05:30

Building the mysql-server Ubuntu package (mysql-dfsg-5.0_5.0.51a-3ubuntu5) on hardy has some issues, even without the Sphinx patches.

The SSL cert used in the tests is expired, failing a bunch of tests with a SSL error (LP: #323755).
~~Some MySQL status and error messages have changed, making some tests fail.~~ (This is not true - see update for details).
Upstream has a debian directory which isn't integrated and causes build failures. (There's a dpatch for this already, but it doesn't go all the way).
Sphinx provides a SphinxSE patch for MySQL 0.37
The debian/rules script skips out on the pre-build automake invocation, and the sphinx patches never make it to the build, or the compilation fails midway.

I managed to get these issues sorted out via a series of dpatches. You can download the debdiff and build the package yourself:

$ apt-get source mysql-server
 .... [will download the source and apply the diffgz etc] ....

$ cd mysql-dfsg-5.0-5.0.1a
$ patch -p1 < /path/to/mysql-dfsg-5.0_5.0.51a-3ubuntu5-sphinx1.debdiff
.... [messages about files patched] ....

$ dpkg-buildpackage -rfakeroot -us -uc
.... [builds and unsigned package. Takes *really* long] ....

update: It turns out that the MySQL error messages didn't change after all. I should have noticed earlier, but the difference in error messages is because the build machine was using OpenDNS, which wasn't returning a NXDOMAIN for invalid/nonexistent hosts. The debdiff is now fixed and the unnecessary dpatch has been removed. But the thing to learn from this is that if your build machine is using OpenDNS in its default configuration (it's possible to fix OpenDNS' behaviour) - the MySQL test suite *WILL FAIL*.

spg - the code

gera — 2009-04-27T12:15:37+05:30

I had written about spg and wrote an initial implementation but never got around to publishing it. Then, I rewrote it as well adding useful stuff like bash completion, but assumed that I had already published it. Turns out that I hadn't, so I'm doing that now.

The "homepage" for spg is here and the git repository (via github) is here. It works on Windows as well, but there are a few caveats - there's no commandline completion and you'd have to change the interpreter string to "#!python' and meddle around with your path. Maybe one day I'll get around to packaging it properly but packaging for Windows looks like more trouble than is worth.

Maybe someone can add a GUI for it for the Windows crowd. Or maybe turn it into a FireFox extension?

CouchDB document update notifications

gera — 2009-04-10T11:02:00+05:30

CouchDB is a very cool document store which uses JavaScript map/reduce views. It is schema free and the docs are JSON. It also has an HTTP API which makes it possible to write purely JavaScript apps with persistence via CouchDB.

Couch has a concept of DB update notifications. A process is launched (and relaunched if it crashes, via an Erlang/OTP supervisor) and is notified of any updates (adds, deletes and changes) to the DBs. That's nice, but not sufficiently useful in all cases.

So I added a "document update notification" mechanism on the lines of db update notifications, which not only tells which document changed, but also what changed in the document. This makes it possible to write various kinds of "triggers" (or even "stored procedures" if you squint just right). The document update notification process gets the type of change (add/delete/update), the db name, the doc id and the bodies of the old and the new doc on each update. It can then use these to determine if something interesting changed and decide whether to take action or not.

The patch was offered upstream (annoucement) but wouldn't land up in the trunk because a feature superseding this functionality is already planned (comet notifications). However, I will maintain the patches out-of-tree till the superseding functionality lands in the trunk. The patches are fairly simple and non intrusive, and are available for both 0.8.1 and 0.9.

DocUpdateNotifications patch for CouchDB 0.9

DocUpdateNotifications patch for CouchDB 0.8.1

These patches were developed as part of my work for BaseCase. BaseCase thinks open source is cool and has gladly agreed to the release of these patches.

Choosing a camera

gera — 2009-04-05T15:47:50+05:30

I often get asked for advice on cameras. The number of choices and differing opinions can confuse anyone, so it is natural to ask someone who has owned a few of them. Deciding which camera to buy isn't an easy decision, but there are a few general things which one should consider. This, of course, is my opinion - feel free to ignore it :)

SLR or a point and shoot? It depends (you'll hear that a lot). A point and shoot is compact and you will end up shooting more pictures (and have more fun) with it. SLR cameras are bulky and you won't be able to carry them everywhere. You wouldn't even *want* to carry them everywhere once the excitement of a new camera is over. Plus, buying an SLR usually becomes a journey into chasing lenses. Sure, SLRs will give you awesome pictures (and that's because of the lenses), but you will miss a lot of pictures because the SLR won't be in your pocket. The very fact that you're asking this question means that you should buy a good P&S. Learn how to take good photos. And if you're still clicking as enthusiastically after a year, but an SLR. Oh, and point and shoots are *good* these days.

OK, which point and shoot? There are some really good P&S cameras there. If you aren't really serious about photography, just buy anything. Buy the camera you like the most which fits into your budget. Buy the slimmest, pinkest, loudest, most megapixels - whatever. Oh, and while you're at it - buy Sony because you're stupid anyway. But seriously, you can buy anything from Canon or Nikon without much worry. Both of them make awesome cameras and have a decent service network. Try out a few cameras and see what you like. Look at the picture modes, see if it has (and you like) features like "color accent" etc. Some things that you might want to look at:

the "zoom range" - but don't fall for the digital zoom trap. Optical zoom is all that matters. And don't fall for the "x" numbers. Look at how wide and how tele you can go. See if you comfortably take in the whole room without having to step back.
you might want to prefer a camera which has a lens that goes wider ("zooms out") over one with a more tele lens ("zooms in"). In all probability, you'll shoot group photos, indoor pictures, pictures of buildings/towers/trees etc, from a close enough distance. A wide lens is *really* useful then. And the more you zoom in, the more stable the camera (in your hands) needs to be. You could carry a tripod, but then why not buy an SLR?
Optical viewfinder - a little inconvenient in a small camera, but saves battery. Not too important these days I guess.
Battery - if you run out of it, can you use standard AAs to get some more shots? And how long does the battery last?
Charging - if the battery charges outside the camera, you can shoot while you charge a spare battery. With charging integrated into the camera, you can't do that.
Storage - does that camera have a SD card or a CF card? MicroSD or standard SD? Which ones do you already own? Do you need a card reader, or does you laptop already have an SD Card slot? Sony is a particularly bad choice because they use MemoryStick which is proprietary, not useful with non-Sony stuff, and the evil corporation will force another format if you upgrade your camera leaving your current storage cards useless.

But remember that friends don't let friends buy Sony.

Which SLR? Nikon or Canon? Sony - if you're stupid. Redundancies aside, you are a serious photographer if you're buying an SLR, and that means in the near future (if not now with the crappy kit lens), it's the lenses that matter. The camera body is just electronics which you may upgrade. The cost of your lens collection will be significantly higher than the cost of the body. And people buy multiple bodies so that they don't have to change lenses.

What that means, is that the choice of the body depends on the choice of the lenses. Lenses are expensive, and you might want to borrow (or swap for a little while) from your friends. If your friends are in the Canon camp, pick Canon. If they're in the Nikon camp, pick Nikon. The difference in the cameras is easily overshadowed by the availability of a lens pool. But you should check if the lens pool *is* available. People are possessive about their lenses (rightly so) and might not want to hand them over to a rookie. A nice way is to give, and then take. Your excellent lens would give them confidence in your ability to take care of lenses. It also helps to discuss things like lens cleaners (materials and services).

If you don't have friends, join Facebook and then come back to reading this after a few months (or minutes, if you take a lot of quizzes). If you don't have friends with SLRs, just jump to any camp. I'm a little biased towards Canon. See, Nikon is a very good company and they make good cameras. But Nikon is a small company with really smart optical engineers. Canon, on the other hand is a really big company with a lot of money. And they make their own sensors. Which means - the Canon cameras will usually have more features, small builds, larger sensors etc. I think the lenses tend to be better per dollar, but a lot of people will debate that. I'm a Canon guy. You also get to joke about "flashing your Canon", and your "big Canon" etc., which is an added bonus :)

One point that many people overlook is the choice between a full frame and an APS camera. This refers to the sensor size. The full frames are more expensive than the APS (or "digital size") sensor cameras. For the same number of pixels, they have a lower sensor density and have lower noise. Which means better low light performance / higher ISOs. But they're more expensive. You can buy any one - that isn't the point. As I said, camera bodies are transient. It's about the lenses.

Since a smaller sensor-size camera has a smaller sensor (no, really!), the lens needs to produce a smaller image. Now, lenses are complex things, and the quality varies. The general rule is that the image quality deteriorates as you go out from the center of the image. Which means that a full frame is more demanding in terms of lenses. Of course, you could always crop out the bad parts to achieve the _exact_same_ results as from a small sensor camera. You're just cropping out after the capture - the small sensor effectively crops out at the time of capture.

The other thing is that there are some lenses "designed for digital cameras" (the "EF-S" lenses in the Canon world). What that means is that they produce an image sufficient only for the smaller sensor size. They won't work on the full frames. Don't be tempted to buy them unless you get a *really* sweet deal. They're cheaper - yes, but they won't work when you upgrade to a full frame. You'll have to buy your lenses all over again. Remember, cameras are transient - lenses stay.

Last note: for similar prices, prefer faster lenses (wider apertures) over "Image Stabilization" or "Vibration Resist" (IS and VR are from the Canon and Nikon worlds respectively). IS/VR helps in low light and gives you a 1-stop advantage, but only when *your hand* is moving. They're useless when the subject is moving. They also make the lens more complex. Complex things break easily and in complex ways. A faster lens is more generally versatile.

Full disclosure: I used to own a Canon 350D with a 50/1.8, a Tamron 28-75/2.8 (excellent lens!), and a Canon 10-22/3.5-5.6 which I sold off a couple of years ago. I now own a Canon 5D (that's a full frame) with a 50/1.8 and a Canon SD750 Point&Shoot.

Engineering culture

gera — 2009-04-05T13:47:42+05:30

quote:

... the phase every programmer goes through during which
he overestimates his abilities, has no sense of scale, and
lacks sense for robustness. In short, he's proud,
ignorant, and dangerous. He believes that libraries are
bloated and slow, that he can out-perform standard
implementations. He optimized prematurely, avoids function
calls, abuses the ternary operator, and doesn't use a
profiler.

Eventually, these programmers grow up, but in the
meantime, they've written a significant amount of horrible
code. I've seen this pattern again and again. As the
parent mentioned, software developers have no "engineering
culture." I imagine that in more established engineering
disciplines, students have the above attitude beaten out
of them before they graduate.

Very very true. And I'm guilty as charged, but I have reformed (or at least I like to believe so).

Recycling

gera — 2009-03-26T10:59:01+05:30

An old (but original) joke:

Q: What coffee does an opportunist drink?
A: Mochaccino

And a limerick:

There was this guy, a hun,
Who gave up all his fun;
He wasn't supposed to use tricks,
Nor to write any limericks,
Therefore this is not a limerick.

As opposed to 'Therefore this isn't one'.