A new decade, a new wrinkle in building open source

February 26, 2010

For about 15 years or so, I’ve been used to the same routine whenever building an open-source package.  You’ve all probably seen it many times, so I’ll put it on a single line: “./configure –prefix=…;make;make install”.

Today I downloaded a new package (using the awesome ‘bzr branch lp:package-name’ command), and was confronted by a slightly different set of files:

AUTHORS    Makefile.am  README        docs        m4         tests
COPYING    NEWS         config        examples    prototest
ChangeLog  PROTOCOL     configure.ac  libdrizzle  support

No ‘configure’ script, just a template file. I figured `autoconf’ would generate a script from the template. It did, but when I tried running `./configure’, I got this error message:

configure: error: cannot find install-sh or install.sh in config .

The solution is straightforward. If the config directory contains an `autorun.sh’ file, here’s the new dance:

config/autorun.sh
./configure 
make
make install

Anyone know when this technique was introduced, and whether this is the way of the new world order?

A new Tabhunter version for the new year

January 1, 2010

The Tabhunter news at the start of 2009 was a mixed review in Lifehacker. A few commenters said it was exactly what they were looking for, while about half said if you needed it something was seriously wrong with the way you surfed the web. Of course Tabhunter isn’t for people who surf the web — it’s for those of us who work on the web.  No wonder so many reviewers were confused — the Lifehacker screenshot shows a session with 6 loaded tabs.  You don’t need this extension for that.  Here’s a more realistic screenshot:

Sample tab search

Searching for Tabhunter

I finally found a few days to add a search-document feature.  The UI is still a bit rougher than the URL/title search that those 10% of us have come to depend on. Because I can’t leave well enough alone, you can search the HTML text with either plain text or JavaScript regular expressions (with an option to ignore case), or if you’re looking at a Roto-Rooter-sized job, you can search for XPath expressions in the document. My first use case was to find which of my loaded Bugzilla entries are for resolved issues, which means I can delete them.  This screenshot shows the search in action:

tabhunter text search

Find resolved bugzilla pages

The screenshot doesn’t show that I have 29 tabs I can close.  As much as I like opening web pages (so it would seem), I like closing them much more.

This version (0.9.*) isn’t available at the Mozilla add-on site yet, but you can pull down the latest XPI at http://code.google.com/p/tabhunter/downloads/list.

ReleaseEngineer.MACROVISION has a Cousin in the Bronx

October 22, 2009

Occasionally at ActiveState we get a bug report from a Windows user that Komodo thinks the current user’s name is “ReleaseEngineer.MACROVISION”, and is disappointed that it can’t find a directory like “c:/Users/ReleaseEngineer.MACROVISION/AppData/Roaming” on the user’s Vista box. The reporter insists that there’s no one of that name at his company, and we don’t have anyone with that name wandering around our company. If you google for “ReleaseEngineer.MACROVISION”, you’d get the idea that plenty of companies have an account for someone with that name, given the number of hits Google reports.

There was always a workaround — to set the KOMODO_USERDATADIR environment variable, so we didn’t push on this. But it does make for a disappointing initial experience. Because we couldn’t reproduce the bug here, we hadn’t escalated it.

Then yesterday someone not only ran into this bug (a similar one, which complained about a bogus username of “gooemp”), but he was running a registry tracker at the same time as Komodo, and saw that we were reading the registry key “HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders”. The reporter pointed out that there’s a subkey in that folder called “!Do not use this registry key”, with the value of “Use the SHGetFolderPath or SHGetKnownFolderPath function instead”. I’ve never seen a deprecation notice hidden in the registry, and obviously our software was ignoring that key as well every time it went in there, but our customer didn’t.

From there it was relatively straightforward to follow the advice. In fact, there was a comment in the registry-walking code stating that it was using the registry instead of the shell command, with a comment I found cryptic at first stating that “SHGetFolderPath” isn’t always available. Digging around a bit more, I found that SHGetFolderPath function was deprecated as of Vista, in favor of the newer SHGetKnownFolderPath. I bit the bullet, and spent a few hours writing code that made dynamic calls, using LoadLibrary, GetModuleHandle, and GetProcAddress, to make the recommended calls to the shell functions. Hopefully no one else will run into the ReleaseEngineer problem after this.

So what about the registry? Well, my first clue is that Macrovision is the company that acquired InstallShield about seven years ago. The second clue is in the nature of MSI technology.  An MSI file is a set of database tables and files, and InstallShield is in some ways nothing more than a tool that helps you fill in these database tables, so that users can install (and uninstall) your products. Many of the fields in these tables can’t be filled in until install time. So it would make sense for those tables to have obvious placeholders in them.  And “ReleaseEngineer.Macrovision” sounds more professional than “Joe User”. Is there a bug where the replacement fails to happen at runtime?

My next question is why the registry is getting overwritten.  I can’t see why any installer would need to update Shell Folder data.  But it’s easy enough to write custom actions in an MSI file, and these custom actions can easily overwrite Registry entries. Is there malice here, or just a common mistake? I can only speculate here (that’s all I’ve been doing in these last couple of paragraphs), but given the frequency we’ve had this bug reported, and the google hits, I’m wondering if InstallShield itself is the victim of a virus. Not necessarily a direct hit, but where some other virus writer is using the name “Macrovision” as a smokescreen.

Okay, enough speculation.  Here’s the C code I replaced the registry lookup with:

static int getSetting(
    size_t nBufferLength,    /* size of buffer */
    char*  lpBuffer          /* path buffer */
) {
    HMODULE hShell32;
    bool freeHandle = false;
    hShell32 = GetModuleHandle(TEXT("shell32.dll"));
    int rc = 1;
    if (!hShell32) {
        hShell32 = LoadLibrary(TEXT("shell32.dll"));
        freeHandle = true;
    }
    fprintf(stderr, "hShell32:%p\n", hShell32);
    lpBuffer[0] = 0;
    if (hShell32) {
        // Try the Vista "SHGetKnownFolderPath" first, and then
        // fall back to the Win2K "SHGetFolderPath"
        // Don't bother with GetVersionEx(), because the user
        // might have installed a service pack for the older OS that
        // added "SHGetKnownFolderPath".
        //
        // This falls in the category of "test for the feature, not the OS
        // version".

        // Vista, W7, future versions...
        const char *funcName = "SHGetKnownFolderPath";
        HRESULT hr;
        // typedef HRESULT (WINAPI * SHGetKnownFolderPathFn)(REFKNOWNFOLDERID rfid,
        typedef HRESULT (WINAPI * SHGetKnownFolderPathFn)(const GUID *rfid,
                                                          DWORD dwFlags,
                                                          HANDLE hToken,
                                                          PWSTR *ppszPath);
        SHGetKnownFolderPathFn shGetFolderPathFunc;
        shGetFolderPathFunc =
            (SHGetKnownFolderPathFn) GetProcAddress(hShell32, funcName);

        if (shGetFolderPathFunc) {
            PWSTR path;
            GUID guid_RoamingAppData = FOLDERID_RoamingAppData;

            hr = shGetFolderPathFunc(&guid_RoamingAppData,
                                     0, NULL, &path);
            if (hr == S_OK) {
                size_t numPrinted = wcstombs(lpBuffer, path, nBufferLength);
                lpBuffer[nBufferLength - 1] = 0;
                HMODULE hOLE32 = GetModuleHandle(TEXT("ole32.dll"));
                bool freeOLEHandle = false;

                if (!hOLE32) {
                    hOLE32 = LoadLibrary(TEXT("ole32.dll"));
                    freeOLEHandle = true;
                }
                if (hOLE32) {
                    typedef void (WINAPI * CoTaskMemFreeFn)(LPVOID pv);
                    CoTaskMemFreeFn coTaskMemFreeFunc;
                    coTaskMemFreeFunc = (CoTaskMemFreeFn) GetProcAddress(hOLE32, "CoTaskMemFree");
                    if (coTaskMemFreeFunc) {
                        coTaskMemFreeFunc(path);
                    }
                    if (freeOLEHandle) {
                        FreeLibrary(hOLE32);
                    }
                }
            }
        }
        if (!lpBuffer[0]) {
            typedef HRESULT (WINAPI* SHGetFolderPathFn)(HWND hwndOwner,
                                                       int nFolder,
                                                       HANDLE hToken,
                                                       DWORD dwFlags,
                                                       char *pszPath);
            SHGetFolderPathFn shGetFolderPathFunc;
            funcName = "SHGetFolderPathA"; // ASCII version
            shGetFolderPathFunc =
                (SHGetFolderPathFn) GetProcAddress(hShell32, funcName);
            if (shGetFolderPathFunc) {
                char path[MAX_DATA + 1];
                hr = shGetFolderPathFunc(NULL, CSIDL_APPDATA,
                                         NULL, 0, path);
                if (hr == S_OK) {
                    if (strlen(path) >= nBufferLength) {
                        return 0
                    } else {
                        strcpy(lpBuffer, path);
                    }
                } else {
                    return 0;
                }
            }
        }
        if (freeHandle) {
            FreeLibrary(hShell32);
        }
    }
    return 1;
}

Debugger variable sorting. Finally.

June 27, 2009

Over at the Komodo team we’ve had a debate for years over how variables should be presented in the debugger.  One team felt each back-end engine should decide how variables should be sorted, and send them to the front-end UI that way.  The other side said that the sorting should all be done in the front-end, so it could give consistent results across languages.  Not to mention allowing people to sort names in reverse order.  At times the PHP users were crying for help, as that language’s debugger engine has usually been the last to return items in sorted order.

I just checked in a fix for bug http://bugs.activestate.com/show_bug.cgi?id=65295, and it fixes a bunch of problems.  Finally, when you click on one of the column headers, Komodo sorts the rows by that column (either by name, value, or for a reason that escapes me, type).  Click once sorts values in ascending order.  Click a second time sorts in reverse order.  Clicking a third time sorts in the “natural” order returned by the debugger engine of the language you’re looking at.

The sort routines are smart enough to know when to do a numeric sort, and when to do a string sort.  The feature will be available in the next nightly.  Finally the long nightmare of seemingly random lists of items is over.

Hello world!

April 24, 2009

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!


Follow

Get every new post delivered to your Inbox.