What catalog is that file in?

That’s what I hear often from my wife Meg; a professional photographer. Meg uses Adobe Photoshop Lightroom for cataloging the pictures she takes for her business as well as the pictures she take of our family. An underlying frustration that I find with Lightroom is the fact that photos become lost inside different catalogs on your hard drive, and Lightroom lacks the ability to search across catalogs. Some photographers believe the answer is to have a few monolithic catalogs. While these catalogs may contain everything in one searchable database, the catalogs become slow and unwieldy in a few short gigs of data. My answer is to have many small catalogs and harness the great search capabilities within the Mac OS — Spotlight. One small problem, Spotlight doesn’t inherently know how to index a Lightroom .lrcat file. In order to overcome this shortcoming, I wrote my own Spotlight plugin; something called an importer (See the Spotlight Importer Programming Guide).

SQLite

Lightroom catalog files are actually SQLite databases and therefore extracting the necessary data was quite simple. I first used SQLite Manager in Firefox to examine the database. I came up with the following simple query to get at the filenames I would need to pass to Spotlight:

SELECT originalFilename FROM AgLibraryFile

The next step was to run this query from within Objective-C. After using the familiar OmniSQLite framework from the larger OmniGroup framework, I settled on a much smaller wrapper, FMDB.

Importer

It took several days before I wrapped my head around Spotlight and the necessary components of an importer. Probably the most difficult concept to understand was Uniform Type Identifiers (UTI). Adobe does not identify a UTI for their catalog files. I had to create my own in order for Spotlight to identify the files and know what importer to use in order to index Lightroom catalogs. I came up with com.adobe.lightroom.library. This UTI is in the Info.plist file of my importer under the Exported Type UTIs section.

Packaging

Take note that the package I created for installing the importer places the importer in ~/Library/Spotlight. I chose the user’s home directory because I can install it without the need for a password. I also run this command

/usr/bin/mdimport -r ~/Library/Spotlight/LightroomSpotlightImporter.mdimporter

after install. That causes Spotlight to index files already on disk. See here for more info.

Source Code

The codes speaks well for itself and that is why the source is open and hosted at Github. I also created a nice Github project page for the code. You can find the page here and the source code here.

Screenshot

Here is the importer in use:

screenshot

Download

Download the installer package

The MultiMarkdown ToC

Do you ever feel like you’re never finished? I do, and I’m not happy with the fact that I haven’t been able to make Scrivener trigger the mmd-xslt script following a compile to MultiMarkdown HTML. Fine. As with the original XML epiphany, I knew there had to be code to perform an XSLT transformation in PHP on the server. Certainly there is, and my content-doc.php content template has further evolved into an XSLT transformation powerhouse. If a custom variable is provided with a path to the XSLT file on disk, and that files exists, the code transforms the XML using the provided XSLT style sheet. Here is a snippet:

<header class="page-header">
    <?php
    $title = get_the_title();

    $docs_path = get_post_meta(get_the_ID(), 'docs_path', true);
    if($docs_path && file_exists($docs_path)) {
        libxml_use_internal_errors(true);
        $xml = simplexml_load_file($docs_path);
        foreach (libxml_get_errors() as $error) {
            error_log($error->message);
        }
        if($xml) {
            $title = $xml->head->title;
            $xslt_path = get_post_meta(get_the_ID(), 'xslt_path', true);
            if($xslt_path && file_exists($xslt_path)) {
                $xslt = simplexml_load_file($xslt_path);
                foreach (libxml_get_errors() as $error) {
                    error_log($error->message);
                }
                if($xslt) { 
                    $proc = new XSLTProcessor;
                    $proc->importStyleSheet($xslt);
                    $xml = simplexml_import_dom($proc->transformToDoc($xml));

                }
            }
        }
    }

    printf('<h1 class="entry-title">%s</h1>', $title);
    ?>
</header><!-- .entry-header -->

Notes

Something to take note of in the PHP code. transformToDoc returns a DOMDocument object and the $xml variable contains a SimpleXMLElement object returned by the simplexml_load_file() method. If you recall, we insert the XML into the page by echoing the proper child element using the asXml() method. Therefore in order to leave that code intact, I import the DOMDocument back into $xml.

To snippet or not to snippet

My last post spoke of the need to use a snippet of XHTML in order to include it in a WordPress Page. This rule was short-lived. Why am I recanting my story? I switched to MultiMarkdown 3 and attempted the use its XSLT transformation capabilities; something of which requires an entire XHTML file (well-formed XML). The process in which one attempts an XSLT transformation with MMD3 is not an easy one at first glance. This is largely because I have yet to see how Scrivener can trigger MMD3 to perform the task at the completion of a successful compile. In order for Scrivener to be aware of the presence of MMD3 (Scrivener ships with MMD 2), an install of the MMD 3 Support package is required. This package installs in /Library/Application Support/MultiMarkdown. Scrivener is inherently aware of the files in this location, and makes use of them in lieu of the MMD 2 files contained within its app package. One small problem, according to the MMD 3 docs, the XHTML XSLT metadata tag has been removed from the spec.

In the absence of the XHTML XSLT: xhtml-toc-h2.xslt metadata tag, I had to run the mmd-xslt utility from the command line — run it from within its bin/ directory when you do. This script transforms the XHTML file into one that includes a ToC at the top. In my drive towards a successful transformation, I realized that if I had a well-formed XML document, then surely there were commands in PHP that would allow me to gain a reference to the BODY node, and inject it and its inner XML into the WordPress Page output. Sure enough, the necessary functions exist. I did however go back to scratch with the template files. I created a dupe of page.php and content-page.php (doc.php and content-doc.php respectively), and modified the content-doc.php thusly:

<article id="post-<?php the_ID(); ?>" <?php post_class(); ?>>
    <header class="page-header">
        <?php
        $title = get_the_title();

        $docs_path = get_post_meta(get_the_ID(), 'docs_path', true);
        if(file_exists($docs_path)) {
            libxml_use_internal_errors(true);
            $xml = simplexml_load_file($docs_path);
            foreach (libxml_get_errors() as $error) {
                error_log($error->message);
            }
            if($xml)
                $title = $xml->head->title;
        }

        printf('<h1 class="entry-title">%s</h1>', $title);
        ?>
    </header><!-- .entry-header -->

    <div class="entry-content clearfix">
        <?php
        if($xml) {
            _e($xml->body->asXml());
        } else {
            the_content( __( 'Continue reading <span class="meta-nav">&rarr;</span>', 'the-bootstrap' ) );
        }
        the_bootstrap_link_pages(); ?>
    </div><!-- .entry-content -->
    <?php edit_post_link( __( 'Edit', 'the-bootstrap' ), '<footer class="entry-meta"><span class="edit-link label">', '</span></footer>' ); ?>
</article><!-- #post-<?php the_ID(); ?> -->

Please take note of the use of libxml_use_internal_errors(true);. I’m keeping the page from showing error in the output. IMHO, it’s a security issue if you expose internal paths to the outside world. Now, back to bending MMD 3 and Scrivener to match my particular requirements. Thanks.

Learning

I’ve been working on this app spec for weeks at work. In an effort to improve on what I’d accomplished, I reached out to my friend, and one of the smartest people I know, Peter Becan. I wanted him to teach me how to do it right. Peter’s been doing this sort of thing for a lot longer than I have, and has a particular knack for it. Learning to correctly prepare an application specification is harder than learning to program, IMHO.

As my spec writing progressed, I’d email Peter PDF output from Scrivener as I accumulate enough new content worthy of a dispatch. marked, which I’m using to preview MultiMarkdown markup in my Scrivener document, generates the formatted PDFs. I realize that as time progresses, keeping Peter in the loop requires a different method. In order to keep him informed, and not have to email him a new PDF every so often, I needed to make use of the web to post the formatted output.

Scrivener

Scrivener supports outputting HTML generated by the MultiMarkdown processor as a Compile For target. There are instructions on how to use MultiMarkdown with Scrivener here, however the instruction appear out of date. I’m using Scrivener 2.2 and some of the preferences mentioned in the instruction have either moved, or no longer exist. Luckily, by setting options using MultiMarkdown metadata and a simple change in the compile settings, I obtained the necessary output.

For posterity’s sake, what I did not find was MultiMarkdown Settings… under the File menu

Two things are necessary to produce the desired output for inclusion in a WordPress page. First, as mentioned in the instructions, I had to enable the exporting of Titles for both Documents and Groups within Scrivener. You do this by checking the check box under Title in the Formatting options of the Compilation Settings sheet.

Second, I opted to use a Meta-Data file at the very top of my Scrivener doc to coerce the MultiMarkdown processor to produce the necessary output. The metadata fields that I use are:

Title:  Contractor Spec   
Format: snippet  

The key is the Format field. What that does is instruct the MultiMarkdown processor to only create the HTML for the given markup and not an entire XHTML page. Clearly if I am including the output in a WordPress page, only the HTML associated with the MultiMarkdown markup is necessary; the cruft associated with a well-formed XHTML page (i.e. HEAD, BODY, etc.) would be in the way. With all the correct metadata and settings in place, I use Compile… with a Compile For of MultiMarkdown -> HTML and save that in its own subfolder within my source tree.

Git

Git. Love it or hate it, it’s the linchpin in the operation. My Git repo resides on the same server as my WordPress installation. Having that scenario started me thinking about how I would get the HTML snippet residing within the repo into a place that I could serve just that content and not the entire source tree. I started Googling update website with git, and sure enough I found what I was looking for. After sifting through several top results, I found that this was the best answer.

I have an addendum to those instructions. For the remote path, I used a file:// path pointing to the path of the real repo on the server. Found that here.

The key to copying the HTML to a place I can include it from is using a post-receive hook in Git. Very simply put:

The post-receive hook runs after the entire process is completed and can be used to update other services or notify users. (taken from Pro Git)

However, if you follow the steps set out in the instructions, you’ll wind up with your entire repo in the web directory. While that may work in most cases, it was not ideal for what I was trying to do. Next pass at Google had me looking for ways to only checkout a subset of the entire repo. The key to that is something entitled a sparse checkout. I used the steps outlined here to only checkout the folder (and it’s content) that contained the HTML snippet. One exception however about the sparse checkout instructions, you will need to include a ‘*’ at the end of the path. Otherwise you will receive from Git:

error: Sparse checkout leaves no entry on working directory

For my “local” repo on the server, I picked a spot outside of the root of the WordPress installation to checkout the files to.

WordPress

Last step in the process. How do I insert a snippet of HTML that resides on disk into a WordPress page? Create your own page template and use a custom field. In the end, this was rather easy, once I learned how to do it. This page at the WordPress codex explains how to create the custom template and where to upload it to your server. Scroll down a bit till you get to the Using Custom Fields section. For this, I’m using a custom field called doc_path. Here is my custom template named docs.php:

<?php
/*
 Template Name: docs
 */
if (is_page() ) {
$docs_path = get_post_meta($posts[0]->ID, 'docs_path', true);
}
get_header();?>

        <div id="primary" class="span8">
            <div id="content" role="main">
                <?php include($docs_path); ?>
            </div><!-- #content -->
        </div><!-- #primary -->

<?php
get_sidebar();
get_footer();
?>

This allows me to specify the full path to the HTML snippet on disk that I want included in the WordPress page.

Make sure, as I forgot this the first time I saved my new Page, to set the template for the Page to your custom template like this:

Final Output

Because the application I’m writing the spec for is proprietary, I can’t share the real fruits of my labor with you. However, what I did do is create another page containing a snippet of sample.html from the MultiMarkdown source at github. My sample page is here. P.S. Yes, I am aware that the sample page has a broken image link.

I had a need to import some CAD drawings into my Visio document. The CAD drawings were provided to me as PDF documents. Visio has no native way to insert a PDF into a drawing. SnagIt to the rescue. Besides being an excellent app for making screenshots, it installs itself as a printer. Well, all I did was print my PDF to the SnagIt printer, saved the image as a TIFF, and then inserted the TIFF into my Visio drawing.

The resolution was quite good and I achieved exactly what I wanted. Gotta love it when shits works out!

Enhanced by Zemanta

I’ve been toying around with SQL Server CE replication. For whatever reason, my code was failing with the following exception when I called Synchronize():

Failure to connect to sql server with provided connection information. sql server does not exist, access is denied because the iis user is not a valid user on the sql server, or the password is incorrect.

As it turns out, if you use the follow form of the SqlCeReplication ctor (as observed using Reflector):

public SqlCeReplication(string internetUrl, string internetLogin, string internetPassword, string publisher, string publisherDatabase, string publication, string subscriber, string subscriberConnectionString)

the PublisherSecurityMode is set to SecurityType.NTAuthentication. Otherwise, if you use the parameterless ctor, PublisherSecurityMode is left to its default, which is SecurityType.DBAuthentication. This assignment is NOT documented.

I am working on a SQL Server 2005 Reporting Services (SSRS) report that has differing row colors based on a value in each data row.  The color value is defined in the database.  When I initially created the report, each row had a variable background color but the foreground color was black.  The first time I ran the report, my dark blue background didn’t contrast well with my black foreground.  I quickly realized that I needed a way to vary the foreground color programmatically based on the background color.  After first discussing things over with Nate, here is the expression I came up with for the Color property of the table row:

=IIF(
((
((CInt(Fields!Status_Color.Value) And &HFF) * 299) +
((CInt(Fields!Status_Color.Value) << 8 And &HFF) * 587) +
((CInt(Fields!Status_Color.Value) << 16) * 114)
) / 1000) &lt; 125,
"White",
"Black"
)

Let me explain where this all comes from.  First off, the color that is stored in the database is used by a VB6 program.  VB6 stores colors as BGR and .NET stores colors as RGB (well, technically aRGB).  The first step is to break down the value from the database to its constituent parts (red, green, and blue) using bitshift operations I learned from Keith Peters and then apply the contrast formula I found from Colin Lieberman‘s website. I then determine that if the blackground is a dark color, then we use white and for a light background, black.  This appears to working like a charm.

Last night we purchase an Apple TV (as well as a Mac Mini, 20″ Cinema Display Monitor, wired keyboard, and wireless mouse). Today I had the unfortunate happenstance of making this little gem of a unit find, and talk to iTunes running on my Mac Pro. I’ll save you the gory details buy my switch is a Linksys SRW224G4P and in the end, I had to disable IGMP Snooping. Otherwise the multicast traffic wasn’t flowing around correctly. This fix came as “well, let’s just see if we turn the helper off”. Well sure as shit, it worked. Yippee for me!

I wanted to report that I succeeded in using iSCSI (on an Openfiler server) with Time Machine via a gigabit link with jumbo frames (MTU of 9000) enabled. The secret to my success? I used the iSCSI initiator from http://www.small-tree.com/. It appears as if the iSCSI initiator from globalSAN is just a plain broke down piece of shit. Well, you do get what you pay for. Hurray for me and my buddy Steve over at Small Tree!