As a longtime amateur astronomer I appreciate good science writing and Mike Brown’s little book How I Killed Pluto and Why It Had It Coming is a wonderful example of the genre. When Pluto was tossed from the pantheon of planets I didn’t care. I knew that in previous centuries when asteroids were first discovered that they were briefly counted as planets. Eventually asteroids lost their planet status; there were too many of them and they were all dinky compared to real planets. Brown notes this bit of astronomical history by pointing to 19th century textbooks with high planet counts. The same holds for Pluto, Eris, Sedna, Quaoar and all the other known baby ice balls that make up the Kuiper belt. Real planets are massive enough to clear their orbits of crap. By this standard Mars barely qualifies and Pluto does not.

The emotional hysterics that greeted Pluto’s demise are still playing out. Some reviews posted here castigate Brown in terms rightly applied to suicide bombers and Obama voters. When Pluto bulks up and starts bullying its neighbors like all manly planets do we’ll talk until then I’d advise the puerile Pluto partisans to plumb up their pie holes otherwise we’ll have to toss you in the crank bin with the creationists and cold fusion nitwits.

cross posted on Goodreads.

J Christmas Wallpaper

I am celebrating Christmas by combining two of my favorite things: the programming language J and the superb image editor Picture Window Pro. The other day the good folks at Digital Light and Color announced 64 bit versions of Picture Window Pro. I was delighted. PWP is my favorite image editor. It’s had 16 bit image support since the cows came home and it simply nukes Photoshop when it comes to flat-out crunching performance.  I was afraid that the good old-fashioned C programmers that had crafted this program were retiring: perhaps overwhelmed by a sea of mediocre consumer oriented overpriced sludge — yes Photoshop Elements I’m talking about you! Fortunately it’s not to be! 64 bit multi-core versions of PWP will dramatically extend the lifetime of PWP for this loyal customer.

Speaking of superb good old-fashioned C programs I must give a shout out to J. I’ve been busy preparing a talk for the up coming J conference so J has been on my mind more than usual. Still all work and no play makes J a dull boy! This morning I rooted around in the J bin directories, sucked some J icons into PWP, and then twiddled transformations to generate some Christmas J monitor wallpaper: download and decorate at will!

Instagram Fingers

I’ve been playing around with the iPhone app Instagram.  I decided to give this freebie a try after a gaggle of soulless self promoting Apple marketing zombies declared it ”imaging app of the year.”  Instagram, like Hipstamatic,  (which incidentally stopped working after the last update), crops iPhone images to squares, applies hokey filters and then offers you the opportunity to “share”  or print your masterpieces on related websites.  Apparently Instagram is already one of the words largest online photo libraries. Considering Instagram images are filtered iPhone shots it’s almost certainly the worlds second largest online collection of crappy throw-away shots.  Facebook is the largest if you’re wondering!

You probably sense that I am not drinking the Instagram cool-aid. Instagram is a fine fun freebie but it strips EXIF data from the images it drops in the iPhone camera roll. This is the second iPhone EXIF exterminator I have encountered.  I have this thing about EXIF data; it annoys me when software massacres it.

Here’s a little poem:

Exterminate EXIF – NOT

Let it be

Set your fricking EXIF free

Alas, with Instagram, it’s not to be!

Hipstamatic water meter

Cameras come in two flavors. There are expensive serious cameras used by serious photographers to take serious photographs and then there are PhD, (Push here Dummy), cameras. Serious cameras meet the highest technical standards, lenses are diffraction limited, chromatic aberration approaches zero, pixel counts match Nyquist limits, ISO’s exceed retinal sensitivity, auto-focus is instantaneous, color spaces are wide, calibrated and range into the infra-red and ultra-violet. PhD cameras don’t go there! PhD cameras provide a button for dummies to push. Over the years I’ve ying’ed and yang’ed from serious to PhD cameras. It’s not that I cannot make up my mind. Camera choice is another eternal struggle sort of like that squabble between good and evil or light and regular beers.

When I’m on a serious photography kick I put time, expertise and money into my pictures. For me serious photography is exhausting. I  seek out worthy subjects, wait for the right light, agonize over composition, experiment with lenses, tweak f-stops, adjust shutter speeds and take as many exposures as light and time allow. Then the post shutter ordeal begins. Digital imaging processing makes anything possible. You can take your exposures and like some mad photo-shopping Rumpelstiltskin spin straw into gold.  But, as in the fairy tale, there is a high price, perfect images will suck years out of you. Photographers often talk about workflow and they aren’t kidding. When I start tweaking my workflow I know I’m reaching the end of my serious rope. Photography is one of my hobbies and I never ruin hobbies by turning them into jobs!

Eventually I snap out of it and resume PhD’ing. Unlike serious photography PhD’ing is effortless and fun. It’s always been fun, remember Brownies and Instamatics, but this is the golden age of PhD photography! In the last decade PhD cameras have evolved and mutated from cheap plastic boxes filled with grainy high-speed ISO film to sophisticated little internet photo-shopping engines that ride around in our pockets as cell phones. I don’t even know why they are called cell phones. My iPhone takes hundreds of images for every phone call it processes. It’s really more of an iCamera+iBrowser that can, on rare occasions, be used as a phone. Based on surveys of web image sites I am not the only iCamera user. PhD photography is growing faster than the national debt.

Mixing up cell phone color channels

Explosive growth attracts armies of sinners and saints. Every two-bit whore chasing a buck is now writing image processing software for cell phones. In the last two years we’ve seen apps that stitch panoramas on the fly, apply cheesy photo-shop effects and automatically create HDR images by blending two or more exposures.  As Sturgeon’s law predicts most of these apps are crap but that’s not a problem for PhD photography. The goal is to offer a fun button for dummies. My favorite PhD app is a great little toy called Hipstamatic. Hipstamatic simulates crappy old PhD film cameras loaded with expired film. You can choose a lens, a film and a flash or you can just shake your iPhone and let it randomly select all these parameters. Shake and shoot breaks all the rules of serious photography; it’s the revenge of the PhD camera.

The undeniable old whiteness of astronomy clubs is a concern because it tells you something about the health of our culture.  If you think I am pulling your leg or being sarcastic consider the following.

1. Club members have strong noncommercial or unprofessional interests in science.  Most are not professional astronomers or even scientists. Few derive any income from their interests in astronomy and many spend insane amounts of time and money building telescopes, observing  the sky and keeping up with findings in astronomy and related sciences. What’s the point? Wouldn’t these people be better off devoting the time they “waste” on astronomy to more productive pursuits?  Mind you the same applies to unprofessional artists and hobbyists of all types that “waste” comparable amounts of time on their own “pointless” pursuits.  Astronomy club members love what they do, and, in my experience, it’s a constant and enduring love that never dims or goes away. Anyone with a real passion knows what I’m talking about.
2. Astronomy club members respect, understand and value scientific arguments. We are constantly reading depressing assessments of the public’s dismal understanding of science. For example, 44% of the US public accepts that, “God created man pretty much in his present form at one time within the last 10,000 years.”  Only 5% of scientists accept this nonsense and even 5% seems to high for my skeptical ass.  Astronomy club members will poll more like scientists than the general public on this question. A society that depends on science yet harbors ignorant masses that do not grasp or appreciate basic scientific findings will not hold.
3. Club members welcome anyone with an interest in astronomy. I have belonged to half a dozen astronomy clubs and without exception they welcome anyone with an interest in the subject. I have never seen somebody turned away, or discouraged, for reasons of race, age, sex, sexual orientation, political or religious affiliations, or social class. Astronomy clubs are about as egalitarian as it gets! It gets better. Most clubs nurture and cherish the youngsters in their midst’s. This is partly because we don’t see a lot of youngsters and by youngster I mean anyone under thirty!

So why are astronomy clubs in North America old and white? Do people think you need expensive telescopes and other pricey gadgets to enjoy amateur astronomy?  I’ve looked at the sky for decades primarily with binoculars. They are still the best way to learn the sky. Has the public’s interest in science declined during the last forty years? Is elementary science education worse than it was when I was a child? Is the XBox’ed and Avatar’ed generation bored with looking at faint smudges in the sky? Finally, do we smell?  I don’t know why astronomy clubs are old and white but I do know that I will probably be associated with one until the day I die.

24 Hour Fitness

Yesterday I found myself back in Saint Luke’s emergency department being asked what day it was. An hour before I had stepped into the shower at 24 Hour Fitness over on Clayton,  I had just finished my noon hour workout.  I try to force myself to the gym at least four times a week.  It’s one of my many delusional projects. I’ve always wanted the lithe, lean muscular body of Spider Man but since I cannot cling to walls I’ve never managed the trick. As I stepped into the shower I suddenly felt sick to my stomach, it was a good pre-vomit feeling, I briefly thought about sitting down but then I stepped into the hot water and then … the next thing I remember  …  I was lying on my back on the shower floor with a few nude guys looking down at me.  A guy off to my side said, ”you were out for bit,”  another added,  ”you hit your head when you fell.”  Apparently,  if the news could be believed,  I had fallen out of the shower stall on my back.  I couldn’t remember a thing. I had passed out in the freaking shower!  My first thought,  ”oh goody one concussion coming up,”  I oddly didn’t care about my naked body inelegantly sprawled over harsh shower floor tiles for all to admire.  See you can get over gym shower shyness.

Of course this alarmed the gym staff, we live in a litigious, sue on the drop of — in this case my body — age.  They dutifully took notes and advised me to ride the ambulance to emergency.  ”It wouldn’t be a good idea to drive away and pass out on the highway.”  They had me there!  The EMT guys arrived, strapped me up to an electrocardiogram,  started an IV and then hauled my ass over to Saint Luke’s where I am almost on first name basis with the staff.  One of the nurses said,  ”I remember you, weren’t you the guy that tore his quadriceps?”  I enjoy being  recognized if only it was somewhere other than emergency.   Emergency staff plugged me into electric doo-dads,  drew my blood, measured my oxygen levels measured and shined lights into my eyes.  An earnest and very nice resident carefully interviewed me and declared I had experienced a syncopal episode.  Google it,  it doesn’t sound good,  I may be a dead man walking or I could have just overdone it at the gym.  Time will certainly tell.

Yousef Nadarkhani

I rarely comment on the “news” but the planned execution of Yousef Nadarkhani for the make-believe crime of Islamic apostasy has really pissed me off. Once again the Iranian branch of the “religion of peace” shows its primitive, ass-backwards, subhuman side to a stunned world that finds it hard to believe that changing your mind about fictitious religious nonsense is punishable by death! It’s rubbish like this that turned me into a hardcore atheist. If this is Sharia then Sharia is shit!

Let’s be perfectly clear. I am not condemning Iranians. I am married to an Iranian!  I have many lovely Iranian in-laws, and not a single one of them approves of this atavistic assinity. Like civilized people everywhere their feelings alternate between disgust, shame and rage. While I forgive the Iranian people I condemn the Iranian regime. The government courts that imposed this sentence are pathetic throwbacks. They don’t belong in the modern world and one day they won’t.

Savages always assume they can bully and terrorize people forever. At first people are easily cowed but even the most egregious assaults fail to deter in the long run. The next time the Iranian masses take to the streets the regime will fall and with the exception of a few islaminals nobody is going to miss it!

P.S.  Before you go all brave fatwa waving kafir killing jihad warrior on me keep in mind that Missouri is a concealed carry state.

“Crom! I have never prayed to you before. I have no tongue for it. No one, not even you, will remember if we were good men or bad. Why we fought, or why we died. All that matters is that today, two stood against many. That’s what’s important! Valor pleases you, Crom; so grant me this one request. Grant me revenge! And if you do not listen, then to hell with you!“

Or learn about what’s good in life:

My friends that golden Apollonic age is past! Now we must content ourselves with eat, pray, love travesties like:

“I live, I love, I slay, I am content.”

When did Conan the Barbarian decide to come out?

UTF8 APL characters within a LaTeX lstlisting environment. Click for *.tex source code

Typesetting APL source code has always been a pain in the ass! In the dark ages, (the 1970′s), you had to fiddle with APL type-balls and live without luxuries like lower case letters. With the advent of general outline fonts it became technically possible to render APL glyphs on standard display devices provided you:

1. Designed your own APL font.
2. Mapped the atomic vector of your APL to whatever encoding your font demanded.
3. Wrote WSFULL‘s of junk transliteration functions to dump your APL objects as font encoded text.

It’s a testament to either the talent, or pig headedness of APL programmers, that many actually did this. We all hated it! We still hate it! But, like an abused spouse, we kept going back for more.  It’s our fault; if we loved APL more it would stop hitting us!

When Unicode appeared APL’ers cheered — our long ASCII nightmare was ending. The more politically astute worked to include the APL characters in the Unicode standard. Hey if Klingon is there why not APL? Everyone thought it was just a matter of time until APL vendors abandoned their nonstandard atomic vectors and fully embraced Unicode. With a few notable exceptions we are still waiting. While we wait the problem of typesetting APL source code festers.

My preferred source code listing tool is the lstlisting package. lstlisting works well for standard ANSI source code.  I use it for J, C#, SQL, C, XML, Ocaml, Mathmematica, F#, shell scripts and source code, i.e. everything except APL! lstlisting is an eight bit package; it will not handle arbitrary Unicode out of the box.  I didn’t know how to get around this so I handled APL by enclosing UTF8 APL text in plain \begin{verbatim} … \end{verbatim} environments. This works for XeLaTeX and LuaLaTeX but you lose all the lstlisting goodies. Then I saw an interesting tex.stackexchange.com posting about The ‘listings’ package and UTF-8. One solution to the post’s “French ligature problem” showed how to force Unicode down lstlisting‘s throat. I wondered if the same method would work for APL. It turns out that it does!

If you insert the following snippet of TeX code in your document preamble LuaLaTeX and XeLaTeX will properly process UTF8 APL text in lstlisting environments. You will need to download and install the APL385 Unicode font if it’s not on your system.  A test document illustrating this hack is available here. The compiled PDF is available here. As always these files can be accessed in the files sidebar.

% set lstlisting to accept UTF8 APL text
\makeatletter
\lst@InputCatcodes
\def\lst@DefEC{%
 \lst@CCECUse \lst@ProcessLetter
  ^^80^^81^^82^^83^^84^^85^^86^^87^^88^^89^^8a^^8b^^8c^^8d^^8e^^8f%
  ^^90^^91^^92^^93^^94^^95^^96^^97^^98^^99^^9a^^9b^^9c^^9d^^9e^^9f%
  ^^a0^^a1^^a2^^a3^^a4^^a5^^a6^^a7^^a8^^a9^^aa^^ab^^ac^^ad^^ae^^af%
  ^^b0^^b1^^b2^^b3^^b4^^b5^^b6^^b7^^b8^^b9^^ba^^bb^^bc^^bd^^be^^bf%
  ^^c0^^c1^^c2^^c3^^c4^^c5^^c6^^c7^^c8^^c9^^ca^^cb^^cc^^cd^^ce^^cf%
  ^^d0^^d1^^d2^^d3^^d4^^d5^^d6^^d7^^d8^^d9^^da^^db^^dc^^dd^^de^^df%
  ^^e0^^e1^^e2^^e3^^e4^^e5^^e6^^e7^^e8^^e9^^ea^^eb^^ec^^ed^^ee^^ef%
  ^^f0^^f1^^f2^^f3^^f4^^f5^^f6^^f7^^f8^^f9^^fa^^fb^^fc^^fd^^fe^^ff%
  ^^^^20ac^^^^0153^^^^0152%
  ^^^^20a7^^^^2190^^^^2191^^^^2192^^^^2193^^^^2206^^^^2207^^^^220a%
  ^^^^2218^^^^2228^^^^2229^^^^222a^^^^2235^^^^223c^^^^2260^^^^2261%
  ^^^^2262^^^^2264^^^^2265^^^^2282^^^^2283^^^^2296^^^^22a2^^^^22a3%
  ^^^^22a4^^^^22a5^^^^22c4^^^^2308^^^^230a^^^^2336^^^^2337^^^^2339%
  ^^^^233b^^^^233d^^^^233f^^^^2340^^^^2342^^^^2347^^^^2348^^^^2349%
  ^^^^234b^^^^234e^^^^2350^^^^2352^^^^2355^^^^2357^^^^2359^^^^235d%
  ^^^^235e^^^^235f^^^^2361^^^^2362^^^^2363^^^^2364^^^^2365^^^^2368%
  ^^^^236a^^^^236b^^^^236c^^^^2371^^^^2372^^^^2373^^^^2374^^^^2375%
  ^^^^2377^^^^2378^^^^237a^^^^2395^^^^25af^^^^25ca^^^^25cb%
  ^^00}
\lst@RestoreCatcodes
\makeatother

So, taking my own advice,  the following is an example of a SQLSever Common Table Expression (CTE) query.

WITH cte1
     AS (SELECT c.obj_id                            AS obj_id,
                a.created                           AS creation_date,
                Datediff(DAY, a.created, Getdate()) AS day_cnt
         FROM   [HIST].[dbo].[History] a
                JOIN [HIST].[dbo].[HistorySummary] b
                  ON a.rowid = b.rowid
                JOIN [HIST].[dbo].[ObjectNames] c
                  ON c.obj_name = b.obj_name),
     cte2
     AS (SELECT cte1.obj_id AS obj_id,
                CASE
                  WHEN cte1.day_cnt <= 30 THEN 1
                  ELSE 0
                END         AS d0,
                CASE
                  WHEN ( cte1.day_cnt > 30 )
                       AND ( cte1.day_cnt <= 60 ) THEN 1
                  ELSE 0
                END         AS d1,
                CASE
                  WHEN ( cte1.day_cnt > 60 )
                       AND ( cte1.day_cnt <= 90 ) THEN 1
                  ELSE 0
                END         AS d2,
                CASE
                  WHEN ( cte1.day_cnt > 90 ) THEN 1
                  ELSE 0
                END         AS d3
         FROM   cte1)
SELECT obj_id,
       SUM(d0) AS OneMonth,
       SUM(d1) AS TwoMonths,
       SUM(d2) AS ThreeMonths,
       SUM(d3) AS Overdue
FROM   cte2
GROUP  BY obj_id

CTE queries essentially create temporary virtual tables during query execution. They are similar to nested SQL queries but are easier to code, more general, (see recursive CTE queries), and often perform better than their convoluted equivalents. This example creates two virtual tables cte1 and cte2 that are then used to compute a quick histogram.

You can call CTE queries with the J ODBC interface. The following assumes a SQLServer ODBC connection dsn history.

NB. odbc interface
require 'dd'

NB. read CTE query
HistoryAgeSQL=. read 'c:/temp/HistoryAge.sql'

NB. connect sqlserver database
ch =. ddcon 'dsn=history'

NB. select with CTE query
sh =. HistoryAgeSQL ddsel ch

NB. fetch results
data=. ddfet sh,_1

As a final note it’s worth comparing the SQL CTE histogram code with J equivalents.  The two following J verbs taken from the J wiki, (here and here), compute histograms.

NB. computes histograms uses right open intervals
histogram=:<:@(#/.~)@(i.@#@[ , I.)

NB. variation on (histogram) uses left open intervals
histogram2=:<:@(#/.~)@(i.@>:@#@[ , |.@[ (#@[ - I.) ])

They scale to tens of millions of data points; returning results in a few seconds on my laptop. The SQL CTE, shown above, takes about three seconds running on 180,000 row tables on my employer’s full warp servers.  If I could convince the masses to adopt languages like J a large part of my job would disappear. Fortunately, the world is hostile to terse elegance!