Review: The Signal and the Noise

There is nothing like being right to make an impression. After calling the majority of congressional districts in the 2012 US election Nate Silver enjoyed his 15 minutes of fame. Before his election prediction I was only dimly aware of Nate Silver. I knew he worked for the New York Times, but that’s no longer an indicator of excellence or even sanity. Heck, even Nobel Prizes no longer guarantee excellence or sanity. Obama, vain narcissist that he is, was embarrassed by the dolts on the Nobel Peace Prize committee that confused existence for accomplishment.

It wasn’t Silver’s employer that led me to his book; it was his stint as a serious poker player that told me he wasn’t a standard NYT brain-dead progressive. Progressives do not bet with their own money! They bet with other people’s money. Anyone that puts their money where their mouth is is worth a hearing and Silver is worth a hearing

The Signal and the Noise is a series of essays about making predictions. It won’t surprise anyone to learn that some fields suffer poor, dare I say idiotic, predictors. Economists and partisan policy wonks are among the worst. Silver’s statistics show many of these people are clueless ideologues or cynical liars. They’re not even reliable contra-indicators. If only Nancy Pelosi, that Botox saturated crone, was consistently wrong, rather than randomly moronic, we might profit from her emissions.

As bad as some predictors are it’s not all bleak. Meteorologists have dramatically improved their forecasts. A few decades ago it was anyone’s guess where hurricanes would hit land. Now it’s possible to forecast landfalls within a hundred miles two days in advance. The weather service called Katrina before it hit New Orleans. It’s too bad so many ignored the warning.

One of the best sections in The Signal and the Noise deals with the dangers of “over-fitting”, over-fitting occurs when a model ends up modeling noise instead of signal. Over-fitting is an egregious statistical error but human beings are evolved over-fitters. If you “predict” the wind is shaking a bush and it’s a tiger you’re cat food. If you predict a tiger is shaking a bush and it’s the wind you have a bad hair day. Evolution favors the latter. If life is short, nasty and brutish, it pays to over-fit immediate dangers. This is not the case when over-fitting tells you something is highly unlikely when it isn’t.

Silver makes a good case for the Fukushima nuclear disaster going down in history as a classic case of the dangers of over-fitting. Earthquakes are currently unpredictable. Silver goes over the history of earthquake prediction and it’s sobering. Forecasts made by 21^st century geophysicists, armed with petaflop supercomputers, are only marginally better than simple historical means. This is a tough scientific problem made orders of magnitude harder by the difficulty of collecting data. We cannot directly measure stresses twenty kilometers underground. Hence the data feeding earthquake models are at best approximate and incomplete. This is unfortunate because models based on sketchy data are essentially conspiracy theories without black helicopters. You won’t find many geophysicists making short-term Vegas bets on the output of their earthquake models.

Power law fit of earthquake intensity – click for details.

This doesn’t mean that earthquakes are random or lack order. Earthquakes are remarkably orderly over geological timescales. They eerily fit a power-law distribution. This excellent fit makes it possible to pick any point on the Earth and compute an earthquake probability.  Such probabilities were computed for the seas near Fukushima but the earthquake model used was over-fitted and it dramatically underestimated the likelihood of magnitude 9 earthquakes. The Fukushima model had been “tweaked” to echo the fact that rare magnitude 9 earthquakes had not been observed near Japan in centuries. Instead of following a nice linear log-log plot the Fukushima plot was “bent” and the bend led to the assumption that it wasn’t necessary to plan for  magnitude 9 earthquakes and potentially huge tsunamis. Here model over-fitting lead to seawall over-topping. This is not your average stats 101 screw-up.

Looking back it’s easy to see where people went wrong. Maybe evil crony capitalists, bent on saving a few yen on concrete, conspired to sabotage earthquake models before submitting low ball Fukushima seawall bids. Doesn’t Lex Luthor do this every other day? Here it’s not necessary to invoke super-villains, good old fashioned short-term thinking, fortified with professional hubris, is all that’s required. Silver makes it abundantly clear that prediction, “especially about the future”, is hard but not necessarily hopeless. This is an excellent book for both lovers and haters of statistics.

The Myth of Sisyphus: Camus’s Absurd Prototype

In 1942, with World II raging, readers of The Myth of Sisyphus could easily identify with Camus’s absurd man. Not only is man absurd he has reduced his entire world to absurdity. Now, seventy-plus years later, Sisyphus readers are more likely to politely yawn and wonder what the fuss is about. Camus’s themes are not trivial or obvious but we, denizens of the early 21^st century, are thoroughly habituated to them. Absurd is now beyond mainstream; how else can one explain facile growths like Facebook. Just like homosexuals kidnapped the word “gay” and changed its meaning Camus abducted “absurd” and changed its meaning. The old “absurd”, synonymous with ridiculous, nonsensical, ludicrous and preposterous, becomes, in Camus’s hands, something few would call absurd. But, part of a great writer’s job description is changing the meaning of words, and it’s easy to see why the best of us have become absurd men.

Camus’s absurd man is a thoroughly honorable creature. He respects reason and wants to understand all things. This is beyond his reach but he doesn’t claim it’s impossible, only that his own limits make it personally impossible. Perhaps you’ve vainly argued with scientific illiterates that assert evolution is wrong because they cannot see how it could work. If I don’t understand it then it cannot be understood. Absurd men recognize, but do not generalize, their limitations. Absurd men also see that in the long run absolutely nothing matters. In a thousand years all but the greatest of us will be forgotten, in a million years even the greatest will be forgotten, in a billion years only precision instruments will detect our remains and in a trillion years it will be like we never existed. Many red dwarf stars will still be shining long after every microscope trace of humanity has disappeared. This is an inescapable scientific truth. It’s a terrifying banality that is often ignored or wrapped up in sky fairy nonsense. Yet, the absurd man faces cosmic futility without flinching or whining.

Instead of being crushed by a vast, difficult to comprehend, cosmos the absurd man soldiers on. He doesn’t curl up, go on food stamps, or complain about lurid Koch brother conspiracies. He gives a middle finger to his fate and then does what he can. Up yours universe: this is Camus’s revolt and, in our time, it’s a universal sentiment. This part of being absurd is easily faked; every brain-dead rapper and air-headed celebrity sports up-yours-airs, but absurd men do not play at revolt! They do their best to create without delusions and what idiot would claim celebrities are free of delusions? Many like to think cathedral masons were honoring god. Even in the Middle Ages this was delusional. Most were simply earning a living: if a pile of well-shaped rocks pleased god who knew or cared. Besides, in a geological blink, the same cathedral stones will weather to mud. Camus is very clear: absurd creation is self-consciously ephemeral.

Camus started The Myth of Sisyphus with what’s become his most famous line:

There is but one truly serious philosophical problem and that is suicide.

This is one exam question we all face. Cowards run and hide; they turn the knob to eleven and pop sweet — there’s an app for that — distractions. The religious create elaborate fantasies and declare war. Pure rationalists drag in Spinoza’s objects that long to persist in their being. They don’t commit suicide because that’s not part of their program. Ironically, only Camus’s absurd answer is remotely satisfying. In our lingo:  man-up, stop whining, pull your weight, don’t take crap, expect nothing, use your head and create. Any modern libertarian would approve.

When casting the prototypical absurd man Camus brilliantly chose Sisyphus. Sisyphus put death in chains, claiming immortality for men, but vindictive and jealous gods freed death and condemned Sisyphus to roll a heavy stone to the top of a hill only to watch it crash down and roll it up again, and again — forever. This sounds awful but Camus saw Sisyphus’s fate as a blessing. Life sucks when you’re pushing your rock up the hill but after it rolls down you get a break. While climbing down the hill Sisyphus has time to think, to absurdly create and, unlike poor doomed mortals, Sisyphus gets an eternity of breaks. This is an absurdly happy fate.

Oz the Gratuitous and Purile

This is not the Oz you’re looking for!

The original Wizard of Oz, the one we have all seen many times, is on a very short list of nearly perfect movies. Everything about the 1939 film is superb in excess. It’s as fun to watch today as it was over seventy years ago and you cannot honestly say that about many films. Because Oz is such a towering film classic it has overshadowed all cinematic attempts to reenter and reimagine Frank L. Baum’s world. Without exception every Oz wanna-be, and there have been many, suck like motivated gay prostitutes. Perhaps if we hadn’t seen the original, Oz the Great and Powerful, might be judged a “good” film, but we have and this overblown homage underscores, yet again, the magnificence of the 1939 Oz. I have some advice for Baum fans. If you want to reimagine his world — read his damn books! Believe it or not the book, The Wizard of Oz, is as good, if not better than the film classic. As for film makers — just stop it! Oz is close to sacred ground; if you’re not a Kubrick-level cinematic genius, and very few of you are, all you’re going to do is embarrass yourself and induce Dorothy nostalgia in your audience.

Watching Oz the Great and Powerful (OGP) has its own minor schizophrenic charms. On the good witch side OGP is another stunning special effects extravaganza. 21^st century movies are now in a weird place; if you can imagine it you can render it on the screen. If the ancient Greeks had a god of special movie effects it’s unlikely he could top your average contemporary big budget — tiny brain — movie. For decades producers and directors have strived to out-effect each other and they’ve finally ended up in a place Sophocles would recognize. Real improvements in modern movies can come from only one place: better stories! It was the telling of the story that distinguished the 1939 Oz. They took brilliant source material and merged it with equally brilliant songs. This is incredibly difficult and rarely achieved. In OGP’s case they didn’t even try. The suits that ruin Disney these days have a reputation for phoning stories in. With OGP they’ve out John Carter’ed themselves. I believe a bad Oz witch went all premenstrual on the screen writers. How else can you explain the transformation of classic source material into the Phantom Menace of Oz?

Books to Ignore

My superpower is indifference. Indifference is the soporific that lets my inner beast nap in peace. Without it I would have long since turned into a murdering psychopath, but I remain calm, rational, nearly ethereal, as I proudly ignore the unhinged idiocy of my fellow human beings. I prize my detachment, my don’t-give-a-shit-ness, my lordly disdain, and I work hard to maintain it. Today I’m sharing one of my, how to resist pummeling the morons around you, secrets. Here it is: don’t read rubbish.

Now, don’t fire up your book Barbie; that’s not what I’m suggesting. I’m asking you to upgrade your standards and scoot past toxic aisles in your favorite book store. Ignorable books are legion and they’re relatively easy to identify. The following rules have served me well.

1. Do not read biographies of living people. There are few, if any, definitive biographies of the living for the simple reason that they’re almost impossible to write without triggering crippling law suits. Historical figures, and their entire sycophantic bottom-feeding cohort, must be long dead before anything approaching perspective is possible. So put down that Prince of Wales or, don’t make me puke, Ted Turner, biography. They’re trash, worse than pornography, at least porn facilitates release.
2. Do not read autobiographies. Autobiographical fiction, and it always is, is akin to masturbating in public. Only one person is going to enjoy it. Everything I said about biographies goes double for autobiographies. We know you’re misleading, omitting, embellishing, polishing, fabricating, composting and just plain lying. If you’re one of the rare true worthies, say a Fields Medalist, a hard science Nobel winner, a Caesar, or a Mark Twain then go ahead and indulge yourself. Your scribbling’s will show future generations that bullshit is the only human universal.
3. Do not read hyphenated anything. Only mediocre twaddle preens as black-history, feminist-politics, gay-literature, Indian-mathematics, lesbian-drama or aboriginal-stories; the good stuff is known as history, politics, literature, mathematics, drama and stories.
4. Skip anything with “New Age” in its title, preamble, introduction, appendices or footnotes. Virtually everything written about new age beliefs, medicine, philosophy and so on is complete and utter garbage. This dreck is for feeble, magical-thinking, childish minds. I genuinely pity the purveyors and swallowers of new-age bilge; they’re sad silly people: come the zombie apocalypse the Deepak Copra’s of the world are starters.
5. Eschew political diatribes written solely to exploit current events. The political diatribe, or insta-history, like autobiographical fiction, may let readers in the future relive our self-deceptions, but we don’t live in the future. Most of these books emit a foul, cash in on my 15 minutes of fame, stench. I weep for the trees that died to print this crap.
6. Ignore books that rehash thoroughly debunked cover-ups and conspiracies. 9/11 was not an inside job, there are no miracle cancer cures, Bigfoot does not exist, the pyramids were not built by aliens, dowsing is crap, Oswald shot Kennedy, and the world did not end on December 21, 2012. Being a hard-ass skeptic clears entire drivel laden bookshelves.
7. Do not read diet books. You are not going to learn anything you don’t already know. We know why we’re fat. We eat too much and move to little. It’s really that simple. Get off you’re stupid obese ass and stop wasting your money on diet books.
8. Do not read computer books with high screen-shot densities. As a programmer I am constantly pawing through computer books and it pains me to report that vast swathes of this technical genre are awful. Stop printing one damn screen shot after another. It’s not helping.
9. Ignore how to get rich books. Get rich whores are worse than insta-history whores and often far more damaging to the pocketbooks of gullible readers.
10. Give any book sharing Steve Job’s secrets a wide berth. The necrophages writing this tripe are stunted little animals. Let’s feed on the dead, rich, white guy. Maybe if we eat his soul we’ll get rich too.

See that wasn’t so bad. With these ten rules you, shrink big-box bookstores to manageable dimensions, speed up online searching, and cut your exposure to rage fomenting mental pollution.  I must warn you that there are exceptions my rules and it delights me  when I find them. If you can point to books that break my rules please drop a note. I enjoy being wrong; it’s an opportunity to learn something and it’s a rare experience.

King Hobbit Kong

The pre-Hobbit hum was not harmonious. It started with the interminable legal battle over who would direct the Hobbit and how to split any spoils. After Peter Jackson’s cinematic Lord of the Rings triumph people assumed he would be The Hobbit guy but an envious and embittered clique of Tolkien’s heirs felt the weren’t getting enough for their stuff and filed suit demanding more. For some reason the spoiled progeny of the famous and accomplished think they’re entitled to feed on their glorious ancestor’s corpse — oops estate — forever. We’ve seen enough of this in the US with the idiot Kennedy clan: none of whom I would trust to drive or fly me anywhere. Even white-bread, holier than thou Canada, cannot shake this affliction. Look at Justin Trudeau — living on his father’s fame and full of it right up to his naturally curly eyebrows. Go back far enough and we all have glorious ancestors; unfortunately, this entitles us to precisely squat which, in a just universe, would constitute the sum total of royalties due to the current parasitic Tolkien generation.

Once the legalities settled and Jackson got control of the project ominous stories started swirling about “changes” to The Hobbit. One particularly disturbing rumor had Legolas making an appearance. Funny, I’ve read the Hobbit twice and missed him in the book! Whenever people start “improving classics” I get very wary. I’m sure the Hobbit screen writers are very smart and capable but they haven’t written a literary classic. If Legolas shows his pointy little ears in The Hobbit they should put bags over their heads and go into hiding for their own safety.

After the legalities and plot rumors the technology of The Hobbit made news. The Hobbit was shot at 48 frames per second: twice the standard rate. This was allegedly done to improve 3D projection. The biggest complaint about 3D movies, other than that they are 3D movies, is that they’re dim and fuzzy. Why this surprises anyone astonishes me. Hey, I’ve got an idea, let’s watch our three hundred million dollar movie through cheap, dark, optically flawed, polarized, throw-away plastic glasses. What could go wrong? The stupidity of hollyweirdos is boundless but what do you expect from Obama worshippers. It’s been claimed that 48 frames per seconds allows 3D projectors to present 24 frames per second to each eye and increase projection brightness, compensating for the those cheap dark glasses, without introducing an annoying flicker that plagues lower frame rates. We shall soon see if this is the case or just more Hollywood bullshit. Still, it would have been nice to test this technology on standard cinematic crap before subjecting classics to it.

Finally, if all of the above wasn’t enough, The Hobbit, by far the shortest of the famous Tolkien books, is being split into two, maybe three films, with the first installment running around three freaking hours. Wasn’t Jackson the guy that took a short, tight, beloved film classic, the original King Kong, and blew it up into a gigantic, tedious, overwrought, hemorrhoid inducing bore that was gutted in less than ten seconds by Robot Chicken! Yes the pre-Hobbit hum was not harmonious so I wasn’t expecting much when I plopped my discerning ass in my cinema seat and braced for The Hobbit.

Then three hours passed.

So what’s the verdict? I’ll try to do this without profanity. First a few warnings:

1. If you have read and enjoyed The Lord of the Rings but haven’t read the Hobbit do not see this film until you have read the book! The book will not spoil the film but the film will spoil the book.
2. If you’re a parent, looking to entertain your sucrose saturated children over the holidays, and you take them to The Hobbit, before they get a chance to read the book, you might as well go whole hog and tell them, there’s no Santa Claus, the Easter Bunny does not lay hardboiled colored eggs, the Tooth Fairy is mom and dad and little children don’t go to an imaginary heaven when they die; they’re just dead.

With that out-of-the-way I will publicly confess to enjoying the film labeled “The Hobbit.” Yes, my worst fears materialized on an epic scale. This is not, “The Hobbit,” but rather Hobbit inspired. So many artistic liberties were taken that I predict a rash of Hobbit, or not Hobbit, critiques will flood the intertubes. The story unfolds with a leaden, worst camping trip ever, pace and even the best scene in the movie, when Bilbo wins his riddle contest with Gollum and makes off with “the precious” ring is tedious. The famous time riddle comes across as a mood-lighting lighting aside and is so far removed from the brilliance of the book that it would have been better to just omit it, but you could say the same for at least half the scenes in this movie.

As I left the cinema surrounded by subdued teenagers, many of whom where commenting on how, “that wasn’t like the book,” I wondered who is this film for? Then it hit me. This is for the fantasy role-playing video game crowd. Many kids spend days immersed in these games. They’re used to laconic meandering pointless plot lines. This demographic will find the Hobbit tight, on plot, and will delight in the impressive special effects. Smaug does have beautiful eyes and I really enjoyed the wood trolls. I would have positively loved this film if the trolls had eaten the dwarves on the spot and then followed it up with a celebrity chef panel discussion. Maybe we’ll get lucky in Hobbit 2, 3, 4, 5, 6, 7, ….

You’ve burnt the plot and added ingredients that the recipe didn’t call for. This isn’t Hobbit stew and you call yourself a chef.

JHS meets MathJax

With the release of J 7.01 Jsoftware “deprecated” COM. J 6.02 can run as a COM automation server but J 7.01 cannot.¹ Throwing COM under the bus is hardly radical. Microsoft, COM’s creator, has been holding COM’s head underwater for years. Many .Net programmers cringe when they hear the word “COM” and the greater nonwindows² world never really accepted it. COM is a complex, over-engineered, proprietary dead-end. Yet despite its bloated deficiencies a lot of useful software is COM based. So with COM going away, at least for J programmers, the hunt is on for viable replacements and while we’re hunting let’s rethink our entire approach to building J GUI applications.

J GUI applications are traditional desktop applications. They’re built on native GUIs like Windows Forms and GTK and when done well they look and act like GUI applications coded in other languages. This is all good but there is a fundamental problem with desktop GUIs. There are many desktop GUIs and they do not travel well. Programmers have spent many dollars and days creating so-called cross-platform GUIs but, if you wander off the Windows, Mac and Linux reservation, the results are not particularly portable. And, as portable GUIs rarely outperform native alternatives, programmers tend to stick in their tribal silos. GUI programming is a bitch, has always been a bitch and will always be a bitch. It’s time to divorce the desktop GUI bitch.

All divorces, even the geeky GUI variety, are hard. When you finally cut the knot you’re not entirely sure what you’re doing or where you’ll end up. All you know is that there is a better way and with respect to J GUI applications I believe that JHS is that way. JHS leverages the large CSS, HTML and JavaScript (CHJ) world and in recent years some impressive browser-based applications have emerged from that world. The application that changed my mind about JavaScript and browser-based applications in general is something called MathJax.

MathJax typesets mathematics. It renders both LaTeX and MathML using fully scalable browser fonts. This is better than what WordPress does. The following Ramanujan identity taken from MathJax examples renders on WordPress as an image.

MathJax renders the same expression with scalable fonts and supports downloading the expression as LaTeX or MathML text. This is pretty impressive for browser JavaScript. I wondered how hard it would be to use MathJax with JHS and was pleased to find it’s easy peasy.

Writing a basic JHS application is a straightforward task of setting up three JHS J nouns HBS, CSS and JS.³ HBS is a sequence of J sentences where each sentence yields valid HTML when executed. JHS generates a simple web page from HBS and returns it to the browser. MathJaxDemo HBS is:

 HBS=: 0 : 0
 navul''           
 '<hr>','treset' jhb 'Reset'

 '<hr>',jhh1 'Typeset with MathJax and J'
 configjax
 oltypeset''
        
 '<hr>',jhh1 'Typeset Random Expression Tables'
 tabledesc
 '<br/>','ttable' jhb'Typeset Random Expression Array' 
 '<br/>','restable' jhspan''        
 )

CSS is exactly what you expect: CSS style definitions. Finally, JS is application specific JavaScript. MathJaxDemo JS matches HBS page events with corresponding JHS server handlers. This demo uses ajax for all event handlers.

JS=: 0 : 0

 function ev_ttable_click(){jdoajax([],"");}
 function ev_tquad_click(){jdoajax([],"");}
 function ev_tmaxwell_click(){jdoajax([],"");}
 function ev_tramaujan_click(){jdoajax([],"");}
 function ev_tcrossprod_click(){jdoajax([],"");}
 function ev_treset_click(){jdoajax([],"");}

 function ev_ttable_click_ajax(ts){jbyid("restable").innerHTML=ts[0]; MathJax.Hub.Typeset();}
 function ev_tquad_click_ajax(ts){jbyid("resquad").innerHTML=ts[0]; MathJax.Hub.Typeset();}
 function ev_tmaxwell_click_ajax(ts){jbyid("resmaxwell").innerHTML=ts[0]; MathJax.Hub.Typeset();}
 function ev_tramaujan_click_ajax(ts){jbyid("resramaujan").innerHTML=ts[0]; MathJax.Hub.Typeset();}
 function ev_tcrossprod_click_ajax(ts){jbyid("rescrossprod").innerHTML=ts[0]; MathJax.Hub.Typeset();}

 function ev_treset_click_ajax(ts){
   jbyid("restable").innerHTML=ts[0]; jbyid("resquad").innerHTML=ts[0];
   jbyid("resmaxwell").innerHTML=ts[0]; jbyid("resramaujan").innerHTML=ts[0];
   jbyid("rescrossprod").innerHTML=ts[0];
 }

Running the JHS MathJaxDemo is a simple matter of:

1. Downloading the J scripts in the MathJaxDemo folder to a local directory.
2. Editing J’s ~config/folders.cfg file and pointing to your download directory with the name MathJaxDemo. You are set up correctly if jpath '~MathJaxDemo' returns your path.
3. Loading the demo: load '~MathJaxDemo/MathJaxDemo.ijs'
4. Browsing to the site:
http://127.0.0.1:65001/MathJaxDemo


It’s not hard to use JHS as a general application web server. JHS provides many common controls right out of the box but to compete with desktop applications it’s necessary to supplement JHS with JavaScript libraries like MathJax. In coming posts I will explore how to use industrial strength JavaScript grids and graphics with JHS.

Screen shot of JHS MathJaxDemo running on Ubuntu.

---

1. On a purely numerical basis there is no greater nonwindows world.↩
2. To learn about JHS programming study the JHS demos and the JHS browser application.↩
3. Bill Lam has pointed out that J 7.01 can function as a COM client. The JAL addon tables/wdooo controls OpenOffice using Ole Automation which is one of the many manifestations of COM.↩

Git me a Hub’bery

Sometime ago I crossed my machine synchronization threshold. I routinely work on four operating systems, three laptops, a few servers, a bunch of phones and so on. I synchronized the directories I cared about while forming deep and rewarding relationships with file sharing services like Dropbox. Dropbox is great but its success has attracted the attention of paranoid IT micromanagers and it’s now frequently blocked on internal corporate networks. The beSOXed imbeciles that set IT policies will not rest until it’s impossible to do useful work on corporate assets and people wonder why there is so little return on IT investment.

Living without Dropbox and its many peers is annoying but tolerable provided humble USB ports are still useful but restricting plug-in drives is now standard not-operating procedure in most companies. So if you cannot share files or use USB what’s left? Would you believe GitHub?

GitHub is close to total global dominance in the geeky code sharing world. I would not have expected a version control system to attract a fiercely loyal and dedicated cult following but it has. Linus Torvalds, the Linux demigod, started Git when he correctly observed that all pre-Git version control system were fundamentally flawed because they enshrine the overlord peon hierarchy. The overload, usually some IT nitwit, manages the entire code submission, review and backup process and the peons, that would be us, bend over take it. Until Git appeared the majority of programmers despised and detested version control. It was just more IT management bullshit.  We put up with it because version control is a necessary evil and paychecks are even more necessary evil. We only wished things could be less evil and then Git appeared.

Git dumped the overlord peon hierarchy and adopted the radical notion that all repositories are created equal. When you synchronize Git repositories everything is synchronized. Your local copy contains everything the source does. This makes Git a superb peer-to-peer file distribution tool. Not only are you distributing files you’re also distributing their complete histories. This, almost biological replication model, has resulted in an explosion of Git repositories and the rise of hub sites like GitHub. Git’s dominance would not be possible if it was centrally managed. It’s succeeded because it’s harnessed market chaos.

I’ve used Git for over a year and I have often thought about pushing JOD source to public repositories. This weekend I bit the bullet and set up public repositories. Now it’s easy to Git me a Hub’bery!

Faith a guilty pleasure

Faith Korean TV

It’s a quiet Labor Day weekend in the drivel dome [1] and your fearless reporter is a tad bored. I could help with the housework or get out and exercise but I have better things to do. Last night while trolling the intertubes for something to watch on Hulu I came across a transcendentally awesome Korean TV series called Faith.  I know what you’re thinking. I haven’t fallen off the skeptical horse. I’m still the same old judgmental know-it-all bombastic boomer asshole you’ve come to know and love. The series Faith has, as far as I can tell after many long hours of couch research, nothing to do with religious faith. This is one of the many reasons I adore this show.

Faith is basically another Asian martial arts epic. After the demise of the demigod Bruce Lee it’s been mandatory for Asians residing east of Himalayas and south of Siberia to work martial arts into the plot whether it makes any damn sense or not. The Chinese, Koreans, Japanese, Vietnamese and other East Asians all follow Bruce’s mandate with various spins. Naturally, the most odious and predictable martial spins come from the mainland Chinese. With few exceptions mainland martial arts goes something like this.

Evil plots are afoot that are threatening the unity of the homeland. Nefarious forces, mostly internal, sometimes external, are plotting to bring down the well-ordered Middle Kingdom. A charismatic badass plans to exploit disunity, dishonor the people and shit all over the ancestors. Something must be done! The ruler, usually a wise emperor, or a really hot empress, tasks some typically reluctant super warrior to off the badass. The super warrior may have ambivalent feelings about the current ruler but never about the homeland. Sure the current ruler is a decadent pussy boy with weak Kung Fu and that’s too bad for him but damn, the country is not going down on my super warrior watch. Predictable mayhem ensues, bodies pile up, evil almost triumphs, gloats too much, and then falls to a combination of super warrior martial arts and old-fashioned hubris. In the end the homeland is saved and the closing credits suggest the super warrior might get some serious tail. I find it interesting that Hollywood is constantly destroying western civilization while mainland Chinese films forcefully reiterate that the homeland will always abide. I think it’s safe to say there hasn’t been an original mainland Chinese martial arts film since Bruce’s glory days.

Thankfully the South Koreans are not like mainland Chinese. Without the burden of an oppressive humorless government forever threatening serious consequences for plot wrong-think South Koreans can show some humor and originality. Faith is an excellent example. We know right away this is not standard martial arts because the bad guys are mainland Chinese threatening to overrun little Korea.  Even odder, our hero and heroine are the oddest of couples. He’s a tall 14^th century ultra-ninja-oid that can shoot lightning bolts from his hands while she is a ditsy 21^st century plastic surgeon.  It’s your basic boy meets time wormhole meets girl story. Faith only gets better after the hero drags the ditsy surgeon back to the 14^th century. The result is a comical, martial arts, chick-flicky, self parodying guilty pleasure.  You can see the cast members thinking WTF between their lines and is there a better endorsement than that?

---

[1] Analyze the Data not the Drivel is not suitable for succinct self-deprecating self-reference.

Turn your iPhone into a jPhone

Jsoftware recently released a free J app for the iPhone. Search for “jsoftware” in Apple’s app store and you will land right on it. There are many excellent free iPhone apps, I have half-a-dozen on my iPhone, but this little jewel sets a new standard for power in your palm.

Let’s start with the good news; this is not a crippled version of J. It’s the same high-caliber interpreter that J programmers have used on Windows, Linux and Mac machines for years. Anything this app’s big desktop brothers can do this little app can do. You won’t see the same blistering desktop speed but I think you’ll be pleasantly surprised at how fast this app munches numbers. I have run desktop J since the 1990’s and J on the iPhone 4 matches or beats what I was seeing on laptops ten years ago. This is the real J deal.

Now for a few caveats — this J app is not a complete J development environment. To meet Apple’s restrictive app store rules C-style APIs, JAL addons, GUI tools and third-party libraries like opengl and lapack are not included. With this app you get the J interpreter and a few of the most useful J addons like plot. Despite these limitations this J app is probably the most powerful, purely local, calculator available for the iPhone.

Don’t believe me? Turn off Wi-Fi and set your iPhone to Airplane mode: Airplane mode cuts off phone and internet access.  Now try computing the following on your favorite iPhone calculator.

NB. generate one million random numbers and average them
(+/ % #) ? 1000000#10000

NB. generate a 50 50 random matrix, invert it and multiply with the
NB. original - rounding to the nearest 0.0001 to form an identity matrix
round=: [ * [: <. 0.5 + %~
matrix=: ? 50 50 $ 10000
invmat=: %. matrix
identity=: 0.0001 round matrix +/ . * invmat

NB. sum of the diagonal elements of matrix (identity) is 50
50 = +/ (<0 1) |: identity

NB. multiply two polynomials with complex number coefficients
polyprod=:  +//.@(*/)

NB. complex number coefficients - AjB is J's notation for A + Bi
poly0=: 2j5 3j7 0 1
poly1=: 1j2 0 3j7 0 0 2
poly0 polyprod poly1

NB. prime factorization table of 50 random integers less than one billion
(<"0 nums) ,: -.&0 &.> <"1 q: nums=.50?1e9

The J app blows through these examples on my iPhone 4 in a few seconds. Of course this is only a tiny taste of what J on the iPhone is capable of. I have managed to run 1000+ line J scripts on this app. The only desktop J code I have not been able to run depends on external compiled libraries like regex.

Cell phones are powerful little computers and it’s gratifying to finally see software that can focus that power on something other than Angry Birds.  If you’re interested in learning an array oriented functional programming language or if you’re already familiar with J and want to pack serious computational heat then this app is for you!

J running on the iPhone

Ooh Promethean Tentacles

Hey let’s wander around in the dark and feed vagina snakes.

Prometheus breaks the first rule of movies: don’t make your audience think! Any movie that violates this taboo gets exactly what it deserves and Prometheus is begging for it.

Let’s get the good stuff out-of-the-way. Prometheus looks great. It throws up one fabulous tableau after another. To all the set designers, CGI programmers and other visual artists that worked on Prometheus take a bow; you did a superb job. I am not kidding when I say this would be a better film if you simply turned off the brain numbing sound track and soaked in the sights. Unfortunately the sound track is left on letting us in on the shallow thoughts of the too stupid to live protagonists.

I won’t dissect Prometheus’s numerous Promethean logical affronts. The blogosphere has already boiled and rendered that beast. Google the phrase “Prometheus too stupid to live”; the deluge of scorn will restore your faith in mankind.  I could join in the script-savaging but I’m a kind, loving and compassionate man. I like science fiction. I want science fiction screen writers to succeed but the poor lost souls need help. Here are three “rules” that might have saved Prometheus.

1. Any “number” that appears in a science fiction movie must be real.

Numbers incite analysis and analysis plops you right back into reality which is not a good place for any movie especially science fiction. Prometheus violates this rule early on.

When the spaceship Prometheus arrived at its multiple-star-grouping destination the film makes a point of informing us that it’s 3.27 x 10¹⁴ kilometers from Earth. My little cinema bound brain went to work. A light year is roughly 10¹³ kilometers thus Prometheus was about thirty light years from Earth.  I’d turned my cell phone off to watch the movie so I didn’t check the Hipparchus star catalogue but I’m pretty sure there are no large visible multiple star groupings that close to Earth.  It’s possible a grouping of nearby brown dwarfs has escaped detection but then the color balance of the giant planet the spaceship was approaching was all wrong.

Flashing bogus numbers distracts and infuriates audiences. If you must insert numbers in your scripts make sure they are 100% scientifically credible.

2. Do not populate your spaceship with suicidal dolts.

It’s hard to relate to complete brain-dead tools. About halfway through Prometheus I realized the only creatures acting sensibly were the tentacle waving predatory aliens.  They were busy going about their prey stalking ways. I can only imagine what they thought.

“Hey Sam,” alien vagina snakes call themselves Sam, “can you believe this fool wants to pet me. I bet his esophagus is going to be really tasty.”

“Uh, I don’t know Sam; something that stupid couldn’t survive in the wild unless it was highly toxic.”

“Yeah, you may be right but I’m going take a chance and stuff myself down his pie hole anyway.”

Nobody mourns the death of imbeciles. Sympathetic characters should have at least room temperature IQs.

3. Never show superior beings.

Good science fiction works best in the imagination of its audience.  Nurturing a sense of mystery, awe and wonder is what this genre is all about.  Showing too much kills the imagination and frankly my imagination took a few sniper rounds to the head when a “superior engineer” turned out to be bald, ripped, roid-raging bodybuilder that was as stupid as every other character in this film.

If the Prometheus of myth was as reckless as this “engineer” then Zeus did us a solid when he chained his dumb ass to that rock.