# The Singularity is Not Near Enough

Personally I don’t have a problem with life extension.  I’m in the Woody Allen camp. “I don’t want to achieve immortality through my work; I want to achieve immortality through not dying.”  Not dying, ever, is a mighty big order. If you place any credence on current Big Bang cosmologies it looks like the entire universe is mortal. In a mere 10100  years the universe might be completely dark with a temperature that is physically indistinguishable from absolute zero. All the stars, galaxies, black holes and unstable elementary particles will have long since decayed. You might find a free electron every few million light years or so if you’re lucky. Such a universe is beyond dead as far as sentient beings are concerned. Religious end-times are for pussyfied wimps, metrosexual girly-men and fat feminists. Plausible universal end states are vastly more terrifying that any Last Judgment or Ragnarök. I don’t expect life extension to produce lifetimes of 10100 years; I would be happy if we eke out a few millennia. There are thousands of trees, deep-sea corals, and freaking sponges that are many thousands of years old?  If a sponge can hack a thousand years I really don’t think I’ll have a problem.

My one serious objection to human life extension is simple. Most of us are dumber than fence posts and even the intelligent have brains infested with patently stupid ideas that deserve immediate and total deletion. Most of us aren’t worth preserving until next week let alone the year 3000. I don’t exempt myself from this harsh judgment. I’ve always assumed that if I can understand something it’s inherently trivial; that if I can do something it’s no big deal; that if I can hack it cannot really hurt. Yet I am constantly amazed at how many cannot meet my pitifully low standards. Do you really want to be surrounded by your grunting, ignorant naked ape neighbors for a thousand years?  It’s hard to make a more cogent argument for mortality. I know what you’re thinking. “That’s not fair; people learn and change.” Oh, if it were only so. I know people who have never changed their minds about anything. They are the true walking dead; after the mind goes the body is just pus and who wants to extend the life of pus?

Greatly extending the lives of horribly flawed and limited people will only perpetuate our screwed up world. Minds do not change, they die. If Caesar was still up and about we would probably still be throwing people to the lions, worshipping Greco-Roman sky fairies and marching off to war every spring. We have to get a lot smarter to use thousand-year lifetimes. Fortunately the singularity is coming.

Ray Kurzweil has absorbed lots of abuse. His book, The Singularity is Near, has been mocked as rapture for nerds. My own view is that his thesis is sound but his timeline is whacked. I don’t think the singularity, the emergence of superior trans-human intelligence, will happen in my lifetime. I doubt it will happen in the 21st century but it’s almost inevitable in the 22nd  and 23rd.  We are entering an end-time of sorts. Standard Homo sapiens will, in a few short centuries, be extinct.[1] We will either be eliminated by trans-humans as a dangerous pest species [2] or we will voluntarily become trans-human ourselves. I agree with Kurzweil that we cannot predict how truly superior beings will live but I’m willing to bet my Bitcoin stash that they will outlive sponges! The singularity is not near enough.

[1] This is why I don’t worry about Global Warming.

[2] They may keep a few of us around like we keep grizzlies in national parks.

# Bitcoin is a Perfect Protest

The most intelligent comment I have read about Bitcoin is that it’s a perfect protest. Bitcoin went live in 2009 shortly after the 2008 financial crisis. The 2008 crisis was a defining moment. Prior to that date I believed that the US government, despite its obvious warts, short comings and long checkered history was still partly accountable to the electorate.  I didn’t buy the widespread cynical notion that modern elections are largely meaningless dog and pony shows that help sell the illusion that the people are in charge. I seriously thought there were important differences between Barak Obama, Hillary Clinton and John McCain. It’s embarrassing to confess such naivety.

Before 2008 I was a good little cog in the machine: obediently paying my taxes and being a productive member of society. I was a chump: a silly stupid chump, but lucky for me, the “crap sandwich” bank bailout cured my naivety. Despite being overwhelmingly rejected by the public and initially rejected by Congress the crap sandwich was forced down our throats and all three presidential contenders voted for it.  When push came to shove there were no significant differences between liberal democrats and conservative republicans, both groups lined up to betray and indebt the public and we’ve been suffering, and will continue to suffer, the consequences for years to come.

The 2008 financial crisis, and the comical US election that followed it, taught me some important lessons:

1. If none of the above is not on the ballot the election is fraudulent. The political systems in the countries I have lived in depend on presenting limited, and frankly insulting choices, to the electorate. If anyone is going to seriously argue that Barak Obama and John McCain were the best that a country of three hundred million souls could offer then we are lost. If I was lost in the woods with Obama and McCain I wouldn’t take a millisecond of direction from either and might consider getting rid of them on the spot to improve my chances of survival. A candidate must be better than nothing, and if nothing is superior to the highly gamed political selections put forward then nothing should be on the ballot! I will return to this theme in future posts. The next time you cast a ballot look for none of the above. If none of the above is not present the election is illegitimate and you are being used to put a stamp of public approval on what’s very probably a vacuous choice.
2. You can tell we’re dealing with a real issue when the ruling class closes ranks. Our idiotic media maelstrom is inconsequential noise that is best ignored.  Will a society with gay marriage manage their finances better than a society without gay marriage?  Will free birth control pills for sluts impact trade balances? Will crosses on public lands constrain money creation?  Does the size of Kim Kardashian’s ass moderate capital controls?  Get in the habit of asking such questions. If the question is absurd, or if the answer doesn’t matter, it’s a distraction.  On the other hand if you see alleged ideological enemies coming together to promote a critical common good beware! In 2008 the flamboyant cosmetic differences between liberals and conservatives vanished removing even the illusion of choice. To bailout, or not bailout, was a real issue and with real issues there is no choice. We’ve recently witnessed rank closing on Edward Snowden. Again, both left-wing democrats and right-wing republicans lined up to declare Snowden a traitor and praise the glories of our NSA surveillance state. Clearly public privacy is another real issue and with real issues there is no choice.
3. Human beings cannot be trusted with money creation.  The 2008 bank bailout was outrageous for two primary reasons. It lavishly rewarded bad behavior and it created money to do it. Money creation is convoluted; many argue that commercial banks create the bulk of money through loans, others claim the Federal Reserve creates money when buying government bonds and treasuries. The food chain is twisted but nobody disputes that at the base of the chain money is created out of nothing. Everything boils down to ledger entries made by sanctioned authorities. There is no mining, there is no collateral, there’s nothing but an invisible yoke that’s eventually placed on the public’s head. The invisible yoke has briefly shown itself in the fiery debt limit fights about the full faith and credit of the United States. What the hell is the full faith and credit of the United States? It’s nothing more than a promise that the government will somehow extract the means to make payments to that long forgotten ledger entry. If the public fully understood that their labor is balanced against nothing they would refuse to pay and the entire system would collapse. The system is such a perfect scam it’s hard not to admire it. Oh, it will eventually collapse; fiat money always goes to zero, but in the meanwhile it affords unlimited fiscal flexibility to the ruling class. Who gets to create money is a real issue and once again there is no choice about real issues.

There has been a lot of nonsense written about Bitcoin but one thing is clear it serves as a brilliant financial foil so I am not surprised to see recent worldwide efforts to suppress it. The most frightening thing about Bitcoin is that it gets people asking questions about money. For example:

1. Exactly what is money?  Every crank has their own definition of money. What amuses me is that both Gold cranks and fiat cranks have lambasted Bitcoin for being arbitrary and made up. One of the best retorts to this confused drivel notes that Bitcoin is to “real money” like the Flying Spaghetti Monster is to “real religion.” Everyone sees the Flying Spaghetti Monster is made up, but – oddly – nobody can mount rational arguments explaining why it’s more made up than the “real thing.” Bitcoin is capable of playing the role of money, so in proper contexts it is money.
2. Why do banking authorities have exclusive money creation rights? The historic rationale was to prevent counterfeiting. Counterfeiting is irresistible to anyone in a position to do it. By giving money creation rights to select authorities and using deadly force on counterfeiters governments could claim they were protecting the “currency of the realm.” It is many orders of magnitude more difficult to counterfeit Bitcoins than US dollars or any national currency. To counterfeit a Bitcoin you have to break a hard cryptographic hash.  Technology has rendered the rationale for central money creation authorities obsolete.
3. Should money be created without limit from nothing? Now that monetary creation restraints, historically ties to gold, no longer exist the only limit on creating money out of nothing is the stupidity of the public. How much debt can you get poor dumb suckers to accept before they rebel? Bitcoins are not created out of nothing. The mining process validates the public ledger, the Blockchain, and insures that nobody is counterfeiting coins or double spending. Mined coins are a reward for valuable network services. Additionally, there is no central creation authority. Competing miners create Bitcoins all over the world. This system is not without fault and Bitcoin variants are exploring technical improvements but the Bitcoin creation process is essentially a mathematically secured network phenomenon and it is much harder to corrupt than bribing a few central bankers.
4. Why do authorities maintain the right to confiscate private funds? A Bitcoin feature that is particularly disturbing to authorities is that it’s not difficult to prevent even powerful entities from seizing coins. A coin cannot be moved or spent unless you get its private key. If you do not know the private key a Bitcoin will just sit in the Blockchain taunting goons that covet it. In a Bitcoin economy it will be difficult to garnish wages, block money transfers and seize assets. How will the state survive?
5. Why must fees be levied when moving money across national borders? The public has never accepted this little rape. How many of us have lied to custom officials when asked about how much cash we’re carrying?  I’m guessing a fair fraction of all travelers. We all know it’s none of their damn business but being good little cogs we bend over and submit to state sodomy. Bitcoin penetrates borders with the same ease that custom authorities conduct cavity searches. Go ahead cut off coin movement! All you have to do is turn off the Internet, commander all USB ports, block old fashioned paper mail and learn how to read people’s minds. Any information storage and transmission device, including the human brain, can be used to move coins. Go fuck yourself customs. One day money will be free to move without your permission or consent!

Obviously we cannot have too many people asking such questions. Mathematically sound, open source, publicly validated and distributed real money like Bitcoin must be ridiculed, harassed and stopped. Left unchecked it will cauterize an important component of state power: arbitrary money creation rights.  By providing an elegant mathematical model of how a world without central banking and national currencies might function Bitcoin is a perfect protest: a good idea that our corrupt “leaders” cannot honestly answer.

# John L. Dobson R.I.P.

At tonight’s meeting of the St. Louis Astronomical Society I learned of John Dobson’s recent death. John Dobson was widely known as the inventor of the homemade “Dobsonian” telescope and the co-founder of the Sidewalk Astronomers: perhaps the most famous and effective amateur astronomy outreach group in modern times. “Big Dob” light buckets are a staple at star parties around the world and most of them derive from John’s original designs. John lived a long life and touched many people including myself.

I briefly met John at a star party in central Texas in the late 1990’s. I cannot remember exactly where we were but it was about two hundred miles west of Fort Worth and was one of the best “dark sky” sites within easy driving distance of the Fort Worth Dallas light pollution wasteland.[1]  Amateur astronomers abhor, detest, loathe and constantly rage, rage against — street lights. When I see the sky sodomized by one ill placed security light or a hideous blinking radio tower I have to suppress Homeric, (Simpson), urges to kill. Light Pollution is an assault on one of the most beautiful things the human eye can behold, a glorious night sky, and most people are completely and utterly oblivious to it.

Because our central Texas star party was far from the maddening crowds, just the way hard-core dark sky connoisseurs  like it,  there weren’t very many people present and most in attendance where armed with state of the art telescopic gadgetry. This did not quite suit John. I remember he remarked that this was an “astronomer’s gathering.” Meaning this was a gathering for people who already knew the majesty of night sky. It was John’s passion to introduce neophytes to the that glory and judging by the accolades coming in from people who caught the astronomy bug at one of John’s sidewalk star parties it’s a passion that will outlive him.

John felt it was vitally important for people to see, with their own eyes, the craters of the moon, the moons of Jupiter, Saturn’s rings, sunspots, the Andromeda galaxy and thousands of other sky wonders. He apparently never tired of watching someone look through a telescope for the first time and until tonight I must confess I didn’t really appreciate just how crucial such “first lights” are.  As I drove home I started thinking about my top astronomical experiences and soon realized they are some of my top experiences — period. Here’s my top ten “first lights” in no particular order. Only the last stands above the  others.

1. First look at the moon through a telescopeRedwash Utah, United States. When I was in grade school my parents got me a 60mm Tasco refractor. It was a simple, and surprisingly good, little telescope. I’ve talked to many amateur astronomers over the years and many fondly remember getting started with a 60mm Tasco. Shortly after I got that telescope I set it up and waited for the sky to darken. The moon was not full when it rose; I didn’t care. I lined up the scope, fiddled with the focus, and then suddenly, the moon’s craters appeared: the sharpness and clarity almost hurt. I’ve been hooked on amateur astronomy ever since. In retrospect the moon’s craters made a bigger impression on me than my first kiss. I don’t remember my first kiss, but I’ll never forget my first telescopic glimpse of the moon. John never tired of introducing strangers to the moon.

2. Seeing the crescent of Venus for the first time: Redwash Utah, United States. It took me awhile to learn how to properly focus my telescope. The moon was easy but for some reason I never got Venus dialed in until one evening when I turned the knob far enough to condense the unfocused blob of Venus into a sharp little crescent. It was mind-blowing. The seeing on the high Uintah plateau was superb. I have seldom seen Venus so steady, sharp and clear. I felt like Galileo — hell for a brief instant I was Galileo!  John loved showing off the bright planets. Venus, Jupiter, Saturn, even Mars and Mercury are all easily visible from the middle of light polluted cities.  Sidewalk Astronomers turned these ancient wanderers into modern celebrities.

3. Seeing Jupiter and its four Galilean moons: Agha Jari Iran. Shortly after getting my first refractor my family moved to Iran. I lugged my little telescope half way around the world. The telescope was my hand baggage. In those days airlines were more tolerant of large carry-ons and being a kid I enjoyed extra latitude.  After we settled in Agha Jari I set up the telescope in our front yard. I wanted to check out a bright star that was hanging about twenty degrees above the hills to the southwest. I knew that looking at stars in the 60mm was, with the exception of binaries, kind of dull. Stars appear as points of light in even the largest of telescopes. Only in modern times, by using long baseline interferometry, has it become possible to resolve details on distant stars.  Not expecting much I aimed the scope at the bright star and focused. Suddenly a new “solar system” sprang into view. I could see a tiny ball surrounded by four bright spots. For a few moments I thought I might be seeing something new. How could people have missed this? Then I realized it was Jupiter. Big J is still my favorite planet.  In John’s Big Dobs Jupiter is a super-planet. You can easily see four or more atmospheric bands, shadows of major moons and the Great Red Spot. Even better, it’s all visible from city sidewalks.

4. Catching Halley’s Comet: Edmonton Alberta Canada. Once you catch the observer’s bug it never really leaves you. It may go dormant but something always wakes it up. The return of Halley’s Comet roused my inner observer. Halley’s Comet is the most famous of all comets. It takes roughly one human lifetime for it to complete its orbit so catching Halley’s Comet is something most of us will only do once. The last time Halley’s Comet zoomed by in 1910 it put on a spectacular show. Astronomers warned that the 1986 passing would be “disappointing.” The comet was further away than it was in 1910. They were right but I was still delighted by what I saw. It was a freezing -30C Edmonton winter night when we drove to the city’s southern outskirts to see the comet. Despite the blistering cold dozens of people were parking along the road and getting out of their cars to look for the comet. I knew exactly where to look and it only took me a few seconds to find the fuzzy ball known as Halley’s Comet. It wasn’t spectacular, but it was historically satisfying. John spent a lot of time educating people about what they would see in the sky. If you understand, even the faintest of objects can thrill.

5. All sky aurora Edmonton Alberta Canada.  Amateur astronomers have mixed feelings about auroras. Amateurs that live in aurora zones, Canadians, Norwegians, Argentines and others often bitch about “natural light pollution” until they witness a full-blown all sky aurora.  It was another cold Edmonton winter night and I was up late watching the idiot box when a local news alert interrupted programming to report an impressive aurora was underway. Northern lights in Edmonton are common and seldom newsworthy. I had to see what the fuss was about so I bundled up, stepped outside and was immediately transfixed. Auroras are usually silent slithering green sheens. Tonight the sky was blazing green, then red, hinting at purple, then back to green, red again, rippling and roaring, from the north to south, east and west: all ablaze. I had never seen a display of such magnitude. Giant auroras are not only beautiful they can shut down power grids.  I love it: natural light pollution shutting down man-made light pollution. I don’t know if John saw great auroras but I know he would have loved them.

6. Comet Bennett near Canmore Alberta Canada. I wasn’t looking for Bennett’s Comet when I saw it. I was driving back to Calgary from Vancouver with friends. We made a road side stop in the Canadian Rockies near Canmore to relieve ourselves. I trudged out into the snow, unzipped my fly,  looked up and saw, just above the dark jagged outline of the mountains, the most exquisite comet. I wasn’t sure what it was. It was so striking that we stopped peeing and admired it. I remember saying, “It looks exactly like the comets you see in textbooks.” I was right.  That passing of Bennett’s Comet was canonical. I’ve seen some great comets since Bennett: Hale-Bopp and Hyakutake were both spectacular but unexpected Bennett is still my favorite. John often reminded people that you don’t always need a telescope: just keep looking up.

7. Total Eclipse of the Moon: Tamale Ghana Africa. I taught mathematics for two years in a northern Ghanaian boarding school after graduating from university. The best total lunar eclipse I have ever seen occurred during my Ghanaian years. I reckoned the eclipse would be a great teaching opportunity for my students.  The school had and old telescope; it was a small reflector that a previous teacher had donated. The scope didn’t have an eye piece so we adapted a microscope eye piece. It worked better than expected.  On the night of the eclipse we set up the scope on a second story veranda with a nice southern view. Lunar eclipses are leisurely events. It takes a long time for the Earth’s shadow to cover the moon. Before the eclipse began students started peaking at the moon through the telescope. Many of them where as delighted as I was with my first telescopic glimpse of the moon. I remember some of the older, and cooler, students had to reign in their obvious excitement. As the Earth’s shadow touched the moon people in small villages around the school started pounding drums. As the shadow crept further and further the drumming got louder and louder and bonfires started popping up all around the school.  I didn’t expect this reaction. It was the best damn star party I’ve ever attended. The eclipse was a good one too. At totality the moon was a deep dark blood-red.  John took advantage of eclipses, nature’s astronomical advertising, to show even more people the greatest show off earth.

8. Annular Solar Eclipse: Syracuse New York United States. 1994 was the year of Shoemaker–Levy 9: the fragmented comet that smashed into Jupiter with such awesome energy that it blinded sensors on large telescopes and left massive bruises that could be seen in small telescopes. Shoemaker-Levy 9 was a rare major event. We were lucky to see such an impact in our lifetimes, but the 1994 event that sticks in my head was the annular eclipse of that year. Annular eclipses are, according to eclipse snobs, failed total eclipses. The moon is too far away to precisely match the angular size the sun so at totality an annular looks like perfect super bright ring in the sky.  It’s a rare celestial event that fits into your work day but the 1994 annular eclipse did just that. Totality occurred during lunch hour and many of my coworkers and strangers on Syracuse sidewalks paused to look through eclipse shades and welding glass at the one ring to bind them all. It was Dobsonian sidewalk astronomy at its finest.

9. Glimpsing the Gegenschein: Grand Teton National Park, United States.  Seeing the gegenschein requires very dark and clear skies. The tiniest hint of light pollution will wash it out. I’d been observing for years before I saw it. I was south of Yellowstone Park looking directly north. There are no large cities directly north of Yellowstone for hundreds of miles and the park is blissfully black. At 3:00am I noticed a slight glowing that steady increased in brightness. Seeing was superb. I could see 7th magnitude stars with averted naked eye vision. I was alone, in the cold, in the dark. Not John’s style but this was a personal first. Sometimes the sky is all you need.

10. Total Solar Eclipse: Zambia Africa. Total solar eclipses are beyond awesome. They utterly wowed our ancient ancestors and still blow away the most jaded and media saturated people today. You have to put yourself in the moon’s shadow and give yourself to the spectacle. It’s worth spending thousands of dollars and going to the ends of the Earth to stand in the moon’s shadow. Of all the wonderful and spectacular things I have seen I’d rate my first total solar eclipse above them all. Of all the planets, moons and other bodies in our solar system only the Earth enjoys pure total solar eclipses. By some freak cosmic accident the moon and sun are almost exactly the same angular size in our sky. It’s only when I’m standing in the moon’s shadow, during a total solar eclipse that I don’t mind being trapped on Earth.

Looking over my list it’s pretty clear that John Dobson had the right idea. The things that stuck were first glances through telescopes and watching special things in the sky with my own eyes. I have spent many long nights tracking objects down in telescopes but after a few years such “serious” sessions blend together. It’s those fleeting first lights that dig in and change how you feel. John Dobson changed how many people feel about the sky. Nice work John.

[1] The best dark sky sites in Texas are in the Davis Mountains near the McDonald Observatory. The Texas Star Party is held nearby every year.

# The Great Verizon Data Famine

The other day I visited my local Verizon store for the fourth freaking time! My mission was simple: upgrade my goddamn phone and change our account from my wife’s name to mine. In sane retail environments long-standing customers with impeccable payment histories get treated like royalty. I know it will come as a shock to all you parasitic socialists out there but it is the paying customer, and only the paying customer, that is keeping civilization’s lights on! I understand and appreciate the need for businesses to make profits and for the last three years Verizon has profited from my patronage and I have benefited from their excellent cell service. We had a mutually beneficial relationship but now I’m wondering if this marriage can be saved.

I have no technical complaints about Verizon. The engineers at Verizon clearly know what they are doing but it looks like the administrative and sales division’s model themselves on the DMV or Obama’s healthcrap.gov. I’ve seen this before. Most software companies harbor competent to brilliant programmers yet are often fronted by ethically challenged sales baboons. My father, a retired petroleum engineer, used to say, “It’s a good thing oil is so valuable and customers are beating down our doors because head office couldn’t sell shit to a house fly.” I know it’s not my place, as a motivated shit seeking house fly, to question the sales practices of multi-billion dollar enterprises but, to quote a very wise old white guy, “you’ve confused me with someone who gives a crap.

When I first walked into the Verizon store I wanted an accurate answer to this question:

How much will my monthly bill be if:

1. I pay the full retail cost of the phone upfront. Old white guys do not buy on credit because old white guys have learned the hard way that buying anything on credit means you eventually pay more. I am not interested in paying more. I have a very bad attitude when it comes to paying more. My butthole has been reamed often enough, long enough and hard enough that it’s now operating on a strict cash upfront basis.
2. And if I have an uncapped 4G data plan. Cell providers constantly go on about their unlimited data plans yet down in the fine print — old white guys always read the fine print — you typically see “limited to two gigabytes per month.” Two gigabytes is not unlimited, four gigabytes is not unlimited, fifty yottabytes is not unlimited; unlimited means arbitrarily high.

It took two trips to the same store to get a simple price quote. The quoted rate was $69.99 per month. This is close to my current rate and since I’m burning another$39.99 per month on Internet cable it looked like I could cancel cable, divert all my residential Internet traffic through an iPhone 4G hot spot and save about thirty bucks a month.

I realized I would have to go on a data diet. 4G connections are faster than 3G but 4G is still much slower than cable Internet. The cable provider in St. Louis, Charter1, runs at 30 megabits per second. This is about five times faster than 4G. 4G is okay for blogging, modest sub-gigabyte downloads, uploading a few dozen high-resolution pictures and normal web browsing. 4G is not up to irritant free HD streaming. You can stream but the image is often downgraded to a blocky low resolution mess. I planned on giving up streaming because TV, whether broadcast or streamed, is still mostly time-wasting garbage. I was looking forward to reallocating my streaming time to good old-fashioned paper2 book reading.

After doing my research, considering the options and allocating funds I returned to the same Verizon store I had visited three times with the intention of plunking down the full cost of an iPhone 5s and signing another two-year service contract at the price I was previously quoted. Then things went horribly wrong. First, we had to call my wife to change the name on our account from hers to mine. The simple act of changing the account name voided my unlimited data. I went from an uncapped plan to a two gigabyte plan. Then, as a final affront, it turns out that you if you actually want to use your iPhone’s hot spot you need to pay another $30.00 per month on top of your normal data plan. In other words my bill would be a few cents shy of$100.00 per month. So, I would pay roughly the same as my current Verizon and Charter bills combined and end up with a connection that is five times slower. Old white guys are slow and stupid but not that stupid.

Instead of walking out of the store with a shiny new iPhone 5s and another two-year contract I left with my old iPhone 4 and a downgraded, but equally expensive data plan. I am now looking at other options. I will probably retain cable and cut off all cell phone data. Most of my cell phone data moves over Wi-Fi so why pay Verizon, or another provider, \$30.00 bucks per month to keep up on Twitter tripe. Verizon’s sales did a bang up job here. They convinced a loyal and reasonably happy customer that it’s time to take a serious look at the competition. I was planning on a data diet but not a data famine. Can you hear me now!

1. Charter Internet is a binary operation. When it’s working it works very well, but over that last three years I’ve watched it go down more often than a cheap street prostitute. Right now it’s down. Charter outages are annoying but they’re usually quickly resolved.
2. EBooks are developing nasty data mining habits. I have no desire to expose the precise details of my reading too busy bodies. This doesn’t mean I am giving up on eBooks but I am giving up on on-line eBooks. I now demand complete control of the eBook file on a device that I can shut off logging and communication. If you don’t control it you cannot trust it.

# Blogging Bad 2013

Another year of blogging bad. My mother’s death and work were major distractions this year; I fell way short of my post goals but still managed to exceed the previous year’s hit count and set a new high.  To show their appreciation the good algorithms at WordPress.com sent us an annual report. Mine follows:

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 11,000 times in 2013. If it were a concert at Sydney Opera House, it would take about 4 sold-out performances for that many people to see it.

Click here to see the complete report.

# APL Software Archaeology .dbi Edition

Have yourself a merry little APL Christmas.

I joke that my job title should be software archaeologist because I often find myself resurrecting, not refactoring, code that dates to primitive and primeval eras. The language I’m typically hired to resurrect is APL. APL, the language with funny symbols, is a software vampire. People keep paying us to kill it but no matter how many stakes we pound through its heart it keeps coming back.

There are good reasons for this. APL embodies many timeless ideas and I’m confident that programming in the future will look a lot more like APL than many expect. If you doubt me just press the Siri button on your iPhone and ask, “Integrate X squared times sine X from 0 to 2.” What comes back has more of an APL than QWERTYUIOP flavor. Strange Unicode characters are creeping into many mainstream languages. This is a good thing because restricting programming to the miserly key sets of ancient typewriters was, is, and always will be a spectacularly bad idea. Ken Iverson deserves rich accolades for pointing this out more than fifty years ago and beating this drum incessantly during his lifetime. Iverson taught that notation is a tool of thought and that if you care about ideas you must care about how they are expressed. Why is this even remotely controversial?

Siri’s results use appropriate mathematical notations. As we move away from keyboards programming languages and mathematical notation will merge. APL was way ahead of its time in this respect.

The genius of APL continues to exert influence on many programming languages but APL’s rise had little to do with its abstract notation and a lot to do with how it was implemented. APL was one of the first programming environments that nonprogrammers could use. It was the spreadsheet of the late 1960’s and 1970’s and just like spreadsheets of today a lot of utterly horrid, poorly structured, lame amateur messes were created with it. If you’ve ever cracked open a gigantic Excel model that looks like it was developed by a roomful of quarreling ADHD afflicted unionized chimpanzees then you know what the standard APL mess feels like. Many programmers blamed APL for this just like gun control advocates blame firearms for shootings. They argued that it would have been impossible to concoct such monsters in clean compiled languages like Pascal. “It wouldn’t even compile.” This is not even wrong. I’ve dealt with plenty of dreadful messes that do compile! The tool is always neutral; don’t blame the paintbrush for the painting.

Allowing rubes to code yields mountains of rubbish and the occasional ruby. It will shock many programmers to learn they are not the only smart people in the world. It turns out that nonprogrammers occasionally have good ideas and, miraculously, some of them can ably express their ideas in code. Before spreadsheets such user rubies congealed in APL where some still run. Part of my day job is extracting these precious stones from layers and layers of kluges, hacks, patch jobs, retro-fits and workarounds and recoding them in modern programming languages like C# and JavaScript.

Recently I recovered1 an ancient inverted file system embedded in the APL systems of my employer and rendered it in C#. This system uses the extension .dbi. I don’t know who created this system; the code is old. The most recent code comments date from the year 2000 but I am pretty sure that .dbi files predate component files in APL+WIN, formerly STSC APL, which pushes the design back to the 1980’s or earlier. I know many APL’ers check this blog. If any of you know who created the original .dbi APL code please leave a note.

Somehow this .dbi system survived unsupported, with few user complaints, for decades of daily use. How is this possible? Astonishingly, good ideas age well and the core .dbi idea is inverted data. Modern high performance databases make heavy use of this method. Inversion is so effective that hoary old interpreted APL code still beats compiled and optimized ADO.Net when fetching large numeric vectors and tables.

Restoring the .dbi system was a two-step process.2 I first converted the APL system to J. I used J because it is a close relative of APL but not so close that you can cut and paste. Translating nontrivial APL to J forces you to understand the APL at the nit-bitty level. The translation to J also allowed me to fix the APL interface. The original system used global variables, rampant branches and other lamentable coding practices that C# will not abide. After matching the APL and J systems I then translated the J to C# and then rematched all three systems.

Comparing multiple systems is a very effective testing technique. I found bugs in all three systems. I fixed the J and C# bugs but left the original APL code unchanged. Software archaeology is a delicate field. You don’t “fix” old code just like you don’t correct errors in cuneiform tablets. Original and important program code belongs in museums with other significant cultural artifacts.

Original inverted file code probably belongs in a museum. This .dbi APL code is old but it certainly derives from earlier programs so it’s not museum worthy. Even if it was the APL and C# .dbi systems belong to my employer. However, I am placing the J scaffold version, which matches the performance of the other systems, into the public domain. The script is available on GitHub and here. The .dbi system gets right down to bits in some cases and illustrates some J techniques for dealing with indexed binary inverted file data. Enjoy!

1.  .dbi files held many gigabytes of actuarially tuned data. Dumping them was not an option. We either had to convert to a new store or produce a component that could read old data in new systems.
2. Restoring old code is somewhat like restoring old pictures. When working on old pictures you’re always tempted to improve them. With pictures you usually have a choice. This may not hold for old code. Changes in software may force updates.