Cubegeek

On MDX, Briefly

I've worked with MSAS a bit and I've had engineers who worked for me tell me things that confirm my brief experiences as I got further from the technology. 

Most people who get up the rather steep curve for MDX admire it for its elegance and prefer it to SQL on that basis. Nevertheless the experience is that it is generally not worth it to learn the language if you have appreciable experience in SQL. 

If you have become something of an expert in tuning databases in the generations of products before the Greenplum and Vertica days, say with Essbase, Microstrategy, Teradata, Oracle Express or Sybase ASE, then you will have some familiarity with arcane 4G languages and dialects of SQL without many cross-product similarities. MDX was the language destined to solve all that, a sort of OLAP Esperanto. Unfortunately, the number of programmers deciding to hack SQL to perform against this and that sort of schema tended to dominate the few cross-platform specialists. In the end, it is my opinion that the lack of a predominating visualization stack obviated the need for widespread MDX adoption. As fat visual programming OLAP clients matured, enterprise customers began demanding thin clients. As full featured stacks became available, developers wanted LAMP, and so on. Now we are met on a battlefield testing whether PHP and MySQL will long endure, with people like me wondering how they ever got started considering the maturity and performance of products like Sybase and Essbase.

My direct experience is that flatly, for applications of any sophistication, on MSAS, stored procedures with T-SQL always performed better than their functional equivalents in MDX. In their aborted product stack, Performance Point, Microsoft engineers created a middle language PEL(?) that would supposedly choose which language was best suited for a task and then generate that code. All jokes about code-generators aside, my engineers always had to second guess PEL and ended up writing it all in T-SQL. So as a practical matter, there was no sense in learning MDX if you had already mastered T-SQL on the platform for which MDX was specifically designed. It would have been nice if MDX performed with speed commensurate with its elegance, but it simply didn't. This was two years ago. 

I know MDX lives on in Essbase but that Essbase expresses it differently than does MSAS. It does rather boil down to 'it depends', because language to language different platforms have different strengths, etc. 

I expect that we will not learn the definitive answer to this question until there is a shakeout of DBs that survive the transition to cloud infrastructure. And in that regard I think a Greenplum or a Vertica or a Hadoop-based solution will win out. In other words it won't come down to the semantic layer. The market never forced it to because there is no real OLAP standard (chicken or egg?). 

In the end I say MDX iff you love MDX.

The OODA Loop of Essbase Six

I have a confession. I still love Essbase Six. It's got 80% of the goodies of the latest Essbases and can be put together with 20% of the effort. Remember native users? Remember Application Manager? Remember cutting and pasting from Excel to the outline? Ahh for the good old days.

Well the good old days are never coming back, but for those of you fortunate enough to have an old copy of Essbase Six lying around there is some good news. Just like SQL Server Express, it's a great prototyping tool and you can still use it on XP. 

Since I've been around several blocks, including what is in retrospect, an ill-advised trip over the the Microsoft dark side of BI, I've had a chance to hear a lot of opinion about Oracle, Essbase and Hyperion. One that sticks with me is the complaint of one well-respected chap that the Oracle stack has just gotten too bloody thick. There are all sorts of implications both good and bad that I won't belabor here except for one and that is speed. 

If you have studied military strategy on any level, sooner or later you will come around to the genius of John Boyd. Boyd's theory of aircraft combat boils down to this, if you can execute faster than the enemy, then you automatically outhink him. So air superiority in dogfights goes to the pilot who can turn quicker.

I have found that with Essbase Six in your toolkit, you can accelerate your development times in Planning and EPMA. Just do a lot of mockup using the old school skills during design and then translate them when it's build time into the newer stack.  It's a bit much to carry around on a laptop, but it can be done. Whereas an installation of EPM 11 is simply not going to fit very well at all.

Speed may seem like the enemy to the biller of time and materials, but working with a model that can be tweaked 30 times a week delivers a great deal more quality in the end than one that can only be changed 5 times a week. Your mileage may vary, void where prohibited, see Oracle for details of support for Essbase Six..

JAAO - Good Stuff

Just Another Analytic Opinion? Good stuff on a new BI blog from a co-conspirator. Conspirator in what you say? Cloud theory for BI and EPM. He's done an infrastructure build on AWS and walks you through it. Why didn't I think of that?

Also, some interesting stories about conflict with IT and finance:

During the near end of the project I realized there had been a difference of opinion between IT and Finance on the product of choice, way before I started as the project lead. With the IT manager saying, 'this project would be so much more simpler with just a simple SQL db and reporting'.

Yike.

We're Not Worthy

I'm almost completely around the bend for Google. 

I'm excited about the new stuff Google is kicking out this way to us consumers. I got my invitation to Google Wave, and even though I thought the demo was insipid, I'm getting some new interesting ideas about the value of flat organizations and collaborations out here in peasant territory. So Google is our friend because they deliver applications for infinite scale, which makes it so easy for us to expand control of our own properties - since Google gives us more than we might generally merit in a more vertically oriented world.

But what really has my knickers in a twist is Google's facial recognition that it has embedded in Picasa 3.5. And even though I'm signing up for Photoshop.com because it integrates Picasa, Facebook and Flickr, the first really useful thing I'm excited about is Google's face tagging. Seeing as I have some 60GB or so of snapshots, I love the idea that I can pick a few pictures of my kids and Google will go through and match up the other thousands for me. We'll see how well it works, but on a small set it has done pretty well. 

Well, since I started writing this four days ago, I've had a lot more experience with the new Picasa, and I have to say that it is successful to a creepy degree. My daughters are in a theater troupe and so we have lots of pictures of lots of kids at lots of performances, offstage etc. It's one thing to say I know three or four of the kids my kids hang out with, it's another to have software that has identified 80 snapshots of them. With a little help from my daughter peering over my shoulder, I've got the faces hooked up with the names. I feel like a CIA analyst. But now I know what a CIA analyst must have. It's obviously not as good as human face recognition, however, combined with a human confirming or denying its recommendations, Picasa is a very powerful tool. 

The only thing Google has done to tweak me wrongly is to discontinue the Google Notebook extension for Firefox, and the service itself for new subscribers. I found it a brilliant way to clip information into a very nice organizational form. They are keeping the service alive, but if they abandon it, I think they should GPL the code and leave it for somebody else. 

I say this because the other thing that I wanted so very much from Google was in fact a 'Google OS'. As soon as I heard about BigTable, I began following that thing which I believe will put my Oracle consulting business at a severe disadvantage, which would be not MySQL but MyDW. But Google was not interested in publishing the 'Google OS'. What we got was in fact better. We got what Cloudera has come to be, the Redhat of Hadoop. All open source stuff for building the ultimate file system.

Google's philosophy of thinking at scale has delivered an entire new kind of computing over the years that I never imagined when people first started talking about grid computing. I expected something like IBM Regattas and Veritas servers with all the approachability of supercomputers. It's nice to be wrong.

Tiptoeing Back From and Towards the Cloud

Well, right about now was the time when I expected to be speaking in tongues to the Oracle Open World. Unfortunately, my proposal didn't get a bite. It was high faluting sounding, but I think they saw through me. It was about Essbase in the Hybrid Cloud, and what it was basically about was using cloud-based storage to inch oh so gingerly into the cloud arena. Now that I've had a bit more time to think about it, I think I know better.

You have to jump in with both feet. At least, that's my attitude.

But first some geekery updates.

I'm backing off of S3 for my own personal use. Yes I have nice fat FIOS bandwidth pipes but I have determined that offsite storage is basically only good for backup & disaster recovery, not file sharing large media libraries with Samba or Windows file shares. So I'm coming down off my Jungle Disk high and transferring all my big stuff back to the new generation of Seagate external hard drives. Now there may still be a chance that I'll experiment with a PogoPlug at some point, but really, what do I want that for? I want it for music when I'm on the road. It's cheaper to buy the drive, or just use the damned iPhone. (or Orb, or Last.fm)

What has turned out to be much more useful than I ever imagined was Dropbox. It has basically put all of my flash drives out of commission. And all I ever really needed was 2GB. Whoda thunk?

I let my Carbonite subscription lapse although I hadn't intended it to. I gave my old machine over to my wife and it just wasn't worth keeping all that stuff there. I moved it over to my new big 1.5TB external and I'll back it up from there. How? It just turns out that I like Backblaze and they allow you to do just that, whereas I seem to recall that Carbonite didn't like mounted drives very much. For 5 bucks a month per computer, I'm digging the pricing model and I like the fact that they're open source.

In other geekery, I finally went the next step and cranked up an EC2 over at Amazon. It was fairly easy to do, but a lot more expensive than I thought it would be. But I'm starting to see the possibilities, which is why now more than ever I think that some of us service providers *are* the new next big thing. Which is to say, the first guys to master the art of hosting enterprise apps and demonstrate super scalability and truly rapid RAD wins big.

Suddenly, I feel like I just might become a hot commodity again because of what I did back in 2005 on that 32 way 64bit Superdome with 500 concurrent ASO cubes. However... I think between that and the 'cloud' are some very significant virtualization issues. Figuring those out is the next step.

You see, even though AWS was telling me my virtual box was 'Large' it certainly wasn't 'Fast'. Getting my favorite marketing database up into SQL 2005 took a lot longer than I expected it to, as in I seem to recall things moving faster on my laptop. So there is an I/O bottleneck getting things into the cloud if you're way down the pipe, as I am most of the time. There's got to be a better way - which is why I'm thinking about the trickly stuff that's built into the Carbonite and Backblaze products, made more transparent by Jungle Disk, and very likely available at a low level with Nirvanix (which I haven't been using at all).

So uptime is costly on EC2, whether or not your box is processing anything. It's the availability of the instance that you pay for. So my next trick is to learn how to park a big old virtual server image so it actually becomes something of an on-demand machine in the way I thought it was. There's a great big financial step from LAMP infrastructure providers like Dreamhost, whom I've been using for about 7 years, to Amazon's EC2. 

I'm also becoming convinced that a trickle server is necessary - just a dedicated machine to Jungle Disk or Nirvanix the required data up to S3 and then trigger something when that job is done. When I use it on a regular working PC, it just gets all bogged down, but part of my problem was that I was doing backups and file transfers intermittently, and I'd have the thing shutting down, sleeping or timing out. What you need, I think, is a machine that just never times out and does nothing but fill your pipe with upstream. An upload server. It shouldn't be so difficult, I should think, to link your S3 to your EC2 and then voila. That is an exercise for me.

--

In other news, I'm becoming basically competent in Planning after many years of barbed remarks. The thing actually works as advertised in 9.3.1. Of course it stands to reason that I'm pushed to the bleeding edge of the very latest feature set and as such will be getting a long luxuriating look at 11.1.1.3. Obviously there are little things that I'm discovering that people are saying 'Duh' to me. But I swear to God I have not had any customers that bothered with Smartview in X years, and I never got to be an expert. Now I see that it does all the things Joe Cajic said it would do back in 2004. Yay. 

Naturally, I jumped at the chance to play around with the OutlineLoader and dropped various parts of that into my Cubegeek's Toolkit. So I'll soon have scripts that generate scripts. And I suspect that I'll try to find ways to automate the dread parts of EMPA as well, time allowing.

Sometime next spring, and certainly by KScope next, I'll have mastered some small portion of the latest Planning scripty bits and present my modest results, board willing. It would be awfully nice to have one of those big ER diagram posters of the Planning repository somewhere wouldn't it? Heh. Listen to me, thinking I might do that HAB kind of development. Is that product still around?

--

Well that's all for now. I'm energized and optimistic.

AT&T, You're Killing Me

Here's an amazing story. My iPhone was updated today so that the MMS feature (picture messaging) will work. We waited until today because even though the iPhone could do it for months now, AT&T couldn't handle the traffic.

So I upgrade.

It has been about 15 minutes since I rebooted my iPhone and I am still searching for an AT&T 3G signal. Now to show you how crazy that is, I have synched the phone and I have sent and received MMS messages, through my wifi connection. I'm going to reboot my phone again and see if I can get an AT&T signal. Any AT&T signal at all.

12:29 - powered down

12:30 - hit power up button.

12:31 - fully booted. (Searching...)

12:31 - sync in progress (Searching...)

12:32 - sync complete. (Searching..) phone goes to sleep.

12:33 - i go to settings and put it in airplane mode and turn it back on.

Aha! There's AT&T. It took you slackers long enough.

So here's your evidence. A picture of my iPhone taken with my wife's Verizon phone. It shows clearly a picture of me taken with my own iPhone and that my phone is still searching for AT&T but connected to wifi.

So the first use of my newly enabled but already functional MMS software goes through my iPhone not through AT&T, but through my Verizon FIOS wifi, to my wife's Verizon phone and then back to me from her phone through my wifi, before I could even get an AT&T signal.

My world, delivered. Like a 10 month pregnancy.

Some Scalability Tips

Artlessly stolen from the High Scalability Blog:

AFK Partners has release what they feel are the Best Practices for Scalability:
1. Asynchronous - Use asynchronous communication when possible.
2. Swim Lanes – Create fault isolated “swim lanes” of hardware by customer segmentation.
3. Cache - Make use of cache at multiple layers.
4. Monitoring - Understand your application’s performance from a customer’s perspective.
5. Replication - Replicate databases for recovery as well as to off load reads to multiple instances.
6. Sharding - Split the application and databases by service and / or by customer using a modulus.
7. Use Few RDBMS Features – Use the OLTP database as a persistent storage device as much as possible.
8. Slow Roll – Roll out new code versions slowly, to a small subset of your servers without bringing the entire site down.
9. Load & Performance Testing – Test the performance of the application version before it goes into production.
10. Capacity Planning / Scalability Summits – Know how much capacity you have on all tiers and services in your system.
11. Rollback – Always have the ability to rollback a code release.
12. Root Cause Analysis - Ensure you have a learning culture that is evident by utilizing Root Cause Analysis to find and fix the real cause of issues.
13. Quality From The Beginning – Quality can’t be tested into a product, it must be designed in from the beginning.

Waterfall Process Mapping

I found this perusing through visualization blogs. This particular method is used generically to measure webpage download performance, but it could measure any number of processes. The 'Waterfall Chart' is a cool way to visualize parallelization bottlenecks, and simple too. I don't know why Excel sorts the chart backwards, but..you'll find it under custom charts as 'Floating Bars'.

Obviously, you can tag each process multidimensionally depending how you want to go through attribute tagging. The metrics for this example is simple. However you could also consider a multistage process as a contiguous stackbar. 

In this example, Process 9 is the bottleneck as no other process can start before it finishes, and the visual clue is very easy to see with this sort by start time. 

To create this data would be simple, given some conversion into UNIX time, or seconds since x, indexed to an arbitary start date of the first process. 

News of the Geek

I didn't realize that there was an app for Google Voice on the iPhone. Alas, it's too late for the Mafia powers that be have whacked it from the iTunes Music Store, which now no longer has the privilege of being known as simply iTunes because it's now all iPolitical and starting to smell like iCrap. But my abusive relationship continues and I'm still buying more iCrap. Is this how you humanize a corporation, make it into a love-hate relationship with their products and services?

It's clear that I love the latest vapor from iCorp, which everybody has pretty much decided is an iTouch on steroids. Considering how slim the normal iTouch is as compare to the iPhone, a really slim iTablet would rock on so many different levels. But more about them later.

I did load up on a couple new apps, including Dopplr, which I think only marginally better than Yelp. So little in fact that I don't use it. Nevertheless, Google Locator is even more worthless to me at the moment, so maybe Dopplr gets the shruggish nod. As to saying where I am, when I want to reveal that, Tweetdeck is the current winner. Still, if I haven't said it already, I'd like a Seesmic for the iPhone.

NY Nick has come through once again. This time he hipped me to a new program called TeraCopy. It is a non-schizophrenic copy program that actually does good estimates of how long it's going to take to copy large amounts of data from one drive to another, unlike the facility native to XP. The drag & drop nature of it is a little un-intuitive but you get used to it.

--

I'm preparing myself for some Planning immersion coming this fall. It's going to be interesting and fun, I think. Planning is about all that's left of poor old Essbase. I've been reminiscing about the good old days at Hyperion eCRM in Foster City and the twisted path of that thing that became Siebel Analytics over the dead bodies of the wonderful products we built there. Now it's OBIEE which is killing off my other old buddy Wired for OLAP, which emerged triumphant through the thick and thin client implementations back in the days when nobody could decide which was better.

iTablet Once Again

It turns out that there have been rumors about the 'iTablet' for years now. There have been a recent spate I caught this week from the Cranky Geeks. They report the rumor and then dis the product idea. I think they are being paid to shutup and say stupid things, like they did about the video iPod. The dimensions of the product?  A $800 machine with a 9.7 inch screen coming in October.

A DS style deal with two screens that turns into a two-page reader is a spectacular idea. An oversized iTouch is a stupid idea. But note how the rumor has the screen at 9.7 inches. That's the exact same size as the Kindle DX. My speculation has been that Apple wants a Kindle killer and I expect that Jobs will want to do with books and texts of all sorts what he has done with music, which is boldly forge alignments that change the industry forever.

Everything that Google has done, somewhat, and failed to do in convincing various publishers to go digital can be done with an Apple/Amazon alliance. Or Apple could do it alone. All you have to do is imagine is Jobs saying with major book publishers what he said with the major music industry players when the iTunes store was first announced. It can be that big.

An iReader with multitouch is reasonably priced at 800 for gearhead first adopters like me, and can come down to Kindle prices next Christmas.

I don't know how anyone can not imagine Apple building a sexier device with multitouch and text to speech that blows the doors off of a Kindle DX. That's all this product needs to do. The problem will be, of course, the liklihood that Apple will overload the iTunes Store. It needs to be a different kind of store, if you ask me, and it needs to be a touch more open so that booksellers can get on board.

If anybody at Apple is as visionary as I am, then they will see a market for 'book developers' which is at least as large as the blogosphere, and they will find a way to monetize writing and lower barriers to self-publication in the same way they have generated a new class of software developers.If they added a bit more brains to that vision, then they would do speech recognition in a devkit that would provide for transcribed podcasts.