Taming the Data Horde

March 6, 2015

By Dennis Kneale

 

For decades data storage has been a control freak’s dream: Data are “structured” and consistent, dutifully tucked away in disk-based systems that are prodigious in capacity but cantankerous and inflexible.

Now comes a data deluge that will make control freaks… freak. A new wave of “unstructured” data is upon us, like one monumental bad hair day, ornery, unmanageable, inconsistent and flighty. The market for this messy mass could grow almost seven-fold to $20 billion in the next few years.

That’s a fat, juicy target for a platoon of private firms presenting at the Montgomery Summit, sponsored by Macquarie Capital and set for Tuesday and Wednesday, March 10-11, at the Fairmont Miramar hotel in Santa Monica, Calif. I’ll be there as MC.

CoreOS, Bracket Computing, PernixData, Nintex and others at Monty ply new techniques in virtualization, generic hardware, cloud computing, flash and cross-platform software to let large enterprises become Jack-be-nimbles as their data-streams swell up with unpredictable, hard-to-tame data.

For two decades, structured data dominated storage; it now is at $13 billion of a $16 billion market, vs. $3 billion for “unstructured,” says Ramana Jonnala, CEO of Coho Data, another presenter at Monty. “But in the next three years, there will be an explosion in unstructured data,” he says. By 2018, that segment could hit $20 billion vs. $16 billion for structured.

The onslaught emanates from smartphones, tablets, servers, Net sensors, apps, wi-fi hotspots and more. Structured data tells a retailer how many pairs of sneakers it sold in California last month; unstructured data, late-breaking and fast-changing, can reveal how many pairs it might sell next month. Harnessing those insights will require new tools.

Coho Data sells an “appliance” to help manage the data morass and turn it into an advantage: $90,000 for a “whitebox” server with 50 terabytes of storage (flash and disk). Companies can keep their critical data close to home and form private clouds, rather than using, say, Amazon’s S3 “public” cloud at 3 cents a month per gigabyte. “Just the assumption that everything belongs in the public cloud is actually a lazy notion,” Jonnala avers.

More than 40 customers, from Wall Street banks to healthcare companies, have bought his DataStream boxes in eight months on the market. A big data site can entail three petabytes of storage (3,000 terabytes or 3 million gigabytes). “In our case that translates to a hundred boxes with our solution,” he says. “The beauty is the customer doesn’t need to buy all three petabytes on Day One. He can take 500 terabytes, and adding the next 500 terabytes is dead simple.” It takes only an hour. “Traditional storage can’t do that.”

While Coho relies on hardware, another Monty presenter, PernixData, bets on software. “It’s easy to go say ‘Take my shiny box, someone else has!’ but enterprises need to realize the power is not in that box—hardware they can get anywhere—it’s in the software,” says CEO Pooja Kumar.

Pernix uses crafty software to help storage play catch-up. Storage has fallen behind the advances in power, virtualization software, smartphones and apps. “The EMC’s of the world haven’t been able to keep up with the demands of the virtual infrastructure on the compute side,” Kumar says.

Pernix software enlists servers to handle critical and urgent storage tasks using flash memory right there inside the box, instead of an EMC disk nearby. “We are disrupting the traditional way of thinking, where I need to solve everything on the storage side. No!” Kumar says.

Flash, so aptly named, can process commands a hundred times as fast as disk storage: 200 I/O commands per second for disk, 20-thousand I/Os per second for flash. Its ubiquitous use in smartphones and tablets has raised its scale and power so much that flash now is moving into the enterprise.

That also owes to the “hyperscale” giants: Google, Amazon, Facebook, et al. They did the impossible, building out massive networks of cheap, homemade servers that run custom software and use flash for superfast access. A corporate network links 10,000 servers; the hyperscalers harness one million, each one 100 times as powerful as a 1990s mainframe. Unbelievable.

The overhaulers at Monty want to bring hyperscale to businesses. “That is a big river of change in the enterprise,” says Rajesh Ghai, senior analyst at summit presenting partner Macquarie. “A lot of hyperscale concepts are going to make it to the enterprise, and that what’s driving investment.”

The trickle-down from Google to the rest of the business world “is changing everything,” says Tom Gillis, CEO of Bracket Computing, also a Monty presenter. “Guess what? Who’s the third-largest server manufacturer in the world? Google, for its own use. That’s an astounding statistic, right?”“We’re helping companies run infrastructure like the big guys,” says Alex Polvi, co-founder and CEO of CoreOS. Its Linux-based software, which debuted last summer, orchestrates batches of servers, from as few as three to hundreds of thousands. One customer testing CoreOS was able to run 100,000 servers at once without any glitches.

Bracket provides a layer of software between enterprises and the cloud, bulletproofing security and managing data flows. “We are like an operating system that runs on top of the cloud,” he says, reluctantly, because “a bunch of people have made that claim, and it’s been bogus.” Bracket launched in October, and DirecTV, GE and Blackstone have been testing its software. Some clients are “rolling it out bigtime,” he says.

The mission at Nintex, another overhaul wizard at Montgomery, is to help businesses tame the mass of unstructured content by automating the impossible. “We automate everyday workflows that haven’t been able to be automated in the past,” says CEO and co-founder John Burton.

Nintex, with more than $50 million in sales, provides a software platform that lets businesses create their own automated processes with surprising ease. In business since 2007, it’s now in 5,000 customer sites with nine million users in 90 countries, in 23 languages.

The average first-time sale for Nintex is $25,000, plus up to $75,000 for partner resellers to install and implement it, which takes only two to four weeks. Each automated workflow shaves 66% off the cost of the paper-based method, and customers earn back their money in six to nine months.

Astoundingly, some Nintex customers have used it to automate 14,000 different paperwork processes, from new-hire “onboarding” to ad-budget authorization to travel approvals. I think it best to withhold comment here on whether any business rightly should even have 14,000 distinct work-flow processes . . . that is a different problem.