Igor Bogachev, executive director of the Information Technologies (IT) Cluster, Skolkovo Foundation, put it this way, “The volume of data in corporate business grows by 30 to 40 percent every year according to different sources. The Big Data problem has been already in the room for some time. We in Skolkovo do not neglect this major trend and give special support to the startups working in this field. The goal is to build the Big Data co-innovation sub-cluster in Skolkovo with the companies like Talksum and other residents. I trust that Talksum is one of the companies that will change the future landscape of the IT industry.”

The Foundation is overseeing the creation of the Skolkovo Innovation Center composed of companies and start-ups, developing innovative technologies (currently numbering almost 1000), a Technopark, the Skolkovo Institute of Technology (Skoltech), a new graduate research University established in collaboration with the Massachusetts Institute of Technology, and Skolkovo city, located near Moscow. Together these entities will establish a vibrant eco-system of technology innovation and entrepreneurship.

Click here to see the entire Skolkovo press release.

First, I’d like to thank all of you who attended our presentation, “A New Approach to Data Management.” We are excited about the future of Big Data and the opportunity we had to share our vision with you, in particular our approach to data management and analytics, which focuses on speed, simplicity, and cost efficiency. Big Data initiatives need more than just new storage platforms and BI solutions. They need to be real time to keep pace with data demands, simple to avoid specialized skills and custom codes, and cost effective to maintain low overhead and offer greater value.

Those are the ingredients built into the Talksum Data Stream Router. For those who missed our presentation, we showed our solution in our booth and discussed various use cases, including those in service delivery, industrial and automotive sensors, financial tracking and analytics, healthcare HL7 compliance, and biogenetics research.

If you didn’t make it to the Summit and would like a copy of the presentation, fill out the Talksum Contact Us form and request the presentation in the “Nature of Interest” box.

At the Summit, it was also fun to listen and discuss innovation with other experts in the field, including those from CERN, Hortonworks, BT, Pfizer, Credit Suisse, Merrill Lynch, and many others.

We are already looking forward to the next Summit!

At the event, which takes place at the Lancaster London Hotel, we will be sharing details about the recent implementation of the Talksum Data Stream solution. In addition to our booth, Talksum CTO Dale Russell will be presenting in the “Big Data Innovation” track at 3 pm on April 30 about the following:

A New Approach to Data Management

In the presentation, Dale will describe how the Talksum Data Stream solution takes a new approach to data management and analytics. Designed to accelerate real-time decisions, the Talksum Data Stream manages data acquisition and transformation, and converts data into flexibly managed event streams. With the ability to output to most external data stores, the Talksum Data Stream can process, filter, and aggregate data in real time, as well as enrich it by correlating events with other external data sources.

We are excited to be a part of the Big Data Innovation Summit and hope to see you there!

This is a big milestone for any young company, and it’s the product of a whole lot of hard work. While we’ve been working with several beta customers on refining requirements and performing POC’s in our research environment, the Mail.ru implementation represents our first customer install.

Mail.ru is using their Talksum Data Stream to aggregate and analyze network traffic data in real time. We look forward to sharing more details about the implementation in a more formal case study in the future, but for now we’re simply excited to say that we’re up and running with our first on-premise implementation.

A very good friend of mine, Pierson Washer (Mercedes repair shop owner and novice AFM racer #983) asked if we could get the same racing telemetry as MotoGP from his SV-650 “Super Dinosaur” class racing bike. He was trying to give me a hard time thinking I would respond with “It’s not that simple, you first have to get the data on a network…” and begin rambling about RFCs and protocols. Well, not this time, earlier that day a question had come up on demonstrating bridging 1st mile of embedded device data complex event processing. I can think of no more interesting fast data platform to analyze in real-time.

A very non-traditional fast data platform…

 The SV-650 has no modern electronics other than the telemetry system we put together from an Arduino and two Digimesh XBee 2.4gHz radios. This test was simply to validate the radios work at the ranges we need them to, and that we get a stream of data back from the track at speed. Sorry Pierson, you’re still not faster than an rf signal… The radios worked flawlessly.

Our Test Pilot and Track Preparation

Pierson Washer, #983 has been kind enough to allow us to use his race bike as a test platform. If you look closely you will see the antenna poking out from between two masonry sponges. We have everything wrapped up in duct tape and securely zip tied  inside the tail fairing.

I will not go in to detail on the actual telemetry package build out here, there will be a link below. 

Snippet from the OSEPP ACCEL-01 Senor

We captured and sent a simple 3-axis message from our OSEPP Mega 2560 R3 board with an OSEPP ACCEL-01 i2c sensor in real-time over rf to the pit. Below is a small snippet of data with the OSEPP example code for the sensor from a data platform moving at 140mph, no wires, and closed network.

Msg ID: 2391, Sensor: 29, X: -28, Y: -239, Z: -105
Msg ID: 2392, Sensor: 29, X: -49, Y: -251, Z: -134
Msg ID: 2393, Sensor: 29, X: 7, Y: -185, Z: -101
Msg ID: 2394, Sensor: 29, X: -54, Y: -152, Z: -43s

To be Continued…

What’s next? We continue our experiment at Buttonwillow Raceway on March 2nd and 3rd where we close the logical gap of doing useful things with these event messages. See you at the races…

link to racing, bike, and electronics – http://techweasel.net

That might sound counterintuitive, or even plain silly. But consider this – as high volume, variegated data environments become the rule, more than just data stores and analytic tools have to change. The core data infrastructure for processing, moving and managing data has to change. Data “grunt work” like ETL, batch processing, quality filtering, data enrichment, legacy integration, and data operations overall has to change. Manual or high touch operations projects aren’t just expensive and painful – they can’t keep pace with data. And if every new data initiative needs custom programming before it gets off the ground, then aren’t you just boxing yourself into a new set of data problems and becoming beholden to a new set of specialized, hard-to-hire skills?

That’s why real time data management matters.  When real time processing is the requirement, the opportunities to simplify and gain flexibility are rich. Treating all data as data in motion results in flexible information streams, ready to be combined, correlated, filtered and routed in a responsive manner. Focus becomes on “feeds and speeds” rather than schema and stored processes. You make decisions on what to do with data instantaneously, and continually evolve your practices. The painful drudgery of data management goes away. And if you do it right, this can all be achieved through an agile configuration framework, without the need for deploying and maintaining custom code.

Granted, most organizations are going to move toward real time data to suit specific real time use cases. We’re happy about that and look forward to helping them. But we’ll continue to spread the word that real time isn’t just for real time anymore.

That said, we think she left a few things unsaid that are really important. While we agree with Ms. Haynes that intelligence applications for business users will be really key, you can’t overlook the importance of core infrastructure technology changes that will be needed to accommodate this Second Act.

In one of the sources Ms. Haynes references, IDC makes some important observations, some of which are laid out in this press release, and some of which you’ll need to get your own copy of the full report:

• While software and services will constitute a larger share of overall revenue, growth rates for infrastructure technologies is projected to be higher.
• “Big Data in Motion” and the high speed messaging technologies needed to support it are one of the key aspects of Big Data growth.
• There’s a Big Data talent gap on both the IT and analytics expertise sides, representing a real opportunity for appliance offerings that make it easy to deploy and utilize next generation concepts in the data infrastructure space.

The bottom line is that there’s more than just some new data stores and cool BI tools out there on the horizon. There are fundamental changes to the core data management, data integration and enterprise information access processes on the way. Innovators who are poised to offer new, easy to deploy platforms to enable Big Data’s Second Act have a bright future, and we’d like to think we’re one of them.

So we were very happy to present along with Fusion-io at the recent MongoSV conference. MongoSV is an annual one-day conference in Silicon Valley, CA, dedicated to the open source, non-relational database MongoDB. This year’s conference included over 50 sessions by 10gen engineers and MongoDB users from companies such as foursquare, Github, Apollo Group (University of Phoenix), AOL, and more.

For our presentation, Talksum’s CTO, Dale Russell, and Platform Architect, Brian Knox, were joined by Fusion-io Solution Architect, Matthew Kennedy. The presentation highlighted how Talksum’s architectural approach, combined with Fusion ioMemory technology, can solve some interesting customer challenges.

Specifically, we highlighted how Talksum’s data aggregation and filtering can simplify Netflow data processing and monitoring. We used MongoDB as the target storage layer, which took advantage of some of the optimization concepts from our platform along with the newest features of MongoDB.

More details can be seen in the presentation recording here, with our portion starting at slide 21:

http://www.10gen.com/presentations/mongiops-your-favorite-data-store-only-faster

First of all, we’ve been building our product and moving from our research prototype to our commercial offering. On that front, we’re very excited to announce that we’re well under way with the private beta program for our appliance-based Talksum Data Stream platform. We’ve been lucky enough to have the opportunity to work with several major enterprises in different verticals to prepare our product to be let loose in the wild, applied to real world data management and analytic problems. We’re working with Netflow data for optimization of data center topology and disaster recovery strategy. We’re enriching data in real time for optimized integration. We’ve been proving how our real time data reduction tools can shrink storage and decrease latency. As I said, we’ve been busy.

Secondly, we’ve been using these experiences to really hone our product vision. As we’ve talked to numerous companies and started working with our initial group of beta candidates, we’ve become increasingly passionate about the need for new approaches to “Big Data.”

We see lots of vendors working on how to do analytics on larger data sets, on how to spin enterprise offerings out of hugely distributed cloud approaches or how to re-tool the Hadoop ecosystem to handle lower latency solutions. However, the more we talk to big companies, the more we see that there’s an underlying need to re-assess larger data management practices. That’s where we see Talksum fitting in – we bring real time processing to the core of data management, data integration and data focused solution development.

We’ll be writing much more about Real Time Data Management and the benefits it brings the enterprise. For now, we’re happy to announce our private beta and to tell the world that we’re excited to be coming out of our stealthy period and into the fray!

We also enjoy sharing our knowledge with our peers, so last week at the monthly meetup of the SF Bay Area Large Scale Production Engineering group (LSPE), I gave a presentation on Salt. The theme was “Command and Control,” so after a general overview I focused on Salt’s remote execution capabilites, but I included a brief demonstration of its configuration management prowess as well.

I’m happy to share the contents of that presentation here:

A full video of the July Meetup can be seen below, thanks to videographer KC Leung.

(My talk on Salt starts around the 46:30 mark, but the other presentations were equally interesting, so the full video is well worth watching!)

Questions? Comments? As always, we’d love to hear from you.

Mike Chesnut, Principal Systems Administrator