Star Trek, Berners-Lee, and DZone’s Ocean of Data

I’m a product of my upbringing, and (using the term loosely) I grew up in the 60’s and 70’s. It was the time of Star Trek and a real-world space program, both of which had tremendous influence in shaping my belief that the pursuit of scientific knowledge leads to good things.

The voyagers of the starship Enterprise had an excellent situation. They simply had to do whatever it was they were good at doing, and somehow resources were available to do keep doing it continuously and with little regard for cost. It’s a model that appeals to the closet utopian in me, but it’s pretty far from the day-to-day economic reality most of us live in.

Today, I listened to Tim Berners-Lee’s TED talk where he urges us all to open up our data, all data, and make it available for linked use. I love the idea of “raw data now”, but it scares me. It happens that DZone is floating on an ocean of data. In our three years online we have tracked how millions and millions of developers have used hundreds of thousands of links from tens of thousands of domains. I imagine that intriguing insights about developer trends could be drawn from this data. It might be even more intriguing if it could be correlated to open source project activity and commit rates or some similar data pool that someone else possesses. Sir Tim’s idea of exposing “raw data now” challenges us to engage in a broad experiment and find out what happens.

I’m close to taking up Sir Tim’s challenge, really I am. My desire to see what we might learn confronts my business training, which suggests that possessing information exclusively is my competitive advantage. My instincts, however, tell me not to sweat it and that things will be alright.

Your input matters a lot, and I’d like to hear your ideas about how you would want to leverage DZone’s data if we opened it up (of course, no personal data would be shared!) What new and interesting possibilities would this create? Are there steps we could take in this direction without throwing the doors wide open and inviting the world into our databases? What would you do?

I’m going to give this serious thought, and I would genuinely like to hear from you. Thanks!