The Coming Explosion Of Data

Paul Kedrosky makes an excellent assessment, about the amount of data that would be generated if every mobile device that could be networked : If most non-stationary device that can be Internet-enabled are made so – with embedded GPS and telemetry like phones, PDAs, cars, trucks, laptops, motorcycles, and cameras, but a host of other devices too, ranging from the sinister to the sublime, we are probably talking about a billion location aware devices, including the 500-million cellphones being sold each year, on out through 20-million cars and trucks being sold annually, and a host of other industrial devices. There is going to be a data explosion from the bottom up as all those data devices are constantly sending GPS and (likely) telemetry data, the data will be flowing inexorably outward. Continuous telemetry (at 10Hz) and GPS (at 1 Hz) requires about 1 kb/s. Two hundred million devices will require, well, 200,000,000 kb/sec of bandwidth - or 200 gb/sec. It is a large load on networks, one that could, in aggregate, soak up considerably otherwise unused bandwidth out there as devices continually ping out location and direction. It is also going to require off-device storage, as most of these devices have no capacity for logging all the data - but keeping the information will be crucial as people search through and find new optima in these large, location-enabled datasets. In a more practical way RFID implementations see explosion of data – with tagging and tracking data continually streaming in – providing for collecting and analysing these data needs huge storage and unique retrieval capabilities.