Big Data: The Next Career Field?

Yesterday I completed the manuscript for the next edition of Best Jobs for the 21st Century. The book actually focuses on the next ten years, but I often wonder about longer-term prospects for career growth in the United States. Where will tomorrow’s jobs come from? Here’s one possibility.

Early in the 20th century, geologists discovered a huge pool of oil beneath the ground near Beaumont, Texas. Other petroleum deposits soon were identified in California and Oklahoma, and these natural resources led to thousands of jobs and billions of dollars of revenue. Our economy has exploited many other natural resources, such as timber, fish, and fresh water, sometimes creating shortages when demand exceeds supply. But perhaps the next huge resource that will be exploited is not a natural resource, not even something tangible, but rather massive quantities of data. This is what a recent report (PDF) from McKinsey Global Institute argues, and the report makes a good case.

It’s estimated that the volume of business data doubles roughly every 1.2 years. Every time you order something online, you’re generating data about your purchasing and payment behavior. The logistics process of getting the product to you generates additional data. Postings on Facebook, geotagged photos on Flickr, items on eBay or Craig’s List, media that stream on YouTube or Internet radio, all these and countless other quantities of data are generated every minute of the day, and the way (including the location from which) people respond to each of these items creates more data.

This pool of data, like a pool of oil, can be exploited for economic value, but it has the additional benefit of never running dry. Not only is new data constantly being generated, but consumption of data doesn’t use it up. It can be analyzed and reanalyzed. Any number of users can exploit it simultaneously.

The most successful current users of big data are Google, Bing, Yahoo, and other search providers. They use it two ways: (1) They create an index of Web content that is ranked according to how many links exist to the content, and (2) when you use this index, they sell information about your clicking behavior to advertisers. But these search providers only scratch the surface of all the data that’s out there.

The authors of the report (try as you may, you just can’t avoid calling a bulletin from McKinsey a “McKinsey report”) emphasize that what makes big data a new resource, different from past uses of business data, is its size: We already have business tools that exploit various databases, but the potential for innovative work lies in finding ways to analyze data at larger scales than have ever been attempted before. In fact, the authors make a point of defining “big data” as a kind of moving target rather than as a fixed number of terabytes, because as the volume of data doubles and redoubles, the scale of the analytical task will continuously create new challenges. Success in this field is not a matter of being able to analyze data, but rather being able to analyze bigger data sets than ever before.

Another dimension of the analytical challenge that defines this new resource (and the occupations that it will spawn) is the speed with which the big-data analysis can be done. We’re already gotten used to being able to track the delivery route of a package within a few hours of real time or the actual arrival time of an airline flight within a few minutes. Wall Street has developed ways to react within milliseconds to fluctuations in the value of securities. Some of the innovations that will be developed for use of big data will be methods of accomplishing instant analysis and response--and doing so in ways that have controls to prevent snowballing events such as the “flash crash” of 2010, in which the Dow Jones plunged about 9 percent within minutes, only to rebound just as quickly.

It’s obvious that marketers will have uses for the outputs of this work. So will governments, which can improve services by better segmenting the population. Law enforcement and defense are already reaping the benefits of using large-scale, real-time monitoring of events to trigger and coordinate a rapid response.

This kind of work will be done by teams that are highly creative and have outstanding technical and communications skills. In other words, it is the kind of work that America has always been good at. The major hurdle that needs to be overcome is the projected shortage of skilled workers. As the report notes, we face “a shortage of 140,000 to 190,000 people with analytical expertise and 1.5 million managers and analysts with the skills to understand and make decisions based on the analysis of big data.” So here is another reason why we need to improve STEM education and career development, make higher education more meritocratic, and (if Americans fail to step up to the plate) facilitate immigration of skilled foreigners.

Do You Mind If I Get Graphic?

This week I was alerted to a skill that I had not thought about or heard about before. I was actually starting to think about this skill when I discovered that someone had already given it a name.

A journalist asked me to respond to some questions about what’s involved in career changes. In the response I composed, I wanted to call attention to my recent book The Sequel, so I emphasized the importance of accumulated knowledge. One of my purposes in that book is to balance the conventional emphasis on skills with this emphasis on knowledge, which is particularly relevant to career changers (as opposed to young people just starting out).

But the journalist specifically asked about important transferable skills, so I included a discussion that began by identifying communication skills as a key to employability. I mentioned that nonverbal skills are a significant part of these skills. When I first wrote that, I was thinking in terms of body language: being able to interpret the physical cues given off by people in conversation (or even when they’re not speaking) and being careful to project the right cues.

However, as I reread this discussion a day later I started thinking about other forms of nonverbal communication--specifically, graphics. If you follow my Twitter feed (LaurenceShatkin), you know that I’m very fond of Web postings with graphs and maps that communicate information about jobs and the state of the economy. A good example is a webpage at the Bureau of Labor Statistics that uses several kinds of colorful graphs--bar, line, pie, and scatter plot--to convey the latest statistics about women in the workforce. Because I use graphics in conference presentations and occasionally in my books, I appreciate that deciding on the most effective use of graphics is not as easy as it appears to those who haven’t tried it. Interpreting graphs also is not as easy as it sometimes appears. Interpretation obviously is a challenge if a graph is poorly composed, but even a well-organized graph can be challenging when it is based on complex concepts, such as the bar chart I found on Paul Krugman’s webpage this week, comparing the projected health-care spending of 65-year-olds under three scenarios, one of which is the budget proposed by Representative Paul Ryan. (The graphic is taken from a report by the Congressional Budget Office—PDF.)

In addition to data-based graphics, pictorial graphics can be very useful in communication. If you’ve ever played the game Pictionary, you know that levels of skill with this kind of graphic can vary widely. People who want to use this type of graphic have access to several styles, from highly realistic portraits to stylized cartoons to symbolic glyphs.

Even as I was thinking about these other nonverbal forms of communication, by pure chance I received a tweet from a friend, Nancy Millichap, about a skill called “graphicacy,” which was exactly the skill I had in mind. Think of graphicacy as analogous to literacy or numeracy. Nancy’s tweet linked to a blog entry by a geographer, Diana Stuart Sinton, which in turn referenced an article written in 2000 by two British psychologists, Frances Aldrich and Linda Sheppard: “Graphicacy”: the Fourth “R”? (PDF). The two psychologists make the case for teaching graphicacy to all schoolchildren to “equip them with a communication skill that will be useful throughout their lives.”

Obviously, some occupations require a high level of graphicacy than others. The need is greatest in occupations that involve the graphic arts or lots of data. But, like reading and writing, this is a fundamental communication skill that is needed at one level or another in all kinds of work. You will not find this among the skills in the O*NET database, but there is a case to be made for adding it.