How Software Is Helping Big Companies Dominate

Throughout the global economy, big companies are getting bigger. They’re more productive, more profitable, more innovative, and they pay better. The people lucky enough to work at these companies are doing relatively well. Those who work for the competition aren’t.

Policymakers have noticed. Antitrust and competition policy are seeing renewed interest, including recent hearings on the subject by the Federal Trade Commission. Headlines in publications ranging from The Nation to The Atlantic to Bloomberg warn of America’s “monopoly” problem with calls to break up big companies such as Google or Amazon or Facebook. “Imagine a day in the life of a typical American,” writes Derek Thompson in The Atlantic. “How long does it take for her to interact with a market that isn’t nearly monopolized?”

Antitrust deserves the attention it’s getting, and the tech platforms raise important questions. But the rise of big companies — and the resulting concentration of industries, profits, and wages — goes well beyond tech firms and is about far more than antitrust policy.

In fact, research suggests that big firms are dominating through their use of software. In 2011, venture capitalist Marc Andreessen declared that “software is eating the world.” Its appetizer seems to have been smaller companies.

What’s Driving Industry Concentration

Most industries in the U.S. have grown more concentrated in the past 20 years, meaning that the biggest firms in the industry are capturing a greater share of the market than they used to. But why?

Research by one of us (James) links this trend to software. Even outside of the tech sector, the employment of more software developers is associated with a greater increase in industry concentration, and this relationship appears to be causal. Similarly, researchers at the OECD have found that markups — a measure of companies’ profits and market power — have increased more in digitally-intensive industries. And academic research has found that rising industry concentration correlates with the patent-intensity of an industry, suggesting “that the industries becoming more concentrated are those with faster technological progress.” For example, productivity has grown dramatically in the retail sector since 1990; inflation-adjusted sales per employee have grown by roughly 50%. Economic analysis finds that most of this productivity growth is accounted for by a few companies such as Walmart who used information technology to become much more productive. Greater productivity meant lower prices and faster growth, leading to increased industry dominance. Walmart went from a 3% share of the general merchandise retail market in 1982 to over 50% today.

All of this suggests that technology, and specifically software, is behind the growing dominance of big companies.

IT Does Matter

In 2003, then-HBR-editor Nick Carr wrote an article (and later a book) titled “IT Doesn’t Matter.” Carr took issue with the common assumption “that as IT’s potency and ubiquity have increased, so too has its strategic value.” That view was mistaken, he argued:

“What makes a resource truly strategic—what gives it the capacity to be the basis for a sustained competitive advantage—is not ubiquity but scarcity. You only gain an edge over rivals by having or doing something that they can’t have or do. By now, the core functions of IT—data storage, data processing, and data transport—have become available and affordable to all. Their very power and presence have begun to transform them from potentially strategic resources into commodity factors of production. They are becoming costs of doing business that must be paid by all but provide distinction to none.”

Carr distinguished between proprietary technologies and “infrastructural” ones. The former created competitive advantage, but the latter were more valuable when broadly shared and so eventually became ubiquitous and were not unique to any company. IT would temporarily create proprietary advantages, he predicted, citing Walmart as an example. Walmart is the country’s largest employer and largest company by revenue and it reached that position through an operating model made possible by proprietary logistics software. But Carr believed that by his writing in 2003 “the opportunities for gaining IT-based advantages are already dwindling” and that “Best practices are now quickly built into software or otherwise replicated.”

It didn’t turn out that way. Although rivals have tried to build their own comparable logistics software and vendors have tried to commoditize it, Walmart’s software acumen remains part of its competitive advantage — fueled now by a rich trove of data. While Walmart faces new challenges competing online, it has maintained its logistics advantage against many competitors such as Sears.

The “Full-Stack” Startup

This model, where proprietary software pairs with other strengths to form competitive advantage, is only becoming more common. Years ago, one of us (James) started a company that sold publishing software. The business model was to write the software and then sell licenses to publishers. That model still exists, including in online publishing where companies like Automattic, maker of the open source content management system WordPress, sell hosting and related services to publishers. One-off licenses have given way to monthly software-as-a-service subscriptions, but this model still fits with Carr’s original thesis: software companies make technology that other companies pay for, but from which they seldom derive unique advantage.

That’s not how Vox Media does it. Vox is a digital publishing company known, in part, for its proprietary content management system. Vox does license its software to some other companies (so far, mostly non-competitors), but it is itself a publisher. Its primary business model is to create content and sell ads. It pairs proprietary publishing software with quality editorial to create competitive advantage.

Venture capitalist Chris Dixon has called this approach the “full-stack startup.” “The old approach startups took was to sell or license their new technology to incumbents,” says Dixon. “The new, ‘full stack’ approach is to build a complete, end-to-end product or service that bypasses incumbents and other competitors.” Vox is one example of the full-stack model.

The switch from the software vendor model to the full-stack model is seen in government statistics. Since 1998, the share of firm spending on software that goes to pre-packaged software (the vendor model) has been declining. Over 70% of the firms’ software budgets goes to code developed in-house or under custom contracts. And the amount they spend on proprietary software is huge — $250 billion in 2016, nearly as much as they invested in physical capital net of depreciation.

How Big Companies Benefit

Clearly, proprietary software is providing some companies advantage and the full-stack model is dominating the software-vendor model. The result is that large firms are gaining market share. But to explain that, one needs to explain why some companies are so much better at developing software than others and why their innovations don’t seem to be diffusing to their smaller competitors the way Carr thought they inevitably would.

Economies of scale are certainly part of the answer. Software is expensive to build but relatively cheap to distribute; larger companies are better able to afford the up-front expense. But these “supply-side economies of scale” can’t be the only answer or else vendors, who can achieve large economies of scale by selling to the majority of players in the market, would dominate. Network effects, or “demand-side economies of scale,” are another likely culprit. But the fact that the link between software and industry concentration is pervasive outside of the tech industry — where companies are less likely to be harnessing billions of users — suggests network effects are only part of the story.

Part of the explanation for rising industry concentration, then, seems to hinge on the fact that software is more valuable for firms in combination with other industry-specific capabilities. These are often referred to as “intangible assets,” but it’s worth getting more specific than that.

Research suggests that the benefits of information technology depend in part on management. Well-managed firms get more from their IT investments, and big firms tend to be better managed. There are other “intangible” assets that differentiate leading firms, and which can be difficult or costly to replicate. A senior executive who has worked at a series of leading enterprise software firms recently told one of us (Walter) that a company’s ability to get more from an average developer depended on successfully setting up “the software to make software” — the tools, workflows, and defaults that allow a programmer to plug in to the company’s production system without having to learn an endless number of new skills.

Patents and copyright also make it harder for software innovations to spread to other companies, as do noncompete agreements that keep employees from easily switching jobs. But one of the biggest barriers to diffusion — and therefore one of the biggest sources of competitive advantage for the firms that excel at software — comes down to how companies are organized.

Architectural Innovation

In 1990, Rebecca Henderson, now a professor at Harvard Business School, published a paper that provides a theoretical basis for the success of full-stack startups. At the time, multiple thinkers were grappling with the question of why big, successful, cash-rich companies were sometimes unseated by new technologies. Incumbent companies aren’t necessarily bad at using new technologies, Henderson argued, based on her study of the photolithography industry. In fact, incumbents were great at using new technologies to improve individual components of their products. But when a new technology fundamentally changed the architecture of that product — the way everything fit together — the incumbent struggled.

Her point was that a company’s way of doing things is often deeply interconnected with the architecture of the products or services it creates. When the architecture changes, all the knowledge that was embedded in the organization becomes less useful, and the company’s way of doing things goes from advantage to disadvantage.

For example, Walmart’s competitive edge depended on having an organization and business model that took advantage of its logistical prowess by emphasizing Everyday Low Prices, large assortment, and rapid response to changes in tastes. Even though its larger competitors such as Sears spent heavily on IT, they could not compete effectively without making fundamental architectural changes. If all Walmart had done was apply IT to one component of the retail system — say, digitizing catalogs or bringing them online — Sears might have been in a better position to compete. But Walmart changed not only how supply chains, product decisions, and pricing worked, but how they related to each other. Sears’ entire existing way of doing things was suddenly a disadvantage.

As Dixon, the VC, clearly recognized, these architectural innovations can create openings for startups. “Before [Lyft and Uber] were started, there were multiple startups that tried to build software that would make the taxi and limo industry more efficient,” Dixon has noted. If Uber had merely created software for dispatching taxis, incumbents would have been well positioned to adopt it, according to Henderson’s theory. One “component” of the service would have been changed by technology (dispatching) but not the entire architecture of the service. But ridesharing startups like Uber and Lyft didn’t didn’t just make taxis more efficient; they fundamentally changed the way the different pieces of the system fit together.

Architectural innovation doesn’t necessarily result in startups displacing incumbents. It can also determine who prevails in a competition between larger, older firms. In November 2007, Forbes put the CEO of Nokia on its cover and asked, “Can Anyone Catch the Cell Phone King?” Apple had launched the iPhone just months before.

Why was Apple, a company with no prior experience in phones, able to overtake the cell phone king? Earlier this year, Harvard Business School professor Karim Lakhani asked this question to a group of conference attendees. One of us (Walter) listened as the technology experts in the audience listed all the ways the iPhone was superior: touch screen, app store, web browser, etc. Lakhani then provided the dates at which Nokia had offered those features: an app store in 2001, a touch screen in 2002, a web browser in 2006. Why, then, did Apple prevail?

Lakhani’s answer is that Apple had the right architecture to bring phones into the internet age. Apple and Nokia both had plenty of the intangible assets necessary to excel in the smartphone business, including software developers, hardware engineers, designers. But Apple’s structure and culture were already based around the combination of hardware and a software ecosystem to which third parties contributed. It already had experience building hardware, operating systems, and software development kits from its PC business. It had built a software platform to deliver content to mobile devices in the form of iTunes. Steve Jobs initially resisted letting developers build apps for the iPhone. But when he eventually gave in, the app store became the iPhone’s key advantage. And Apple was able to manage it because of its existing “architecture.”

Like any theory, architectural innovation can’t explain everything. If experience building operating systems and SDKs were so key, why didn’t Microsoft invent the winning smartphone? Apple’s particular acumen in product design clearly mattered, too. But architectural innovation helps explain why certain capabilities are so tough to replicate.

Spreading the Benefits of Software

The challenge for policymakers worried about industry concentration, markups, and the power of giant companies is to spread the benefits of the digital economy – of software – more broadly. Antitrust may be able to help in extreme cases, including in reining in the tech platforms and their ability to buy up competitors. But policymakers should also consider ways to help software and software capabilities diffuse throughout the economy. To some degree, economies of scale will simply increase the average size of firms, and that’s ok. But banning non-competes would help employees spread their knowledge by moving jobs. Reforming patents, which aren’t always necessary to protect software innovation and are abused by patent trolls to the detriment of nearly everyone, would help, too. Anything governments can do to encourage the use of open source software could help as well. For example, the French government mandates that public administrative bodies thoroughly review open source alternatives when revising or building new information technology and to use the savings realized to fund further open source development.

Encouraging startups is another promising avenue, as these firms are able to organize around software capabilities to take on incumbents. Doing so through public policy isn’t always easy, but government funding can help when done well, and at the state and city level policymakers can encourage the formation of technology clusters. These policies would pair well with more aggressive merger review, to ensure that promising startups are not all swallowed up by the incumbents they’re challenging.

For companies, the takeaway is more obvious. Even if you’re not in the software industry, there’s a good chance your success hinges on your ability not just to use but also to build software. Using vendors often still makes financial sense, of course. But consider what makes your company unique, and how software might further that advantage. Investing in proprietary solutions that complement your strengths might be a good idea, especially for medium and large companies and for growth startups.

A Cloud on the Horizon

There is some good news: research suggests that cloud computing is helping smaller, newer firms to compete. Also, some firms are unbundling their advanced capabilities. For example, Amazon now offers complete fulfillment services including two-day delivery to sellers, large and small, on its Marketplace. It may be that Carr was right in principle but just had the timing wrong. But we wouldn’t bet on it. Some aspects of software will be democratized, including perhaps some areas where companies now derive competitive advantage. But other opportunities will arise for companies to use software to their advantage. One in particular stands out: even when machine learning software is freely available, the datasets to make it valuable often remain proprietary, as do the models companies create based on them. Policy may be able to help level that playing field. But companies that don’t invest in software and data capabilities risk being left behind.