Ericsson to WhatsApp : The Story of Erlang

How a programming language from Sweden powered one of the most valuable startup of the 2010s.

In this post, we’re going to look at a piece of technology that was banned by the company that created it. It was kept alive by a small team of enthusiasts. Then, almost thirty years after its first development, it became the core technology underpinning one of the most important and lucrative startups of the 2010s.

Today, it plays a key role in services that are used by billions across the world.

We’re talking about the Erlang programming language.

Why is Erlang interesting? Well, one of the themes of this newsletter is that old technology can be incredibly valuable and useful. Erlang was often seen as an esoteric curiosity, until a small group of entrepreneurial engineers used it to create a start-up that they sold for billions of dollars.

Erlang’s history also has a number of lessons. The spread of general purpose hardware and software into specialist areas like telecoms. The power of individuals and small teams using the right software tools. The resilience of open source software. And more.

This is our first post on a programming language. Except it isn’t entirely about software, it’s about how software can make the most of the hardware resources available and the links between the hardware of computing and that of telephony.

So, what is Erlang, how did it come into being, and why has it had a renaissance recently?

We’re going to start at the place where Erlang was born. In Stockholm in Sweden.

Ericsson

If you’re old enough, you may remember Ericsson mobile phone handsets which were a popular choice in the pre-smartphone era. Today, Ericsson’s business is focused on telecommunications equipment.

Ericsson was founded by Lars Magnus Ericsson in Stockholm, Sweden in 1876. In 1878, it started making and selling telephones and switchboards. Over the following decades, Ericsson had mixed fortunes and, after a period of financial difficulties, it was rescued by banks controlled by the Wallenberg family. By the 1960s, the Wallenbergs had full control of Ericsson.

We’ve previously met Ericsson in recounting the story of the first RISC computer, the IBM 801. IBM wanted to to enter the telecoms equipment market and considered joining forces with Ericsson, with the development of a new computer at the center of a possible joint venture. After a series of secretive meetings at London’s Claridges Hotel, that came to nothing.

Ericsson Computer Science Laboratory

But telephony in general, and Ericsson in particular, needed computing.

In 1980, a small group of Ericsson engineers - Bjarne Däcker, Mike Williams, Göran Båge and Seved Torsten-dahl - proposed creating a Computer Science Laboratory (CSL) within the company. Their objectives were to create software technology for future telecoms systems and to help introduce that technology into existing Ericsson systems.

Their hardware mixed industry standard computers with specialised telecoms hardware, with a DEC VAX 11/750 minicomputer linked to an Ericsson telephone exchange.

The original setup used to develop Erlang with a VAX 11/750 minicomputer running Unix connected to an Ericsson telephone exchange

Earlier work to program telecom systems at Ericsson had used a range of languages including PL163, CHILL and modified versions of the Pascal programming language.

Now the team started to look at a range of existing programming languages, each of which brought its own approach. These languages included then mainstream languages like Ada and lesser known ones such as Concurrent Euclid, PFL, LPL, OPS4, Frames and CLU.

As the team's work progressed, it became clear that none of these individual languages by itself would meet all the team’s requirements.

In 1985, the group was joined by Joe Armstrong and Robert Virding.

Then in 1988, the Lab moved from Ericsson to Ellemtel, a joint venture between Ericsson and the Swedish national telecoms provider Televerket.

The focus now was on building software to program AXE, Ericsson’s telephone exchange, which was programmed in a language called PLEX. The motivation was to “to make something like PLEX, to run on ordinary hardware, only better.”

Carrying forward some properties of PLEX was considered essential. In particular, according to Armstrong:

Firstly, it should be possible to change code “on the fly;” in other words, code change operations should be possible without stopping the system.

Software systems need to be updated and maintained with new code replacing old. A telephone system can’t be taken down completely whilst a set of code updates are introduced. Customers would not be happy if they can’t make their calls!

The language also needed to support many simultaneous activities, as calls were routed between users, and it had to be able to do this efficiently. Again, according to Armstrong, the language had to take replicate this property of telecoms switching systems:

A switching system is made from a number of individual switches. Individual switches typically handle tens to hundreds of thousands of simultaneous calls. The switching system must be capable of handling millions of calls and must tolerate the failure of individual switches, providing uninterrupted services to the user.

To this was added the conclusions of the earlier work to test a variety of languages. In particular:

• Small languages were thought desirable.

• Functional programming was liked.

• Logic programming was best in terms of elegance.

• Concurrency was viewed as essential for this type of problem.

Armstrong experimented with Smalltalk, but his attention was drawn to Prolog. Experimenting with building a system in Prolog soon led to something distinct:

What started as an experiment in “adding concurrency to Prolog” became more of a language in its own right and this language acquired a name “Erlang,” which was probably coined by Bjarne Dacker. What did the name Erlang mean? Some said it meant “Ericsson Language,” while others claimed it was named after Agner Krarup Erlang (1878 – 1929), while we deliberately encouraged this ambiguity.

Naming Erlang after Agner Krarup Erlang the Danish Mathematician who worked in the fields of traffic engineering and queuing theory, was surely appropriate for a language used to route telephone calls!

Robert Virding was now working with Armstrong on Erlang and by the end of 1988 most of the key ideas in Erlang had been implemented.

What was Erlang?

So what made Erlang different, and how did it support the Ericsson team’s requirements?

At the centre of Erlang was the idea of thousands and thousands of ‘processes’. These are separate pieces of code that run largely independently of each other. Each process communicates with other processes using ‘messages’, small amounts of data that are sent and received by each process.

Each process has a ‘mailbox’ which it can check from time to time and take action depending on the contents.

Furthermore, each process is extremely ‘lightweight’ in that it uses a very small proportion of the resources of the computer. A modern server can support millions of Erlang processes running at the same time.

The team built resilience into the language, allowing processes to fail, either due to bugs or to hardware failures, without bringing down the whole system – which of course would be a disaster for a telephony switching system.

They also built in the ability to scale. Processes can run on multiple physical machines, sending messages to other processes either running on the same machine or on other machines. So, an Erlang program can easily grow by just adding more physical machines.

Life Inside Ericsson

Word of the development of Erlang spread, and Ericsson engineers wanted to try it out:

The team wanted to prototype a new software architecture called ACS3 designed for programming telephony services on the Ericsson MD110 PABX4 and were looking for a suitable language for the project, which is how they got to hear about Erlang. A project called ACS / Dunder was started to build the prototype.

When the work on ACS / Dunder was evaluated, there was a striking conclusion: development in Erlang was a lot faster than the alternatives.

Erlang was revealed to the world at a conference in 1989. Work continued inside Ericsson on to develop and improve Erlang over the following years. Then in 1992 there was a seemingly small, but in fact immensely significant, step forward for Erlang. A book was published on the language.

PLEX had been proprietary to Ericsson, as it was presumed that it gave the company a commercial advantage. For Erlang, it was decided to follow the example of AT&T and the C language.

The decision to publish an Erlang book and to be fairly open about what we did was therefore to avoid isolation and follow the AT&T/C path rather than the Ericsson/PLEX path. Also in 1992 we ported Erlang to MS-DOS windows, the Mac, QNX and VxWorks.

So the Erlang team started to promote the language. Including on video. We can see some of the original designers of Erlang, showcasing the system, in ‘Erlang: The Movie’. In it Bjarne Decker, Joe Armstrong, Mike Williams and Robert Virding show off the capabilities of their new language.

‘Declarative Real Time Programming Now!’, ‘Try It, You’ll like it!’

Mike Williams, Joe Armstrong and Robert Virding

The Collapse of AXE-N

In December 1995, a large project at Ellemtel, called AXE-N, collapsed. According to Armstrong:

AXE-N was a project aimed at developing a new generation of switching products ultimately to replace the AXE10 system. The AXE-N project had developed a new hardware platform and system software that was developed in C++.

It was decided to replace the C++ software with code written in Erlang.

Again, according to Armstrong:

This new project was to be the largest-ever Erlang project so far, with over 60 Erlang programmers. At the start of the AXD project, the entire Erlang system was the responsibility of half a dozen people in the Lab. This number was viewed as inadequate to support the needs of a large development project and so plans were immediately enacted to build a product unit, called OTP, to officially support the Erlang system.

OTP or the ‘Open Telecoms Platform’ provided capabilities such as ‘hot code reloading’, databases (he ‘Mnesia’ package) and many other facilities. Despite the use of the word ‘telecom’ in the name OTP is really a general purpose set of programs and doesn’t really have anything that is peculiar to telecoms systems.

So with increasing adoption Erlang’s future within Ericsson looked bright.

Ericsson Ban

Then suddenly, in 1998, Erlang was banned within the Ericsson. The reason? To avoid the costs of development of a language that was unique to Ericsson, and instead to access the shared effort being put into the development of other languages.

The selection of an implementation language implies a more long-term commitment than the selection of a processor and OS, due to the longer life cycle of implemented products. Use of a proprietary language implies a continued effort to maintain and further develop the support and the development environment. It further implies that we cannot easily benefit from, and find synergy with, the evolution following the large-scale deployment of globally used languages.

In other words, Erlang was almost killed by the success of open-source languages.

Open Source Erlang

But all was not over for Erlang.

The language continued to be maintained within Ericsson, as it was needed to support existing products.

And the efforts of the Erlang team outside Erlang would bear fruit. According to Armstrong:

For some time, we had been distributing Erlang to interested parties outside Ericsson, although in the form of a free evaluation system subject to a non-disclosure agreement. By 1998, about 40 evaluation systems had been distributed to external users and by now the idea of releasing Erlang subject to an open-source license was formed. Recall that at the start of the Erlang era, in 1986, “open source” was unheard of, so in 1986 everything we did was secret.

Following the ban, the Erlang team lobbied for Erlang, no longer seen as a sensitive commercial secret for Ericsson, to be released as open source. On 2 December 1998, “Open-Source Erlang” was announced.

The original members of the Erlang team soon left Ericsson to form their own company, Bluetail, to work on Erlang. It was natural for the team to turn to internet services, as this was just at the end of the era of the internet bubble.

Given that the Bluetail system was programmed by most of the people who had designed and implemented the Erlang and OTP systems, the project was rapidly completed and had sold its first system within six months of the formation of the company. This was one of the first products built using the OTP technology for a non-telecoms application.

Erlang had broken away from its telecoms roots and made its way onto the internet.

Other uses for the language started to appear. It found a niche in finance, helping firms that rely on ultra-fast trading to ensure that their systems were robust. Illustrious Wall Street firm Goldman Sachs used Erlang to implement their messaging system, with its robustness and reliability again key.

Open-source Erlang-based projects started to appear. One of these was ejabberd, originally developed by Alexey Shchepin starting in 2003, which provided high-quality software to implement a system that allowed chat messages to be sent between a large number of users.

WhatsApp

Then in 2009, Jan Koum came up with an idea. What about an internet-based messaging system for the iPhone?

The iPhone was launched in 2007 and the App Store in July 2008. The first version of Android was released in September 2008.

iPhone and Android users could send text messages to each other, but doing so was costly, with charges on a per-text basis. By contrast, the amount of data used for each message was tiny, even in the context of the small data transfer allowances granted to users at the time.

Koum was joined by Brian Acton, who he’d met whilst working at Yahoo!

¹

Koum and Acton set out to build a system where users could send each other messages using their data allowances. This would dramatically undercut the pricing of the telecoms operators. With dedicated apps on each of the new mobile platforms, they could also build in more features to make using the application more useful and fun.

The underlying software base to power the ‘back end’ of this system was already available in the form of ejabberd. Koum and Acton took the open-source messaging system and adapted it to provide the underpinnings of the product that they envisaged.

So, WhatsApp was born in January 2009.

WhatsApp Logo

WhatsApp’s growth was spectacular. From 10 million users in 2010 it grew to 100 million at the end of 2012. Thanks to the efficiency of Erlang and their careful refinement of both ejabberd, the FreeBSD operating system and Erlang itself, they were able to scale quickly whilst keeping growth of both their hardware and engineering resources down.

Their ability to scale at low cost also underpinned their business strategy. They avoided showing ads to users, instead charging each user $1 each year after the first year. $1 per user may not sound much, but with their growing user base, it would soon represent significant income. It was also affordable for users even in lower income countries, and users liked the ad-free WhatsApp user experience.

By 2013, Facebook’s Mark Zuckerberg started to see WhatsApp as a threat to his own business. Facebook had its own messaging application, Facebook Messenger, but WhatsApp was winning the messaging battle.

Critically, Zuckerberg would have seen how fast WhatsApp was scaling and presumed that it would be able to continue to scale. All thanks to Erlang.

So Facebook bought WhatsApp in 2014 for $16 billion, making Koum and Acton billionaires overnight. In the previous year, WhatsApp had revenues of only $10.2 million had incurred losses of $138 million (still small for a company with 400 million users at the end of the year – and with much of this stock-based compensation rather than underlying costs). Immediately after the acquisition, Facebook dropped what had been WhatsApp’s major revenue source, by removing the $1 per year fee. At the time of the acquisition WhatsApp employed only 35 engineers.

Under Facebook’s ownership, WhatsApp has continued to grow, with more than 2 billion users worldwide, all supported by Erlang.

WhatsApp wasn’t the only messaging system to use Erlang. Like WhatsApp, Facebook Messenger had used Erlang and ejabberd but moved away from it after finding it difficult to recruit engineers. WeChat in China also used Erlang for a messaging system operating with billions of users.

Elixir

But messaging wasn't the only area where Erlang could be built on.

Jose Valim had been a core member of the group developing the Open-Source Ruby on Rails (commonly known as ‘Rails’) web framework. Rails was started by David Heinemeier Hansson in 2003 a part of the Basecamp project management web application.

Rails provides much of the software infrastructure that is needed to build a modern web application: the ability to create web pages, access a database with user data and so on. Rails is built using the Ruby programming language.

Valim saw the advantages of Erlang and how useful they would be to someone developing a modern web application. In particular, its scalability made it suited to applications where users need to communicate frequently with web servers.

He also saw that Erlang had some major limitations in this context, including an unfamiliar syntax. So het set out to create a language that dealt with these issues whilst retaining the core advantages of Erlang and OTP.

So Elixir was born in 2013. Elixir provides the capabilities of Erlang (in fact, the language ‘cross compiles’ to Erlang) in a syntax that is largely familiar to Ruby programmers.

Just as Ruby on Rails built on Ruby, Elixir soon formed the base for a new ‘web framework’, developed by Chris McCord, Jose Valim and others, Phoenix, provides open-source software that is broadly equivalent to Ruby on Rails in its capabilities whilst being less resource intensive and making it more straightforward to build web applications, such as chat, that need frequent connection to a web server.

Innovation on top of Elixir and Phoenix has continued to this day. Two notable examples are LiveView, that removes the need for the user to write JavaScript running in the browser, and Nx which adds the base capabilities required for machine learning applications.

Today, many companies are using Elixir as the basis for their web applications, including large companies such as Twitch and a number of smaller start-ups. Elixir and Phoenix provide everything that is needed for even a small team to build an application that can be rapidly scaled.

Erlang Today

Erlang continues to be developed and supported by individuals and teams both inside and outside Ericsson. In 2022, it received a notable update that speeded up the execution of Erlang programs significantly through the introduction of an updated ‘just in time’ compiler.

Elixir and Phoenix have both reached maturity. They have both attracted a small but dedicated and enthusiastic group of supporters. The ecosystem around both continues to expand, although it still can’t match that around a framework such as Ruby on Rails.

But Elixir and Phoenix haven’t taken over the world and are still relatively niche. We’ll discuss why this is and some of the lessons that can be learned from the history of Erlang, WhatsApp, and Elixir in the paid supplement to this post coming on Tuesday. This post also contains loads of links to sources and supplementary material, including the briefest introductions to programming in Erlang and Elixir.

1. Ironically both men had been turned down for jobs at Facebook. Koum would subsequently become a director of Facebook.