the open source Cluster System - the fastest, cheapest way to build reliable clusters of computers

What is a Cluster System?

An Operating System (O/S) is a set of libraries that abstract your hardware and which mean that your unwritten software can speak to persistent storay, networks, input/output devices, etc, etc.

Similarly a Cluster System (C/S) is a set of libraries that run across many computers (with different O/Ses) which means that your unwritten software can scale horizontally, failover when a computer goes down, have hardware replaced under it ...comprende?

Are There Other C/S's?

Yes, the best known one is the Google App Engine.

But, like in the early days of operating systems, most running C/S are in-house and custom-built in big companies like Facebook, Twitter etc.

Amazon provide a lot of components from which you can build your own proprietory C/S - things like AWS, S3, EBS, and many more.

Erlang, Erlang/OTP? Eh?

Erlang is a programming language

You can use Erlang to programme an Erlang/OTP cluster.

It is perfectly possible to use Erlang on a single computer (a cluster-of-one).

There are other languages you can use to programme an Erlang/OTP cluster, like Elixir, LFE and Joxa, as well an stage experiment in enclustering browers - LuvvieScript.

But Nobody Uses Erlang/OTP Anyway...

(apart from these companies, obviously)

Hey CEO! Hey CTO!

all you need to know about Cluster Systems

Why Use Erlang/OTP?

If your business is going to run on computers then you will need to have a cluster system of some sort.

You can either pay people to write and maintain your C/S or you can use an open source battle-tested one like Erlang/OTP

In the next 5 years everyone will be using Cluster Systems - just like everyone uses Operating Systems

The Top 3 Reasons

  • Cost cheaper to use an open source C/S than write or rent one
  • Speed To Market quicker to use an C/S than write one
  • Availability And Reliability Erlang/OTP systems have been measured at 99.9999999% uptime (31ms a year downtime)

Decisions, Decisions...

Q: Do I need many computers?
A: No
Use an excellent O/S languages, like Ruby, Java, PHP, Perl, Clojure, Scala, C++ (or Erlang)
A: Yes
Choose between the Google App Engine (a commercial C/S), Erlang/OTP (the open source one) or write your own.

Lets Get Started

if you can't build a cluster in 2 minutes I will eat your hat

You won't believe it is this easy, no you really won't

Its super easy to set up your first cluster - you should be able to do it in 5 minutes (once you have Erlang/OTP installed). Erlang/OTP runs on Linux, FreeBSD, most Unices, Mac OS/X and Windows.

We will install erlang and then open two terminals.

In terminal 1 type:

erl -sname alice -smp

An erlang VM will start with a running shell and command line, which looks like:

Erlang R16B01 (erts-5.10.2) [source] [smp:1:1] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.2  (abort with ^G)

You know have node alice running. Now type observer:start(). into the console. A system console will start like so:

Erlang/OTP Observer

Lets start bob. In terminal 2 type:

erl -sname bob -smp

Erlang VM's admit other VM's to their cluster if they share a common cookie. You need to check that alice@yourcomputer and bob@yourcomputer have the same cookie with the command erlang:get_cookie(). If they do then you can join them into a cluster by running net_adm:ping('bob@yourcomputer'). into alice@yourcomputer. If you type nodes(). into one console you should see the other VM's name.

If you need to sync the cookies use erlang:set_cookie(node(), somecookie). - cookies that don't start with a lowercase letter needed to single quoted.

See, told you it was easy. You can put your hat away now, I'm not going to eat it, am I?

Exploring Your Cluster

Poke around the Observer and see what you can see.

Different panels show you different things that look vaguely familiar from operating systems, process lists, memory allocation, applications, but also strange stuff like tracing.

By using the node menu you can point the observer to a different VM in the node.

This is all the familiar/unfamiliar stuff you need to fold into your operations team. Erlang/OTP supports SNMP alerting and a host of other stuff to integrate it into your ops support world.

Processes, Processes, Processes

The process tab in the observer shows all the running processes in the VM.

It is the Erlang/OTP version of Unix's ps aux command.

Whereas in Unix each process Id (PID) is a single numeric value all the Erlang/OTP PID's are these stange three-parters <0.50.0>. The three parts describe which:

  • process it is on its home node
  • VM it is on its home machine
  • machine it is in the cluster

Applications in Erlang-the-language works by processes sending each other messages - and that ultimately depends on knowing each other PID's.

An Erlang processes doesn't distinguish between a process on its VM, or one on another member of the cluster. This network transparency is the rock on which Erlang/OTP is built.

Operational Debugging

Have a squint at the trace panel. Erlang/OTP has the built in ability to trace a message between one process and another - across the cluster.

Trace matches can be applied to process and function calls to trigger a trace process. You can trace messages which are sent, which trigger other messages and the spawning of new processes.

The consequence of this is that you have powerful built in tools inside your C/S which enable you to address the most intractable problems.

But I Heard Its Too Freaky-Weird!

well you heard wrong buster...

Lets Start With Hello World, Shall We?

A lot of people get confused here. Erlang-the-language is very small and easy to master. Erlang/OTP-the-cluster-system is large. Remember when you first started learning Linux - all that stuff, networks, cron jobs, /dev, filesystem mounting /etc/init.d - all that stuff you need to master for O/S programming?

The good news is that there isn't as much stuff to master to programme a C/S but there is still a lot...

Don't believe me?, well here's the full Erlang/OTP documentation.

Hello world in Erlang is simple:

#!/usr/bin/env escript

main(_Args) ->
    io:format("Hello World!~n").

But Erlang Is Wierd, Right?

Whisper it not in Gath, but it is a bit weird: if you don't know Prolog (and who does these days?). If you are used to Algol-style languages (like most programmers) its a bit scary. It feels like the difference between reading a book and reading an opticians chart. Here are some of the stranger bits.

overloaded = and variables which don't vary

Sometimes = means let this thing equal that thing and sometimes it means check that this thing equals that thing, and sometimes it means both.
A = 1, % A doesnt have a value (it is unbound) so let A = 1
A = 2, % A does have a value (it is bound to 1) so confirm A = 1 or throw error

These two meanings can be combined into one equals sign:

A = 1,          % let A = 1 it is unbound
{A, B} = {1, 2} % check that A = 1 and also let B = 2

This is the basis of pattern matching - which is a key weird and adictive thing. You might know it from CoffeeScript where it is called destructuring.

the strange case of case

Erlang loves case statements. There are 5 ways to write case statements and you can merge some of them.

case X of
   1 -> do something();
   2 -> do somethingelse();
   _ -> ok


  X = 1                   -> dosomething();
  X = 2                   -> dosomethingelse();
  X =/= 1 andalso X =/= 2 -> ok

multi-headed function hydras

The multiheaded function (what a strange beast is this) with pattern matching:

chose(1) -> dosomething();
chose(2) -> dosomethingelse();
chose(_) -> ok.

The variable _ means I don't care about the value

Or the same multihead function with guards:

chose(X) when X == 1 -> dosomething();
chose(X) when X == 2 -> dosomethingelse();
chose(_)             -> ok.

You can also consider a pattern-match as a case-statement with one clause:

X = 1,


If X is 1
   then continue
   else throw an error 'X isn't 1'

whats for for anyway?

There is no for operator and no loops. You use recursion.

You are probably used to seeing constructs like this in other languages:

for (i; i < N; i++) {

In Erlang this would be written as:

do_fun(0) ->
do_fun(N) when is_integer(N) ->
   do_fun(N - 1).

The function calls itself and counts down from N to 0

People also freak out about the , ; . stuff but that is easy, they just mean AND OR and END

A = 1,
B = 2,
   case {A, B} of
      {1, 2} -> dosomething();
      {2, 2} -> dosomethingelse()
{A, B}.

You can read this as:

A = 1 AND
B = 2 AND
   case {A, B} of
      {1, 2} -> dosomething() OR
      {2, 2} -> dosomethingelse()

And freaky list comprenensions look like an ascii art dance of death:

L = [1, 2, 3]
[X * 2 || X <- L]

This means take the list L get all the variables X in that list and multiply them by two. This returns:

[2, 4, 6]

Muslims say that all human things are flawed, and only God is perfect. The great Muslim caligraphers would make a deliberate mistake in their work so God wouldn't think them too proud. Getting the final semicolon wrong in Erlang (which we all do) is the Erlang developer's acknowlegement of their own human frailty.

That's all the weirdness, perfectly surmountable by a software developer of your calibre (schmooze, schmooze...)

That Syntax Still Too Freaky For Ya?

We got plenty others

Seriously Erlang is not the only language you can programme your C/S with. There are loads of others

Need Yourself Something Ruby-Like? Try Elixir

Elixir has been developed by one of the core rails team, and it shows...

defmodule Hello do
  IO.puts "Defining the function world"

  def world do
    IO.puts "Hello World"

  IO.puts "Function world defined"

Run and you will get the output:

Defining the function world
Function world defined
Hello World

Lisp more to your taste? Try Joxa

Joxa is a crisp Lisp...

joxa-shell> (defn foo () :hello)
joxa-shell> (foo)

And There's More...

For a very traditional Lisp try LFE (Lisp-Flavoured Erlang) by one of the inventors of Erlang. You want traditional Scheme, we got it or you like Lua scripting, good to go. Perhaps you are even one of those freaks who thinks Erlang is not-prolog-y enough, well here's prolog, step right up.

But Some People Are Never Satisifed

<-- Insert Your Name Here --> has written their own language <-- Your Cool 4 Letter Domain --> inspired by <-- What Tickles Your Fancy-->.

Erlang has a hidden gem called Core Erlang which is a verbose but very simple human-readable language - designed for humans to read (but not optimised for them to write).

It is also designed to make it super-easy to write your own compile-to-Core-Erlang-source language - and paticularly to iterate on the design.

<-- Insert Your Name Here --> used Core Erlang to develop and prototype their <-- Whatever Tickles Your Fancy -->-flavoured <-- Your Cool 4 Letter Domain-- > language giving it all the goodness of Erlang/OTP.

Break Glass For Emergency Tweet

Dear @rvirding @rich_4711 @josevalim @gordonguthrie @ericbmerrit I am writing <-- Your Cool 4 Letter Domain --> an Erlang/OTP C/S language. Help!

You Like Your Steak Bloodier Still?

Erlang has a concept called C-node's where things like special hardware, the Emacs editor, or Unicode libraries written in C can be exposed as junior members of the cluster

You can just send and receive messages from C-node's.

The bleeding-edge LuvvieScript Project is trying to wrap the browser as a C-node so the server-side components of a cluster and can just transparently send and receive messages

All the failure handling (ie my phone signal has died) will be managed in the background.

It also wants to wrap the DOM as a C-node so the front-end programmer will be abstracted from the browser.

Thanks to the miracle of Core Erlang, when it works for normal Erlang it will work for Elixir, LFE, Joxa and <-- Your Cool 4 Letter Domain --> as well.

That steak's still a-mooing, yeah-hargh!

Blog Posts

Read More!