Ruby and Erlang, Talking to Each Other

A presentation for Ruby Users of Minnesota, March 2008.

OSS Packages Currently Available for Ruby/Erlang Communication

• RBridge, https://rubyforge.org/projects/ruby-mnesia/
  • Get the source, gem doesn't install correctly (on Linux/x86 at least)
• Erlectricity, https://rubyforge.org/projects/erlectricity/
  • Originally at Google Code, but now out-of-date!
  • gem install erlectricity
• Fuzed, use Erlang's Yaws HTTP server for Rails
  • Uses Erlectricity
  • http://fuzed.rubyforge.org/

Erlang<->C Communication

• Via an Erlang "driver" port
  1. External process, communicate via pipes, all comm is serialized
     • Safe (can't crash Erlang VM), slow (pipe data xfer, OS context switching, etc.)
  2. Shared library loaded into Erlang VM's memory space
     • Fast, unsafe, must still use driver port API to serialize data in both directions (mostly true)
• "C-node", a process that implements the Erlang message passing protocol
  • C-nodes look exactly like Erlang nodes
• Via TCP, UDP, SCTP, files, named pipes, ....

RBridge, Simple Communication

• TCP client-server design, no chicken-and-egg problem
• Basic design (if it were glue to Ruby instead): TCP server for "irb"
  • Data conversion to Erlang: ASCII
  • Data conversion from Erlang: ASCII S-Expression formatting -> Ruby types
• Data type conversion is limited, several Erlang types unsupported

RBridge Example

% /usr/local/lib/ruby/gems/1.8/gems/rbridge-0.1.2/rulang_server/rulang 9900 &
[RBridge Server Started]

% irb
irb> require 'rbridge'
=> true

irb> a = RBridge.new(nil, "localhost", 9900)
=> #<RBridge:0xb7f0ac14 @erlang=#<ErlangAdapter:0xb7f0ab38 @host="localhost", @port=9900>, @mod="erlang", @async=false>

irb> a.erl("[70, 71, 72].")
=> "FGH"

irb> a.erl("[70, 71, 172].")
=> "FG\254"

irb> a.erl("[70, 71, 272].")
=> [70, 71, 272]

irb> a.erl("length(erlang:processes()).")
=> 22

irb> a.erl("erlang:processes().")
RuntimeError: [Error]=>Error: {function_clause,[{local,to_ruby,[<0.0.0>]}]}

        from ./rbridge.rb:77:in `erl'
        from (irb):3

irb> a.erl("spawn(fun() -> ets:new(slf, [set, named_table, public]), receive die -> ok end end).")
RuntimeError: [Error]=>Error: {function_clause,[{local,to_ruby,[<0.68.0>]}]}
        from ./rbridge.rb:77:in `erl'
        from (irb):25

irb> a.erl("ets:insert(slf, {\"scott\", 39, \"louise\"}).")
=> "true"

irb> a.erl("ets:lookup(slf, \"scott\").")
=> [["scott", 39, "louise"]]

RBridge Data Conversion To Ruby

Critique of RBridge

• Any Ruby -> Erlang data must be flattened to ASCII, using Erlang syntax
• Easy to call arbitrary Erlang functions
• Impedence mismatch: somewhat high

Erlectricity

1. Start an Erlang virtual machine
2. Open a driver port to a new Ruby OS process
3. Send (nearly) arbitrary Erlang term to port
4. Read Erlang term from port (optional)

• The Chicken and Egg:
  • The Erlang VM runs first, Ruby's is a child OS process.
  • Ruby's STDIN and STDOUT is used for communication with Erlang

Simple Example

See code in http://www.snookles.com/erlang/ruby.mn-Mar2008/.

% erl
Erlang (BEAM) emulator version 5.5.5 [source] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.5.5  (abort with ^G)
1> c(slf1).
{ok, slf1}

7> Port = slf1:start().
#Port<0.111>
Before top of loop

8> slf1:msg(Port, {hello, world}).
you_betcha
At top of loop
Before when 1
After when 1
After when 1b
After when 2
got :hello :world

9> slf1:msg(Port, {hello, <<"world">>}).
you_betcha
got :hello "world"      

11> slf1:msg(Port, {hello, "world!"}).                         
you_betcha
got :hello [119, 111, 114, 108, 100, 33]

13> slf1:msg(Port, {hello, [1, 2, 3, {foo, [], bar}]}).
you_betcha
got :hello [1, 2, 3, [:foo, [], :bar]]

15> slf1:msg(Port, {hello, this, is, a, 6, tuple}).   
you_betcha
./slf1.rb:46: warning: multiple values for a block parameter (6 for 1)
   from /usr/local/lib/ruby/gems/1.8/gems/erlectricity-0.2.0/lib/erlectricity/matcher.rb:14
catchall: [:hello, :this, :is, :a, 6, :tuple]

Reading a Reply Term From Erlang

16> slf1:get(Port).
no_data_in_mailbox

18> slf1:msg(Port, {sleep, 1}).
you_betcha
Going to sleep for 1 seconds ... done

19> slf1:get(Port).            
{done_sleeping,1}

20> slf1:get(Port).
no_data_in_mailbox

32> [ slf1:msg(Port, {sleep, 0.5}) , slf1:get(Port), timer:sleep(1000), slf1:get(Port) ].
Going to sleep for 0.5 seconds ... done
[you_betcha,no_data_in_mailbox,ok,{done_sleeping,0.500000}]

Critique of Erlectricity

• Ruby code is creating Erlang terms, communication is fairly efficient.
• Mostly-straightforward mapping of types, back and forth
• Lists and tuples are the exception
  • Ruby Array -> Erlang tuple
  • Ruby Erlectricity::List -> Erlang list
• Not so easy call arbitrary Erlang functions
  • But easy enough to fix on app-by-app basis
• Impedence mismatch: fairly low
• Wouldn't be too much work to create an independent node
  • Use TCP/IP instead of local pipes
  • Remote nodes can't tell the difference: Erlang, Ruby, Folger's Crystals, ...

Fuzed, Not Mongrel

• Yes, it works.
• As of 30-March-2008, don't use the gem. Fetch the code via "git".
• Packaging is targeted at interactive/development environment right now.
• Maintains a dynamic pool of client Erlang nodes for processing Rails requests.
  • Dispatch from client pool is LIFO.
  • Each client pool member runs two Ruby OS processes.

-- Main.fritchie - 30 Mar 2008