I load tested (using wrk) nginx serving a two line, static HTML file, against a basic Cowboy service that parsed parameters, did an ETS lookup, and rendered several hundred json elements, on my EC2 micro.
nginx came in at 14612 requests per second, and the dynamic code under Cowboy option came in at 13336 per second.
The same server was pulling 147 requests per second for a "hello world" rails application (Unicorn, production mode). Cowboy really isn't joking when they talk about performance.
I guess you can either see it as how much faster Cowboy is or how much more useful Rails is. It's nice if you have a lot of Erlang code already, but if you're making a web application from scratch, performance is pretty far down the list and I would stick to the half dozen stacks that are proven. Erlang can make sense sometimes, but it's probably not(!) comparable to Rails in those cases.
Edit: Eh, I forgot a "not" in the last sentence, which is now added.
> Erlang can make sense sometimes, but it's probably not(!) comparable to Rails in those cases.
I have no dog in this fight, but I've seen many people say that -despite the newness of the library "ecosystem"- they get an awful lot done with Elixir/Phoenix, and have a lot of fun doing it.
(Remember that Elixir is -like Erlang- a BEAM language. This means that Elixir gets the ease-of-understanding and interesting features of its syntax and has access to all Erlang software ever written. Similarly, any and all Elixir code can be called from Erlang with little trouble.)
Edit: (Also remember that the BEAM equivalent of Ruby is Elixir, and the (most popular?) BEAM equivalent of Rails is (AIUI) Phoenix. Phoenix uses Cowboy as its HTTP(S)/SPDY server.)
Yeah don't get me wrong, this wasn't some blog spam about "why I don't use Rails".
In fact I recommend it for a lot of cases. Problems that were solved years ago in Rails, like sane and default CSRF protection, are showing up in newly written PHP7 and Node applications that I'm seeing, because the things that Rails does are generally there for your own good.
I'm also aware it's not a direct comparison because Rails actually had an edge as far as its codebase.
The point I believe we can still take away from the original comment is - Cowboy is fast.
I'm very much looking forward to v2, the way it handles parameters is a much more natural fit imo.
The Phoenix Framework uses Cowboy at its core along with a Rack like middleware called Plug. There isn't much bloat in the stack so I'd assume performance is quite close to bare bones Cowboy. It doesn't quite have everything that Rails does, but it's getting there and fast!
Almost. I believe that half of the people that approach Phoenix coming from Rails stop when they realize there isn't any registration and authentication framework as good as Devise. The closest match is Addict but it covers only the basics and requires too much work. If one has to write his own product that could be OK. If one has to code for a customer it erodes the profit.
1) The Elixir community is still in its infancy and the community is growing rapidly and flourishing. If the concern is there isn't an equivalent Elixir library for X gem, it's only a matter of time before it is. So please join!
2) I attended the Phoenix training at ElixirConf this week and an easy answer to registration and auth is "hey, write a plug". Once you dive into how straightforward plugs (1) and what (init/1, call/2) do, it's really simple.
You will appreciate the explicit nature of plugs compared to say, Rack middleware...
I'm rewriting a Rails app in Phoenix and overall, I've found it to be a really pleasant experience.
I'm not so concerned by the lack of libraries as with that no one seem to know the state of Elixir, Phoenix or BEAM. Security vulnerabilities are found all the time in established stacks and things like robustness, reliability, maintenance and growth are ongoing issues. Yes, BEAM is proven in enterprise use, but most of us don't have those resources. The first time I tried Elixir the welcome site broke, which wasn't a big deal, but it still didn't exactly instill a lot of confidence.
> Security vulnerabilities are found all the time in established stacks and things like robustness, reliability, maintenance and growth are ongoing issues.
If you stick to writing code only in a BEAM language, you're insulated from all errors except for those of system resource exhaustion (edit: or -obviously- system hardware failure ;) ) or programmer logic errors. [0]
The parts of Elixir and other such projects that are written in another language that compiles down to native code, [1] are vulnerable to the usual array of issues endemic to the language selected.
> Yes, BEAM is proven in enterprise use, but most of us don't have those resources.
I'm not sure what you mean by this, and what concerns are implied by this statement. Would you care to expand this statement?
The WhatsApp folks have found that they got a lot done with Erlang with a small staff and (what is reported to be) very few machines when compared to the amount of work performed.
[0] Erlang can neither save you from your own faulty logic, nor can it add resources to your system. ;)
[1] Erlang provides a mechanism called NIF that lets you write performance-critical (or whatever) code in C or another language, and link it in to Erlang. From there, you can access your other-language code with from your Erlang code what appears to be a regular Erlang function call.
I guess in case of a web application it wouldn't be so much BEAM itself as the libraries. It hard to know if e.g. the crypto library is secure and used correctly. The same goes for serialization and other things that normally leads to problems further up the stack.
"Would you care to expand this statement?"
It seem proven enough if you have dedicated highly skilled developers. Not necessarily if you're a smaller startup that have to do a lot of other things and are relying on the ecosystem. With one of the more well know stacks you have a long history of not only security fixes, but knowledge about how to do things.
It kind of boils down to if I would implement say a payment system with the stack could I feel confident we wouldn't get compromised? Currently I have a lot higher confidence in e.g. Python + Django + Nginx than Elixir + Phoenix + Cowboy.
I understand and largely agree with your core points and concerns. I, myself am cautious and suspicious of new and/or unfamiliar software stacks. :)
So, the following disjointed commentary might be entirely superfluous. It also makes no mention of Elixir or Phoenix, as I've not yet used them:
AFAIK, everything shipped in Erlang/OTP has a test suite that you can run and -if you've the time and technical chops- inspect and evaluate for completeness and correctness.
Given that the Erlang community tends to be concerned about safety and reliability, I have substantially more confidence in the correctness of some random Erlang library on Github than of some random JavaScript (or -ye gods- Perl) library. ;)
Given that neither Cowboy nor Ranch appear to make use of any NIFs, I would be fairly confident in their safety in the face of garbage or malicious input.
The erlang-questions mailing list is a good place to ask questions of folks who use Erlang professionally and non-professionally for a wide variety of things. I've heard that the Elixir equivalents are similarly helpful.
Yeah, the routing system is. Fred Hebert (aka ferd, the author of "Learn You Some Erlang" and "Stuff Goes Bad: Erlang in Anger") is one of the folks working at Heroku on exactly that.
> One lesson I took from the first .com wave in the early 2000 was never to use a scripting language for application software again.
A couple of things:
Both Erlang and Python compile down to bytecode. Python compiles the first time you run a given script, Erlang won't execute Erlang source code; you must compile it to run it.
As go1979 mentions, Erlang has HiPE, which compiles Erlang down to the host platform's native code. If you find that your program is spending a lot of time in the Erlang code in a given module, you can -at any time- recompile that module with HiPE and reload it with either a system restart, or the hot code reloading machinery.
As an additional escape hatch, Erlang makes it rather easy to push performance-critical code out to a faster language -like C or C++ or whatever- and call that directly from Erlang. [0] Such code is called an NIF [1].
If most or all of what you're doing is hard-core number crunching, Erlang is probably not appropriate. If what you're doing requires tricky logic, needs to contain failures, or you're building a concurrent or distributed system, Erlang may very well be the right tool for the job.
Erlang's HiPE compiler is apparently AOT. The challenge seems to be dynamic typing. At points, it has to guess types. I assume compiled Python would have similar issues.
In the distant past, some guy performed such a benchmark on then-current versions of several popular web frameworks in Ruby, Go, maybe Python, and Elixir. Phoenix wasn't the fastest of the bunch, but it was also not the slowest, and -IIRC- provided the most consistent response time under heavy load.
I was either unaware (or had forgotten that) he had taken a second stab at the benchmark. I do hope he updates the document with the root cause of the enormous variance in response time.
I note with interest that the two frameworks which are faster than Phoenix (in both throughput and latency) are Gin, written in Go, and Play, written in Scala. In this run of the benchmark, they also have more consistent latency than Phoenix, although the author notes a suspected problem with his Erlang installation; previous runs had consistency better than Play, but still not as good as Gin.
I note with amusement that:
> Play consistently appeared to generate hundreds of socket read errors (see the detailed output), so I believe it should probably be semi-disqualified from the results for now.
I'd be interested to see another JVM framework tested. Play is really rather complicated, and it's written in Scala, which is also rather complicated. I'd love to see a comparison with a Sinatra-for-Java type framework like Spark, or perhaps Undertow (since it does so well on the TechEmpower benchmark). Oh well, maybe it's pull request time ...
Until he figures out what was up with his terrible Elixir consistency numbers, I would treat the second round of results with some suspicion. Maybe something changed in Phoenix/Elixir/Erlang, or maybe something's up with his machine.
If you do submit a PR, I do hope that you have the time and gumption to re-run all of his round 2 benchmarks, in addition to any additional ones you care to see. :)
nginx came in at 14612 requests per second, and the dynamic code under Cowboy option came in at 13336 per second.
The same server was pulling 147 requests per second for a "hello world" rails application (Unicorn, production mode). Cowboy really isn't joking when they talk about performance.