Happy 2014!

New beginnings are exciting and I'm delighted to announce that I've joined a terrific team of people at Enova who are working hard to innovate and push Ruby well beyond its comfort zone.

I'm looking forward to sharing the journey with you as we build fantastic developer tools, migrate giant monolithic Rails apps, and create next-generation distributed applications that scale efficiently to all the cores. There are huge challenges given the engineering tasks, but also tremendous opportunities to demonstrate how powerful Ruby is and can be.

At Enova, I'll be continuing my OSS work on Rubinius and RubySpec. This is a generous contribution from Enova to the Ruby community. I'll also be devoting time to Rubinius X as we explore and address the many deficiencies in Ruby that continue to drive developers and businesses to other languages like Go, Clojure, and Node.js.

For those still running on Ruby 1.8.7, we'd love for you to collaborate with us as we build tools to migrate to newer Ruby versions. If you're already on Ruby 1.9.3 or later, we'd love to help you explore migrating from MRI to a Ruby implementation with a modern garbage collector, JIT compiler, and good support for concurrency. If you're considering rewriting your Ruby app in some other language, please let us know why.

If you've already rewritten your Ruby apps in another language, I'd especially like to hear from you. Rewrites always involve new architecture decisions as well. I'd like to understand if the Ruby language prevented your new architecture decisions or if better technology in Ruby would have saved the cost of rewriting an application.

Kumiko, Miwa, and I will be moving to Chicago. If you're in the area, we look forward to meeting you. If not, hopefully you'll come visit Chicago and say hello, maybe at RailsConf 2014!

Happy New Year!

Testing Your Project with Rubinius on Travis

Travis CI has been a tremendously useful tool for automating testing and has provided Ruby implementations with valuable feedback. However, it can be a trial-and-error process to find the right incantations for .travis.yml to get your preferred selection of Ruby implementations running. This post explains how to test your project on Rubinius.

With the Rubinius 2.0 release, there are no longer language modes. Rubinius is presently working on compatibility with the upcoming MRI 2.1 release. There are still compatibility issues to be fixed, primarily keyword syntax and some core library API changes. What this means for your .travis.yml file is that 'rbx-18mode' and 'rbx-19mode' are no longer supported. We submitted a patch to the Travis lint tool to check for this, so please use the linter!

So, if those language modes are no longer available, what should you add to your .travis.yml file? Remember that Rubinius is releasing new versions every week or so. Together with recent changes to RVM, this gives four options for how specific you want to be about the Rubinius version you test against. The following list of Rubinius options is from least-to-most specific:

  1. 'rbx' - This means the most recent Rubinius release. Every time your tests run, the most recently released binary will be used.
  2. 'rbx-X' - This means the most recent Major release. For example, if you use 'rbx-2' (which is the only one available right now), your tests will run on the most recent 2.Y.Z release, but would not run on 3.Y.Z.
  3. 'rbx-X.Y' - This means the most recent Minor release. If you use 'rbx-2.2', your tests will run on the most recent 2.2.Z release, but will not use the 2.3.Z or 2.1.Z release.
  4. 'rbx-X.Y.Z' - This means precisely the specified release. So, 'rbx-2.2.1' will run only on the 2.1.1 releease and no other release.

This method of designating the version of Rubinius you wish to run against should look similar to specifying versions of gems with the pessimistic operator (~>). It is intended to give you the same flexibility while allowing for stability constraints that you choose.

Since each of these designations is independent, you can mix and match them as you wish. For example, if you know that Rubinius 2.2.1 is green for your project, you can specify both 'rbx-2.2.1' and 'rbx'. Or you can always stay on the cutting edge by just using 'rbx'.

If you need help getting your project set up or updated on Travis, please let us know. I've created a simple project called travis-canary that you can check to see if the ways of specifying Rubinius listed above are currently working on Travis.

Happy testing!

The Parisian Release

Rubinius 2.1.0 has been released. I'm naming this "The Parisian release" in honor of the dotRB conference in Paris and today being Evan's birthday. I wanted this to be the 2.0 release, but circumstances, so sorry about that.

Following the release scheme I detailed in the 2.0 release, I've rev'd the minor version because we added a few C-API functions and different standard library loading stubs. The rest of the changes are basically bug fixes.

I'll be posting more soon about gems, packaging, and releases.

Enjoy! Preferably with a glass of nice wine but surely with something that makes you happy.

Introducing Rubinius X

ED: Removed a "frequently misused English phrase" to improve readability.

Today, I'm announcing the Rubinius X project. Please read about the details at the website. Here's why I started the project.

Ruby is a dying language. Business is over its dalliance with Ruby. No major startup is lauding their use of Ruby and existing businesses are migrating away or simply writing new applications in a different language.

Businesses care about providing customers value so the businesses earn profits. That's the harsh reality. Ruby is failing to help businesses engage customers. It is seen as inefficient and inferior to other languages.

It's not true that people are leaving Ruby because the language is no longer new. Javascript is not new and it's not a particularly good language. People use it because it helps them solve problems. Java is also not new and many people dislike writing Java code. But Java is evolving and businesses pay a lot of people to write Java. No business is rewriting their Java applications in Ruby because the developers are happier.

Ruby has great potential to help businesses compete in tumultuous markets by reducing the cost to experiment with products that may interest consumers. It has potential but fundamental things about Ruby must change. Simply writing some libraries or a new framework like Rails isn't enough.

I really want Ruby to be a language that businesses choose so that people who are happy writing Ruby will have a job. I'm going to test whether this is possible and I hope you will help.

I want to thank a number of people who have inspired me: Yehuda Katz, James Coglan, Gary Bernhardt, Evan Phoenix, Wilson Bilkovich, Rich Hickey, Michael Bernstein, Gerlando Piro, and many other.

The Once and Future Rubinius

Engine Yard has posted their statement about ending sponsorship for Rubinius, which gives me the opportunity to clearly address the future of Rubinius.

First of all, Engine Yard deserves great respect and admiration for their contribution to Rubinius and the entire Ruby community. I had the pleasure of interacting often with three of the Engine Yard founders: Tom Mornini, Lance Walley, and Ezra Zygmuntowicz. I have rarely had the good fortune to work with people as ethical, careful, and visionary as these folks. They endeavored to build community and business together, and they were highly influential in both.

Engine Yard's sponsorship of Rubinius certainly accelerated development and brought the project to the attention of many developers. Additionally, Engine Yard's sponsorship contributed to the success of RubySpec as an idea and tool for unifying Ruby compatibility across more than a half-dozen significant implementations of Ruby for the benefit of the Ruby community.

So, thank you very much, Engine Yard!

The simplest statement about the status of Rubinius is that there are now zero people paid to work on the project. This fact has several implications, none of which are inherently negative.

On the one hand, Rubinius is free to aggressively pursue the goals of the project in helping build the future of Ruby. On the other hand, I have significantly less time to devote to the project. While unfortunate, I'm not discouraged. I worked on Rubinius for over a year before Engine Yard hired me and we accomplished a tremendous amount.

We still have numerous things yet to do. Over the past several weeks, I have been working to simplify and focus the project so that all the time we can invest pays significant rewards for developers and businesses. We'll continue to streamline and accelerate delivering value to the people investing their time to use Rubinius.

Rubinius has a broad and ambitious vision. Since Evan Phoenix created it, Rubinius has been pushing the envelope. It was one of the first projects in the Ruby community to use git. One of the first big projects on GitHub. One of the first projects to use LLVM outside of the LLVM ecosystem. There have always been skeptics voicing their opinions about Rubinius using Ruby, building RubySpec, building our own virtual machine and garbage collector, removing the global interpreter lock, using gems, about almost every aspect of the project.

Despite this, Rubinius keeps moving forward. People are experiencing the tremendous value of running concurrent applications on modern hardware, saturating the CPU cores instead of blowing out the memory. It's trivial to migrate from MRI to Rubinius, continuing to use familiar platform tools and running C-extensions. The terrific response to the 2.0 announcement has been ample validation of our vision for Rubinius. We're just getting started.

Visit us in the #rubinius channel on Freenode and check out ways you can contribute to the project. The simplest, and always the most fun, way to contribute is to use Rubinius to do something you find interesting.

The future is, by definition, undefined. Let's define it.

Rubinius 2.0 Released

We are thrilled to announce the release of Rubinius 2.0. There are many exciting things to share. We'll review what Rubinius is, look at what you can expect from this release and future releases, and talk about plans for the future.

Many Thanks!

Rubinius 2.0 would not be possible without the tremendous support and contributions from so many amazing people.

Thanks Matz for creating Ruby. Thanks David Heinemeier Hansson for creating Ruby on Rails, which spread Ruby happiness around the world. Thanks Evan Phoenix for creating Rubinius and inspiring us to think beyond current limitations.

Thanks to the hundreds of contributors for their time, efforts, and frustration while improving Rubinius and RubySpec. There isn't room to list them all, but their indelible mark on Rubinius lives on in the source code.

Thanks to the Ruby developers, students, and businesses that all contribute to making Ruby better. Thanks to Engine Yard for financially supporting Rubinius development and sponsoring all those rad tshirts and stickers people loved so much.

All of these people and organizations, not just Ruby developers, are the Ruby community. Rubinius owes you a huge debt of gratitude.

The 2.0 Story

With the 2.0 release, Rubinius regains a laser focus on supporting the future of Ruby. Rubinius 2.0 is expected to be compatible with Ruby 2.1.

While MRI hasn't released 2.1 yet, Rubinius will continue improving compatibility as more features are finalized. Last RubyConf, Matz urged people to upgrade as soon as possible to Ruby 2.0. Significant effort has been dedicated to making the upgrade from 1.9 as simple as possible. Rubinius supports the effort to move Ruby into the future.

Rubinius started life with the goal of bringing modern technology to Ruby's implementation, giving developers more power, and businesses who rely on Ruby a faster, more stable and more efficient platform on which to build products and services.

Over time, we've tried to support multiple Ruby language versions, many different projects, old and new code, code that abuses every corner of MRI's ad hoc semantics, and every random, undocumented MRI C function with the Rubinius C-API compatibility layer. Unfortunately, this is unsustainable and not in the best interests of Ruby or Rubinius.

Starting with 2.0, Rubinius will concentrate on providing exceptional Ruby support for building modern concurrent and distributed applications. Not every legacy Ruby program or quirky Ruby feature will be suitable for Rubinius. Instead, we'll prioritize the performance and stability of concurrent applications to make Ruby competitive with Go, Erlang, Clojure, Scala, and Node.

Versions and Releases

Starting with Rubinius 2.0, we're changing the way releases are done.

Every week or so, we'll release a version of Rubinius. We are not following a pre-determined release schedule. We will continue to keep the master branch extremely stable, as we have done for years. If there are only bug fixes on master since the last X.Y.Z release, the new version will be X.Y.Z+1. If there are other changes, the new version will be X.Y+1.0.

We're moving to this release process to get updates into your hands as quickly as possible. No matter how much work we do, there is always more. A release never seems ready. Releases are painful to get right. So we are following the advice, "If something is painful, push it to the front and work to reduce the pain." There will be bugs with our Ruby 2.1 compatibility. If something is broken for you, please file an issue. Hopefully it will be fixed and released within days.

The goal is to semantically version the Rubinius core starting with version 3.0. During the 2.x to 3.0 transition, we'll be very careful about introducing breaking changes, but we'll do so when the benefits outweigh the risks. Obviously, the more we know about how Rubinius is being used, the better we can evaluate these decisions.

We've added a subdomain http://releases.rubini.us for hosting release tarballs. We expect that over the next few days the many Ruby installers and switchers will be updated to install Rubinius.

Below, in the section on future plans, I'll explain what we hope to accomplish with these ambitious release plans.

Rubinius Parts

Many people have heard that Rubinius is an implementation of the Ruby programming language. There's a lot wrapped up in that simple description. Let's review the major pieces of technology in Rubinius today. Later, we will look at plans to improve these in the future.

The Rubinius architecture is fairly standard for a modern language runtime.

The bytecode virtual machine (VM) runs the bytecode produced by the Ruby compiler. A notable feature is that every Ruby method essentially gets its own interpreter. This enables powerful features like the full-speed built-in debugger. Only the specific Ruby method with debugger breakpoints runs the "debug" interpreter while the rest of the methods run at normal speed.

The generational garbage collector (GC) has a very fast young generation collector, usually pausing for less than 15 ms to complete a collection. Applications running on Rubinius typically see shorter GC pauses times and many fewer noticeable GC pauses because the entire heap needs to be collected far less often. Rubinius also has a partially concurrent mark phase for the mature generation which further reduces the GC pause times when a full collection is required.

Rubinius implements native operating system threads for concurrency and has no global interpreter lock (GIL). Ruby code can run in parallel on multi-core or multi-CPU hardware.

The Rubinius just-in-time compiler (JIT) turns Ruby bytecode into machine code. The JIT thread is mostly independent of the Ruby threads so the JIT operation doesn't impact the running code's performance. The JIT framework tracks which methods are often used and what types of objects are seen. Using this runtime data, the JIT is able to combine application methods and core library methods, generating highly optimized machine code that runs several times faster than the bytecode interpreter.

The Rubinius core libraries (e.g. Array, Hash, Range, etc.), as well as Rubinius tools like the bytecode compiler, are written in Ruby. The Rubinius systems treat them just like Ruby application code (e.g. the JIT combining core library methods and application methods to best optimize running code). This consistency also improves understanding of the entire Rubinius system. Ruby developers can contribute to significant parts of Rubinius simply by writing Ruby code.

Plans, Meet Future

For Rubinius to have a place in future application development, it must serve the needs of those applications.

The world is rapidly changing and the rate of change is accelerating. Building software today is different than it was just five years ago. Continuous delivery and A/B testing are becoming commonplace. Businesses must experiment to discover how to compete in changing conditions. Driving down the cost of experimenting is essential.

More now than ever, time is money. The time scale to deliver features must be hours or days, not weeks or months. To meet the required velocity, building concurrent and distributed applications in a heterogenous environment is no longer optional, it is essential.

Ruby is more suited than many languages to rapidly deliver features, reducing the cost to experiment and the time needed to begin engaging customers. Unfortunately, Ruby development has not kept pace with the software as a service revolution.

Ruby became popular because Ruby on Rails accelerated the delivery of value by an order of magnitude. This influence is rapidly declining. Efficiencies that Rails introduced, things like convention over configuration and full-stack integration, also encouraged monolithic application architectures. Applications built this way are difficult to change and difficult to scale, which means that under changing conditions, their costs tend to quickly outweigh any value they deliver. Businesses are rapidly learning this lesson.

Future, Meet Plans

Roadmaps are notoriously painful because, oddly enough, predicting the future continues to be an inexact science. Given the future that Rubinius wants to support—concurrent and distributed applications—the following are specific areas we plan to improve in the coming weeks.

  1. Rubinius has no global interpreter lock, but we can significantly improve concurrency coordination in the system. During some phases of garbage collection, some operations of the JIT, and during fork/exec, we have to stop all the threads. We intend to improve this significantly so that less coordination is required.
  2. In place of the GIL, Rubinius uses finer-grained locks internally in various places. These can be reduced further, improving multi-core efficiency, by using modern lock-free concurrent data structures.
  3. While the garbage collector has reasonably low pauses, we can improve this by making the GC more concurrent and parallel.
  4. The JIT compiler already often improves Ruby code performance by 2-4x over executing bytecode. We can do even better. We will improve communicating Ruby semantics to the JIT to avoid unnecessarily allocating objects and doing unnecessary bookkeeping that slows performance. We will expose more of the JIT framework to Ruby to enable rapidly coding and testing new ideas for optimizing Ruby.

Gems as Components

The ability to compose independent components is one of the best means to manage high complexity. In Ruby, the natural way to package, distribute, and compose components is gems.

Rubinius wants to bring the advantages of continuous delivery and the "evergreen browser" idea to Ruby developers. The Rubinius approach to this is to fully leverage gems.

Rubinius itself has been dramatically simplified. Major components, like the bytecode compiler, Ruby parser, debugger, etc. have been moved to gems. These components can be updated easily and quickly without requiring a Rubinius release. These components participate in the Ruby ecosystem, for example Bundler or dep, like all the gems that Ruby developers are familiar with.

In Rubinius 2.0, the Ruby standard library has also been converted to gems. The Ruby standard library is some of the oldest Ruby code that exists. Due to being bundled with MRI all these years, it was far more difficult to change than a library or gem. Every change required MRI to accept it, and any changes required waiting for a new release of MRI, something which did not happen frequently.

It's not surprising that many people simply went around these obstacles and made alternate libraries. Instead of being the pinnacle of excellent Ruby design and idioms, parts of the standard library bit-rotted and retained anachronisms like embedded tests at the end of files. Further, many of the critical libraries were written as C extensions, making them unusable by JRuby, Topaz, IronRuby, Opal, and early MagLev (which now has some C-API support).

In Rubinius 2.0, the components and standard library are just gems. There is nothing special about them. They are installed as gems. They participate in gem sets and Bundler workflow as gems. There are some challenges required to bootstrap the gems, but that requires an internal Rubinius command, not changes to the RubyGems infrastructure.

Providing the standard library as gems opens the opportunity to rebuild it in Ruby and improve the code rapidly so that multiple implementations can share it. However, we are also not bound to using any of these libraries. Since they are just gems, they can be put in a Gemfile, or not, if there are better libraries available. There's no point spending time "improving" things that no one wants. Supporting newer libraries with better code and APIs may be much more beneficial. This is now an option.

We'll be monitoring what people do and where pain points are. Building on all the gem infrastructure, we open up a world of possibilities for Ruby developers.

Rubinius Inspirations

As 2.0 is a big milestone and transition, it's interesting to reflect on some significant Rubinius contributions to the Ruby community.

Rubinius has inspired a number of projects that have benefited Ruby far beyond Rubinius. The best known of these is RubySpec, which is used by every significant Ruby implementation, including promising new ones pushing the limits, like Topaz and Opal.

Rubinius also created the initial FFI spec that opened the world of native libraries to Ruby with a simple API across implementations and without needing to write C-extenions.

Evan created Puma to meet the need for a fast web server that would promote the Rubinius parallel thread support. Puma also works well on MRI and JRuby. It provides Ruby applications excellent performance and multi-core scaling, especially when there's no global interpreter lock.

Rubinius as a language platform has inspired many projects and encouraged people who may have thought language design was beyond their skills to experiment and discover the tremendous joy in creating their own programming language.

It is a joy that Rubinius has been a part of these efforts and we will continue improving developer experience in these areas.

Ready, Set, Ruby!

Concurrent and distributed applications aren't the future anymore, they are the present. They are vital to business success. The many talented developers that are passing over Ruby for Erlang, Go, Clojure and Node are draining Ruby of talent and vitality.

Ruby is an excellent language. Rubinius is dedicated to providing Ruby developers with excellent tools and technology competitive with these other languages. Developers who are happy writing Ruby shouldn't be forced to leave it because of technical limitations.

If you are one of these developers, let's build the future.