This blog has been archived. Our writing has moved to makandra cards.
The blog of , a Ruby on Rails development team

Our new e-book: Growing Rails Applications in Practice

Find below the opening chapter of our new e-book: Growing Rails Applications in Practice. The complete book will be published this summer. Enjoy!


How we got here

When you started doing Rails some years ago, it all seemed so easy. You saw the blog-in-ten-minutes video. You reproduced the result. ActiveRecord felt great and everything had its place.

Fast forward two years. Your blog is now a full-blown CMS with a hundred models and controllers. Your team has grown to four developers. Every change to the application is a pain. Your code feels like a house of cards.

You turn to the internet for assistance, and many solution seem to be available. You should move away from fat controllers, it says. But do avoid fat models. And use DCI. Or CQRS. Or SOA. As you cycle through patterns, your application is becoming a patchwork of different coding techniques. New team members are having a hard time catching up. And you're beginning to question if all those new techniques actually help, or if you're just adding layers of indirection.

You start missing the early days, when everything had seemed so easy and every new piece of code had its place. You actually liked ActiveRecord before it drowned you in a sea of callbacks. If only there was a way to do things "the Rails way" without having it fall apart as your application grows.

The myth of the infinitely scalable architecture

We'd like to show you one path to write Rails apps that are a joy to understand and change, even as your team and codebase grows. This book describes a complete toolbox that has served us well for all requirements that we have encountered.

But before we do that, we need to let you in on an inconvenient secret: Large applications are large. The optimal implementation of a large application will always be more complex than the optimal representation of a smaller app. We cannot make this go away. What we can do is to organize a codebase in a way that "scales logarithmically". Twice as many models should not mean twice as many problems.

Here is a chart that we made up:

Vanilla Rails vs. structured Rails

In order to walk the green line in the chart above, you don't necessarily need to change the way your application is built. You don't necessarily need to introduce revolutionary architectures to your code. You can probably make it with the tools built into Rails, if you use them in a smarter way.

Compare this to sorting algorithms. When a sorting function is too slow, our first thought is not "install a Hadoop cluster". Instead we simply look for an algorithm that scales better. In a similar fashion this book is not about revolutionary design patterns or magic gems that make all your problems go away. Instead we will show how to use discipline, consistency and organization to make your application grow more gently.

We have three principles that guide our design decisions:

1. Be boring

We have a universally applicable default design that maps any user-faced interaction to CRUD and ActiveModel. By being consistent in your design decisions we can reduce the cognitive overhead required to follow your code. This makes it easier for colleagues and new developers to navigate and change our source.

Read about this principle in the following sections:

  • Why controllers should be boring
  • Using ActiveRecord effectively
  • User-facing models without a database: ActiveModel
  • Taming style sheets

2. Be organized

We organize code in a way that encourages the creation of new classes. By isolating screen-specific concerns from a slim core model we can reduce the side effects that models can have on other parts of your application.

Read about this principle in the following sections:

  • Dealing with fat models
  • Taming style sheets
  • Taming javascripts
  • Organizing large codebases using namespaces
  • Don't let authorization infect your application

3. Be humble

Where we suggest using before/after code comparisons to avoid being sidetracked by the design fad of the day. We will make a case for adopting new technologies and patterns with care, and take full responsibility for those techniques and technologies you do choose to adopt.

  • How to test effectively
  • Hipster Patterns
  • Owning your stack
  • Surviving the upgrade pace of Rails

Are you interested in this book?

We're currently finishing up the final chapters. Growing Rails Applications in Practice will be released this summer. If you're interested in the book, please enter your e-mail address into our notification form. We will let you know as soon as we publish!

Growing pains

First off: You might have noticed that the name of this site has changed from Gem Session to simply makandra blog. It's still us, makandra. Please don't unsubscribe.

You might also have noticed that the last post before this one is from 2011. That's three years ago. Yes, we're still alive. Yes, we're still doing Ruby on Rails. We've just been super busy.

1. We've become a supplier for the Ruby industry

I'm not sure if there even was a Ruby industry a few years ago, but by now Ruby and Rails has become a huge technology sector. We now see massive Rails projects facing serious challenges of scale and security. We have moved on from being a pure Ruby consultancy to supporting owners of commercial Rails projects.

When official support for Ruby on Rails 2.3 and 3.0 ended with the release of Rails 4, we picked up the torch with Rails LTS. We now continue to provide security patches for Rails 2.3 and 3.0, protecting existing applications or providing project owners with more time for upgrading.

As a companion service we launched Ruby Backports. By backporting a gem you can use modern libraries without having to upgrade your Rails or Ruby. Or you might want to port a security patch to a version branch that is no longer maintained by the community.

By offering services like Rails LTS and Ruby Backports we hope to bring some calm into an ecosystem that traditionally had issues with API stability.

2. We're writing a book about growing Rails projects

OK, everyone's writing a book. But ours is special. Super-promise.

How to make large Rails applications not fall apart is the number one challenge Rails teams all over the globe are facing right now. There are a number of approaches like SOA, DCI or embedded engines, which all come with their benefits and drawbacks. What's missing is a good default for the average Rails team: What low-ceremony measures can you take to keep a Rails application maintainable as it grows into its second or third year of development?

By taking a dozen Rails applications from inception to deployment each year, we had the unique opportunity to experiment with different architectural patterns. Since we often maintain these projects for years afterwards, we have a pretty good idea which techniques tend to work out and which don't.

By distilling our experiences into a single book we want to show you one path to create a Rails app that is a joy to understand and change, even as your team and code base grows. We won't talk about revolutionary design patterns, hip techniques, or awesome gems that automagically solve all your problems. Instead we will show how to use discipline, consistency and organization to make your application grow more gently.

We're currently finishing up the final chapters. Growing Rails Applications in Practice will be released this summer. If you're interested in the book, please enter your e-mail address into our notification form. We will let you know as soon as we publish!

3. We've been blogging elsewhere

Remember when we told we how we open-sourced our internal knowledge base and built makandropedia? When we did this in 2010 we had about 200 HOWTOs for Ruby, Rails, RSpec, Cucumber and Javascript. Since then we've been blogging there. A lot. Today makandropedia contains over 3000 cards. We liked it so much that we sort of forgot about this blog.

We will continue to post technical guides and HOWTOs to makandropedia, so if you're following us for that, you should subscribe to makandropedia's feed. We will use this blog for rants and commentary about software craftsmanship, technology and the world of Ruby and Rails in particular.

4. We've just been busy

When we entered our third year as a company in 2011, we experienced some growing pains that are the last thing on your mind when you launch a startup. Despite this transition and all the product development we launched so many Rails projects in those years that we're having a hard time remembering them all.

We're sorry for falling off the grid. We're sorry we didn't call. But sometimes you just need to sit down and get shit done.

Expect more posts on this blog soon. Make sure you subscribe to our feed. If you're not into feed readers you can also follow us as @makandra_de on Twitter or on Facebook. We will re-post links to new blog articles on both networks.

Meet us in Berlin this weekend

We're on our way to Berlin for the Plat_Forms 2011 results presentation and awards ceremony. It will take place tonight (Friday) at the Freie Universität Berlin. You can meet us there if you’re interested. No word yet about results, but we’re rooting for the Ruby teams.

The results presentation will be followed by an unconference this Saturday. It’s free to attend, just add yourself to the attendee list. I will give a crash course into Test Driven Development. Thomas will talk about how we’re sharing knowledge at makandra and how makandropedia has eventually evolved into a knowledge management solution for everyone. There will be some other sessions as well.

If you have trouble finding us, just e-mail me your cellphone number (henning dot koch at makandra dot de). I will occasionally check my e-mail and give you a call.

I hate infinite scrolling

From the very bottom of my heart, I hate infinite scrolling. I hate how it makes the scrollbar jump while I'm dragging it, I hate how it obscures how much of a document I've already read, or how much content there is left. I hate how it seems like such a good idea at first glance, but ultimately just annoys the shit out of everyone. I believe that every time a developer implements infinite scrolling, a kitten orphanage catches fire, everyone dies, and the world becomes a worse place.

Don't implement infinite scrolling.

Scalable, redundant failsafe Rails hosting infrastructure

Over the past two and a half years the applications we host for customers grew constantly up to several dozen projects today. We insisted on our principles at each point of time over the years: Keeping things simple. We successfully develop software using this principle and we successfully operate our customers' applications using this principle. Dijkstra knew that "simplicity is prerequisite for reliability" and we state that it is the prerequisite for security, maintainability, scalability and basically whatever else you want.

Up to now we hosted our projects on a single more or less powerful machine, having a backup on a secondary server that kept database and files in sync. Failover (which we fortunately never had to use) was handled by manually routing traffic from the primary to the backup machine. As the number of projects grew we were searching for a (simple but stable, powerful but cheap) solution that offers

  • fully automated, real-time fail-over for HTTP traffic
  • scalable design
  • economical efficiency
  • ready for our upcoming Ruby on Rails cloud™ infrastructure

What came out of what you see in Figure 1. The setup consists of a publicly accessible layer where DNS points to (web) servers. All traffic is terminated there. Incoming traffic is routed to the background application servers internally.

makandra Rails hosting infrastructure

For the web slaves, we have two machines running FreeBSD 8.2 that act as reverse proxies (utilizing nginx) to the backend application servers (Passenger). As most of our applications use SSL there are quite a lot of public IP addresses configured on both BSD machines. Failover for the web servers¹ (on IP level) is handled by the amazing CARP protocol.

CARP can be configured to run in MASTER/BACKUP mode which means the MASTER gets all the traffic and BACKUP only comes into play when the master fails. CARP can also run in load balancing mode where traffic is distributed among the participating machines. To decide whether the servers are up, the master machine sends out advertisement packets so the backup slaves know it is alive. If the backups do not see the advertisement packets for some time, one of the backup slaves becomes the new master. See FreeBSD handbook and OpenBSD FAQ for more details.

The web servers have both an interface within our public VLAN (that is Internet) as well as the private one, which is used for connections to the application servers. Failover (or load balancing) for the backend application servers is carried out by nginx' proxy module.

This failover on IP level is so important for us because we have dozens of domains pointing to unique IPs (as mentioned, because of SSL) at our infrastructure and we do not even have control over some of the DNS servers. In case of hardware problems with the web servers, we have the secondary waiting to take over automatically with nearly no downtime. The very same for the application servers: If one of them dies, the secondary automatically takes over.

A nice side effect of this setup is that we can easily make use of nginx' caching possibilities and SSL traffic is loaded off to separate machines. We're currently planning to add a layer to our infrastructure where customers can have dedicated rails instances in the backend and profit from the failover web slaves, too.

We rolled out this new infrastructure a few weeks ago using Puppet for automated configuration of all involved parts. We are optimistic to give a detailled report about our (positive) experience at some point in the future.

¹ In case you wonder about the hostnames: We name our machines after professors of the computer science department at the University of Augsburg where most of us studied (or are still enjoy studying in case of our student trainee and those writing their thesis at makandra :)).

Our address:
makandra GmbH
Werner-von-Siemens-Str. 6
86159 Augsburg
Germany
Contact us:
+49 821 58866 180
info@makandra.de
Commercial register court:
Augsburg Municipal Court
Register number:
HRB 24202
Sales tax identification number:
DE243555898
Chief executive officers:
Henning Koch
Thomas Eisenbarth