Ruby on Rails – Varyonic

Nine Years A Rubyist

This last year has seen continuing maturation of Ruby on Rails with Ruby 2.6 and Rails 6.0. Ruby 2.6 included an initial release of JIT compilation for YARV, and Rails 6.0 adopts Webpacker as the default JavaScript asset manager instead of Sprockets. Ruby, however, continues increasingly overshadowed by JavaScript and Python, and Rails is challenged by the popularity of microservices and Node. Neither Matz’s mruby 2.0 nor Crystal have gained widespread attention, but at least AWS Lambda Ruby support and FaastRuby are now available.

At a practical level, for our mature in-house codebase a switch to microservices is not necessary or cost effective. This year, however, has seen Stephan Hagemann’s ‘Component Based Rails Applications’ ebook published (endorsed) as a softback by Addison Wesley and a very similar approach embraced by Shopify. Our own internal codebase remains ~60% gemified from 2017 in a simple layered architecture while we have focused on consistently packaging business logic using interactors (service objects) for both foreground and background processes.

New gems (to us) used this year included roxml, and replacing the unmaintained god process monitor with eye. On the open source side I stepped back from contributing to ActiveAdmin and published instead a couple of API clients for AmEx and PayPal.

Eight Years A Rubyist

Year 8 with Rails for me has been an unglamorous affair of maintenance upgrades (Ruby 2.3, Rails 5.x), refactors and bug fixes.

It has been discouraging to see StackOverflow characterize Ruby as shrinking and unpopular and compare Ruby on Rails with ASP.net. Nate Berkopec’s findings that vanilla Rails apps are perfectly capable of 1,000 requests/minute is cold comfort in the face of shrinking developer mindshare: this decline has real consequences in terms of open source contributions and Ruby gem maintenance.

On the other hand Ruby remains #4 on Github and developers are still very much in demand. JavaScript continues ascendant but Derek Prior of Thoughtbot has mentioned cooling enthusiasm for Elixir, and Go too has slid in popularity, at least according to TIOBE. Crystal 1.0 remains delayed. An experimental JIT has now shipped for YARV foreshadowing further performance improvements in Ruby 2.6.

On the Rails front end Basecamp continued their contrarian tradition by offering Stimulus.js as a more limited and pragmatic alternative to React’s dominant framework. The Majestic Monolith remains relevant for smaller businesses and early stage startups despite ambitious engineers everywhere (myself included) wanting to use the same tech stack as the largest, most successful companies from Apple, Salesforce and Netflix to… Twitter. At Railsconf GitHub, now owned my Microsoft, recommitted to easier scalability enhancements for Rails 6.

On the infrastructure side the Docker ecosystem has continued to mature with the embrace of Kubenetes at Dockercon EU in Oct ’17. It has also been good to see a more enlightened approach to open source by AWS with EKS and ApacheMQ.

At my employer we continue to support a somewhat modular monolith with a 50 kloc monorepo, 40 ActiveAdmin resources, 60 background jobs, 70 direct and 240 total gem dependencies. We have fully caught up with Rails 5.2 and edged forward with Ruby 2.3 for Bootsnap support and React-Apollo 1.x. Procedural code has continued to migrate to standardized Interactors. New gems used included Bootsnap, Combustion, Dalli (memcached) and AWS SDK v3 SQS, S3, SES & SNS.

What has Rails 5.x brought us? More secure controller parameters, still scarily subvertible by less experienced developers, but at least awareness has been raised. More restricted autoloading, for the best. Better callback management, though we try to avoid them as much as possible. Bootsnap with a significant improvement in startup times, yay!

Open source contributions this year have included a couple of minor ActiveAdmin releases including Rails 5.2 support and various PRs for the AuthorizeNet Ruby SDK.

AWS features I have been working more with this year have included ClassicLink, private subnets, piculet, Data Pipeline and managed SSL certs. Concerns over production container management have been mitigated with a mass of metrics collection with CloudWatch, Librato and Skylight.

What will the next year bring? A pragmatically managed evolution towards ‘mini-services’, Rails 6 with polished multi-database support, Ruby 2.6 with improving JIT support, Webpack as the default JavaScript handler and increasing adoption of Kubernetes.

Railsconf 2018 – Pittsburgh

Tuesday 17 April. Snow. DHH poking fun at Twitter and raising alarms about Facebook. Nick Quaranto with a GraphQL intro: best for new APIs as a mobile backend. Sean Griffin trying to encourage Rails contributions by labeling it a legacy app. Fastly presenting their not yet released alternative to docker-compose for developers. Olivier Lacan talking about the life and imminent death of CodeSchool.com. Taylor Jones with an introductory overview of Webpacker. Akira Matsuda teasing with his unfinished performance hack gems. Mark Imbriaco with a stellar career from AOL to Decisiv, 37signals, Heroku, Living Social (with Chad Fowler), GitHub, Digital Ocean and Pivotal.

Wednesday. Eileen Uchitelle keynote on upcoming Rails 6 features for scalability: baked in parallel tests (minitest) and polished multi-database support. Initially skeptical but ultimately inspired by her appeal for more companies to upstream their work. Justin Searles and Ted Kaufman talking about growing their agency. Chris Hoffman reviewing Optoro’s experiences adopting services: start with new business services, not an extraction, do something large enough to be noticed but as simple as possible, establish infrastructure standards and have developers support their services. Exhibition hall with booths by BTCOTC, Procore, GitHub, Heroku, Engine Yard, Cloud 66, Shopify, Scribd and LendingHome amongst others. After lunch James Adam recounting his struggle to popularize Rails engines and the forces shaping the framework’s evolution. Leanardo Targon of Plataformatic with an introduction to Warden. Admin frameworks BOF discussing modular monoliths, TrailBlazer and CBRA. Lightening talks including Clearwater and jsonapi-suite

Thursday. Snow again and chill winds. Sarah Mei filling in a keynote for Bari Williams: application development is interior decorating, not architecture. Sam Phippen with another well presented RSpec talk with the latest fixes to support system tests. Graham Conzett reminding how much can still be done with UJS alone. After lunch Ariel Caplan introduction to Swagger (OpenAPI) and (unmaintained?) apivore, Michael Crismali giving a quick review of JavaScript strategies, followed by Ross Kaffenberger giving actionable advice on dealing with Webpack. Finally Aaron Patterson proposing to upstream easier performance tuning into Rails.

Attendance looked up (2,000+), and more diverse. No more ‘Rails is dying’ and React/Elixir FOMO. Webpack/services/JSONAPI for sure, but not too much beyond that. Standards of the technical talks were good, especially Akira and Eileen. I missed the talks by Vladimir Dementyev and Justin Weiss and I look forward to the inspiring videos of Ernie Miller and Nicolas Means.

Year 7 with Rails

In my seventh year with Rails I started allocating ’20% time’ to maintenance of Active Admin and related open source gems. As the universe of web technologies continues to expand and fragment there has been increasing acknowledgement at GitHub and elsewhere of the stresses on volunteer contributors and the consequent rise in the number of projects falling into disrepair. Active Admin at least now has an official 1.0 release and timely support for Rails 5.1.

On the plus side competition and the exodus from Rails has brought back innovation, most notably Rails 5.1’s integration of NPM, Yarn and Webpack. Beyond Rails in the wider Ruby community an object-functional style of programming is gaining popularity, championed by dry-rb and embraced by Hanami and Trailblazer.

This year has been one of considerable progress (and hype) around containers and microservices. Microservices are not without tradeoffs but Docker containers have proven themselves useful in both development/CI and production.

At my employer our codebase has reached 50kloc, now 60% componetized into gems, but a new joint venture has doubled our infrastructure and provided incentive to switch to Docker (and Elastic Beanstalk). Containerizing legacy applications is not always trivial, however, as adopting a 12-factor approach means simplifying and standardizing sometimes disparate logging and configuration solutions, as well as changing to process supervision in a stripped down POSIX environment instead of a full blown Linux host. Our monitoring stack now includes Pingdom, Airbrake, Loggly, PagerDuty, Skylight, (CloudWatch) and Librato. Moving CI to Docker has been challenging, with initially only CodeShip providing a SaaS solution with parallel tests each with multiple sets of containers.

Looking forward Rubyconf last November continued to raise hopes that performance and concurrency will be improved… eventually. Meanwhile there are numerous projects working on integration with compiled languages: helix with Rust, swig with C/C++, Loren Segal’s gorb with Go, and rubex. Crystal has gained attention as a Ruby-ish experiment on LLVM with its own MVC frameworks.

On the front end React may now be a ‘safe’ choice for front end development but possibly disruptive innovation continues with VueJS, CycleJS and Elm. The next twelve months will see either our further adoption of React on Rails or a shift to a different approach.

RailsConf 2017, Phoenix, AZ

Tuesday. Day 1. Keynote by DHH on belief systems. Fascinating talk by Lance Ivy on JSON Web Tokens, Justin Weiss on sessions, Dave Tapley with an entertaining exploration of VueJS, ActionCable and image processing. Eileen Uchitelle with a much anticitpated presentation on adding a significant feature (SystemTestCases) to Rails. Alex Kitchens with a great primer on using Git to explore the history of the Rails codebase. Finally Sam Phippen, now with Digital Ocean, in much better health than last year, giving an upbeat tour of fixing some RSpec issues.

Wednesday. Day 2. Justin Searles keynote with a tongue-in-cheek programmer personality test inspired by Myers Biggs. Back to back presentations on Elm, Phoenix, React and Helix highlighting various Rails related technologies. Nate Berkopec with clear and practical guidance on configuring app servers on Heroku or elsewhere. Evening lightening talks, most notably Isaac Sloan building a Crystal Kemal blog in five minutes.

Thursday. Three talks on distributed teams, including Maria Gutierrez, Glenn Vanderburg, formerly of Living Social, and David Copeland of Stich Fix. Simon Enskilden on the decision at Shopify to embrace flash sales and scale from MySQL sharding to multi-datacenter ‘pods’. Tony Drake pitching his active_reporting gem, then Nate Berkopec and Eilieen Uchitelle again with Rafael Franca and Richard Schneeman on a performance panel before the final keynote from Aaron Patterson with his traditional trolling of DHH and, eventually, h2o with Rack 3.0.

Active Admin 1.0

Note: This post is by Varyonic, not the ActiveAdmin organization.

So after nearly seven years Active Admin 1.0 is out. It has been a long and sometimes uncertain journey of which I have witnessed only part. The original author has moved on, subsequent maintainers have burned out, changed jobs, started families… The bug list remains longer than we would like, the features list remains longer than we can handle, but Active Admin has been in production use for years at companies, NGOs and agencies, and there is commitment to keeping the project alive through quiet pragmatism rather than grand plans.

Not all of us have the time, money, or even need for the next shiny thing, and even if Active Admin sometimes cuts across ‘the Rails Way’ it can be a significantly productive tool. I encourage you to give it a try, to upgrade if you are already using it, to give us feedback if you find issues. General help is available on StackOverflow (tag #activeadmin); search the GitHub issues, both open and closed (~5,000 in total), before submitting anything new. If you are using Active Admin regularly consider looking through the code, issues and the commit log, maybe you will be inspired to contribute, even if just documentation or translations. If you are working for an agency with multiple customers you may be able to provide valuable insight and encourage adoption.

So thank you to all 500+ contributors so far and especially Timo for the past couple of years, here’s to the next 1M downloads.

Year 6 with Rails

In my sixth year with Rails the Ruby community has been experiencing a sea change: gem innovation has peaked and leading Rails developers have been moving on to JavaScript 6 Node, Dockerized microservices in Go, functional Clojure on the JVM or highly concurrent Elixir Phoenix on the fault tolerant Erlang VM. Despite these defections Ruby adoption has resurged (as, coincidentally, has Java) given new versions that have improved garbage collection and performance. Rails 5 has embraced concurrent-ruby and websockets, while HTTP/2 looks set to drive further significant change to all web application frameworks in the near future.

Meanwhile our in-house multi-currency e-commerce solution has grown to 45kloc, 2,100 tests and nearly 100 gem dependencies thanks to ever expanding integration with fraud detection services, fulfillment providers, payment gateways, email marketing solutions and an autodialer. Between daily business requirements we have been addressing technical debt by straggling to Ruby 2.2, Rails 4.2, RSpec 3.4 and ActiveAdmin 1.0.0.pre2. We have adopted CBRA and gemified the persistence core of the application. New gems adopted have included VCR, Wisper and React on Rails.

On the tools side this year has seen widespread adoption of Slack for distributed teams, but we have stuck with Atlasssian HipChat, configuring integration with Airbrake for production exceptions, Assembla for tickets, BitBucket for code reviews, CodeShip for CI and Coveralls for test coverage. Increasingly global sales have raised expectations of operations to provide 24×7 uptime, monitored by PagerDuty and requiring MySQL hot backups.

‘In Container’ Rails Development With Docker

Why Docker? Because it promises to make dev-ops easier. Dockerizing an existing production environment is not trivial, moving development to Docker is perhaps a useful stepping stone. It might also make developer on-boarding easier but I haven’t tried that yet.

Docker can be installed on OS X in various ways, most notably Docker Toolbox or homebrew. I was previously using docker-machine with Parallels but recently I discovered Dlite which simplifies using docker by creating a local OS X docker.sock that forwards requests to the docker VM. Dlite also sets up a an NFS server that exports /Users to the VM. Instead of VirtualBox or Parallels Dlite leverages a lightweight hypervisor called xhyve which requires Yosemite and hardware virtualization support (sysctl kern.hv_support). Note that as of writing Docker for Mac and Windows in public beta is using a similar approach.

With Dlite started I can get a usable bash session with:

docker run -v /Users:/home -it ruby:2.2 bash

I can then cd to the project directory and bundle.

To run browser based tests I was previously building a docker image with Xvfb, Chrome and Selenium Server installed. Recently, however, I found a different approach that uses standalone selenium-server as another docker service. On a Mac workstation installing selenium-server as a service is as simple as

brew install selenium-server-standalone chromedriver
selenium-server

Once tests are passing with a local selenium-server then adding selenium/standalone-chrome to docker-compose.yml and running browser based tests inside the container is quite achievable. The location of the selenium-server is exported to the app container as SELENIUM_PORT, eg. tcp://172.0.0.5:4444

RailsConf ’16, Kansas City, MO

Wednesday. Day 1. Keynote introduction by Jeremy Daer, nee Kemper, DHH thankfully racing. Heroku presenation on Kafka, Shopify presentations on shipit-engine, request_interceptor and BuildKite, Rafael França on Sprockets, and well attended presentation by Akira Matsuda with ideas for assorted Rails performance hacks. Closing keynote from Nickolas Means with a history of Lockheed’s original skunkworks.

Thursday. Day 2. Ruby Heroes including Avdi Grimm, Charles Nutter, Akira Matsuda and Richard Schneeman. Surprisingly few vendor booths, including Indeed Prime, Hired, Skylight, RubyMine, Stitch Fix and Codacy. Rails 5 presentations on ActionCable, Turbolinks and Sean Griffin on miscellaneous enhancements. Richard Schneeman on maintaining Sprockets and David Copeland of Stitch Fix on emergent software architecture. Lightening talks on Scrivito, RocketJob, Appfolio Farcy, Railscamp, Bundler Gemstash, multi_zip, Brakeman, RubyBookClub, Reinteractive Wallaby and Benjamin Fleischer on JSONAPI.

Friday. Day 3. Aaron Patterson, now at GitHub, workshops on Hashicorp tools and RedPotion and presentations on Percy and Sandi Metz on refactoring. Paul Lamere from Spotify with a wonderful presentation on music data analytics… in Python.

P.S. I never guessed the most interesting presentation on Rails would be by Justin Searls.

From Rails 3.2 to Rails 4.2 If We Really Must

Rails 5 is almost upon us and we’ve spent the last two months, amongst other things, moving our 45,000 lines of Rails e-commerce code from 3.2 to 4.2. For better or worse the move has been less about embracing new features and more about compliance and maintaining security bug fix support. The move has been a rapid succession of stages, from 3.2 to 4.0 in development, 4.1 in production, and finally 4.2 in production. Overall it has been a significant effort and a learning experience.

The biggest challenge we encountered was not fixing our 175 models and 40 controllers, but dealing with our nearly 100 open source gem dependencies. The project started from a proof of concept elegantly scaffolded with ActiveAdmin’s DSL and has accreted functionality and third party interfaces for the past four years. ActiveAdmin remains the leading Rails admin interface, but with the departure of Greg Bell momentum has slowed. Sean Linsley, Timo Schilling and others have put a significant amount of effort into answering questions and applying bug fixes, but the ambitious 1.0 milestone has remained unmet for two years, and gem releases have been infrequent. To update ActiveAdmin first we switched from stable 0.6 gems to the master branch in GitHub, replacing meta-search with ransack, then we worked our way down the master branch to 1.0.0.pre1 in 90 day increments, isolating and working around the issues that impacted us (#2233, #2679, #2771, #3038/#3324, #3145, #3147, #3486, #3591, #3776, #3784). Critical to this effort was our test suite coverage, which remains a disappointing, but so far adequate, 90% (and yes, we upgraded to RSpec 3.x as well).

Apart from gem dependencies the most dramatic issue updating Rails was the breakage of non-strict mode used with our old MySQL database (and thus nearly everything). For Rails 4 we needed to configure ‘strict: false’ to ignore our true mode setting in the database, then with Rails 4.2 this changed again and we needed to explicitly set variable sql_mode to the required value.

So what has Rails 4.x brought us? ActiveRecord foreign keys and enums, but we embraced the foreigner and enumerize gems long ago. Spring, but we so loved Zeus. secrets.yml but we already use environment variables and dotenv. Web Console, but we are happy with Better Errors and Pry. An improved cookie serializer that fouled up our ‘seamless’ rolling production upgrade. Autoloader vexation. Millisecond DateTime rounding errors (or is that due to OS X?). ActionMailer::Preview and ActionMailer#deliver_later, but we have already built out more sophisticated solutions. Strong parameters for sure. New bin/ scripts. Stricter routing with Patch. An end to ActiveRecord’s old dynamic finder API and tweaks to callbacks and ActiveRecord::Relation. Better performance, please, from Aaron’s AdequateRecord. ActiveJob as a wrapper for Resque or Sidekiq. ActiveSupport::Testing::TimeHelpers. Perhaps for us most importantly a step closer to greater concurrency with Puma (and LiveStreaming?), Sidekiq, Celluloid, who knows?

More detailed ActiveAdmin update notes

1 2 3

Posts navigation