Goodnight Staircase, hello BlueGenes

TL;DR: InterMine 2.0 will be based on our new BlueGenes framework. We’ve answered a lot of the common questions about it below.

You may recall seeing various preview screenshots or demos of Staircase (also sometimes referred to as Steps) over the last year or so. We were all pretty excited about it, with plans to release it as InterMine 2.0 beta early this year.

Not all was well in Staircaseland

Towards the end of 2015, though, we realised we weren’t going to be able to continue with Staircase in its current format. The biggest reason for this was AngularJS. Angular 1.x is a popular Javascript web framework, and we’d chosen to  base Staircase on it. The Angular team had recognised a few problems with their 1.x implementation, particularly speed/performance issues, something we’d noticed in Staircase too. So they decided that Angular 2.0 would be a slimmer, faster framework. Sounds great! Only problem? Angular 2.0 was designed to be so radically different from 1.x that it wasn’t backwards compatible – so we couldn’t upgrade Staircase without some serious work.

Time for a re-think

Feeling a bit in the lurch, we had quite a few technical architecture meetings, trying to decide how to take this forward, whether it meant going through the Angular upgrade pain or starting from scratch. Our biggest priorities were:

  • Choosing an architecture that was going to be maintainable in the long-term, not something fly-by-night that would leave us in an awkward position again.
  • Making sure whatever we chose was fast: it needed to feel speedy and responsive in the browser, and quick for devs to update the code, too.
  • Other important things in no particular order: modern/attractive look, usable, and generally fun to implement.

Welcome, BlueGenes

We ended up choosing a complete re-write, code-named BlueGenes, with the following setup:

Language: Clojure and ClojureScript

Dependencies: Reframe and Reagent (a Clojurescript wrapper around React.js). Whilst we’ve been focusing on the front-end technologies, it looks like server-side data will be stored in MongoDB.

Deployment: Effortless, using Docker and dokku.

FAQ

What’s the ETA? We’re aiming to release to public alpha with a basic featureset this summer. Private alpha prototype user testing is already ongoing.

Will there still be workflows? Workflows as re-playable histories won’t be present in the summer release, but they’re still very prominent in the roadmap.
Can I see it? Sure. Caveat? It’s not release-ready yet, and some sections are just mockups. The best way you can help right now is to run through our user-testing questionnaire to give us some feedback. The link’s in there, along with some guidance pointing you towards the bits we’re ready for you to try out. Send responses/feedback to yo@intermine.org.
Can I see it with my own mine? Not yet, but it’ll be one of our next additions. You’ll be able to test it without disturbing your existing InterMine 1.6+ webapp, since BlueGenes uses your existing webservices. If you want to try it out, you’ll need to make sure you’ve merged InterMine 1.6, ideally the 1.6.6.6 branch that’ll be released in the next few weeks. 
What happens to InterMine 1.x? Will PostgreSQL and/or the web services change? There are no plans to retire the existing webapp, API, or PostgreSQL data store. Bluegenes uses the InterMine web services, so even when Bluegenes is deemed ready for official release, we’ll still be fully dependent on the existing APIs.
In the longer term, it’s possible that we’ll stop creating major updates for the JSP side of the webapp, but there are no current plans to retire the data layer of the InterMine you’ve come to know and love.

A bit more about the technology stack we’ve chosen:

Re-frame

Re-frame is web app framework written in/for Clojurescript.

Unlike Angular, Re-frame has a very simple set of rules for handling the “model/controller” part of an application. The basic rules are:

  1. The entire state of the application is stored in one object.
  2. Accessing the state is always performed via a “subscription”.
  3. Mutating the state is always performed via a “handler”.

That’s it really.

The UI subscribes to different slices of the state object and are notified when the data changes. This makes the application almost entirely reactive where the underlying data is the “truth” and the UI is just its reflection. With a reactive approach we don’t need to worry about the data and the view getting out of sync.

Reagent

Reagent is a thin wrapper written in Clojurescript for Facebook’s ReactJS javascript library.

It handles the rendering of the view in such a way that elements on the page are only re-rendered when the data that backs them changes. This has a huge advantage over Angular 1’s approach of constantly thrashing the page to re-display data when parts of the model change. Angular 2 adopted React’s philosophy as well, but the nice thing about using Reagent/React is that it can be used independently or in conjunction with any framework.

Clojure

We chose to use Clojure for many reasons. Some of the big advantages include:

  • Uses immutable data structures.
  • Promotes the use of pure functions which makes for reliable testing.
  • Compiles to both Java and Javascript and therefore can interoperate with Java and Javascript. We can still natively include third party applications written in Javascript. For example, we’re using IMJS extensively.
  • Impressive code optimization / dead-code removal.

Featured image, “Blue Jeans” by RetroWench via Flickr. Licence: CC BY-NC 2.0, no modifications made.

Advertisements

2 thoughts on “Goodnight Staircase, hello BlueGenes”

    1. Never fear! The mongo bit is mostly for user profiles and saving workflows, things like that. Since the original Java/Tomcat/PostgreSQL webapp will still be functioning as the data layer for BlueGenes, Postgres will still remain in place for the forseeable future.

      Liked by 1 person

Comments are closed.