Status update for BlueGenes

It’s been a while since we posted our last (rather optimistic) update around BlueGenes, so we thought we’d share a quick update, starting with the basics.

As a reminder, the long-term goal of BlueGenes is to replace the existing JSP-based UI with a more modern interface – one that works well with mobiles, one that hopefully responds more quickly and is easier to use, and perhaps most importantly, is easy to update and customise.

Some of the questions we’ve had in the last few months:

Q: Will BlueGenes replace the current JSP UI?

A: Yes, eventually. Once we reach official beta/prod release (we’re currently in alpha), we anticipate running them concurrently for a couple of years, but we probably will only provide small fixes for the JSP UI during this period, focusing most of our development effort on BlueGenes.

Q: Do I have to run my own BlueGenes, or can I use the central one at apps.intermine.org?

A: Since BlueGenes is powered purely by web services, it will probably be possible to run your InterMine as a server/api-only service and use BlueGenes at bluegenes.apps.intermine.org/. You can also run your own BlueGenes on your servers and domains, allowing you to customise it so it’s suitable for your data, and not having to rely on our uptime. Either (or both) should work fine. There will be some version requirements related to what version of InterMine can access all the features of BlueGenes – see the next point.

Q: What version of InterMine do I need to have to run BlueGenes?

A: BlueGenes will require a minimum version of InterMine to run. The original release of InterMine web services focused primarily on providing a way to give JSP users access to their data programmatically, but at the time there wasn’t an anticipated need for application level services such as superuser actions. There are a few web services and authentication-layer services we still need to implement, so it’s likely BlueGenes will need API version 31+ or higher in order to be fully-featured. InterMines with API version 27 or higher can run a basic version of BlueGenes. You can check out this table to see if your InterMine is configured to work with BlueGenes.

Q: Ok, so what’s left to do before BlueGenes is released as a public beta?

A: Mostly authentication, superuser and MyMine features – things  like saving and updating personal templates, sorting lists in folders, updating preferences and passwords. Some of these features require updates to InterMine itself in order to work – hence the minimum version noted in the previous question. Once these are ready we’ll move to the public beta stage.

Your input here will be incredibly welcome, too – the more feedback we get early on, the more polished we hope BlueGenes can be.

Q: Will BlueGenes work nicely with HTTPS InterMines?

A: You will be able to run BlueGenes without HTTPS, but in order to avoid inadvertently exposing user passwords, the login button will only be available over HTTPS connections. We’re also working with a student over the next few months, to implement a pilot InterMine Single Sign On service. You can read about it in our interview with Rahul Yadav.

Q: Will I be able to customise the way BlueGenes looks?

A: Totally! There are two ways you can do this. One is to make sure you have your logo and colour settings configured in your web properties. We have a nice guide for that. This’ll tell us what your preferred highlight colours are – FlyMine is purple, HumanMine green, etc. If you’re really dedicated and would like to write your own CSS, you can do that too, if you’re running your own InterMine/BlueGenes combo.

Q: I have some nice custom visualisation tools in my InterMine. I don’t want to have to re-write them!

A: We don’t want you to re-write them either! It depends how they’re implemented in your mine, but we’ve designed the BlueGenes Tool API with you in mind, and many Javascript-powered tools will require only a few lines of code to become BlueGenes ready.

As an example, the Cytoscape interaction viewer currently used in some InterMines only requires 20 lines of code to import into BlueGenes, plus a few lines of config – all the other files (and most of the config too) is boilerplate that we auto-generated.

JavaScript everywhere – the BlueGenes Tool API version 1 is released!

If you attended the 2017 InterMine developer workshop, you may recall the discussion we had about embedding tools in InterMine’s new UI, BlueGenes. One of the biggest priorities was to make sure that it was easy and fun to create visualisations for your mine.

It’s taken a lot of sweat, toil, testing, and iteration, but I’m incredibly excited to announce that Version 1.0 of the Tool API is released today. This means that you’ll be able to view all of your favourite client-side tools in BlueGenes, hopefully with just a few quick tweaks. It’ll also be relatively straightforward to install tools that other people have created.

bluegenes_protein_report_page_for_A0A0B4KEJ0_DROME
Preview of a protein report page for A0A0B4KEJ0_DROME, with the InterMine Cytoscape interaction viewer and ProtVista protein feature viewer both embedded.

 

You can try it out yourself, at bluegenes.apps.intermine.org – just search for a gene or protein report page.

How does it work?

We didn’t want to re-invent the wheel, and JavaScript definitely has package managers out there already. We use npm (node package manager) to package and install all BlueGenes tools. You can find all BlueGenes compatible tools by browsing the tag bluegenes-intermine-tool, and once you’ve configured your InterMine to work with the tool API, installing a new tool is often as simple as typing npm install @intermine/my-new-tool-name --save into a terminal. Equally, updating tools is as simple as running npm update from your tools folder. BlueGenes then looks inside a file in your tool folder called package.json, and outputs all installed tools listed there. package.json is an npm configuration file which contains a manifest of all the installed packages.

Getting started

Running tools in BlueGenes

If you have an InterMine that’s at least at version 2.1, you can start bluegenes with the ./gradlew blueGenesStart task. See our docs for details.

It’s also possible to run tools on standalone BlueGenes.

Once tools are installed, you can see the entire list of them under the developer menu in BlueGenes (top right > click the cog > developer > tool “app store”).

Developing tools and converting your existing tools

So, you’ve gotten some tools installed and you’d like to add some more of your own? We have the full tool API specs, a tutorial to walk you through creating your first tool, and a nice tool-scaffolder yeoman generator that will create most of the boilerplate files you need automatically, so you can spend time on more important things like eating cake 🎂 and feeding the cat.

Some credits and thanks

A huge thanks to Vivek Krishnakumar for the very first draft of the Tool API specs, and to Josh Heimbach for further work on the spec. I’d also like to thank Julie, who patiently tested the Tool API installation process and helped me iron out a lot of the bugs.

Future plans

What’s next for BlueGenes and the Tool API? Well, we have some updates planned specifically for the Tool API, including: 

  • extending the tool support to list result pages working with legacy tools that aren’t packaged as Javascript modulesBetter integration with BioJS.

You can see a roadmap here for the Tool API. First, though, we’ll probably be thinking about some of the final bits of polish BlueGenes needs before it can be officially launched as a non-beta UI, including: 

  • Authentication. When the InterMine web services were initially implemented, it was with an eye to enable data scientists and bioinformaticians to access data from InterMine. Some authentication-related services had been implemented, such as token-based authentication, but given the fact that the web services weren’t designed with a full application layer in mind, we need to add some more, including as user registration.
  • A MyMine section. We started with some prototypes late last year / early this year, but ended up rolling them back due to unexpected complications.
  • Speedier and more configurable report pages. Is there anything you’ve always wished for in a report page that’s not a tool? Feel free to ping us with ideas.

 

Questions, concerns, confusion, ideas?

Drop by the InterMine chat, email the group at info@intermine.org, or drop me a mail directly – yo@intermine.org.