JavaScript everywhere – the BlueGenes Tool API version 1 is released!

If you attended the 2017 InterMine developer workshop, you may recall the discussion we had about embedding tools in InterMine’s new UI, BlueGenes. One of the biggest priorities was to make sure that it was easy and fun to create visualisations for your mine.

It’s taken a lot of sweat, toil, testing, and iteration, but I’m incredibly excited to announce that Version 1.0 of the Tool API is released today. This means that you’ll be able to view all of your favourite client-side tools in BlueGenes, hopefully with just a few quick tweaks. It’ll also be relatively straightforward to install tools that other people have created.

bluegenes_protein_report_page_for_A0A0B4KEJ0_DROME
Preview of a protein report page for A0A0B4KEJ0_DROME, with the InterMine Cytoscape interaction viewer and ProtVista protein feature viewer both embedded.

 

You can try it out yourself, at bluegenes.apps.intermine.org – just search for a gene or protein report page.

How does it work?

We didn’t want to re-invent the wheel, and JavaScript definitely has package managers out there already. We use npm (node package manager) to package and install all BlueGenes tools. You can find all BlueGenes compatible tools by browsing the tag bluegenes-intermine-tool, and once you’ve configured your InterMine to work with the tool API, installing a new tool is often as simple as typing npm install @intermine/my-new-tool-name --save into a terminal. Equally, updating tools is as simple as running npm update from your tools folder. BlueGenes then looks inside a file in your tool folder called package.json, and outputs all installed tools listed there. package.json is an npm configuration file which contains a manifest of all the installed packages.

Getting started

Running tools in BlueGenes

If you have an InterMine that’s at least at version 2.1, you can start bluegenes with the ./gradlew blueGenesStart task. See our docs for details.

It’s also possible to run tools on standalone BlueGenes.

Once tools are installed, you can see the entire list of them under the developer menu in BlueGenes (top right > click the cog > developer > tool “app store”).

Developing tools and converting your existing tools

So, you’ve gotten some tools installed and you’d like to add some more of your own? We have the full tool API specs, a tutorial to walk you through creating your first tool, and a nice tool-scaffolder yeoman generator that will create most of the boilerplate files you need automatically, so you can spend time on more important things like eating cake 🎂 and feeding the cat.

Some credits and thanks

A huge thanks to Vivek Krishnakumar for the very first draft of the Tool API specs, and to Josh Heimbach for further work on the spec. I’d also like to thank Julie, who patiently tested the Tool API installation process and helped me iron out a lot of the bugs.

Future plans

What’s next for BlueGenes and the Tool API? Well, we have some updates planned specifically for the Tool API, including: 

  • extending the tool support to list result pages working with legacy tools that aren’t packaged as Javascript modulesBetter integration with BioJS.

You can see a roadmap here for the Tool API. First, though, we’ll probably be thinking about some of the final bits of polish BlueGenes needs before it can be officially launched as a non-beta UI, including: 

  • Authentication. When the InterMine web services were initially implemented, it was with an eye to enable data scientists and bioinformaticians to access data from InterMine. Some authentication-related services had been implemented, such as token-based authentication, but given the fact that the web services weren’t designed with a full application layer in mind, we need to add some more, including as user registration.
  • A MyMine section. We started with some prototypes late last year / early this year, but ended up rolling them back due to unexpected complications.
  • Speedier and more configurable report pages. Is there anything you’ve always wished for in a report page that’s not a tool? Feel free to ping us with ideas.

 

Questions, concerns, confusion, ideas?

Drop by the InterMine chat, email the group at info@intermine.org, or drop me a mail directly – yo@intermine.org.

Advertisements

InterMine 3.0 – Solr search

InterMine 3.0 is now available and features a brand new search powered by Solr.

Default search configuration will work well, but Solr allows for endless configuration for your specific needs.

Now the first search after deployment is instant, you can inspect the search index directly (via http://localhost:8983/solr/) and there’s a facet web service (via /service/facet-list and /service/facets?q=gene). Certain bugs, e.g. searching for the gene “OR”, are also now fixed.

New Configuration Option – optimize

There is a new keyword search configuration setting: index.optimize. If set to `true`, reorganises the index so chunks are placed together in storage which might improve the search time. (Similar to defragmentation of a hard disk.) See the configuration docs for more details.

Docs

Installing Solr

Configuring the keyword search

InterMine 3.0 upgrade instructions | release notes

A big thank you to our clever and hard-working 2018 Google Summer of Code student Arunan Sugunakumar — who did the bulk of the work as part of his summer project. Great job!

HumanMine 5.0 released!

HumanMine has been updated to the latest version of NCBI Entrez Gene. All other data sets have also been updated to the newest versions and we have fixed a few bugs. See the data sources page for a full list of data and their versions. All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

 

New Data Source: GTEx

We added a new expression data set, GTEx. Here’s an example search:

Gene –> Tissue Expression

Tissue –> Gene expression

Data model changes!

Our data model has changed slightly to make querying easier between mines.

  • Protein molecular weight is a float instead of an integer
  • We’ve added URLs for GO term evidence codes
  • Sequence Ontology (the basis for the InterMine data model) is updated, so lots of new data types added.

See our previous blog post for complete details of updates.

 

We have docs and videos, and for a full list of data sources available in HumanMine see the data sources list.

However, please do not hesitate to contact us should you require any further assistance. For all types of help and feedback email info@intermine.org.

HumanMine 3.0

HumanMine has been updated to the latest version of NCBI Entrez Gene. All other data sets have also been updated to the newest versions and we have fixed a few bugs. See the data sources page for a full list of data and their versions. All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

 

New Data Source: ClinVar

We added a new data source linking genes with their alleles and associated diseases. Here’s an example query:

http://www.humanmine.org/humanmine/template.do?name=Gene_Alleles_Disease

Human Data Sources Switch

We switched Entrez and Ensembl gene identifiers around. Please see our blog for details. If you have questions or problems, please contact us.

Complex Interaction Viewer

We’ve added a nice viewer for complexes. Source: http://interactionviewer.org

71e3498e-18f5-11e6-8422-5e4486ab4b67

 

We have docs and videos, and for a full list of data sources available in HumanMine see the data sources list.

However, please do not hesitate to contact us should you require any further assistance. For all types of help and feedback email support@humanmine.org

FlyMine 43.0

FlyMine has been updated to the latest version of FlyBase. All other data sets have also been updated to the newest versions and we have fixed a few bugs. See the data sources page for a full list of data and their versions. All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

If you have any questions, please see our docs and videos. Please do not hesitate to contact us should you require any further assistance. For all types of help and feedback email support@flymine.org

FlyMine 41.0

We are pleased to announce that FlyMine has been updated to release 6 of the BDGP sequence assembly and all annotations are now mapped to this sequence version. See FlyBase (http://flybase.org/static_pages/feature/previous/articles/2014_07/FB2014_04.html) for full details of the sequence changes.

All other data sets have also been updated to the newest versions and we have fixed a few bugs.  See the data sources page (link below) for a full list of data and their versions. All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

Also, check out the new interactions network displayer, esyN, now available on gene report and list analysis pages. Editing the network within esyN allows further analysis and exploration of the network, including expansion from any selected node. For an example see:

http://www.flymine.org/flymine/report.do?id=453007817&trail=|453007817

and navigate to Interactions.

Full documentation for FlyMine is available at:

User docs: http://flymine.readthedocs.org
Videos: http://flymine.readthedocs.org/en/latest/videos/Documentationvideos.html

For a full list of data sources see http://www.flymine.org/flymine/dataCategories.do

However, please do not hesitate to contact us should you require any further assistance.

For all types of help and feedback email support@flymine.org

FlyMine 40.0

We are pleased to announce a new release of FlyMine.  We’ve updated all of our data sets to the newest versions and fixed a few bugs.  See the data sources page (link below) for a full list of data and their versions.  All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

Please also take a look at our updated user documentation and videos – here you can also find a cookbook with some more advanced queries explained:

User docs:  http://flymine.readthedocs.org

Videos:  http://flymine.readthedocs.org/en/latest/videos/Documentationvideos.html

However, please do not hesitate to contact us should your require any further assistance.

 

For a list of all changes see http://blog.flymine.org

For a full list of data sources see http://www.flymine.org/release-40.0/dataCategories.do

For all types of help and feedback email support@flymine.org