🎉 BlueGenes 1.0.0 release 🎉

We are super excited to announce that BlueGenes 1.0.0 has been officially released!

BlueGenes is a new user interface to InterMine utilising its extensive web services combined with a modern web application stack, to provide a more interactive experience for exploring and analysing your data. In contrast with the previous interface, BlueGenes is separate from the InterMine server, increasing the ease of further development to both.

The new interface provides the same feature set you’re already familiar with, wrapped in a new design offering an enhanced user experience. In addition, it allows you to quickly change the InterMine server you’re connected to, to any other instance in your organisation or that is available through the InterMine Registry. While the old webapp acted as a front-end to a single InterMine, BlueGenes can be configured to serve multiple InterMine servers.

A new Tool API for adding custom visualisations and analyses tools to your InterMine has also been introduced. This allows you to create your own tiny JavaScript apps (or adapt the ones we have developed for HumanMine and FlyMine) for your InterMine data, which BlueGenes will display for list analysis, query results and/or report pages, depending on what you specify it supports.

While this is the first major release, we still have lots more coming. Some admin tasks, notably creating templates, is not available in BlueGenes yet so you will have to access the legacy webapp for this. The web services have yet to be separated from the legacy webapp, so it will continue to be available for a while (if you wish to wait with switching to BlueGenes, we recommend using the latest InterMine 4.x release instead).

This new release requires InterMine 5.0.0. To upgrade to InterMine 5.0.0 please read the upgrade instructions carefully.

InterMine 5.0.0 release

This releases contains new web services to support BlueGenes’s new features, in particular:

• Login using OAuth2 (e.g. Google, Elixir)
• Upgrading lists
• Generate external links
• Submit feedback
• Reset password

This release includes the Frictionless Data Package specifications implemented by Nikhil Vats. Please see the documentation here for more details.

All the InterMine artifacts have been published on the Maven Central repo instead of Bintray’s JCenter due to the recent jcenter/bintray shutdown announcement.

For more details about this release, please see the release notes.

Documentation

The user documentation has been adapted to the new interface by Hanan (Outreachy intern) and is available here.

The InterMine Server documentation (aka dev documentation) has been revised by Mardhiyah (Outreachy intern) and is available here.

FlyMine and HumanMine

FlyMine has been updated to use BlueGenes. If you use Firefox, you might experience an issue due to the browser cache, in that case you need to delete the cache. The legacy user interfaces is still available here.

HumanMine will be updated soon.

Thanks to …

… all our users/contributors for their support, feedback and help. Thanks to Joshua Heimbach and Yo Yehudi for their great and essential past contributions to this project. Thanks to our excellent interns who have directly contributed to BlueGenes; Akshat for the 2019 GSoC project Javascript Data Visualisations, Sakshi Srivastava for the 2020 Outreachy mid-year project Javascript Data Visualisations, Roshni Prajapati for the 2020 Outreachy mid-year project Improve UI & UX of BlueGenes, and Hanan Younes for the 2020 Outreachy end of year project Create InterMine user training docs.

Contact us

If you have any questions or concerns about any of these changes, please do not hesitate to contact us.

InterMine 4.3.0 release

We are pleased to announce the new InterMine release 4.3.0.
It includes a few improvements and bug fixes.
This is a non-disruptive release.

Improvements and bug fixes

1. Added a new generic obo source which can be configured in project.xml (Sam Hokin)
2. Removed the gretty plugin from testmine which now uses gradle 4.9
3. Removed duplication of so.obo file
4. Fixed the Google Charts API which broke various things, including the humanmine expression viewers
5. Updated Google auth and userinfo endopoints
6. Fixed the NullPointerException in the report page caused by an empty value in the field key used to generate the permanent URI (Share button)
7. Fixed the home page loading when bioschemas markup are enabled (markup.webpages.enable=true) and JAVA 11 is used
8. Fixed the webservice query/results when the format is jsonobjects and the root class is a simple object – with no id (Sam Hokin)

See release notes for detailed information.

Upcoming releases

For more information about the upcoming releases, please visit the InterMine roadmap here.

InterMine 4.2.0 release

We are pleased to announce the new InterMine release 4.2.0.
It includes new functionalities to support the upcoming BlueGenes release 0.10.0, some improvementes on FAIR side and a few bugs fixes.
This is a non-disruptive release.
Thank you so much to our contributors: Ahmed Hafez, Asher Pasha and Sam Hokin!

BlueGenes related improvements

1. Added /login web service that merges the anonymous session with the user logged in.
2. Added /logout web service.
3. Added a new webservice to change the users’s password.
4. Updated the existing /lists webservice which allows modifying the list description.
5. Improvements on the Date type (to support CovidMine).

BlueGenes 0.10.0 will be released soon and announced in a separate blog.

FAIR related improvements

1. Simplified the webservice that generates Bioschemas markup for the report page.
2. Adopted DataRecord in the report page.
3. Added Gene, Protein markup in the report page.
4. Added BioChemEntity markup in the report page (only if configured).
5. Added the ontology licences to the obo converters.

Bug Fixes / Improvements

1. Added a new bio source to load ISA files in json format
2. Fixed organism short name generation (Ahmed Hafez)
3. Fixed a bug related to long fields in the report page (Asher Pasha)
4. Removed BioEntity.ontologyAnnotations because redundant (Sam Hokin)
5. Fixed src.data.dir.include (gff3 and xml) ans src.data.dir (intermine-items-xml-file)
6. UniProtFastaLoader works with organism names longer than 2 words (for example Severe acute respiratory syndrome coronavirus 2)

See release notes for detailed information.

Upcoming releases

For more information about the upcoming releases, please visit the InterMine Development Roadmap. More details on the roadmap here.

InterMine 4.1.2 – patch release

We’ve released a small batch of bug fixes and improvements.

If you host your own CDN please update it with the latest version of imjs (v.3.18.1) and im-tables (v 2.1.0).

Thank you to our contributors Joe Carlson, Paulo Nuin and Asher Pasha!

Fixes

• DataSet URLs appear in tables
• LOOP query in webapp has been fixed
• Complex Displayer fixed
• Updated chebiWS-client and jami-interactionviewer-json versions
• Licence dataset doesn’t display null
• runtime exception in BagManager.getBags catched
• Fixed bug in the report page which allowed to execute javascript (Asher’s contribution)
• Cast conversion corrected when updating serial (Joe’s contribution)

Enhancements

• Update to Java11 (Asher’s contribution)
• WebservicePythonCodeGenerator updated according to Python’s code styling PEP8 (Paulo’s contribution)
• In the Export section, option “Upload to GenomeSpace” removed
• ThaleMine updated to to psi BioSource and BioGrid (Asher’s contribution)
• From FAIR side: json-ld home page updated + use of the registry to set provider/support in the home page markup, ‘Shared link’ configuration improved
• Libraries as im.js, imtables.js, imtables-dep.js removed from intermine-webapp
• gff source added to the bio/source multi gradle project
• Improved the logs when post processes related to solr fail

This is a non-disruptive release.

See release notes for detailed information.

InterMine 4.1.0

InterMine team has just released InterMine 4.1.0.

The new release includes a better integration with Galaxy: we can import data into Galaxy from any InterMine of our choice (either starting from InterMine or Galaxy), and we can export a list of identifiers from Galaxy to any InterMine of our choice through the InterMine registry. No need to configure anything any more: all the Galaxy properties have been moved to InterMine core. No need to create a mine-specific Galaxy tool anymore, use the NEW intermine tool instead. Please read here for more details. A simple InterMine tutorial will be published soon in the Galaxy Training Material, under the Data Manipulation topic.

This release offers the integration with ELIXIR AAI (Authentication and Authorisation Infrastructure) allowing the researchers to log in the InterMine instances using their ELIXIR profile. You will need:

1. an ELIXIR identity
2. register the InterMine client in order to obtain the client-id and the client-secret which must be set in the mine properties file.

More details here in the OpenAuth2 Settings section of the documentation.

Also new in this version is the gradle wrapper 4.9, which is compatible with Java11. This only effects the users which compile/install InterMine code.

Thank you so much to our contributor Joe Carlson for improving the generateUpdateTriggers task.

The release contains also a few bug fixes.

Bug Fixes

• Solved the error caused by obsolete terms in the gene ontology
• Fasta query result: CDS translation option + extra view parameter
• The ONE OF constraint works properly when editing a template
• The default queries configuration have been migrated to json
• The task generateUpdateTriggers has been improved

See the release notes for the complete list and detailed information.

This is a non-disruptive release. To update your mine with these new changes, see the upgrade instructions.

InterMine 4.0 – InterMine as a FAIR framework

We are excited to publish the latest version of InterMine, version 4.0.

It’s a collection of our efforts to make InterMine more “FAIR“. As an open source data warehouse, InterMine’s raison d’être is to be a framework that enables people to quickly and easily provide public access to their data in a user friendly manner. Therefore InterMine has always strived to make data Findable, Accessible, Interoperable and Reusable and this push is meant to formally apply the FAIR principles to InterMine.

What’s included in this release?

1. Generate globally unique and stable URLs to identify InterMine data objects in order to provide more findable and accessible data.
2. Apply suitable ontologies to the core InterMine data model to make the semantic of InterMine data explicit and facilitate data exchange and interoperability
3. Embed metadata in InterMine web pages to make data more findable
4. Improve accessibility of data licenses for integrated sources via web interface and REST web-service.

More details below!

How to upgrade?

This is a non-disruptive release, but there are additions to the data model. Therefore, you’ll want to increment your version, then build a new database when upgrading. No other action is required.

However, keep reading for how to take advantages of the new FAIR features in this release.

Unique and stable URLs

We’ve added a beautiful new user-friendly URL.

Example: http://beta.flymine.org/beta/gene:FBgn0000606

Currently this is used only in the “share” button in the report pages and in the web pages markup. In the future, this will be the only URL seen in the browser location bar.

For details on how to configure your mine’s URLs, see the docs here.

See our previous blog posts on unique identifiers.

Decorating the InterMine data model with ontology terms

InterMine 4.0 introduces the ability to annotate your InterMine data model with ontology terms.

While these data are not used (yet), it’s an important feature in that it’s going to facilitate cross-InterMine querying, and eventually cross-database analysis — allowing us to answer questions like “Is the ‘gene’ in MouseMine the same ‘gene’ at the EBI?”.

For details on how to add ontologies to your InterMine data model, see the docs here.

Embedding metadata in InterMine webpages

We’ve added structured data to web pages in format of JSON-LD to make data more findable, and these data are indexed by Google data search. Bioschemas.org is extending Schema.org with life science-specific types, adding required properties and cardinality on the types. For more details see the docs here.

By default this feature is disabled. For details on how to enable embedding metadata in your webpages, see the docs here.

Data licences

In our ongoing effort to make the InterMine system more FAIR, we have started working on improving the accessibility of data licences, retaining licence information supplied by the data sources integrated in InterMine, and making it available to humans via our web application and machines via queries.

See our previous blog post on data licences.

For details on how to add data licences to your InterMine, see the docs.

Future FAIR plans

1. Provide a RDF representation of data stored, lists and query results, and the bulk download of all InterMine in RDF form, in order to allow the users to import InterMine resources into their local triplestore
2. Provide an infrastructure for a SPARQL endpoint where the user can perform federated queries over multiple data sets

Upcoming Releases

The next InterMine version will likely be ready in the Fall/Winter and include some user interface updates.

Docs

To update your mine with these new changes, see upgrade instructions. This is a non-disruptive release.

See release notes for detailed information.

InterMine 3.1.2 – patch release

We’ve released a small batch of bug fixes and small features. Thank you so much to our contributors: Sam Hokin, Arunan Sugunakumar and Joe Carlson!

Features

• Templates can be tagged by any user, not just the super user. (Via webservice only – for now)

Fixes

• When searching our docs, some times the “.html” extension was dropped. This was fixed by our beautiful documentation hosters – readthedocs.org
• Installing the “bio” project via Gradle does not fail if you do not have the test properties file.
• Gradle logs error fixed
• Removed old GAF 1.0 code
• Fixed XML library issue:  java.lang.ClassCastException for org.apache.xerces
• Set converter.class correctly
• Updated the protein atlas expression graph
• Handle NULL values returned by NCBI web services
• Updated Solr to support new Solr versions
• Removed unneeded Gretty plugin
• Better error handling for CHEBI web services
• Publication abstract is longer than postgres index
• Removed phenotype key, it’s not in the core model and has conflicting key
• Updated ObjectStoreSummary to handle ignored fields consistently.

Upcoming Releases

InterMine 4.0 is scheduled for release the week of 7 May 2019.

Docs

To update your mine with these new changes, see upgrade instructions. This is a non-disruptive release.

See release notes for detailed information.

InterMine 3.1.1 – patch release

We’ve released a small batch of bug fixes and small features. Thank you so much to our contributors: Sam Hokin, Paulo Nuin and Joel Richardson!

Features

• Added access to the GFF header in the GFF parser
• GFF sequence handler has access to feature now
• Added DOES NOT CONTAIN constraint
• Added a few end points for BlueGenes

Fixes

• InterPro source handles DTD correctly
• Updated to new GitHub URL for Gretty Plugin
• Fixed OMIM link outs
• NCBI going to update their GFF files (at our request! Thanks Wayne!)
  • fix spelling on feature “DNaseI_hypersensitive_site” i
  • change “recombination_region” to “recombination_feature”
• Updated external links on enrichment widget
• Handle NULL search index correctly
• Fix publication with NULL title
• Fixed log library dependency conflict
• Removed deprecated Yahoo login link
• Fixed Panther source to handle proteins

Upcoming Releases

• 3.1.2 – More small bug fixes
• 4.0.0 – FAIR release

Docs

To update your mine with these new changes, see upgrade instructions. This is a non-disruptive release.

See release notes for detailed information.

InterMine 3.1 – Extending the Core InterMine Data Model with Multiple Genome Versions, Strains

Advances in sequencing technologies mean that genome sequence and annotation data for multiple strains of a species are now often available. An update to the InterMine core data model was decided that would allow addition of Strain data should it be available without affecting InterMines which do not have this data.

It was decided that the addition of a new class, Strain, which is referenced by Organism and Sequence feature and vice versa, would allow both the flexibility required and allow for addition of further data and expansion if required.

The Strain class has the following features/advantages:

• SequenceFeature entities, such as Genes, would continue to reference Organism, but would also reference the new Strain class, allowing for queries returning SequenceFeatures for a specific strain.

• Providing strain information as a separate class allows individual InterMine’s to reference other information as required, such as Genotype and Stocks.

• The Strain class extends BioEntity so will include strain-relevant attributes such as PrimaryIdentifier and Name and will reference other collections such as synonym.

• Minimal changes to the user interface will be required as, to our knowledge, SequenceFeatures in individual strains always have a unique identifier. With the help of templates if necessary, users will be able to identify particular SequenceFeatures and which strain they originate from.

To update your mine with these new changes, see upgrade instructions. This is a non-disruptive release.

See release notes and the notes from the community call for more details. Please join our community calls if you’d like to be part of future data model decisions! (Details of upcoming calls are available via our developer mailing list).