InterMine 2.1.0 release

We pushed out a few bug fixes and improvements:

  • FIX – “Update publications” data source failed when too many PubMed IDs were sent to the Entrez web service (Thank you to Norbert Auer!)
  • FIX – small bug for generating Python code
  • FIX – FASTA query web service times out when extensions are used (Thanks to Joel Richardson!)
  • FIX – Discontiguous CDS sequences and lengths not set properly
  • FIX – Some SO terms were not updated in 2.0 release
  • FIX – Region search: trying to leave “extended region” field blank results in error

See GitHub for details:

https://github.com/intermine/intermine/releases

How to Upgrade

Throughout the InterMine code, the InterMine version number is set via a global variable. Here’s an example:

# Maven will download the bio-core JAR with the correct version
compile group: 'org.intermine', name: 'bio-core', version: System.getProperty("bioVersion")

To change which InterMine version you are using , you will want to increment the value of the system property “imVersion” and “bioVersion“. These are located in  the “gradle.properties” file for your mine:

# gradle.properties in your mine
systemProp.imVersion=2.1.+
systemProp.bioVersion=2.1.+

Maven will now download, for example, the bio-core JAR of the latest version, e.g. “bio-core-2.1.0.jar”.

If you set the property to “2.1.+” you will get any small point releases that are published in the future. You can set the property to be 2.1.0 if you ONLY want to use version “2.1.0” and do not want to receive updates:

# gradle.properties to only get specific version
systemProp.imVersion=2.1.0
systemProp.bioVersion=2.1.0

Here is an example:

HumanMine upgrade to use the latest version.

InterMine 3.0 – SOLR

The next InterMine release will be InterMine 3.0 which will include SOLR. See our SOLR blog post for details.

We are currently testing SOLR with InterMine and should have a version ready for public beta testing early next month.

 

Advertisements

InterMine 2.0

We are excited to announce the official release of InterMine 2.0!

InterMine 2.0 includes some model updates, a big change in how InterMine itself is built, lots of new features, like a new UI, and a long list bug fixes. See the full list of updates here.

This release represents a large milestone for the InterMine team! Not only because we made big fundamental changes to the core InterMine data model and build system, but also because this release represents a major shift in philosophy for us. Previously InterMine was a big, monolithic, single piece of software. You downloaded the whole InterMine, you compiled the whole InterMine, you got the whole of InterMine. Instead, we are moving towards this idea of modularity and responsiveness. Smaller, independent libraries that are interconnected but can be used for tools and features separately or linked together.

Smaller decoupled InterMine packages will allow us to develop more features faster with less errors. InterMine maintainers might then have the flexibility to include (or not) the features in their mine, plug in their own tools, etc.

Version 2.0 represents a big step towards this goal!

A New Interface

A new feature in InterMine 2.0 is the ability to run our new UI, nicknamed “Blue Genes”. This app is in addition to the current webapp and offers a new and responsive search environment for your InterMine data.

Blue genes is a modern UI built in Clojure and provides a modern user experience.

  • Super fast response times
  • Interactive list upload
  • Redesigned “My account” section
  • Search autocomplete
  • Template and query builder result previews
  • .. lots more!

Once you have your InterMine updated to InterMine 2.0, there is a single command that will launch Blue Genes for your mine.

We are actively seeking feedback on Blue Genes, it’s still very much in the beta phase still, so please get in touch once you have some opinions!

Special Thanks

Thanks to everyone who helped test this release! Thanks Howie Motenko at MGI for your alpha testing and model insights. And a BIG thank you goes to Sam Hokin from the NCGR who spent a lot of time and effort helping improve InterMine! Thanks Sam and Howie! You are much appreciated.

Helpful Links

What exactly we changed (blog post)

Full list of GitHub tickets included this release

Docs on how to upgrade to 2.0

 

As always, please contact us if you have any questions or comments! We have an active twitter account, a discord server at chat.intermine.org, and a low traffic mailing list.

FlyMine 46.0 Released!

FlyMine has been updated to the latest version of FlyBase. All other data sets have also been updated to the newest versions and we have fixed a few bugs. See the data sources page for a full list of data and their versions. All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

 

Data model changes!

Our data model has changed slightly to make querying easier between mines.

  • Protein molecular weight is a float instead of an integer
  • We’ve added URLs for GO term evidence codes
  • Sequence Ontology (the basis for the InterMine data model) is updated, so lots of new data types added.

See our previous blog post for complete details of updates.

No more Anopheles

After listening to community feedback, we have decided to stop loading Anopheles data into FlyMine. However, as always, if there is a specific data set you are interested in, please contact us!

 

We have docs and videos, and for a full list of data sources available in FlyMine see the data sources list.

However, please do not hesitate to contact us should you require any further assistance. For all types of help and feedback email info@intermine.org.

HumanMine 5.0 released!

HumanMine has been updated to the latest version of NCBI Entrez Gene. All other data sets have also been updated to the newest versions and we have fixed a few bugs. See the data sources page for a full list of data and their versions. All data can be accessed through our comprehensive library of template searches or by building your own queries using the query builder.

 

New Data Source: GTEx

We added a new expression data set, GTEx. Here’s an example search:

Gene –> Tissue Expression

Tissue –> Gene expression

Data model changes!

Our data model has changed slightly to make querying easier between mines.

  • Protein molecular weight is a float instead of an integer
  • We’ve added URLs for GO term evidence codes
  • Sequence Ontology (the basis for the InterMine data model) is updated, so lots of new data types added.

See our previous blog post for complete details of updates.

 

We have docs and videos, and for a full list of data sources available in HumanMine see the data sources list.

However, please do not hesitate to contact us should you require any further assistance. For all types of help and feedback email info@intermine.org.

InterMine 2017 Fall Workshop – Biological Data Analysis using InterMine

University of Cambridge is hosting an InterMine workshop 27 October 2017.

The course is aimed at bench biologists and bioinformaticians who need to analyse their own data against large biological datasets, or who need to search against several biological datasets to gain knowledge of a gene/gene set, biological process or function. The exercises will mainly use the fly, human and mouse databases, but the course is applicable to anyone working with data for which an InterMine database is available.

The workshop is composed of two parts:

Part 1 (2.5 – 3 hours) will introduce participants to all aspects of the user interface, starting with some simple exercises and building up to more complex analysis encompassing several analysis tools and comparative analysis across organisms. No previous experience is necessary for this part of the workshop.

The following features of the InterMine web interface will be covered:

  • Search interfaces and advanced query builder
  • Automated analysis of sets, e.g gene sets, including enrichment statistics
  • Analysis workflows
  • Tools for cross-organism analysis between InterMine databases.
  • Web services

Part 2 (1 hour) will focus on the InterMine API and introduce running InterMine searches through Python and Perl scripts. While complete beginners are welcome, some basic knowledge of Perl, and/or Python would be an advantage. The InterMineR package will also be introduced. Those not interested in this part of the workshop are welcome to leave or there will be a more advanced exercise using the web interface available as an alternative.

See here for details: https://www.gen.cam.ac.uk/events/intermine-training

 

 

InterMine 2.0

InterMine 2.0 is a large, disruptive release. There are lots of exciting features, but they will require InterMine maintainers to update their mines. Usually devs are able to update their mines with a simple git pull request. In this case, they’ll have to take specific actions to make sure their software is up to date.

Model changes

Several changes and additions to the core InterMine data model were discussed and approved by the community. See here for specific details on the new core data model.

This means that it’s likely that an InterMine 2.0 webapp will require a database built by InterMine 2.0 code.

Blue Genes

InterMine 2.0 will come with detailed instructions on how to deploy the new InterMine user interface.

Come to the next InterMine community call to see a demo of the latest features!

Gradle

We’ve got a new software build system in the works. This will change the commands you use to build a data source and deploy your webapp. See a previous blog post for details.

Closer to the time, we’ll release detailed instructions on how to update your build system to work with the new tools. And as always the InterMine team will be on hand to answer any questions or issues on the community calls and the dev list and chat.

We hope to make the transition as easy as possible!

Software Dependencies

All software dependencies will need to be on the latest version.

  • Java 8
  • Tomcat 8.5.x
  • Postgres 9.4+

API Changes

We are making some non-backwards compatible changes to our API.

/user/queries will be moved to /queries

These three end points have a parameter called xml which holds the XML query. We are going to rename this parameter to be query (as we now accept JSON queries!) to match the syntax of all the other end points.

/query/upload
/template/upload
/user/queries (POST)

If this update is going to cause you any trouble at all, please let us know ASAP!

 

If you have any questions or concerns about any of these changes, please contact us or come along to the community calls.

 

 

 

Toxygates: exposing toxicogenomics datasets and linking with InterMine

This is a guest post from our colleague Johan Nyström-Persson, who works with ToxyGates and the NIBIOHN in Japan.

Toxygates (http://toxygates.nibiohn.go.jp) has been developed as a user-friendly toxicogenomics analysis platform at the Mizuguchi Lab, National Institutes of Biomedical Innovation, Health and Nutrition (NIBIOHN) in Osaka since 2012. The first public release was in 2013. At this time, the main focus of Toxygates was exposing the Open TG-GATEs dataset, a large, systematically organised toxicogenomics dataset compiled during more than a decade by the Japanese Toxicogenomics Project (http://toxico.nibiohn.go.jp). This dataset consists of over 24,000 microarray samples. To make use of such a large dataset without time-consuming data manipulation and programming, it is necessary to have a rich user interface and access to many kinds of secondary data.

Toxygates allows anyone with a web browser to explore and analyse this data in context. Various kinds of filtering and statistical testing are available, allowing users to discover and refine gene sets of interest, with respect to particular compounds. For a reasonably sized data selection, hierarchical clustering and heat-maps can be displayed directly in the browser. Through TargetMine (http://targetmine.nibiohn.go.jp) integration (based on the InterMine framework), enrichment of various kinds is possible. Compounds can also be ranked according to how they influence genes of interest.

To support all of these functions, we came up with the concept of a “hybrid” data model which recognises that, while gene expression values by themselves may be viewed as a large matrix with a flat structure, secondary annotations of genes and samples, such as
proteins, pathways, GO terms or pathological findings, have an open-ended structure. Thus, we combine an efficient key-value store (for gene expressions) with RDF and linked data (for gene and sample annotations) to allow for both high performance and a flexible data structure.

Today, the project continues to evolve in new directions as a general transcriptomics data analysis platform. We have integrated Toxygates not only with TargetMine, but also with HumanMine, RatMine and MouseMine. Recently, users can also upload their own transcriptomics data and analyse it in context alongside Open TG-GATEs data. We may
also add more datasets in the future.

P1000874The current project members are Kenji Mizuguchi (project leader) and Chen Yi-An (NIBIOHN), Johan Nyström-Persson and Yuji Kosugi (Level Five), and Yayoi Natsume-Kitatani and Yoshinobu Igarashi (NIBIOHN).