Updating bioinformatic software on Bioconda - a reference guide

In this second part of a three part guide, this post aims to provide an (opinionated) guide on how to update an existing tool or package that is already on Bioconda.

• For part one of this guide, see adding a new tool or package to Bioconda.
• For part three of this guide, see debugging a Bioconda build.

The conda package manager combined with the Bioconda repository has become a de facto gold-standard way for distributing bioinformatics software (Grüning et al. 2018). The associated Biocontainer project serves to provide complementary Docker and Singularity containers from the same conda (da Veiga Leprevost et al. 2017).

Updating a Bioconda recipe is often a relatively easy process as much of the ‘hard work’ and problems has been solved in the initial recipe creation. Typically updates to a Bioconda recipe consist of updating the version number of the tool and the hash of the tool or packages source code tarball. In some cases you may need to add a few dependencies (easy), and in rare cases change the build process (more complex). However in all cases you can refer to part one of this guide to understand these more complex scenarios.

In general, however, the process for updating or fixing an existing recipe is similar to the later steps in the first part of this guide: adding a new tool or package to Bioconda.

Note that if we use GitHub releases for our tool/package, Bioconda tries to _automatically update Bioconda recipes for us, so we may not need to do many of the steps this manually._ Of course, this works if there are no changes to the dependencies or tests that can cause the tests and thus the recipe building to fail.

Prerequisite

Make sure to familiarise yourself part one of this three part guide to understand the basics of adding a new tool to Bioconda.

Updating a Bioconda recipe

Otherwise, to manually update or fix a recipe:

1. Make sure our bioconda-recipes fork is up to date with the main.
2. Make a new branch for the update.
3. Edit the meta.yaml, build.sh files of the recipe with our changes.

4. Update the build number:

   • If it is simply fixing a recipe with no version change of the tool, bump the build_number by +1.
   • If this is a new version of the tool, set the build_number to 0.

   For most updates, the differences would simply look like this in the meta.yaml file:

   - {% set version = "2.0.6" %}
   + {% set version = "2.0.7" %}
   
   package:
       name: cami-amber
       version: {{ version }}
   
   source:
       url:  https://pypi.io/packages/source/c/cami-amber/cami-amber-{{ version }}.tar.gz
   -  sha256: d2d3d13a135f7ce4dff6bc1aab014945b0e5249b02f9afff3e6df1d82ef45d5a
   +  sha256: 01f11fbab7cb0f24497932669b00981292b1dc0df2ce6cd4b707a7ddd675bf8d
   
   build:
       noarch: python
   

5. Add all files, commit and push to our fork.
6. Open the PR on bioconda-recipes, wait for the CI to to complete successfully, and tag for review with ‘ please add label’ as above. For more details see part one of this guide.
   • If something goes wrong and something does not complete successfully, check the hash and build numbers are correct
   • If linting goes wrong, this is typically related to a missing run_exports section, see the opening instructions on the pango-collapse PR.

In case something goes wrong during step 6 above, see part three of this guide on how to debug a Bioconda build in case something goes wrong.

If the tool needs a new build procedure, see part one of this guide for more information on how to write build.sh scripts.

Conclusion

This part three of this guide, we given you enough pointers for anyone to be able to update an existing Bioconda recipe.

In the third part, we will go through how to manually debug the build process if things go wrong.

As with all bioinformatics and software development in general, things rarely just ‘work’ straight out of the box. My three biggest points of advice:

• Always copy and paste from other similar tools or packages on the Bioconda recipes repository.
• Take the time to read through the whole log messages (sometimes you can find critical clues hidden amongst the verbose information).
• Take the time to go step by step trying to follow exactly what Bioconda does during it’s own building on Azure with local building.

I found by taking the time, I very quickly learnt common issues and how to solve them.

Worst comes to worst, you can always ask the very friendly Bioconda team on the Bioconda gitter/matrix channel.