Galaxy Conference – Admin 25/06/2018

bit.ly/galaxyadmin
bit.ly/adminvms
bit.ly/gadminchat

Galaxy Release Schedule

3 releases per year: January, May and September

Install Galaxy using Ansible

https://github.com/galaxyproject/dagobah-training/blob/2018-gccbosc/sessions/14-ansible/ex2-galaxy-ansible.md

sudo pip install ansible
git clone https://github.com/ARTbio/GalaxyKickStart
cd GalaxyKickStart
git checkout 2018-gccbosc
ansible-galaxy install -r requirements_roles.yml -p roles –force

curl https://gist.githubusercontent.com/raonyguimaraes/e70220921504d26a0627050ade17bc24/raw/327f952683dc550613c4c60b2b74d32427dde0ea/galaxy_ansible_install.sh | sh

$ sudo su galaxy
$ vi /srv/galaxy/config/galaxy.yml
# Add the following line under galaxy: section
    admin_users: your@email.address
$ exit  # change back to ubuntu user
$ sudo supervisorctl restart galaxy:

galaxy.ansible.com/explore#

Time Topic Links Instructor
09:00 Welcome and introduction Slides (Č)
09:15 Deployment and platform options Slides (Č)
9:30 Using Ansible to deploy Galaxy SlidesExercise (E)(G)
10:20 Extending installation SlidesExercise (G)
10:40 Defining and importing genomes, Data Managers SlidesExercise (E)
11:00 Galactic Database Slides (M)(N)
11:15 Web Servers nginx/Apache Slides (M)(N)
11:30 Close Morning Session

https://github.com/galaxyproject/dagobah-training/blob/2018-gccbosc/sessions/05-reference-genomes/ex1-reference-genomes.md#exercise-3-install-a-datamanager-from-the-toolshed

https://github.com/galaxyproject/dagobah-training/blob/2018-gccbosc/sessions/14-ansible/ex2-galaxy-ansible.md

Galaxy admin -> local data: Create DBKey and Reference Genome – fetching

Install dbkey from saccer2 data_manager_fetch_genome_dbkeys_all_fasta

Install BWA data_manager_bwa_mem_index_builder

https://ephemeris.readthedocs.io/en/latest/

Admin -> create bwa index

Second Session

ubuntu@2018-gcc-training-0:~⟫ sudo vim /srv/galaxy/config/galaxy.yml

In /srv/galaxy/config/galaxy.yml, uncomment #nginx_x_accel_redirect_base: False and change it to nginx_x_accel_redirect_base: /_x_accel_redirect. Remember, this file is owned by the galaxy user so be sure to use sudo -u galaxy when editing it.

https://github.com/galaxyproject/dagobah-training/blob/2018-gccbosc/sessions/03-production-basics/ex3-nginx.md

sudo supervisorctl restart nginx galaxy:

Google’s PageSpeed Tools can identify any compression or caching improvements you can make.

If configuring SSL (out of scope for this training), out-of-the-box SSL settings are often insecure!

Use the Mozilla SSL config generator to create a default config and Qualys SSL Server Test to check it.

https://planemo.readthedocs.io/en/latest/

https://planemo.readthedocs.io/en/latest/readme.html

https://planemo.readthedocs.io/en/latest/writing_cwl_appliance.html?highlight=write%20appliance

$ planemo test –no-container –engine toil seqtk_seq.cwl

galaxy-tool-test

planemo o

#this will open the browser

https://planemo.readthedocs.io/en/latest/writing_advanced.html

https://stats.galaxyproject.org/login

https://grafana.com

https://telescope.galaxyproject.org

cd /srv/galaxy/server/lib/galaxy/jobs/runners

Plugins

Correspond to job runner plugins in lib/galaxy/jobs/runners

Plugins for:

  • local
  • Slurm (DRMAA subclass)
  • DRMAA: SGE, PBS Pro, LSF, Torque
  • HTCondor
  • Torque: Using the pbs_python library
  • Pulsar: Galaxy’s own remote job management system
  • Command Line Interface (CLI) via SSH
  • Kubernetes
  • Go-Docker
  • Chronos

https://galaxyproject.github.io/dagobah-training/2018-gccbosc/15-job-conf/job_conf.html#3

https://research.cs.wisc.edu/htcondor/

Need a shared file system, nfs, ceph and etc.

Exception is Pulsar!

sudo cat job_conf.xml.sample_basic 
<?xml version="1.0"?>
<!-- A sample job config that explicitly configures job running the way it is configured by default (if there is no explicit config). -->
<job_conf>
 <plugins>
 <plugin id="local" type="runner" load="galaxy.jobs.runners.local:LocalJobRunner" workers="4"/>
 </plugins>
 <destinations>
 <destination id="local" runner="local"/>
 </destinations>
</job_conf>

https://docs.galaxyproject.org/en/master/admin/jobs.html#dynamic-destination-mapping

https://docs.galaxyproject.org/en/master/admin/dependency_resolvers.html

http://galaxyproject.github.io/training-material/topics/admin/tutorials/connect-to-compute-cluster/slides.html#1

https://github.com/galaxyproject/galaxy-hub/blob/master/src/galaxy-updates/2018-03/index.md

https://pulsar.incubator.apache.org/

http://galaxyproject.github.io/training-material/topics/admin/tutorials/connect-to-compute-cluster/slides.html#13

https://github.com/galaxyproject/galaxy-kubernetes

http://galaxyproject.github.io/training-material/topics/admin/tutorials/connect-to-compute-cluster/tutorial.html#section-4—statically-map-a-tool-to-a-job-destination

Edit:/srv/galaxy/config/tool_destinations.yml

---
tools:
  multi:
    rules:
      - rule_type: file_size
        lower_bound: 16
        upper_bound: Infinity
        destination: slurm-2c
    default_destination: slurm_cluster
default_destination: local_no_container
verbose: True

gdb

attach 28376

https://www.gnu.org/software/gdb/

 

A Django Async Roadmap – Aeracode

I think that the time has come to start talking seriously about bringing async functionality into Django itself, and so I have been working on a draft “roadmap” for what I think this might look like. I’ve run this past a few people – some of who were Django core members, and some who weren’t – but I’m now posting it up for public feedback (see the end for where to discuss this).

 

Source: A Django Async Roadmap – Aeracode

Machine Learning: The High Interest Credit Card of Technical Debt – Google AI

Abstract

Machine learning offers a fantastically powerful toolkit for building complex systems quickly. This paper argues that it is dangerous to think of these quick wins as coming for free. Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning. The goal of this paper is highlight several machine learning specific risk factors and design patterns to be avoided or refactored where possible. These include boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, changes in the external world, and a variety of system-level anti-patterns.

Machine Learning: The High Interest Credit Card of Technical Debt

Source: Machine Learning: The High Interest Credit Card of Technical Debt – Google AI

On Intelligence in Cells: The Case for Whole Cell Biology

Biology needs revolution. All my adult life, I have been lost with admiration
for the achievements in molecular biology and genetics, and I have come to
know many of the main proponents. Yet there is an alternative aspect: in
studying the minutiae, we have lost sight of the whole cell as organism.
Living cells within the body are modelled in this paper as coordinated but
essentially autonomous entities. We shall see how independent cells in
nature have remarkable abilities to make decisions and take constructive
action, which correlate with the definitions of intelligence.

Source: a-ISR_Ford.pdf

EU copyright reforms draw fire from scientists

EU copyright reforms draw fire from scientists

An influential committee of the European Parliament is due to vote this month on changes to copyright regulations in the European Union, but the latest drafts of the rules have triggered a wave of criticism from open-science advocates. They say that the proposals will stifle research and scholarly communication.

Source: EU copyright reforms draw fire from scientists

State of React Native 2018 · React Native

It’s been a while since we last published a status update about React Native.

At Facebook, we’re using React Native more than ever and for many important projects. One of our most popular products is Marketplace, one of the top-level tabs in our app which is used by 800 million people each month. Since its creation in 2015, all of Marketplace has been built with React Native, including over a hundred full-screen views throughout different parts of the app.

We’re also using React Native for many new parts of the app. If you watched the F8 keynote last month, you’ll recognize Blood Donations, Crisis Response, Privacy Shortcuts, and Wellness Checks – all recent features built with React Native. And projects outside the main Facebook app are using React Native too. The new Oculus Go VR headset includes a companion mobile app that is fully built with React Native, not to mention React VR powering many experiences in the headset itself.

Naturally, we also use many other technologies to build our apps. Litho and ComponentKit are two libraries we use extensively in our apps; both provide a React-like component API for building native screens. It’s never been a goal for React Native to replace all other technologies – we are focused on making React Native itself better, but we love seeing other teams borrow ideas from React Native, like bringing instant reload to non-JavaScript code too.

State of React Native 2018

Source: State of React Native 2018 · React Native