Monit: Monitoring your Services

Bioinformatics Applications are moving more in the direction of "Microservices" Architectures where services should be fine-grained and the protocols should be lightweight. Microservices Architectures decomposed the application into different smaller services improving the modularity; making the application easier to develop, deploy and maintain. It also parallelizes development by enabling small autonomous teams to develop, deploy and scale their respective services independently.

With more services (Databases, APIs, Web Applications, Pipelines) more components should be trace, monitor, to know the health of your application. There might be different roles that are played by different services (in different physical/logical machines) that can be even geographically isolated from each other. As a whole, these services/servers might be providing a combined service to the end application. A particular issue or problem on any of the server should not affect the final application behavior and must be found and fixed before the outage happens.

Multiple applications allow developers/devops and sysadmins to monitor all the services in a microservices application, but the most popular ones are Nagios and Monit.

What is BioHackathon 2014?

In a week BioHackathon 2014 will start (http://www.biohackathon.org/). It will be my first time ins this kind of "meeting". I will give a talk about PRIDE and ProteomeXchange and future developments of both platforms (below the complete list of talks).

But first, a quick introduction of BioHackathon. National Bioscience Database Center (NBDC) and Database Center for Life Science (DBCLS) have been organizing annual BioHackathon since 2008, mainly focusing on standardization (ontologies, controlled vocabularies, metadata) and interoperability of bioinformatics data and web services for improving integration (semantic web, web services, data integration), preservation and utilization of databases in life sciences. This year, we will focus on the standardization and utilization of human genome information with Semantic Web technologies in addition to our previous efforts on semantic interoperability and standardization of bioinformatics data and Web services.

Who is a senior developer anyway?

Originally posted in Java Code Geeks

by David Green on October 28th, 2013 | Filed in: Software Development

What makes you a “senior developer”? Everyone and their dog calls themselves a senior developer these days. From fresh graduates to the CTO, everyone is a senior developer. But what the hell does it even mean?

Technologists

Some developers are avid technologists. They got into programming really because they like tinkering. If it hadn’t been 7 languages in 7 weeks, it would have been a box of meccano or they’d be in their shed busy inventing the battery operated self-tieing tie. These people are great to have on your team, they’ll be the ones continually bombarding you with the latest and greatest shiny. If you ever want to know if there’s an off the shelf solution to your problem, they’ll know the options, have tried two of them, and currently have a modified version of a third running on their raspberry pi.

The trouble with technologists is more technology is always the answer. Why have a HTTP listener when you can have a full stack application server? Why use plain old TCP when you can introduce an asynchronous messaging backbone? Why bother trying to deliver software when there’s all these toys to play with!

Installing standalone SpectraST in linux

Some tips to install SpectraST in linux in standalone:

1. -  Download TTP latest version.
2. -  go to SpectraST folder and run make:
     
    cd TPP-x.x.x/trans_proteomic_pipeline/src/Search/SpectraST
    make -f Makefile_STANDALONE_LINUX

Start a startup or Work for someone else?

Originally posted on P4P:

When you look online for advice about entrepreneurship, you will see a lot of "just do it": 

The best way to get experience... is to start a startup. So, paradoxically, if you're too inexperienced to start a startup, what you should do is start one. That's a way more efficient cure for inexperience than a normal job. - Paul Graham, Why to Not Not Start a Startup

There is very little you will learn in your current job as a {consultant, lawyer, business person, economist, programmer} that will make you better at starting your own startup. Even if you work at someone else’s startup right now, the rate at which you are learning useful things is way lower than if you were just starting your own. -  David Albert, When should you start a startup?

This advice almost never comes with citations to research or quantitative data, from which I have concluded:

The sort of person who jumps in and gives advice to the masses without doing a lot of research first generally believes that you should jump in and do things without doing a lot of research first. 

Adding CITATION to your R package

Original post from Robin's Blog:

Software is very important in science – but good software takes time and effort that could be used to do other work instead. I believe that it is important to do this work – but to make it worthwhile, people need to get credit for their work, and in academia that means citations. However, it is often very difficult to find out how to cite a piece of software – sometimes it is hidden away somewhere in the manual or on the web-page, but often it requires sending an email to the author asking them how they want it cited. The effort that this requires means that many people don’t bother to cite the software they use, and thus the authors don’t get the credit that they need. We need to change this, so that software – which underlies a huge amount of important scientific work – gets the recognition it deserves.

ProteoStats: Computing false discovery rates in proteomics

By Amit K. Yadav (@theoneamit) & Yasset Perez-Riverol (@ypriverol):

Perl is a legacy language thought to be abstruse by many modern programmers. I’m passionate with the idea of not letting die a programming language such as Perl. Even when the language is used less in Computational Proteomics, it is still widely used in Bioinformatics. I’m enthusiastic writing about new open-source libraries in Perl that can be easily used. Two years ago, I wrote a post about InSilicoSpectro and how it can be used to study protein databases like I did in “In silico analysis of accurate proteomics, complemented by selective isolation of peptides”. 

Today’s post is about ProteoStats [1], a Perl library for False Discovery Rate (FDR) related calculations in proteomics studies. Some background for non-experts:

One of the central and most widely used approach for shotgun proteomics is the use of database search tools to assign spectra to peptides (called as Peptide Spectrum Matches or PSMs). To evaluate the quality of the assignments, these programs need to calculate/correct for population wise error rates to keep the number of false positives under control. In that sense, the best strategy to control the false positives is the target-decoy approach. Originally proposed by Elias & Gygi in 2007, the so-called classical FDR strategy or formula proposed involved a concatenated target-decoy (TD) database search for FDR estimation. This calculation is either done by the search engine or using scripts (in-house, non-published, not benchmarked, different implementations). 

So far, the only library developed to compute FDR at spectra level, peptide level and protein level FDRs is MAYU [2]. But, while MAYU only uses the classical FDR approach, ProteoStats provides options for 5 different strategies for calculating the FDR. The only prerequisite being that you need to search using a separate TD database as proposed by Kall et al (2008) [3]. Also, ProteoStats provides a programming interface that can read the native output from most widely used search tools and provide FDR related statistics. In case of tools not supported, pepXML, which has become a de facto standard output format, can be directly read along with tabular text based formats like TSV and CSV (or any other well-defined separator). 

Java Optimization Tips (Memory, CPU Time and Code)

There are several common optimization techniques that apply regardless of the language being used. Some of these techniques, such as global register allocation, are sophisticated strategies to allocate machine resources (for example, CPU registers) and don't apply to Java bytecodes. We'll focus on the techniques that basically involve restructuring code and substituting equivalent operations within a method.  

EntrySet vs KeySet

-----------------------------------------

More efficient:

for (Map.Entry entry : map.entrySet()) {
    Object key = entry.getKey();
    Object value = entry.getValue();
}

than:

for (Object key : map.keySet()) {
    Object value = map.get(key);
}

Avoid to create threads without run methods

------------------------------------

Usage Example: 

public class Test
{
 public void method() throws Exception
 {
  new Thread().start();  //VIOLATION
 }
}
Should be written as:

public class Test
{
 public void method(Runnable r) throws Exception
 {
  new Thread(r).start();  //FIXED
 }
}

Initialise the ArrayList if you know in advance the size

--------------------------------------------

 
For example, use this code if you expect your ArrayList to store around 1000 objects:

List str = new ArrayList(1000)

Use ternary operators

----------------------------------------

class Use_ternary_operator_correction
{
 public boolean test(String value)
 {
  if(value.equals("AppPerfect"))  // VIOLATION
  {
   return true;
  }
  else
  {
   return false;
  }
 }
}

Should be written as:


class Use_ternary_operator_correction
{
 public boolean test(String value)
 {
  return value.equals("AppPerfect"); // CORRECTION
 }
}

Always declare constant fields Static

public class Always_declare_constant_field_static_violation
{
 final int MAX = 1000; // VIOLATION
 final String NAME = "Noname"; // VIOLATION
}

Should be written as:

public class Always_declare_constant_field_static_correction
{
 static final int MAX = 1000; // CORRECTION
 static final String NAME = "Noname"; // VIOLATION
}

Solving Invalid signature in JNLP

I have this error each time i run my jnlp:

invalid SHA1 signature file digest for

I found some discussions about possible solutions:

http://stackoverflow.com/questions/8176166/invalid-sha1-signature-file-digest

http://stackoverflow.com/questions/11673707/java-web-start-jar-signing-issue

But he problem was still there. I solved the problem using plugin option (<unsignAlreadySignedJars>true</unsignAlreadySignedJars>) and removing previous signatures to avoid possible signature duplications:

  <plugin>
     <groupId>org.codehaus.mojo.webstart</groupId>
       <artifactId>webstart-maven-plugin</artifactId>
         <executions>
           <execution>
             <id>jnlp-building</id>
             <phase>package</phase>
               <goals>
                 <goal>jnlp</goal>
               </goals>
            </execution>
         </executions>
         <configuration>
           <!-- Include all the dependencies -->
           <excludeTransitive>false</excludeTransitive>
           <unsignAlreadySignedJars>true</unsignAlreadySignedJars>
           <verbose>true</verbose>
           <verifyjar>true</verifyjar>
           <!-- The path where the libraries are stored -->
           <libPath>lib</libPath>
           <jnlp>
             <inputTemplate>webstart/jnlp-template.vm</inputTemplate>
             <outputFile>ProteoLimsViewer.jnlp</outputFile>
             <mainClass>cu.edu.cigb.biocomp.proteolims.gui.ProteoLimsViewer</mainClass>
           </jnlp>
           <sign>
             <keystore>keystorefile</keystore>
             <alias>proteolimsviewer</alias>
             <storepass>password</storepass>
             <keypass>password</keypass>
             <keystoreConfig>
               <delete>false</delete>
               <gen>false</gen>
             </keystoreConfig>
           </sign>
              <!-- building process -->
              <pack200>false</pack200>
              <verbose>true</verbose>
         </configuration>
     </plugin>

News: JBioWH WebServices

We decided to develop a JBioWH webservice to provides the JBioWH data through internet. The source code is under development now but you can test the server on:
 

   http://hydrax.icgeb.trieste.it:8080/jbiowh-webservices/
 

Only the DataSet module is available and you can retrieve the dataset info 
using the server URL.  The webservices is able to send data in XML and JSON.

We are open to develop any webservices requested by users. So, let me know if 
your specific needs.

Regards 

Retrieve weka weitghts from Linear Regression

Weka function to retrieve the linear regression weights:

import java.util.List;
import weka.classifiers.Evaluation;
import weka.classifiers.functions.SMOreg;
import weka.classifiers.functions.supportVector.Kernel
.....

public double[] getCoefficients(){
       String svmClassifier = svm.toString();
       List<Double> coefficents = new ArrayList<Double>();
       double[] coeficcientsArray = null;
       if(svmClassifier.contains("weights (not support vectors):")){
          String[] svmClassStrings = svmClassifier.split(System.lineSeparator());
          int i = 0;
          boolean lastFound = false;
          while( i < svmClassStrings.length && !lastFound){
              String currentString = svmClassStrings[i];
              if(currentString.contains("weights (not support vectors):")){
                  i++;
                  while(i < svmClassStrings.length && !lastFound){
                      currentString = svmClassStrings[i];
                      String[] parts = currentString.split("\\s+");
                      if(parts.length < 5){
                          lastFound = true;
                      }else{
                          String number = parts[1] + parts[2];
                          coefficents.add(Double.parseDouble(number));
                      }
                      i++;
                  }
              }
              i++;
          }
        }