tag:blogger.com,1999:blog-40595813337840293642024-03-13T03:14:33.111-07:00Coding and More - keeping track of annoying things I run into while codingGert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.comBlogger151125tag:blogger.com,1999:blog-4059581333784029364.post-19553239725598768512015-04-01T17:52:00.001-07:002015-04-01T17:52:08.308-07:00adding program shortcuts to the context menu of windowsever since I got the surface pro 3, loving it, I ended up doing a lot of programming under windows and rediscovering this platform.<br />
<br />
So one of my first things I had todo was to add/optimize some context menu entries to simplify the progress of getting console2 to open at specific directories. Basically trying to replace the cmd program as much as possible.<br />
<br />
<div>
regedit.exe</div>
<div>
<br /></div>
<div>
got to:</div>
<div>
<br /></div>
<div>
<span style="background-color: #f9f9f9; font-family: monospace; font-size: 14.8199996948242px; line-height: 21px;">HKEY_CLASSES_ROOT\Folder\shell</span></div>
<div>
<span style="background-color: #f9f9f9; font-family: monospace; font-size: 14.8199996948242px; line-height: 21px;"><br /></span></div>
<div>
<span style="background-color: #f9f9f9; font-family: monospace; font-size: 14.8199996948242px; line-height: 21px;">edit 'Command Prompt here' </span></div>
<div>
<span style="background-color: #f9f9f9; font-family: monospace; font-size: 14.8199996948242px; line-height: 21px;"><br /></span></div>
<div>
<span style="background-color: #f9f9f9; font-family: monospace; font-size: 14.8199996948242px; line-height: 21px;">set as new command</span></div>
<div>
<span style="background-color: #f9f9f9; font-family: monospace; font-size: 14.8199996948242px; line-height: 21px;"><br /></span></div>
<div>
<span style="background-color: #f9f9f9;"><span style="font-family: monospace;"><span style="font-size: 14.8199996948242px; line-height: 21px;">"C:\terminal\Console.exe" -d "%L"</span></span></span></div>
<div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
</div>
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-49109374601161753622015-02-24T15:32:00.000-08:002015-02-24T15:32:16.912-08:00simple oneliner to create an index over all the massbank indexesI'm quite often playing around with massbank these days and always like to have a simple index of all there files in a single directory instead of dealing with all there sub directories.<br />
<br />
<blockquote class="tr_bq">
<div class="p1">
<span class="s1">find ./ -name '*.txt' -exec bash -c "f=\$(basename {} ); ln -fs {} ./all/\$f" \;</span></div>
</blockquote>
<br />
basically takes care of all these issues for youGert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-56825839452152813692014-05-19T22:40:00.000-07:002014-05-19T22:43:21.545-07:00Using grails for restful servicesWell if you use grails for restful services, it saves a lot of time to ensure that you add the following mapping to your controller.<br />
<br />
<br />
<pre class="prettyprint" name="code">
class SubmitterController extends RestfulController{
//response format avoid the error that views are not found
static responseFormats = ['json']
static allowedMethods = [list:"GET", save: "POST", update: "POST", delete: "DELETE", show: "GET"]
public SubmitterController(){
super(Submitter)
}
}
</pre>
once you did this, everything works as it should and it's rather obvious. But sometimes we forget. Nearly as stupid as font-awesome not working fonts, if you use...
<pre class="prettyprint" name="code">
< i class="fa-minus-square"></i>
</pre>
you will only see a stupid utf square. Instead you need to use...
<pre class="prettyprint" name="code">
< i class="fa fa-minus-square"></i>
</pre>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-19103058090589220732014-02-07T15:36:00.001-08:002014-02-07T15:36:20.087-08:00adding the head node of rocks as compute node<h2>
happily stolen from:</h2>
<div>
<br /></div>
<div>
https://wiki.rocksclusters.org/wiki/index.php/Sun_GridEngine</div>
<div>
<br /></div>
<div>
cause I keep forgetting it...</div>
<h2>
<br /></h2>
<h2>
Add Frontend as a SGE Execution Host in Rocks </h2>
To setup the frontend node to also be a SGE execution host which
queued jobs can be run on (like the compute nodes), do the following:
<br />
<div class="editsection" style="float: right; margin-left: 5px;">
[<a href="https://wiki.rocksclusters.org/wiki/index.php?title=Sun_GridEngine&action=edit&section=9" title="Sun GridEngine">edit</a>]</div>
<h3>
Quick Setup </h3>
<pre># cd /opt/gridengine
# ./install_execd (accept all of the default answers)
# qconf -mq all.q (if needed, adjust the number of slots for [frontend.local=4] and other parameters)
# /etc/init.d/sgemaster.frontend stop
# /etc/init.d/sgemaster.frontend start
# /etc/init.d/sgeexecd.frontend stop
# /etc/init.d/sgeexecd.frontend start
</pre>
<div class="editsection" style="float: right; margin-left: 5px;">
[<a href="https://wiki.rocksclusters.org/wiki/index.php?title=Sun_GridEngine&action=edit&section=10" title="Sun GridEngine">edit</a>]</div>
<h3>
Detailed Setup </h3>
1. As root, make sure <code>$SGE_ROOT</code>, etc. are setup correctly on the frontend:
<br />
<pre># env | grep SGE
</pre>
It should return back something like:
<br />
<pre>SGE_CELL=default
SGE_ARCH=lx26-amd64
SGE_EXECD_PORT=537
SGE_QMASTER_PORT=536
SGE_ROOT=/opt/gridengine
</pre>
If not, source the file <code>/etc/profile.d/sge-binaries.[c]sh</code> or check if the SGE Roll is properly installed and enabled:
<br />
<pre># rocks list roll
NAME VERSION ARCH ENABLED
sge: 5.2 x86_64 yes
</pre>
<br />
2. Run the <code>install_execd</code> script to setup the frontend as a SGE execution host:
<br />
<pre># cd $SGE_ROOT
# ./install_execd
</pre>
Accept all of the default answers as suggested by the script.
<br />
<br />
<br />
<ul>
<li> <b>NOTE:</b> For the following examples below, the text <code><frontend></frontend></code> should be substituted with the actual "short hostname" of your frontend (as reported by the command <code>hostname -s</code>).
</li>
</ul>
For example, if running the command <code>hostname</code> on your frontend returns back the "FQDN long hostname" of:
<br />
<pre># hostname
mycluster.mydomain.org
</pre>
then <code>hostname -s</code> should return back just:
<br />
<pre># hostname -s
mycluster
</pre>
<br />
3. Verify that the number of job slots for the frontend is equal to the
number of physical processors/cores on your frontend that you wish to
make available for queued jobs by checking the value of the <code>slots</code> parameter of the queue configuration for <code>all.q</code>:
<br />
<pre># qconf -sq all.q | grep slots
slots 1,[compute-0-0.local=4],[<frontend>.local=4]
</frontend></pre>
The <code>[<frontend>.local=4]</frontend></code> means that SGE can run up
to 4 jobs on the frontend. Be aware that since the frontend is
normally used for other tasks besides running compute jobs, it is
recommended that not all the installed physical processors/cores on the
frontend be available to be scheduled by SGE to avoid overloading the
frontend.
<br />
For example, on a 4-core frontend, to configure SGE to use only up to 3 of the 4 cores, you can modify the slots for <code><frontend>.local</frontend></code> from 4 to 3 by typing:
<br />
<pre># qconf -mattr queue slots '[<frontend>.local=3]' all.q
</frontend></pre>
If there are additional queues besides the default <code>all.q</code> one, repeat the above for each queue.
<br />
Read <code>"man queue_conf"</code> for a list of resource limit parameters such as <code>s_cpu</code>, <code>h_cpu</code>, <code>s_vmem</code>, and <code>h_vmem</code> that can be adjusted to prevent jobs from overloading the frontend.
<br />
<br />
<br />
<ul>
<li> <b>NOTE:</b> For Rocks 5.2 or older, the frontend may have been default configured during installation with only 1 job slot (<code>[<frontend>.local=1]</frontend></code>) in the default <code>all.q</code> queue, which will only allow up to 1 queued job to run on the frontend. To check the value of the <code>slots</code> parameter of the queue configuration for <code>all.q</code>, type:
</li>
</ul>
<pre># qconf -sq all.q | grep slots
slots 1,[compute-0-0.local=4],[<frontend>.local=1]
</frontend></pre>
If needed, modify the slots for <code><frontend>.local</frontend></code> from 1 to 4 (or up to the maximum number of physical processors/cores on your frontend that you wish to use) by typing:
<br />
<pre># qconf -mattr queue slots '[<frontend>.local=4]' all.q
</frontend></pre>
<br />
<br />
<ul>
<li> <b>NOTE:</b> For Rocks 5.3 or older, create the file <code>/opt/gridengine/default/common/host_aliases</code> to contain both the .local hostname and the FQDN long hostname of your frontend:
</li>
</ul>
<pre># vi $SGE_ROOT/default/common/host_aliases
</pre>
<pre><frontend>.local <frontend>.mydomain.org
</frontend></frontend></pre>
<br />
<br />
<ul>
<li> <b>NOTE:</b> For Rocks 5.3 or older, edit the file <code>/opt/gridengine/default/common/act_qmaster</code> to contain the .local hostname of your frontend:
</li>
</ul>
<pre># vi $SGE_ROOT/default/common/act_qmaster
</pre>
<pre><frontend>.local
</frontend></pre>
<br />
<br />
<ul>
<li> <b>NOTE:</b> For Rocks 5.3 or older, edit the file <code>/etc/init.d/sgemaster.<frontend></frontend></code>:
</li>
</ul>
<pre># vi /etc/init.d/sgemaster.<frontend>
</frontend></pre>
and comment out the line:
<br />
<pre>/bin/hostname --fqdn > $SGE_ROOT/default/common/act_qmaster
</pre>
by inserting a <code>#</code> character at the beginning, so it becomes:
<br />
<pre>#/bin/hostname --fqdn > $SGE_ROOT/default/common/act_qmaster
</pre>
in order to prevent the file <code>/opt/gridengine/default/common/act_qmaster</code> from getting overwritten with incorrect data every time <code>sgemaster.<frontend></frontend></code> is run during bootup.
<br />
<br />
4. Restart both qmaster and execd for SGE on the frontend:
<br />
<pre># /etc/init.d/sgemaster.<frontend> stop
# /etc/init.d/sgemaster.<frontend> start
# /etc/init.d/sgeexecd.<frontend> stop
# /etc/init.d/sgeexecd.<frontend> start
</frontend></frontend></frontend></frontend></pre>
<a href="" name="Quick_Setup"></a>
<a href="" name="Detailed_Setup"></a>
<br />
<br />
And everything will start working. :)Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-81968710684110093572013-09-20T15:34:00.007-07:002013-09-20T15:35:08.441-07:00postgres killing queriesonce in a while our database server get's terrible overloaded, cause people are running thousands of long running queries against it.<br />
<br />
1. How to find out who is generating queries:<br />
<br />
<b>select datname, client_addr from pg_stat_activity;</b><br />
<br />
<br />
2. Killing queries from a specific IP:<br />
<br />
<b>select pg_terminate_backend(procpid) from pg_stat_activity where client_addr='IP';</b><br />
<br />
This can be done as postgres user.Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-13637000750779685912013-08-20T14:40:00.001-07:002013-08-20T14:40:14.977-07:00playing with logfiles...After setting up our zabbix system at work to monitor most of the servers, I'm still trying to analyze our database server and logs a bit more. Since the system seems to be under quite a high load recently.<br />
<br />
The easiest way todo this, was like always a little awk scripts to analyze some statistics from our log files and point me in the right directions<br />
<br />
<blockquote class="tr_bq">
cat pg_log/postgresql-2013-08-*.log | grep duration | awk -F ':' '{ print $7, $0 }' | grep execute | awk '{ total += $1; count++; print $0 } END { print "average query speed: ", total/count, " count of queries: ", count }'</blockquote>
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-6268093407517741012013-07-18T12:50:00.002-07:002013-07-18T12:50:42.093-07:00X-Forwarding in qlogin on rocks linux > 5to enable X forwarding from your nodes to your original remote session:<br />
<br />
<blockquote class="tr_bq">
<div style="background-color: white; border: 0px; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px; margin-bottom: 1em; margin-top: 1em; padding: 0px; vertical-align: baseline;">
In rocks 4.3,I get the output<br /># qconf -sconf | grep qlogin<br />qlogin_command /opt/gridengine/bin/rocks-<wbr></wbr>qlogin.sh<br />qlogin_daemon /usr/sbin/sshd -i</div>
<div style="background-color: white; border: 0px; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px; margin-bottom: 1em; margin-top: 1em; padding: 0px; vertical-align: baseline;">
But in rocks 5.3,I get<br /># qconf -sconf | grep qlogin<br />qlogin_command builtin<br />qlogin_daemon builtin</div>
<div style="background-color: white; border: 0px; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px; margin-bottom: 1em; margin-top: 1em; padding: 0px; vertical-align: baseline;">
So I changed it in rocks 5.3<br /># qconf -mconf global<br />qlogin_command /opt/gridengine/bin/rocks-<wbr></wbr>qlogin.sh<br />qlogin_daemon /usr/sbin/sshd -i</div>
</blockquote>
<br />
And modify the script:<br />
<br />
<blockquote class="tr_bq">
<span style="background-color: white; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px;">/opt/gridengine/bin/rocks-</span><wbr style="color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px;"></wbr><span style="background-color: white; color: #222222; font-family: Arial, Helvetica, sans-serif; font-size: 13px;">qlogin.sh</span></blockquote>
<br />
to include this part<br />
<br />
<blockquote class="tr_bq">
/usr/bin/ssh -Y -p $PORT $HOST</blockquote>
The Y basically allows none secured X authentification.<br />
<blockquote class="tr_bq">
</blockquote>
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-41175274498959783422013-06-20T15:49:00.001-07:002013-07-08T15:23:46.697-07:00Playing with ChefRecently I got a bit overwhelmed with all the servers I maintain and have to update and so decided to ease my life and start using an automation system. Don't really care that much about my real servers. But having quite a lot of dedicated virtual machines, I guess it was time to simplify this process.<br />
<br />
<b>Bootstrapping a system</b><br />
<blockquote class="tr_bq">
knife bootstrap systemname-x username -P password --sudo</blockquote>
<b>Installing a cookbook from the central repository</b><br />
<blockquote class="tr_bq">
knife cookbook site install name-of-cookbook</blockquote>
<b>Uploading a cookbook to the server</b><br />
<blockquote class="tr_bq">
knife cookbook upload name-of-cookbook</blockquote>
<b>Adding a recipe to a system</b><br />
<blockquote class="tr_bq">
knife node run_list add HOSTNAME 'recipe[recipe]'</blockquote>
<br />
<b>Updating all systems</b><br />
<blockquote class="tr_bq">
knife ssh "*:*" "sudo chef-client" -x username -P password</blockquote>
<b>Updating/Adding a data bag</b><br />
<blockquote class="tr_bq">
knife data bag from file BAG_NAME ITEM.json</blockquote>
<b>Changing the environment for nodes</b><br />
<div class="section" id="move-nodes" style="-webkit-text-stroke-width: 0px; background-color: white; color: #212c35; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: 21px; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px;">
<div class="highlight-python">
<blockquote class="tr_bq">
knife exec -E 'nodes.transform("chef_environment:dev") { |n| n.chef_environment("production") }'</blockquote>
</div>
</div>
<b>Adding a role to all nodes in a certain environment</b><br />
<blockquote class="tr_bq">
knife exec -E 'nodes.transform("chef_environment:Fiehnlab") {|n| puts n.run_list << "role[user-management]"; n.save }'</blockquote>
<b>Overriding the runlist for a single run</b><br />
<blockquote class="tr_bq">
chef-client -o recipe["rocks-cluster-6.1::computenode"]</blockquote>
<br />
Just a small overview for me to remember some commands.Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-46004560711267869512013-04-19T15:39:00.000-07:002013-04-19T15:40:03.343-07:00cdk and grails - why don't get they along. This time using maven for dependency management......<br />
<br />
Recently I posted about <a href="http://codingandmore.blogspot.com/2013/03/cdk-and-grails-why-dont-they-get-along.html" target="_blank">grails2 and the CDK using the BuildConfig.groovy</a> approach.<br />
<br />
Now it's only fair that I show the maven approach as alternative. It's aint pretty...<br />
<br />
But it works and allows you to build grails 2.1 project with the CDK. Now i just need to integrate scala into the mix...<br />
<br />
<br />
<div>
<pre class="prettyprint" name="code"><?xml version="1.0" encoding="utf-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>edu.ucdavis.fiehnlab.cts</groupId>
<artifactId>cts</artifactId>
<packaging>grails-app</packaging>
<version>2.0-beta-2</version>
<name>cts</name>
<description>2.0-beta-2</description>
<parent>
<artifactId>web</artifactId>
<groupId>edu.ucdavis.fiehnlab.cts.web</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<dependencies>
<dependency>
<groupId>org.grails</groupId>
<artifactId>grails-dependencies</artifactId>
<version>${grails.version}</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>org.grails</groupId>
<artifactId>grails-test</artifactId>
<version>${grails.version}</version>
<type>pom</type>
</dependency>
<dependency>
<groupId>org.grails</groupId>
<artifactId>grails-plugin-testing</artifactId>
<version>${grails.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>net.sf.jni-inchi</groupId>
<artifactId>jni-inchi</artifactId>
<version>${jni.inchi.version}</version>
</dependency>
<dependency>
<groupId>org.openscience.cdk</groupId>
<artifactId>cdk-inchi</artifactId>
<version>${cdk.version}</version>
<exclusions>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xmlParserAPIs</artifactId>
</exclusion>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
</exclusion>
<exclusion>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
</exclusion>
<exclusion>
<groupId>java3d</groupId>
<artifactId>vecmath</artifactId>
</exclusion>
<exclusion>
<groupId>xmlpull</groupId>
<artifactId>xmlpull</artifactId>
</exclusion>
<exclusion>
<groupId>cml</groupId>
<artifactId>cmlxom</artifactId>
</exclusion>
<exclusion>
<groupId>xom</groupId>
<artifactId>xom</artifactId>
</exclusion>
<exclusion>
<groupId>jdom</groupId>
<artifactId>jdom</artifactId>
</exclusion>
<exclusion>
<groupId>xpp3</groupId>
<artifactId>xpp3</artifactId>
</exclusion>
<exclusion>
<groupId>jama</groupId>
<artifactId>jama</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.openscience.cdk</groupId>
<artifactId>cdk-fingerprint</artifactId>
<version>${cdk.version}</version>
<exclusions>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xmlParserAPIs</artifactId>
</exclusion>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
</exclusion>
<exclusion>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
</exclusion>
<exclusion>
<groupId>java3d</groupId>
<artifactId>vecmath</artifactId>
</exclusion>
<exclusion>
<groupId>xmlpull</groupId>
<artifactId>xmlpull</artifactId>
</exclusion>
<exclusion>
<groupId>cml</groupId>
<artifactId>cmlxom</artifactId>
</exclusion>
<exclusion>
<groupId>xom</groupId>
<artifactId>xom</artifactId>
</exclusion>
<exclusion>
<groupId>jdom</groupId>
<artifactId>jdom</artifactId>
</exclusion>
<exclusion>
<groupId>xpp3</groupId>
<artifactId>xpp3</artifactId>
</exclusion>
<exclusion>
<groupId>jama</groupId>
<artifactId>jama</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.openscience.cdk</groupId>
<artifactId>cdk-renderawt</artifactId>
<version>${cdk.version}</version>
<exclusions>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xmlParserAPIs</artifactId>
</exclusion>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
</exclusion>
<exclusion>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
</exclusion>
<exclusion>
<groupId>java3d</groupId>
<artifactId>vecmath</artifactId>
</exclusion>
<exclusion>
<groupId>xmlpull</groupId>
<artifactId>xmlpull</artifactId>
</exclusion>
<exclusion>
<groupId>cml</groupId>
<artifactId>cmlxom</artifactId>
</exclusion>
<exclusion>
<groupId>xom</groupId>
<artifactId>xom</artifactId>
</exclusion>
<exclusion>
<groupId>jdom</groupId>
<artifactId>jdom</artifactId>
</exclusion>
<exclusion>
<groupId>xpp3</groupId>
<artifactId>xpp3</artifactId>
</exclusion>
<exclusion>
<groupId>jama</groupId>
<artifactId>jama</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.openscience.cdk</groupId>
<artifactId>cdk-render</artifactId>
<version>${cdk.version}</version>
<scope>runtime</scope>
<exclusions>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xmlParserAPIs</artifactId>
</exclusion>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
</exclusion>
<exclusion>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
</exclusion>
<exclusion>
<groupId>java3d</groupId>
<artifactId>vecmath</artifactId>
</exclusion>
<exclusion>
<groupId>xmlpull</groupId>
<artifactId>xmlpull</artifactId>
</exclusion>
<exclusion>
<groupId>cml</groupId>
<artifactId>cmlxom</artifactId>
</exclusion>
<exclusion>
<groupId>xom</groupId>
<artifactId>xom</artifactId>
</exclusion>
<exclusion>
<groupId>jdom</groupId>
<artifactId>jdom</artifactId>
</exclusion>
<exclusion>
<groupId>xpp3</groupId>
<artifactId>xpp3</artifactId>
</exclusion>
<exclusion>
<groupId>jama</groupId>
<artifactId>jama</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.codehaus.groovy.modules.http-builder</groupId>
<artifactId>http-builder</artifactId>
<version>0.5.2</version>
<scope>runtime</scope>
<exclusions>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xmlParserAPIs</artifactId>
</exclusion>
<exclusion>
<groupId>xerces</groupId>
<artifactId>xercesImpl</artifactId>
</exclusion>
<exclusion>
<groupId>xalan</groupId>
<artifactId>xalan</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>8.2-504.jdbc3</version>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>cache</artifactId>
<version>1.0.1</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>jquery-ui</artifactId>
<version>1.8.24</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>jquery</artifactId>
<version>1.8.3</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>export</artifactId>
<version>1.5</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>resources</artifactId>
<version>1.1.6</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>jquery-ui-extensions</artifactId>
<version>0.5.8</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>hibernate</artifactId>
<version>${grails.version}</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>spock</artifactId>
<version>0.6</version>
<type>zip</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-grails-support</artifactId>
<version>0.6-groovy-1.8</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>functional-test</artifactId>
<version>1.2.7</version>
<type>zip</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>webxml</artifactId>
<version>1.4.1</version>
<type>zip</type>
</dependency>
<dependency>
<groupId>org.grails.plugins</groupId>
<artifactId>tomcat</artifactId>
<version>2.1.0</version>
<type>zip</type>
<scope>runime</scope>
</dependency>
</dependencies>
<build>
<pluginManagement/>
<plugins>
<!-- Disables the Maven surefire plugin for Grails applications, as we have our own test runner -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>surefire-it</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-clean-plugin</artifactId>
<version>2.4.1</version>
<configuration>
<filesets>
<fileset>
<directory>plugins</directory>
<includes>
<include>**/*</include>
</includes>
<followSymlinks>false</followSymlinks>
</fileset>
</filesets>
</configuration>
</plugin>
<plugin>
<groupId>org.grails</groupId>
<artifactId>grails-maven-plugin</artifactId>
<version>${grails.version}</version>
<configuration>
<!-- Whether for Fork a JVM to run Grails commands -->
<fork>true</fork>
</configuration>
<extensions>true</extensions>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>grails</id>
<name>grails</name>
<url>http://repo.grails.org/grails/core</url>
</repository>
<repository>
<id>grails-plugins</id>
<name>grails-plugins</name>
<url>http://repo.grails.org/grails/plugins</url>
</repository>
<repository>
<id>ambit-public</id>
<name>ambit-public</name>
<url>http://ambit.uni-plovdiv.bg:8083/nexus/content/repositories/public/</url>
</repository>
<repository>
<id>ambit-thirdparty</id>
<name>ambit-thirdparty</name>
<url>http://ambit.uni-plovdiv.bg:8083/nexus/content/repositories/thirdparty/</url>
</repository>
<repository>
<id>codehaus</id>
<name>codehaus</name>
<url>http://snapshots.repository.codehaus.org</url>
</repository>
</repositories>
</project>
</pre>
</div>
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-25834561373250044162013-04-10T10:57:00.001-07:002013-04-10T10:57:20.313-07:00scala yield vs map which is nicer?currently I'm playing a bit more with scala in an attempt to leverage the power of my new pair of radeon 7970 for GPU calculations and I'm a touch unsure, which syntax I prefer and is easier to read.<br />
<br />
example a:<br />
<br />
<br />
<i>scala> 1 to 10 map ( _ * 2)</i><br />
<i>res4: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10, 12, 14, 16, 18, 20)</i><br />
<br />
<br />
versus example b:<br />
<br />
<i>scala> for (i <- 10="" 1="" 2="" i="" nbsp="" to="" yield=""></-></i><br />
<i>res5: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10, 12, 14, 16, 18, 20)</i><br />
<br />
<br />
They both produce the same result, but I feel the b is easier to understand from a none scala perspective. But a. Is shorter and somewhat clearer<br />
<br />
Now if we change this to:<br />
<br />
<br />
<i>scala> val result = 1 to 10 map ( _ * 2)</i><br />
<i>result: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10, 12, 14, 16, 18, 20)</i><br />
<br />
and<br />
<br />
<i>scala> val result = for (i <- 10="" 1="" 2="" i="" nbsp="" to="" yield=""></-></i><br />
<i>result: scala.collection.immutable.IndexedSeq[Int] = Vector(2, 4, 6, 8, 10, 12, 14, 16, 18, 20)</i><br />
<br />
now the first version seems to make a lot more sense and the second looks unwieldy complicated.<br />
<br />
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-25610131828164282422013-03-14T15:04:00.001-07:002013-03-14T15:04:49.596-07:00cdk and grails, why don't they get along...so apperently the cdk depends on some xml libaries, which causes us all kind of error messages like this in grails:<br />
<br />
<br />
<pre class="prettyprint" name="code">| Loading Grails 2.1.0
| Configuring classpath
| Error Error executing script Help: loader constraint violation: when resolving overridden method "org.apache.tools.ant.helper.ProjectHelper2$RootHandler.setDocumentLocator(Lorg/xml/sax/Locator;)V" the class loader (instance of org/codehaus/groovy/grails/cli/support/GrailsRootLoader) of the current class, org/apache/tools/ant/helper/ProjectHelper2$RootHandler, and its superclass loader (instance of <bootloader>), have different Class objects for the type org/xml/sax/Locator used in the signature (Use --stacktrace to see the full trace)
IDEA hook: Grails not found!
| Error java.lang.NullPointerException
| Error at org.jetbrains.groovy.grails.rt.Agent$2.run(Agent.java:135)
| Error at java.lang.Thread.run(Thread.java:680)
</bootloader>
</pre>
<div>
which are rather annoying since all the mailing list say, to just run the dependency report to see what causes this problem. Great idea exact that grails crashes before it can generate the report...
<br />
<br />
So after a while of searching and goggling and shouting random words at my imac, I found an ugly solution which works rather well for now.
<br />
Basically make your BuildConfig.groovy file look like this and it should fix your current issues...
</div>
<pre class="prettyprint" name="code"> dependencies {
runtime 'postgresql:postgresql:8.2-504.jdbc3'
compile('xmlpull:xmlpull:1.1.3.1')
compile('org.openscience.cdk:cdk-fingerprint:1.4.16') {
transitive = false
}
compile('org.openscience.cdk:cdk-inchi:1.4.16') {
transitive = false
}
compile('org.openscience.cdk:cdk-standard:1.4.16')
compile('org.openscience.cdk:cdk-interfaces:1.4.16')
compile('org.openscience.cdk:cdk-annotation:1.4.16')
compile('org.openscience.cdk:cdk-io:1.4.16')
compile('org.openscience.cdk:cdk-isomorphism:1.4.16')
compile('org.openscience.cdk:cdk-render:1.4.16')
compile('org.openscience.cdk:cdk-renderbasic:1.4.16')
compile('org.openscience.cdk:cdk-renderawt:1.4.16')
compile('org.openscience.cdk:cdk-smarts:1.4.16') {
transitive = false
}
compile('org.openscience.cdk:cdk-extra:1.4.16') {
transitive = false
}
compile('org.openscience.cdk:cdk-dict:1.4.16') {
transitive = false
}
compile('jama:jama:1.0.2') {
transitive = false
}
compile('org.openscience.cdk:cdk-formula:1.4.16')
compile('org.openscience.cdk:cdk-smsd:1.4.16')
compile('xpp3:xpp3:1.1.4c')
compile('java3d:vecmath:1.3.1')
compile('net.sf.jni-inchi:jni-inchi:0.7')
}
</pre>
<div>
Maybe this help someone, who has the same issues with old XML-Apis and Xerces and so
</div>
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-68501325210542503572013-03-11T16:14:00.001-07:002013-03-11T16:20:11.543-07:00Lift - getting access to a databaseSince I recently started to play more and more with lift. I discovered that it's not always as easy as you think it is and once you find a solution it was easy indeed.<br />
<br />
For example I want to connect to my database server, but have no desire to actually use any form of OR mapping.<br />
<br />
So after googleing a bit and woundering how todo this. It's all rather simple.<br />
<br />
In your Boot.scala file<br />
<br />
<br />
<pre class="prettyprint" name="code">
def boot {
// where to search snippet
LiftRules.addToPackages("code")
// Build SiteMap
LiftRules.setSiteMap(sitemap)
//rest based stuff
/**
* connection url for lift
*/
val vendor = new StandardDBVendor("org.postgresql.Driver","jdbc:postgresql://****/****",Full("username"),Full("password")) with ProtoDBVendor{
override def maxPoolSize = 100
override def doNotExpandBeyond = 200
}
DB.defineConnectionManager(DefaultConnectionIdentifier, vendor)
}
</pre>
<br />
This basically defines what to connect to and a connection pool with a max of 100 connections.<br />
<br />
To actually access this connection and todo something with it, well this is remarkable easy.<br />
<br />
<br />
<pre class="prettyprint" name="code">
DB.performQuery("select * from data")
</pre>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-39521197391319297322013-01-29T16:32:00.001-08:002013-01-29T16:33:07.016-08:00scala + lift + jsonSo since I really really really dislike soap and Java, I decided to play a bit more with scala and rest based web services. Basically parse some JSON strings with scala.<br />
<br />
Now after browsing countless articles and remembering the JSON structure, I finally found the 'lift-json' project, which simplifies my live as busy developer a bit.<br />
<br />
How to parse JSON with scala?<br />
<br />
<br />
<code>
<div class="p1">
<span class="s1">package</span> edu.ucdavis.fiehnlab.alchemy.core.communication.services.reader</div>
<div class="p1">
<span class="s1">import</span> edu.ucdavis.fiehnlab.alchemy.core.process.types.LibrarySpectra</div>
<div class="p1">
<span class="s1">import</span> net.liftweb.json.parse</div>
<div class="p1">
<span class="s1">import</span> net.liftweb.json.DefaultFormats</div>
<div class="p2">
/**</div>
<div class="p2">
* connects to the alchemy <span class="s2">webservice</span> and fetches all registered <span class="s2">co</span></div>
<div class="p2">
*/</div>
<div class="p1">
<span class="s1">class</span> WebserviceLibraryReader {</div>
<div class="p3">
<br /></div>
<div class="p2">
<span class="s3"> </span>/**</div>
<div class="p2">
* internal converter class to simplify things</div>
<div class="p2">
*/</div>
<div class="p1">
<span class="s1">case</span> <span class="s1">class</span> JsonCompound(name: String, inchikey: String, retentiontime: Double, theoretical: Double, massspectra: String)</div>
<div class="p3">
<br /></div>
<div class="p1">
<span class="s1">case</span> <span class="s1">class</span> CompoundWapper(compound: Array[JsonCompound])</div>
<div class="p3">
<br /></div>
<div class="p1">
<span class="s1">implicit</span> <span class="s1">val</span> formats = DefaultFormats</div>
<div class="p3">
<br /></div>
<div class="p2">
<span class="s3"> </span>/**</div>
<div class="p2">
* downloads the compounds for the given URL and returns it as a set of library spectra</div>
<div class="p2">
*/</div>
<div class="p1">
<span class="s1">def</span> fetchCompounds(url: String): Set[LibrarySpectra] = {</div>
<div class="p3">
<br /></div>
<div class="p1">
<span class="s1">val</span> compounds: CompoundWapper = parse(<span class="s4">"""</span></div>
<div class="p4">
</div>
<div class="p5">
{"compound":[{"name":"test","inchikey":"ADVPTQAUNPRNPO-UHFFFAOYSA-N","retentiontime":11111,"theoretical":11,"massspectra":"111:1 112:2 113:3"},{"name":"test","inchikey":"ADVPTQAUNPRNPO-UHFFFAOYSA-N","retentiontime":11111,"theoretical":11,"massspectra":"111:1 112:2 113:3"}]}</div>
<div class="p4">
</div>
<div class="p1">
<span class="s4"> """</span>).extract[CompoundWapper]</div>
<div class="p3">
<br /></div>
<div class="p1">
compounds.compound.foreach { x: JsonCompound =></div>
<div class="p1">
println(x)</div>
<div class="p1">
}</div>
<div class="p3">
</div>
<div class="p3">
</div>
<div class="p1">
<span class="s1">null</span></div>
<div class="p1">
}</div>
<div class="p3">
<br /></div>
<div class="p1">
}</div>
<div class="p3">
<br /></div>
<div class="p1">
<span class="s1">object</span> WebserviceLibraryReader {</div>
<div class="p3">
<br /></div>
<div class="p1">
<span class="s1">def</span> main(ars: Array[String]) = {</div>
<div class="p5">
<span class="s3"> println(</span><span class="s1">new</span><span class="s3"> WebserviceLibraryReader().fetchCompounds(</span>"http://localhost:8080/alchemy-admin/services/queryAllCompoundsForMethod/test"<span class="s3">))</span></div>
<div class="p1">
}</div>
<div class="p1">
}</div>
<div class="p1">
<br /></div>
<div class="p1">
</code>
The complete tutorial can be found <a href="https://github.com/lift/lift/tree/master/framework/lift-base/lift-json/" target="_blank">here</a></div>
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-53874433240776746552013-01-24T21:17:00.002-08:002013-01-24T21:17:57.749-08:00using ssd's in a raid0 over 4 drives.Recently I played a bit more with ssd's, since frankly they are getting affordable enough to use as scratch drives for our new software product.<br />
<br />
Basically I was wondering, if a raid0 over 4 ssd's is fast enough for us, or if I should use a PCI-Express card.<br />
<br />
<code></code><br />
<br />
root@****:/mnt/luna/scratch/gert# dd if=/dev/zero of=/mnt/scratch/bs.img bs=8048 count=81920<br />
81920+0 records in<br />
81920+0 records out<br />
659292160 bytes (659 MB) copied, 0.529044 s, 1.2 GB/s<br />
root@****:/mnt/luna/scratch/gert# dd of=/dev/null if=/mnt/scratch/bs.img bs=8048 count=81920<br />
81920+0 records in<br />
81920+0 records out<br />
659292160 bytes (659 MB) copied, 0.129838 s, 5.1 GB/s<br />
<br />
<br />
Sure the sample size is not perfect, but for a first test I'm rather surprised by this result and looking forward, if the speed keeps up like this.<br />
Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-62249299095638659632012-12-13T14:18:00.001-08:002012-12-13T14:18:43.195-08:00grails 1.3.8 and java 1.6since we still have a lot of grails 1.3.8 application, we are forced to use them and apparently sometimes all the test fail with the following exception:<br />
<br />
<br />
<code>
java.lang.IllegalAccessError: tried to access class org.apache.xml.serializer.XSLOutputAttributes from class org.apache.xalan.transformer.TransformerImpl
at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2387)
at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2255)
at org.apache.xalan.lib.Redirect.write(Redirect.java:212)
at org.apache.xalan.extensions.ExtensionHandlerJavaClass.processElement(ExtensionHandlerJavaClass.java:495)
at org.apache.xalan.templates.ElemExtensionCall.execute(ElemExtensionCall.java:230)
at org.apache.xalan.templates.ElemApplyTemplates.transformSelectedNodes(ElemApplyTemplates.java:395)
at org.apache.xalan.templates.ElemApplyTemplates.execute(ElemApplyTemplates.java:177)
at org.apache.xalan.transformer.TransformerImpl.executeChildTemplates(TransformerImpl.java:2336)
at org.apache.xalan.transformer.TransformerImpl.applyTemplateToNode(TransformerImpl.java:2202)
at org.apache.xalan.transformer.TransformerImpl.transformNode(TransformerImpl.java:1276)
at org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:673)
at org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:1192)
at org.apache.xalan.transformer.TransformerImpl.transform(TransformerImpl.java:1170)
at org.apache.tools.ant.taskdefs.optional.TraXLiaison.transform(TraXLiaison.java:187)
at org.apache.tools.ant.taskdefs.XSLTProcess.process(XSLTProcess.java:709)
at org.apache.tools.ant.taskdefs.XSLTProcess.execute(XSLTProcess.java:333)
at org.apache.tools.ant.taskdefs.optional.junit.AggregateTransformer.transform(AggregateTransformer.java:264)
at org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator.execute(XMLResultAggregator.java:158)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:288)
at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at _GrailsEvents_groovy$_run_closure5.doCall(_GrailsEvents_groovy:58)
at _GrailsEvents_groovy$_run_closure5.call(_GrailsEvents_groovy)
at _GrailsTest_groovy$_run_closure1.doCall(_GrailsTest_groovy:199)
at TestApp$_run_closure1.doCall(TestApp:82)
at gant.Gant$_dispatch_closure5.doCall(Gant.groovy:381)
at gant.Gant$_dispatch_closure7.doCall(Gant.groovy:415)
at gant.Gant$_dispatch_closure7.doCall(Gant.groovy)
at gant.Gant.withBuildListeners(Gant.groovy:427)
at gant.Gant.this$2$withBuildListeners(Gant.groovy)
at gant.Gant$this$2$withBuildListeners.callCurrent(Unknown Source)
at gant.Gant.dispatch(Gant.groovy:415)
at gant.Gant.this$2$dispatch(Gant.groovy)
at gant.Gant.invokeMethod(Gant.groovy)
at gant.Gant.executeTargets(Gant.groovy:590)
</code>
<br />
fix in the BuildConfig.groovy file:<br />
<br />
<br />
<code>
inherits("global") {
// uncomment to disable ehcache
// excludes 'ehcache'
excludes 'serializer'
}
</code>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-59309976922831125982012-12-13T12:23:00.001-08:002012-12-13T12:23:04.846-08:00git deleting remote branchdeleting a remote branch is rather clunky, but easy:<br />
<br />
git push origin :branch<br />
<br />
please ensure that you have the colon (:) infront of your branch name to ensure that the branch will deleted.Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-29617771290697876632012-12-13T12:03:00.002-08:002012-12-13T12:03:24.268-08:00git memorizing branches<br />
<div style="background-color: white; border: 0px; clear: both; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px; margin-bottom: 1em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">
First, you must create your branch locally</div>
<pre style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;">git checkout -b your_branch
</code></pre>
<div style="background-color: white; border: 0px; clear: both; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px; margin-bottom: 1em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">
After that, you can work locally in your branch, when you are ready to share the branch, push it. The next command push the branch to the remote repository origin and tracks it</div>
<pre style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;">git push -u origin your_branch
</code></pre>
<div style="background-color: white; border: 0px; clear: both; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px; margin-bottom: 1em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">
Teammates can reach your branch, by doing:</div>
<pre style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;">git fetch
git checkout origin/your_branch
</code></pre>
<div style="background-color: white; border: 0px; clear: both; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px; margin-bottom: 1em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">
You can continue working in the branch and pushing whenever you want without passing arguments to git push (argumentless git push will push the master to remote master, your_branch local to remote your_branch, etc...)</div>
<pre style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;">git push
</code></pre>
<div style="background-color: white; border: 0px; clear: both; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px; margin-bottom: 1em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">
Teammates can push to your branch by doing commits and then push explicitly</div>
<pre style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;">... work ...
git commit
... work ...
git commit
git push origin HEAD:refs/heads/your_branch
</code></pre>
<div style="background-color: white; border: 0px; clear: both; font-family: Arial, 'Liberation Sans', 'DejaVu Sans', sans-serif; font-size: 14px; line-height: 18px; margin-bottom: 1em; padding: 0px; vertical-align: baseline; word-wrap: break-word;">
Or tracking the branch to avoid the arguments to git push</div>
<pre style="background-color: #eeeeee; border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; font-size: 14px; line-height: 18px; margin-bottom: 10px; max-height: 600px; overflow: auto; padding: 5px; vertical-align: baseline; width: auto;"><code style="border: 0px; font-family: Consolas, Menlo, Monaco, 'Lucida Console', 'Liberation Mono', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', 'Courier New', monospace, serif; margin: 0px; padding: 0px; vertical-align: baseline;">git checkout --track -b your_branch origin/your_branch
... work ...
git commit
... work ...
git commit
git push</code></pre>
Found at <a href="http://stackoverflow.com/questions/1519006/git-how-to-create-remote-branch" target="_blank">stackoverflow</a>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-40772645640467902542012-10-10T14:37:00.001-07:002012-10-11T11:44:43.142-07:00submitting all files in a directory to qsubrecently we needed a small script to submit all files in a directory to a script, executed by qsub.
<br>
<br>
<code>
#!/bin/bash
if [ $# -lt 2 ]; then
echo Missing arguments...
echo "Use: process.sh 'input dir' 'output dir'"
exit 1
fi
if [ -d $1 ]; then
for file in `ls $1`
do
qsub -cwd -p -512 run.sh $1$file $2
done
else
echo "Missing or incorrect input directory..."
exit 1
fi
</code>
<br>
<br>
and the actual run.sh script is just a small java program, which takes our two parameters.
<br>
<br>
<code>
java -Xmx1024m -jar $HOME/data/jars/DataExtractor-0.1.jar $1 $2
</code>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-12328794792594540952012-10-03T00:18:00.000-07:002012-10-03T00:18:40.663-07:00maven-assembly-plugin - stackoverflowcurrently I'm developing several helper tools for the <a href="http://code.google.com/p/alchemy">alchemy</a> project and promtply run into the issue of a stackoverflow exception with maven.
Basically I try to assemble several archive, including all the required jars and generate the corresponding MANIFEST.MF files for a couple of main classes.
apparently with maven3, you have to set the following flag:
<code>
export MAVEN_OPTS=-Xss2m
</code>
to avoid having a stackoverflow exception. Annoying...
solution found <a href="http://stackoverflow.com/questions/7329545/maven-assembly-throwing-stackoverflowerror">here</a>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-36882627461488490902012-09-21T13:27:00.000-07:002012-09-21T13:28:10.269-07:00Maven - generate executable jar fileRecently I started to work a bit more with scala again and also finally upgraded to my new nemesis, maven3. More torture than maven2, but I could not live without it.<br />
<br />
So what I wanted todo is rather simple, build an archive containing all my classes, a generated manifest and make it executeable.<br />
<br />
10 minutes later we found the solution:<br />
<br />
<br />
<br />
<code>
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<executions>
<execution>
<id>convert-to-asci</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<archive>
<manifest>
<mainClass>edu.ucdavis.fiehnlab.alchemy.core.process.util.ConvertMZXmlToAscii</mainClass>
<packageName>edu.ucdavis.fiehnlab.alchemy.core.process.util</packageName>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<descriptors>
<descriptor>src/main/descriptor/jar.xml</descriptor>
</descriptors>
<finalName>ConvertMzXmlToASCII-${project.version}</finalName>
<appendAssemblyId>false</appendAssemblyId>
</configuration>
</execution>
<execution>
<id>alchemy-converter-zip</id>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
<configuration>
<descriptors>
<descriptor>src/main/descriptor/zip.xml</descriptor>
</descriptors>
<attach>true</attach>
<finalName>alchemy-converter-${project.version}</finalName>
</configuration>
</execution>
</executions>
</plugin>
</code>
The important part is to set the appendAssembyId to false, otherwise you get all kinds of annoying exceptions....Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-80929372843530009002012-08-23T11:20:00.002-07:002012-08-23T11:20:49.404-07:00relocating a git repositoryRecently I started creating a new project, with the amazing name 'alchemy'. Sadly it turned out that this name was already taken on google code. So to start working on it I had to create a temporary project on google code and once I managed to get my greedy fingers on the others project name (successful I might add!) ...<br />
<br />
...I had now the challenge to move my dozen or so lines of codes over to the new repository.<br />
<br />
Now I'm the first one to admit, I'm still kinda green with 'git' and have only used it in the past on 1 or 2 little project and never on anything real. So how complicate will it be to move the code and not loose your complete history of commit (all 7 of them...)<br />
<br />
Surprisingly easy it turns out, just issue the following magic commands and you are all set.<br />
<br />
<ol>
<li>git remote add alchemy https://code.google.com/p/alchemy/</li>
<li>git push alchemy master</li>
<li>git remote rm origin</li>
<li>git remote add origin https://code.google.com/p/alchemy/</li>
<li>git remote rm alchemy</li>
</ol>
<div>
Now looking at this, I'm quite sure I could had simplified this even more, but for now it got the job done.</div>
<br />
<br />
<br />Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-18889526531259579202012-06-26T02:02:00.001-07:002012-06-26T02:02:19.566-07:00VMWare and keyboards and mappingcurrently I'm doing a lot of international traveling for our little company and to install BinBase systems at different locations in the world and keep running in the issue that my keyboard mapping is completely off.<br />
<br />
For example denmark:<br />
<br />
<blockquote class="tr_bq">
We have a danish server with a danish keyboard connected, which starts a vmware player instance. This instance has an english keyboard setup and an english keyboard layout. Now we are remotely connecting to this using a TeamViewer instance, with a US keyboard.</blockquote>
What is happening all the key maps are completely random and neither english or danish. The solution<br />
<br />
<blockquote class="tr_bq">
in the configuration of the virtual machine, we need to explicitly set the hardware keyboard model to a danish model and the mapping to an english mapping to make it all work correctly using a remote session.</blockquote>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-69347459095773867122012-05-07T16:13:00.001-07:002012-05-07T16:13:25.643-07:00replacing space in filename under osx/linuxjust a little handy script to replace all the spaces in a filename with an underscore<br />
<br />
<br />
<code>
for file in *.txt
do
newname=$(echo $file | tr ' ' _)
mv "$file" $newname
done
</code><br />
<span style="font-family: monospace;"><br /></span><br />
<span style="font-family: monospace;"><br /></span><br />
<span style="font-family: monospace;">just a little handy tool</span>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-33601943537461595202012-05-07T13:43:00.001-07:002012-05-07T14:16:51.859-07:00where is my space?so since 24h I try no to discover why my harddrive is full... All I did was deleting a VMWare image and now all my space is gone, which look like there were some error during the delete option...<br />
<br />
for example:<br />
<br />
<br />
<code>
sh-3.2# du -shcx /*<br />
11G<span class="Apple-tab-span" style="white-space: pre;"> </span>/Applications<br />
8.4G<span class="Apple-tab-span" style="white-space: pre;"> </span>/Developer<br />
5.7G<span class="Apple-tab-span" style="white-space: pre;"> </span>/Library<br />
0B<span class="Apple-tab-span" style="white-space: pre;"> </span>/Network<br />
2.8G<span class="Apple-tab-span" style="white-space: pre;"> </span>/System<br />
4.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/User Guides And Information<br />
28G<span class="Apple-tab-span" style="white-space: pre;"> </span>/Users<br />
4.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/Volumes<br />
4.1M<span class="Apple-tab-span" style="white-space: pre;"> </span>/bin<br />
0B<span class="Apple-tab-span" style="white-space: pre;"> </span>/cores<br />
4.5K<span class="Apple-tab-span" style="white-space: pre;"> </span>/dev<br />
4.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/etc<br />
1.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/home<br />
0B<span class="Apple-tab-span" style="white-space: pre;"> </span>/lost+found<br />
15M<span class="Apple-tab-span" style="white-space: pre;"> </span>/mach_kernel<br />
1.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/net<br />
49M<span class="Apple-tab-span" style="white-space: pre;"> </span>/opt<br />
8.6G<span class="Apple-tab-span" style="white-space: pre;"> </span>/private<br />
2.3M<span class="Apple-tab-span" style="white-space: pre;"> </span>/sbin<br />
4.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/tmp<br />
1.4G<span class="Apple-tab-span" style="white-space: pre;"> </span>/usr<br />
4.0K<span class="Apple-tab-span" style="white-space: pre;"> </span>/var<br />
65G<span class="Apple-tab-span" style="white-space: pre;"> </span>total<br />
</code>
<br />
<div>
<br /></div>
<div>
means that the system should have a total size of 65GB.</div>
<div>
<br /></div>
<div>
Now on the other hand,</div>
<div>
<br /></div>
<div>
<div>
<code>
sh-3.2# df -h</code></div>
<code>
</code><br />
<div>
<code>Filesystem Size Used Avail Capacity Mounted on</code></div>
<code>
</code><br />
<div>
<code>/dev/disk0s2 238Gi 181Gi 57Gi 77% /</code></div>
<code>
<div>
devfs 182Ki 182Ki 0Bi 100% /dev</div>
<div>
map -hosts 0Bi 0Bi 0Bi 100% /net</div>
<div>
map auto_home 0Bi 0Bi 0Bi 100% /home</div>
</code></div>
<code>
</code><br />
<div>
<code><br /></code></div>
<code>
</code>
<br />
<div>
it says that 181GB are used, which is a difference of ca 120GB. Now where is this space?</div>
<div>
<br />
Disabling the local backups in OSX, verifying the disk and rebooting using Command-R. I was able to reapair the disk, which cleaned up the space.<br />
<br />
<br />
<code>
Filesystem Size Used Avail Capacity Mounted on<br />
/dev/disk0s2 238Gi 68Gi 170Gi 29% /<br />
devfs 181Ki 181Ki 0Bi 100% /dev<br />
map -hosts 0Bi 0Bi 0Bi 100% /net<br />
map auto_home 0Bi 0Bi 0Bi 100% /home<br />
</code>
<br />
</div>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0tag:blogger.com,1999:blog-4059581333784029364.post-26972233541761671372012-05-03T20:23:00.000-07:002012-05-03T20:25:10.874-07:00Basic Rocks linux stuff.Some basic Rocks Linux Cluster stuff...
just some stuff I need to remember and always forget:
<br />
<br />
<b>list all host interfaces:</b><br />
<code><br /></code><br />
<b></b><code>rocks list host interface</code><br />
<br />
<b>changing the mac address of a network card: </b><br />
<code><br /></code><br />
<code>
rocks set host interface mac HOSTNAME iface=eth1 mac=00:00:00:00:00:02<br />
</code>
<br />
<code><br /></code><br />
<code>please be a aware that you also have to remove the mac address definition in the following file:</code><br />
<code><br /></code><br />
<code>/etc/sysconfig/networking/devices/ifcfg-eth*</code><br />
<code><br /></code><br />
<code>you should be able to just uncomment the mac address line, without any ill effects and this will simplify this process next time.</code><br />
<code><br /></code><br />
<code>and possible in the dhcpd.conf file</code><br />
<code><br /></code><br />
<code>vim /etc/dhcpd.conf</code><br />
<code><br /></code><br />
<code>in case you modify the eth0 interface. To ensure that the dhcpd configuration still works as supposed to.</code><br />
<code><br /></code><br />
<code><br /></code><br />
<span style="font-family: monospace;"><br /></span>Gert Wohlgemuthhttp://www.blogger.com/profile/02906619451323734545noreply@blogger.com0