rocks linux cluster - adding a new parallel environment

by default rocks ships with a couple of environments, which execute stuff on different nodes. But sometimes you just want to have a node all to your self and take over all it's slots.

Todo this you can just create a new environment and which gives you a defined number of cpus for a specified job.

1. create a file which describes the paralell environment like this

pe_name threaded
slots 999
user_lists NONE
xuser_lists NONE
start_proc_args /bin/true
stop_proc_args /bin/true
allocation_rule $pe_slots
control_slaves FALSE
job_is_first_task TRUE
urgency_slots min
accounting_summary FALSE


2. register this on the head node

qconf -Ap file.txt


3. add it to the list of available envionments

qconf -mq all.q
pe_list make mpich mpi orte threaded


4. test it with qlogin

qlogin -pe threaded 4

playing around with threads

currently I got back to my hobby and play a bit with multithreading to tune an algorithm.

So the first step was to write a smpl class to test if the threading api works on my system and what is better than to calculate primes?



import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class Test {

public static void main(String args[]) throws InterruptedException {

ExecutorService service = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
for (int i = 0; i < 500000000; i++) { final int postion = i; service.execute(new Runnable() { @Override public void run() { try { int i = postion; int i1 = (int) Math.ceil(Math.sqrt(i)); boolean isPrimeNumber = false; while (i1 > 1) {

if ((i != i1) && (i % i1 == 0)) {
isPrimeNumber = false;
break;
}
else if (!isPrimeNumber) {
isPrimeNumber = true;
}

--i1;
}

if (isPrimeNumber) {
System.out.println(Thread.currentThread().getName() + " - prime " + i);
}

}
catch (Exception e) {
e.printStackTrace();
}
}
});
}
service.shutdown();
service.awaitTermination(4000, TimeUnit.DAYS);
}
}




and the output is nicely:

pool-1-thread-2 - prime 469237
pool-1-thread-2 - prime 469241
pool-1-thread-2 - prime 469253
pool-1-thread-4 - prime 466553
pool-1-thread-4 - prime 469267
pool-1-thread-4 - prime 469279
pool-1-thread-2 - prime 469283
pool-1-thread-3 - prime 467869
pool-1-thread-3 - prime 469303
pool-1-thread-2 - prime 469321

while all 4 CPU's are at 100% use.

translation, java executor api seems to work quite well.

Now time to tune the binbase algorithm...