Installing Messenger For Multi-node Task-Spooler

Messenger Server #

On a base apt-based distro install the Task-Spooler Messenger Server:

1
2
3
4
5
sudo apt install -y libboost-dev libboost-system-dev libboost-thread-dev libboost-filesystem-dev

pushd messenger/messenger-server
sudo ./install
popd

Configure the messenger-server ms_server by creating ~/.hosts_ports with a port for listening to client messages.

1
echo "xxx $THISHOSTPORT" >> ~/.hosts_ports

Start the server with the command ms_server.

Messenger Client #

On a base apt-based distro install the Messenger Client:

1
2
3
pushd messenger/messenger-client
sudo ./install
popd

Create a file called ~/.servers_ports to configure task-spooler hosts to use with the messenger-client.

1
echo "$nickname1 $HOST1 $HOSTPORT1" >> ~/.servers_ports 

To use messenger-client ms:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
>>> ms -h
usage: ms [messenger-flags] [task-spooler-flags] [command] 
       messenger-flags: [--cd directory] [--env "FLAG1=VALUE1;FLAG2=VALUE2;..."] [--host/-H host_name] 
                        [--show_gpus] [--show_free_gpus] [--num_free_gpus] [--auto_server] 
                        [--kill] [--sync directory] [--sync_dest directory] 
                        [--excludes pattern1,pattern2,...] 
       task-spooler-flags: [-h] [--set_gpu_wait seconds] [--get_gpu_wait] [--get_label] 
                           [--count_running] [--last_queue_id] [--gpus num] [--full_cmd job_id] 
                           [-K] [-C] [-l] [-S num] [-t job_id] [-c job_id] [-p job_id] [-o job_id] 
                           [-i job_id] [-s job_id] [-r job_id] [-w job_id] [-k job_id] [-T] 
                           [-u job_id] [-U job_id1-job_id2] [-B] [-V] [-n] [-E] [-g] [-f] [-m] [-d] 
                           [-D job_id] [-L label] [-N num] 

Messenger - A multi-server plugin for Task Spooler

positional arguments:
  command               job to be run

optional arguments:
  -h, --help            show this help message and exit
  --cd directory        change directory. For e.g., ``ms --cd /home/justanhduc/Documents``.
  --env FLAG1=VALUE1:FLAG2=VALUE2:...
                        set environment variable flags.
  --host host_num, -H host_num
                        host to select. Value corresponds to the order specified in the ".servers_ports" file.
  --show_gpus           show all GPUs info.
  --show_free_gpus      show current available GPUs info.
  --num_free_gpus       show the number of available GPUs.
  --auto_server         auto-magically choose a server based on the number of available GPUs.
  --kill                kill Messenger server.
  --sync directory      whether to sync the selected working directory to a temp directory before executing the
                        command.
  --excludes pattern1,pattern2,...
                        exception patterns when moving files to server.
  --set_gpu_wait seconds
                        set time to wait before running the next GPU job (30 seconds by default)
  --get_gpu_wait        get time to wait before running the next GPU job.
  --get_label, -a       show the job label. Of the last added, if not specified.
  --count_running, -R   return the number of running jobs
  --last_queue_id, -q   show the job ID of the last added.
  --full_cmd job_id     show full command. Of the last added, if not specified.
  -K                    kill the task spooler server
  -C                    clear the list of finished jobs
  -l                    show the job list (default action)
  -S num                get/set the number of max simultaneous jobs of the server.
  -t job_id             "tail -n 10 -f" the output of the job. Last run if -1.
  -c job_id             like -t, but shows all the lines. Last run if -1.
  -p job_id             show the pid of the job. Last run if -1.
  -o job_id             show the output file. Of last job run, if -1.
  -i job_id             show job information. Of last job run, if -1.
  -s job_id             show the job state. Of the last added, if -1.
  -r job_id             remove a job. The last added, if -1.
  -w job_id             wait for a job. The last added, if -1.
  -k job_id             send SIGTERM to the job process group. The last run, if -1.
  -T                    send SIGTERM to all running job groups.
  -u job_id             put that job first. The last added, if not specified.
  -U job_id1-job_id2    swap two jobs in the queue.
  -B                    in case of full queue on the server, quit (2) instead of waiting.
  -V                    show the program version
  -n                    don't store the output of the command.
  -E                    Keep stderr apart, in a name like the output file, but adding '.e'.
  -z                    gzip the stored output (if not -n).
  -f                    don't fork into background.
  -m                    send the output by e-mail (uses sendmail).
  -d                    the job will be run after the last job ends.
  -D job_id             the job will be run after the job of given id ends.
  -L label              name this task with a label, to be distinguished on listing.
  -N num                number of slots required by the job (1 default).
  --gpus num, -G num    number of GPUs required by the job (1 default).

For more information go to the repository at https://github.com/justanhduc/messenger/.