Installing the OAR batch system
===============================

Overview
--------

There are currently 3 methods to install OAR (All of them are documented in this page) :

 - with the debian packages
 - with the rpm packages
 - with the sources


The first thing you have to know is about the OAR architecture. A common OAR
installation is composed of:

    - a **server node** which will hold all of OAR "smartness". This node will run the oar server daemon;
    - **frontend nodes** on which you will be allowed to login, then reserve some
      computing nodes (oarsub, oarstat, oarnodes, ...);
    - several **computing nodes** (a.k.a. the nodes), on which the jobs will run.
    - and optionally a **visualisation node** on which all the visualisation
      web interfaces (monika, draw-gantt, ...)  will be accessible ;

Computing nodes
---------------

Installation from the packages
______________________________

**Instructions**

*For redhat like systems*::

        # Add the oar repository
        cat <<EOF > /etc/yum.repos.d/oar.repo
        [oar]
        name=OAR Packages for Enterprise Linux 6 - \$basearch
        baseurl=http://oar-ftp.imag.fr/oar/2.5/rpm/stable/
        enabled=1
        gpgcheck=0
        EOF

        # Install OAR node
        yum install oar-node

*For the debian like systems*::
  
        # Add the OAR repository (choose the right one. See http://oar.imag.fr/repositories/)
        echo "deb http://oar-ftp.imag.fr/oar/2.5/debian squeeze main" > /etc/apt/sources.list.d/oar.list
        curl http://oar-ftp.imag.fr/oar/oarmaster.asc | sudo apt-key add -
        apt-get update

        # Install OAR node
        apt-get install oar-node

Installation from the tarball
_____________________________

**Requirements**

*For redhat like systems*::

          # Build dependencies
          yum install gcc make tar python-docutils

          # Common dependencies
          yum install Perl Perl-base openssh 
          
*For debian like system*::

          # Build dependencies
          apt-get install gcc make tar python-docutils
          
          # Common dependencies
          apt-get install perl perl-base openssh-client openssh-server 
          
**Instructions**

Get the sources::

        OAR_VERSION=2.5.2
        curl http://oar-ftp.imag.fr/oar/2.5/sources/stable/oar-${OAR_VERSION}.tgz | tar xzvf -
        cd oar-${OAR_VERSION}/

build/install/setup::

        # build
        make node-build
        # install
        make node-install
        # setup
        make node-setup


Configuration
_____________
 
oar node ssh access
~~~~~~~~~~~~~~~~~~~

You need to ensure that the oar user can access to each nodes through ssh. To
ensure that, you can just copy the ``/var/lib/oar/.ssh`` folder from the oar
server to each nodes (ensure that ``/var/lib/oar/.ssh`` has the right
permissions).

Init.d scripts
~~~~~~~~~~~~~~

If you have installed OAR from sources, you need to become root user and
install manually the {init.d,default,sysconfig} scripts present in the folders::

    $PREFIX/share/doc/oar-node/examples/scripts/{init.d,default,sysconfig}

Then you just need to use the script ``/etc/init.d/oar-node`` to start
the ssh daemon dedicated to oar-node.

Server node
-----------

Installation from the packages
______________________________

**Instructions**

*For redhat like systems*::

        # Add the epel repository (choose the right version depending on your operating system)
        rpm -i http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm
        
        # Add the oar repository
        cat <<EOF > /etc/yum.repos.d/oar.repo
        [oar]
        name=OAR Packages for Enterprise Linux 6 - \$basearch
        baseurl=http://oar-ftp.imag.fr/oar/2.5/rpm/stable/
        enabled=1
        gpgcheck=0
        EOF
        
        # Install OAR server for the PostgreSQL backend
        yum install oar-server oar-server-pgsql
        
        # or Install OAR server for the MySQL backend
        yum install oar-server oar-server-mysql

*For the debian like systems*::
  
        # Add the OAR repository (choose the right one. See http://oar.imag.fr/repositories/)
        echo "deb http://oar-ftp.imag.fr/oar/2.5/debian squeeze main" > /etc/apt/sources.list.d/oar.list
        curl http://oar-ftp.imag.fr/oar/oarmaster.asc | sudo apt-key add -
        apt-get update

        # Install OAR server for the PostgreSQL backend
        apt-get install oar-server oar-server-pgsql
        
        # or Install OAR server for the MySQL backend
        apt-get install oar-server oar-server-mysql

Installation from the tarball
_____________________________

**Requirements**

*For redhat like systems*::

          # Add the epel repository (choose the right version depending on your operating system)
          rpm -i http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm

          # Build dependencies
          yum install gcc make tar python-docutils

          # Common dependencies
          yum install Perl Perl-base openssh Perl-DBI perl-Sort-Versions
          
          # MySQL dependencies
          yum install mysql-server mysql perl-DBD-MySQL
          
          # PostgreSQL dependencies
          yum install postgresql-server postgresql perl-DBD-Pg


*For debian like system*::

          # Build dependencies
          apt-get install gcc make tar python-docutils
          
          # Common dependencies
          apt-get install perl perl-base openssh-client openssh-server libdbi-perl libsort-versions-perl
          
          # MySQL dependencies
          apt-get install mysql-server mysql-client libdbd-mysql-perl
          
          # PostgreSQL dependencies
          apt-get install postgresql-server postgresql-client libdbd-pg-perl

**Instructions**

Get the sources::

        OAR_VERSION=2.5.2
        curl http://oar-ftp.imag.fr/oar/2.5/sources/stable/oar-${OAR_VERSION}.tgz | tar xzvf -
        cd oar-${OAR_VERSION}/

Build/Install/Setup the OAR server::

        # build
        make server-build
        # install
        make server-install
        # setup
        make server-setup

Configuration
_____________

The oar database
~~~~~~~~~~~~~~~~

Define the database configuration in /etc/oar/oar.conf. You need to set the
variables ``DB_TYPE, DB_HOSTNAME, DB_PORT, DB_BASE_NAME, DB_BASE_LOGIN,
DB_BASE_PASSWD, DB_BASE_LOGIN_RO, DB_BASE_PASSWD_RO``::

        vi /etc/oar/oar.conf

Create the database and the database users::

        # General case
        oar-database --create --db-admin-user <ADMIN_USER> --db-admin-pass <ADMIN_PASS>
        
        # OR, for PostgreSQL, in case the database is installed locally
        oar-database --create --db-is-local 
        

Init.d scripts
~~~~~~~~~~~~~~


If you have installed OAR from sources, you need to become root user and
install manually the init.d/default/sysconfig scripts present in the folders::

    $PREFIX/share/doc/oar-server/examples/scripts/{init.d,default,sysconfig}
    
Then use the script ``/etc/init.d/oar-server`` to start the OAR server daemon.

Adding resources to the system
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want to **automatically** initialize your cluster then you just need to
launch ``oar_resources_init``. It will detect the resources from the nodes that
you put in a file and store right OAR commands to initialize the database with
the appropriate values for the memory and the cpuset properties. Just try...

There is also a tool to help you managing your oar resources and admission 
rules : ``oaradmin``.
Take a look at the oaradmin documentation in the administrator commands section 
for more details. You can also read this tips::
    
    http://wiki-oar.imag.fr/index.php/Customization_tips#Using_oaradmin_to_initiate_the_resources

*Otherwise:*

To add resources to your system, you can use (as root) the command oarnodesetting.
For a complete comprehension of what does this command, type man
oarnodesetting. For now, the two options you will need will be **-a** (means
add a resource) and **-h** (defines the resource hostname or ip adress).

For example, to add a computing resource on the node <NODE_IP> to OAR
installation, you can type::

        oarnodesetting -a -h <NODE_IP>

This will add a resource with <NODE_IP> as host IP address.


You also can modify resources properties with **-p** option, for example::

        oarnodesetting -r 1 -p "deploy=YES"

will allow the resource #1 to accept jobs of the type deploy.

Notes
_____

Security issues
~~~~~~~~~~~~~~~

For security reasons it is hardly **recommended** to configure a read only
account for the OAR database (like the above example).  Thus you will be able
to add this data in DB_BASE_LOGIN_RO and DB_BASE_PASSWD_RO in *oar.conf*. 

PostgreSQL : autovacuum
~~~~~~~~~~~~~~~~~~~~~~~
Be sure to activate the "autovacuum" feature in the "postgresql.conf" file (OAR
creates and deletes a lot of records and this setting cleans the postgres
database from unneeded records).

PostgreSQL : authentication
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In case you've installed a PostgreSQL database remotly, if your PostgreSQL
installation doesn't authorize the local connections by default, you need to
enable the connections to this database for the oar users. Supposing the OAR
server has the address <OAR_SERVER>, you can add the following lines in the
``pg_hba.conf``:

    # in /etc/postgresql/8.1/main/pg_hba.conf or /var/lib/pgsql/data/pg_hba.conf
        host    oar         oar_ro            <OAR_SERVER>/32    md5
        host    oar         oar               <OAR_SERVER>/32    md5

About X11 usage in OAR
~~~~~~~~~~~~~~~~~~~~~~

The easiest and scalable way to use X11 application on cluster nodes is to open
X11 ports and set the right DISPLAY environment variable by hand.  Otherwise
users can use X11 forwarding via ssh to access cluster frontal. After that you
must configure ssh server on this frontal with ::

    X11Forwarding yes
    X11UseLocalhost no

With this configuration, users can launch X11 applications after a 'oarsub -I'
on the given node or "oarsh -X node12".

Using Taktuk
~~~~~~~~~~~~

If you want to use taktuk to manage remote administration commands, you have to
install it. You can find information about taktuk from its website: 
http://taktuk.gforge.inria.fr.

**Note**: Taktuk is scalable remote command execution without the need to
install special stuffs on nodes. So it is very useful to administer a large
amount of server.

Then, you have to edit your oar configuration file and to fill in the different 
related parameters: 

  - TAKTUK_CMD (the path to the taktuk command)
  - PINGCHECKER_TAKTUK_ARG_COMMAND (the command used to check resources states)
  - SCHEDULER_NODE_MANAGER_SLEEP_CMD (command used for halting nodes)

CPUSET feature
~~~~~~~~~~~~~~

OAR uses the CPUSET features provided with the Linux kernel >= 2.6. This
enables to restrict user processes only on reserved processors and to clean
correctly the nodes after the end of the jobs.

For more information, look at the CPUSET file.

Energy saving
~~~~~~~~~~~~~

Starting with version 2.4.3, OAR provides a module responsible of advanced
management of wake-up/shut-down of nodes when they are not used.
To activate this feature, you have to:

    - provide 2 commands or scripts which will be executed on the oar server
      to shutdown (or set into standby) some nodes and to wake-up some nodes
      (configure the path of those commands into the
      ENERGY_SAVING_NODE_MANAGER_WAKE_UP_CMD and
      ENERGY_SAVING_NODE_MANAGER_SHUT_DOWN_CMD variables into oar.conf)
    - configure the "available_upto" property of all your nodes:

      - available_upto=0           : to disable the wake-up and halt      
      - available_upto=1           : to disable the wake-up (but not the halt) 
      - available_upto=2147483647  : to disable the halt (but not the wake-up) 
      - available_upto=2147483646  : to enable wake-up/halt forever            
      - available_upto=<timestamp> : to enable the halt, and the wake-up until 
        the date given by <timestamp> 

    - activate the energy saving module by setting ENERGY_SAVING_INTERNAL="yes"
      and configuring the ENERGY_* variables into oar.conf
    - configure the metascheduler time values into SCHEDULER_NODE_MANAGER_IDLE_TIME,
      SCHEDULER_NODE_MANAGER_SLEEP_TIME and SCHEDULER_NODE_MANAGER_WAKEUP_TIME
      variables of the oar.conf file.
    - restart the oar server (you should see an "Almighty" process more).

You need to restart OAR each time you change an ENERGY_* variable.
More informations are available inside the oar.conf file itself. For more
details about the mechanism, take a look at the "Hulot" module documentation.


Disabling SELinux
~~~~~~~~~~~~~~~~~

On some distributions, SELinux is enabled by default. There is currently no OAR
support for SELinux. So, you need to disable SELinux, if enabled.


Intel cpuset id issue
~~~~~~~~~~~~~~~~~~~~~
The cpuset ids on an intel platform are not persistent across reboot. So you need
to update the cpuset ids in the resource database at startup for each computing
node. You can do this by using the ``/etc/oar/update_cpuset_id.sh`` script. The
following page give more informations on how configuring it:

    http://wiki-oar.imag.fr/index.php/Configuration_tips#Start.2Fstop_of_nodes_using_ssh_keys


Other issues
~~~~~~~~~~~~

You can take a look at the "Customizaion tips" on the OAR Wiki:
    
    http://wiki-oar.imag.fr/index.php/Customization_tips


Frontend nodes
--------------

Installation from the packages
______________________________

**Instructions**

*For redhat like systems*::

        # Add the epel repository (choose the right version depending on your operating system)
        rpm -i http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-5.noarch.rpm

        # Add the oar repository
        cat <<EOF > /etc/yum.repos.d/oar.repo
        [oar]
        name=OAR Packages for Enterprise Linux 6 - \$basearch
        baseurl=http://oar-ftp.imag.fr/oar/2.5/rpm/stable/
        enabled=1
        gpgcheck=0
        EOF

        # Install OAR user for the PostgreSQL backend
        yum install oar-user oar-user-pgsql
        
        # or Install OAR user for the MySQL backend
        yum install oar-user oar-user-mysql

*For the debian like systems*::
  
        # Add the OAR repository (choose the right one. See http://oar.imag.fr/repositories/)
        echo "deb http://oar-ftp.imag.fr/oar/2.5/debian squeeze main" > /etc/apt/sources.list.d/oar.list
        curl http://oar-ftp.imag.fr/oar/oarmaster.asc | sudo apt-key add -
        apt-get update

        # Install OAR server for the PostgreSQL backend
        apt-get install oar-user oar-user-pgsql
        
        # or Install OAR server for the MySQL backend
        apt-get install oar-user oar-user-mysql


Installation from the tarball
_____________________________

**Requirements**

*For redhat like systems*::

          # Build dependencies
          yum install gcc make tar python-docutils

          # Common dependencies
          yum install Perl Perl-base openssh Perl-DBI
          
          # MySQL dependencies
          yum install mysql perl-DBD-MySQL
          
          # PostgreSQL dependencies
          yum install postgresql perl-DBD-Pg


*For debian like system*::

          # Build dependencies
          apt-get install gcc make tar python-docutils
          
          # Common dependencies
          apt-get install perl perl-base openssh-client openssh-server libdbi-perl
          
          # MySQL dependencies
          apt-get install mysql-client libdbd-mysql-perl
          
          # PostgreSQL dependencies
          apt-get install postgresql-client libdbd-pg-perl

**Instructions**

Get the sources::

        OAR_VERSION=2.5.2
        curl http://oar-ftp.imag.fr/oar/2.5/sources/stable/oar-${OAR_VERSION}.tgz | tar xzvf -
        cd oar-${OAR_VERSION}/

Build/Install/setup::

        # build
        make user-build
        # install
        make user-install
        # setup
        make user-setup


Configuration
_____________

Coherent configuration files between server node and user nodes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You need to have a coherent oar configuration between the server node and the
user nodes. So you can just copy the /etc/oar directory from to server node to
the user nodes.


OAR RESTful API Installation
____________________________

Since the version 2.5.2, OAR offers an API for users and admins interactions. This api
must be installed on a frontend node (with the user module installed).

From the packaging
~~~~~~~~~~~~~~~~~~

*For redhat like systems*::

        # Add the oar repository
        cat <<EOF > /etc/yum.repos.d/oar.repo
        [oar]
        name=OAR Packages for Enterprise Linux 6 - \$basearch
        baseurl=http://oar-ftp.imag.fr/oar/2.5/rpm/stable/
        enabled=1
        gpgcheck=0
        EOF

        # Install apache FastCGI module (optional but highly recommended)
        FIXME:

        # Install OAR Restful api
        yum install oar-restful-api

*For the debian like systems*::
  
        # Add the OAR repository (choose the right one. See http://oar.imag.fr/repositories/)
        echo "deb http://oar-ftp.imag.fr/oar/2.5/debian squeeze main" > /etc/apt/sources.list.d/oar.list
        curl http://oar-ftp.imag.fr/oar/oarmaster.asc | sudo apt-key add -
        apt-get update
 
        # Install apache FastCGI module (optional but highly recommended)
        apt-get install libapache2-mod-fastcgi 

        # Install OAR Restful api
        apt-get install oar-restful-api

From the sources
~~~~~~~~~~~~~~~~

**Requirements**:

*For redhat like systems*::

          # Build dependencies
          yum install gcc make tar python-docutils

          # Common dependencies
          yum install perl perl-base perl-DBI perl-CGI perl-JSON perl-YAML perl-libwww-perl httpd

          # FastCGI dependency (optional but highly recommended)
          FIXME:
          
          # MySQL dependencies
          yum install mysql perl-DBD-MySQL
          
          # PostgreSQL dependencies
          yum install postgresql perl-DBD-Pg


*For debian like system*::

          # Build dependencies
          apt-get install gcc make tar python-docutils
          
          # Common dependencies
          apt-get install perl perl-base libdbi-perl libjson-perl libyaml-perl libwww-perl httpd-cgi libcgi-fast-perl 
          
          # FastCGI dependency (optional but highly recommended)
          apt-get install libapache2-mod-fastcgi

          # MySQL dependencies
          apt-get install mysql-server mysql-client libdbd-mysql-perl
          
          # PostgreSQL dependencies
          apt-get install postgresql-server postgresql-client libdbd-pg-perl

**Instructions**

Get the sources::

        OAR_VERSION=2.5.2
        curl http://oar-ftp.imag.fr/oar/2.5/sources/stable/oar-${OAR_VERSION}.tgz | tar xzvf -
        cd oar-${OAR_VERSION}/

build/install/setup::

        # build
        make api-build
        # install
        make api-install
        # setup
        make api-setup


Configuration
~~~~~~~~~~~~~

*Configuring OAR*

    For the moment, the API needs the user tools to be installed on the same host
    ('``make user-install``' or oar-user package). A suitable ``/etc/oar/oar.conf`` should
    be present. For the API to work, you should have the oarstat/oarnodes/oarsub
    commands to work (on the same host you installed the API)

*Configuring Apache*

    The api provides a default configuration file (``/etc/oar/apache-api.conf``) that
    is using a identd user identification enabled only from localhost.  Edit the
    ``/etc/oar/apache-api.conf`` file and customize it to reflect the authentication
    mechanism you want to use. For ident, you may have to install a "identd" daemon
    on your distrib. The steps may be:

        - Install and run an identd daemon on your server (like *pidentd*).
        - Activate the ident auth mechanism into apache (``a2enmod ident``).
        - Activate the headers apache module (``a2enmod headers``).
        - Activate the rewrite apache module (``a2enmod rewrite``).
        - Customize apache-api.conf to allow the hosts you trust for ident.

*YAML, JSON, XML*

    You need at least one of the YAML or JSON perl module to be installed on the host running the API.

*Test*

    You may test the API with a simple wget::

        wget -O - http://localhost/oarapi/resources.html

    It should give you the list of resources in the yaml format but enclosed in an
    html page.  To test if the authentication works, you need to post a new job.
    See the example.txt file that gives you example queries with a ruby rest
    client.


Visualization node
------------------

Description
___________

There are two different tools. One, named Monika which displays the current
cluster state with all active and waiting jobs. The other, named drawgantt
which displays node occupation in a lapse of time. These tools are CGI scripts
and generate HTML pages.

Installation from the packages
______________________________

**Instructions**

*For redhat like systems*::

        # Add the oar repository
        cat <<EOF > /etc/yum.repos.d/oar.repo
        [oar]
        name=OAR Packages for Enterprise Linux 6 - \$basearch
        baseurl=http://oar-ftp.imag.fr/oar/2.5/rpm/stable/
        enabled=1
        gpgcheck=0
        EOF

        yum install oar-web-status

*For the debian like systems*::
  
        # Add the OAR repository (choose the right one. See http://oar.imag.fr/repositories/)
        echo "deb http://oar-ftp.imag.fr/oar/2.5/debian squeeze main" > /etc/apt/sources.list.d/oar.list
        curl http://oar-ftp.imag.fr/oar/oarmaster.asc | sudo apt-key add -
        apt-get update

        apt-get install oar-web-status


Installation from the tarball
_____________________________

**Requirements**:

*For redhat like systems*::

          # Build dependencies
          yum install gcc make tar python-docutils

          # Common dependencies
          yum install perl perl-base perl-DBI ruby-GD ruby-DBI perl-Tie-IxHash perl-Sort-Naturally perl-AppConfig
          
          # MySQL dependencies
          yum install mysql perl-DBD-MySQL ruby-mysql
          
          # PostgreSQL dependencies
          yum install postgresql perl-DBD-Pg ruby-pg


*For debian like system*::

          # Build dependencies
          apt-get install gcc make tar python-docutils
          
          # Common dependencies
          apt-get install perl perl-base ruby libgd-ruby1.8 libdbi-perl libtie-ixhash-perl libappconfig-perl libsort-naturally-perl
          
          # MySQL dependencies
          apt-get install libdbd-mysql-perl libdbd-mysql-ruby
          
          # PostgreSQL dependencies
          apt-get install libdbd-pg-perl libdbd-pg-ruby

**Instructions**

Get the sources::

        OAR_VERSION=2.5.2
        curl http://oar-ftp.imag.fr/oar/2.5/sources/stable/oar-${OAR_VERSION}.tgz | tar xzvf -
        cd oar-${OAR_VERSION}/

build/install/setup::

        # build
        make monika-build draw-gantt-build www-conf-build
        # install
        make monika-install draw-gantt-install www-conf-install
        # setup
        make monika-setup draw-gantt-setup www-conf-setup

Configuration
_____________

**Drawgantt configuration**

 - Edit ``/etc/oar/drawgantt.conf`` to fit your configuration.

**Monika configuration**

 - Edit ``/etc/oar/monika.conf`` to fit your configuration.
 
**httpd configuration**

 - You need to edit ``/etc/oar/apache.conf`` to fit your needs and verify that you
   http server configured.


Further informations
--------------------

For further information, please check the documentation section on the OAR
website http://oar.imag.fr/.

