Upgrade

28/11, 2014

  • This is a significant upgrade and some binaries are not compatible anymore, thus before opening support case please recompile your code.


  • Older compilers will have to use the following compiling recipe:

    module swap cray-mpich cray-mpich2
    module swap cray-libsci cray-libsci/12.2.0
    module load craype-barcelona


  • Only current version of PGI will work without using the above mentioned recipe.


  • xt prefixed modules are deprecated and replaced by cray prefixed modules.
    ex:
    xtpe-interlagos

    - replaced by
    craype-interlagos


  • Cray CCE is not working currently because of missing license. We are in contact with the vendor.
  • 01/11, 2013

    Old fimm work folder is mounted on new fimm login node(/old_work) only.

    We suggest that you only move necessary files from old work to your new work folder, to avoid creating huge load on login node.

    old_work is NOT mounted on compute nodes.

    Regards
    16/02, 2012

    Update 19.03 18:40: Upgrade is finished, machine is open for SSH access.

    Update, Monday 19th: We are finalizing the upgrade, the machine is up and we expect to allow logins later today. When logging in for the first time, please remember to recompile ALL your applications and libraries to be compatible with the new system.


    Hexagon will get a major hardware and software upgrade in the first week of March.

    The current schedule is for the upgrade to start on March 9th 2012 at 8:00 (a delay of 1 week from initial announcement) and to last for about 1 week.

    NOTE: A reservation is set in the queue system. Thus, jobs must have a walltime set so that they can finish before the maintenance to be allowed to start.

    The upgraded hexagon will have the following specs:
    * Cray XE6m-200
    * 204.9 TFlops peak performance
    * 22272 cores
    * AMD Opteron 6276 (2.3GHz "Interlagos")
    * 1392 CPUs (sockets)
    * 696 nodes
    * 32 cores per node
    * 32GB RAM per node (1GB/core)
    * New interconnect: Cray Gemini
    * New topology: 2.5D Torus
    * OS: Cray Linux Environment, CLE 4.0 (Based on Novell Linux SLES11sp1)

    Although the user experience will be very much the same after the upgrade (with just newer versions of familiar software, and a faster machine) please observe the following critical point:

    IMPORTANT! All applications MUST be recompiled to be compatible with the new and upgraded hexagon.

    You can expect that the software list that is available via "modules" to be short right after the upgrade for then to grow during the next few weeks. Please be patient while we recompile and install the necessary applications and libraries.

    We remind you that you have to move all files not related to your current runs out from the /work file system. Please see our previous email for details.

    IMPORTANT! The old /work will be available on new hexagon only up to April 9th. On April 11th it will be completely DESTROYED!
    It is therefore very important that you participate in moving data out of hexagon or transfer it to the new file-system. The old /work will be mounted back after a reformat and used as secondary storage.

    You can follow the upgrade at our Syslog:
    http://computing.uni.no/syslog

    Please contact support-uib at notur.no if you have any questions regarding the upgrade.
    14/01, 2010

    Hexagon will have a scheduled maintenance on Monday Feb. 8th from 13:00 to approx. Tuesday late evening Feb. 9th.
    The following operations will be performed during maintenance slot:
    * Base software upgrade from CLE2.1 to CLE2.2
    * Optimization of /work filesystem metadata
    * Hardware maintenance
    The queue have a reservation in place such that only jobs that can complete (according to asked for walltime) before the maintenance will start.
    This note will be updated when we have more information.

    NB! Users are encouraged to recompile all binaries after performed maintenance. This is due to a new CLE release.

    Update: We will try to start maintenance at 12:30 instead of 13:00 since only few jobs are running.

    Update: 09/02/2010 21:25 Maintenance finished. Hexagon is online.
    Please remember to recompile all your programs!
    Before contacting support please be sure that you have recompiled your code, this will speed up your case processing.
    31/08, 2009

    Several libraries and have been updated on hexagon.
    For maximum performance and stability users are encouraged to log out and in again and then recompile their programs and libraries.

    Updates:
    xt-asyncpe 3.2
    xt-libsci 10.3.8
    petsc 3.0.0.5
    libfast_mv 1.0.5
    mpt 3.4.1
    java-jdk 1.6.0_15
    PGI compiler 9.0.2
    Intel compiler 11.1.046

    Removed
    xt-libsci 10.3.5
    petsc 3.0.0.3
    libfast_mv 1.0.3
    mpt 3.1.2
    24/07, 2009

    Several libraries have been updated on hexagon.
    For maximum performance and stability users are encouraged to log out and in again and then recompile their programs and libraries.

    Updates:

    MPI:
    xt-mpt 3.3.0 -> 3.4.0

    Libs/math:
    xt-libsci 10.3.6 -> 10.3.7
    hdf5 1.8.2.3 -> 1.8.3.0
    netcdf_hdf5parallell 4.0.0.3 -> 4.0.1.0
    netcdf 4.0.0.3 -> 4.0.1.0
    petsc 3.0.0.3 -> 3.0.0.4
    acml 4.2.0 -> 4.3.0

    Compiler:
    xt-asyncpe (wrapper) 3.0 -> 3.1
    pgi 8.0.6 -> 9.0.1

    NOTES:

    xt-mpt
    Bug fixes related to SHMEM as well as Intel compiler.

    hdf5/netcdf
    New features and bugfixes.

    See here for more information:
    http://www.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.8.3-RELEASE.txt

    http://www.unidata.ucar.edu/software/netcdf/release-notes-4.0.1.html

    petsc
    Feature update: CASK 1.1, plus bugfixes.

    CASK 1.1 includes new sparse matrix vector multiplication for
    transposed matrices improves the performance by 5-30% depending
    on nonzero pattern of the matrix.

    Support for the Intel compiler.

    ACML
    New feature release. Note that this release does not support the pathscale compiler.

    See ACML site for more information:
    http://developer.amd.com/cpu/Libraries/acml/features/pages/default.aspx

    PGI
    New major release.

    See PGI site for more information:
    http://www.pgroup.com/doc/pgiwsrn901.pdf