Direkt zum Inhalt | Direkt zur Navigation

Personal tools

Sektionen

outcome

last modified Jul 10, 2012 09:53 AM
This contains a summary on the results and the decisions made at the I/O workshop held in Hamburg on Feb 27 and Feb 28 2012.

Decisions made

MPI-Met:

  • Provide benchmark toy simulating halo exchanges and computation
  • Support ICHEC in implementing “cache/concentrator node”-concept into CDI
  • In cooperation with IPSL write unified I/O server meta data API (needs to be finished until 08.2012)
  • Put CDI into RAPS benchmark (to be done until 11.2012-01.2013)

DKRZ:

  • Write a summary that describes and compares all presented IO-Server solutions
  • Create collaboration website for information exchange on I/O issues between developers
  • Moderate I/O collaboration website
  • Organise follow-on workshop (12.2012 or 02.2013)

CSCS:

  • Moderate I/O collaboration website
  • Write NetCDF(?3/4?) interface for in-memory files (?name?tempFS?)
  • Support I/O-Server developers (MPI, IPSL, ?Met Office?, ?) in using ?tempFS?

ICHEC:

  • Implementation for “cache/concentrator node”-concept to overcome problems machines that have little memory per node. This is to be implemented into CDI and XIOS, with the support of MPI-Met and IPSL.
  • Integrate either CDI or XIOS into EC-Earth

IPSL:

  • Support ICHEC in implementing “cache/concentrator node”-concept into XIOS
  • In cooperation with MPI-Met write unified I/O server meta data API (needs to be finished until 08.2012)
  • Might put XIOS into RAPS benchmark (to be done until 11.2012-01.2013)

Unified I/O server meta data API:

  • Unified API for CDI and XIOS to provide all required meta data
  • To be used by ICHEC

Knowledge exchange

Programming language:

An object oriented programming (OOP) approach seems to be the programming paradigm of choice

  • Comparison of languages used
    • Fortran 90
      • F90 is more common among climate scientists that C or C++
      • F90 is wide spread in the existing infrastructure (Met Office)
      • OOP is possible to some extent
      • Compilers seem to have problems with this language
    • C
      • C is used in related tools (MPI-Met)
      • Allows low-level programming (can be advantage and disadvantage)
      • Highly portable
      • Good supported by compilers
      • OOP is possible to some extent
    • C++
      • Support for stl and boost
      • Is OOP-language
      • Might have portability issues

Parallel writing to disc (for example using pNetCDF or NetCDF4):

  • Out-of-the-box performance is very bad
  • Hard to tune
    • Highly system dependent (software and hardware)
    • Few information available -> we will try to build a platform for information exchange on this and similar issues
    • MPI needs to be tunes for this...hardly documented

Issues in the I/O-server development:

  • Being general/robust to cope with all different kinds of configurations
  • Integrate into existing software architecture
  • Parallel writing to disc (for example using pNetCDF or NetCDF4):
    • Out-of-the-box performance is very bad
    • Hard to tune
      • Highly system dependent (software and hardware)
      • Few information available -> we will try to build a platform for information exchange on this and similar issues
      • MPI needs to be tunes for this...hardly documented

isend/irecv vs. get/put:

  • isend/irecv:
    • high control about what the MPI-implementation is actually doing
  • Get/put:
    • Get/put have the ability to be completely one-sided
    • Can be risky due to hidden rendezvous points
    • Hard to tune due to bad documentation
    • Hard to make it work on multiple platforms

PIO (NCAR):

  • Plan to implement I/O-server solution into PIO -> hopefully can benefit from experiences gain in the presented developments