View on GitHub

ecp-data-vis-sdk

DOE Exascale Computing Project Data and Visualization SDK

ECP Data and Visualization SDK

The Data and Visualization SDK is an integration effort for the collection of I/O, compression, visualization, and analysis software developed under the DOE’s Exascale Computing Program (ECP). Software developed under ECP targets using Spack, an HPC targeted source-based package manager, for installation on HPC platforms. The primary product of this project is the ecp-data-vis-sdk Spack meta-package which allows the set of SDK member packages to be built together in a way that enables optimal features for ECP target environments as well as interoperable features provided by other packages within the SDK.

Included Projects
Integration Status

Included Projects

The following projects are included as part of the Data and Visualization SDK:

I/O and Data Management

ADIOS2: An adaptable framework for HPC I/O supporting files, in situ, and in transit data movement. ADIOS2 also generates an HDF5 VOL Adapter.
Darshan: An HPC I/O characterization tool.
HDF5: A data model, library, and file format for storing and managing data.
- CUDA support (where applicable) is provided by the HDF5 GPUDirect Storage VFD. Refer to the documentation for more information on enabling GDS VFD for your application. The SDK will install the package and populate HDF5_PLUGIN_PATH (only).
- HDF5 VOL plugins will be installed with HDF5_PLUGIN_PATH populated:
  - ADIOS2: when ecp-data-vis-sdk +adios2 +hdf5 is installed, the ADIOS2 VOL is installed (by the adios2 package).
  - hdf5-vol-cache, hdf5-vol-async, and hdf5-vol-log are installed** when ecp-data-vis-sdk +hdf5 ^hdf5@1.14: is installed. (6)
  - hdf5-vol-daos is a VOL adapter used for direct interfacing with the Distributed Asynchronous Object Storage (DAOS) system, bypassing both MPI I/O and POSIX for efficient and scalable I/O. This VOL adapter is currently being tested on Pre-Aurora systems with DAOS support.
PNetCDF: A high-performance parallel I/O library for accessing Unidata’s NetCDF, files in classic formats, specifically the formats of CDF-1, 2, and 5.
UnifyFS: A filesystem for burst buffers.
VeloC: A multi-level checkpoint-restart runtime for HPC supercomputing infrastructures and large-scale data centers.

Visualization and Analysis

Ascent: An open source many-core capable lightweight in situ visualization and analysis infrastructure for multi-physics HPC simulations.
Cinema: An image-based approach to extreme scale in situ visualization and analysis.
ParaView: An open-source, multi-platform data analysis and visualization application.
SENSEI: An interface for scalable in situ analysis and visualization of simulation data.
VisIt: An open source, interactive, scalable, visualization, animation and analysis tool.
VTK-m: A toolkit if scientific visualization algorithms for emerging processor architectures.

Compression

SZ: An error-bounded lossy data compressor for floating-point and integer datasets.
cuSZ: A CUDA based implementation of the SZ lossy compressor.
ZFP: An open-source library for compressed floating-point arrays that support high throughput read and write random access.

Integration Status

General Spack Support

Project

CPU

CUDA

ROCm

SYCL (*)

ADIOS2

(1)

Darshan

HDF5

PNetCDF

UnifyFS

VeloC

(13)

(14)

Ascent

(2)

(3)

(22)

Cinema

ParaView

(7)

SENSEI

(10)

(11)

VisIt

(16)

VTK-m

SZ

cuSZ

ZFP

(20)

Legend

Available

Not Available

Failing

ECP DAV SDK Support

Project

CPU

CUDA

ROCm

SYCL (*)

ADIOS2

(1)

Darshan

(25)

HDF5

PNetCDF

UnifyFS

VeloC

(13)

(14)

Ascent

(2)

(3)

(22)

Cinema

ParaView

(7)

SENSEI

(12)

(11) (12)

VisIt

([15]

(16)

(7)

VTK-m

SZ

cuSZ

ZFP

(20)

Legend

Verified	In Progress	Not Applicable	Failing

Extreme-scale Scientific Software Stack (E4S)

E4S Deployment

This table is an overview of the status of the different deployments of the Data and Vis SDK via E4S. Some items may be marked as “In Progress” but have fully completed specific stacks on those systems. The following tables will fill in more details on the facility deployments.

Project

Desktop

Docker

Frontier

Perlmutter

Pre-Aurora

Smoke Test

ADIOS2

Darshan

HDF5

(6)

PNetCDF

UnifyFS

VeloC

Ascent

(21)

(4)

Cinema

(5)

ParaView

(8) (9)

SENSEI

(12)

VisIt

VTK-m

SZ

cuSZ

ZFP

Legend

Verified	In Progress	Not Applicable	Failing

Frontier

One of the primary challenges on Frontier is handling the compiler wrappers for the CCE and AMD compilers, in particular how they wrap MPI and HIP.

Info	GCC Toolchain	CCE Toolchain	AMD Toolchain
Version	v11.2	v15.0.0	v5.3.0
Cray MPICH	v8.1.23	v8.1.23	v8.1.23
ROCm	v5.3.0	v5.3.0	v5.3.0

Project

GCC

GCC + ROCm

CCE

CCE + ROCm

AMD

AMD + ROCm

ADIOS2

(1)

Darshan

HDF5

PNetCDF

UnifyFS

(23)

VeloC

(14)

Ascent

(3)

(21)

(3)

(21)

(3)

Cinema

(25)

ParaView

(8)

SENSEI

(12)

VisIt

(16)

VTK-m

(18)

(18) (19)

SZ

ZFP

(20)

Legend

Verified	In Progress	Not Applicable	Failing

Perlmutter Toolchains (CUDA)

Info	GCC Toolchain	NVHPC Toolchain
Version	v11.2	v22.4
Cray MPICH	v8.2.3	v8.2.3
CUDA	v11.3	v11.3

Project

GCC

GCC + CUDA

NVHPC + CUDA

ADIOS2

Darshan

HDF5

PNetCDF

UnifyFS

VeloC

(13)

Ascent

(2)

Cinema

ParaView

SENSEI

VisIt

(16)

VTK-m

SZ

cuSZ

ZFP

Legend

Verified	In Progress	Not Applicable	Failing

Aurora/Sunspot Toolchains

Info	GCC	OneAPI
Version	v11.2.0	v2023.0.0

Project

GCC

OneAPI

SYCL (*)

ADIOS2

(**)

Darshan

(**)

HDF5

(**)

PNetCDF

(**)

UnifyFS

(**)

VeloC

(**)

Ascent

(**)

Cinema

(**)

(5)

ParaView

(8) (9)

(7)

SENSEI

(10) (12)

(11) (12)

VisIt

(16)

VTK-m

(**)

SZ

(**)

ZFP

(**)

Legend

Verified	In Progress	Not Applicable	Failing

Notes

* SYCL extensions are implemented and tested using OneAPI.

** Due to the similarity between the toolchains on Aurora and Sunspot, the results for the Aurora system are currently marked as verified as builds on Aurora progress. As issues are found on Aurora, these values will be updated to reflect the target system status.

ADIOS2

1. ROCm/HIP support is not expected in near term release.

Ascent

2. Blocked by build errors in RAJA package.

3. The spack recipe does not have support for ROCm. It is in under development in the Alpine Spack fork.

4. The way MPI is set up on Perlmutter conflicts with Ascent’s spack recipe and CMake. Fixes for this are being developed.

21. Using the CCE toolchain has issues locating MPI with CMake.

22. Ascent currently does not have any known plans for explicit SYCL support.

Cinema

5. Failure to build a number of Python modules with OneAPI.

25. Failure to build working python modules for some of Cinemas dependency modules.

Darshan

24. Darshan with HDF5 integration enabled does not support use cases that do not link HDF5 in the same runtime environment.

ParaView

7. ParaView and VisIt do not support building SYCL kernels for VTKm filters in released versions.

8. OSPRay support has been disabled due to build errors.

9, Failure to build a number of Python modules with OneAPI.

SENSEI

10, SENSEI is developing native OSPRay rendering support.

11. SENSEI is developing Kokkos support for ROCm, CUDA, and SYCL interop.

12. SENSEI is developing updates to support ParaView v5.11 and Catalyst 2.

UnifyFS

23, UnifyFS has a link error when using the Cray wrappers for the Cray and AMD toolchains.

VeloC

13. CUDA support for VeloC is under development.

14. Kokkos/ROCm support for VeloC is experimental in kokkos-resilience

VisIt

15. VisIt utilizes a VTK version locked to a Python that is not compatible with the Python requirements of PyH5, the HDF5 python interface used by Cinema.

16. VTK-m enabled GPU support is available VisIt, but is not officially tested as part of the Data and Vis SDK due to (15) only recently being resolved.

VTK-m

18. Spack ROCm does not provide OpenMP correclty for VTKm HIP modules.

19. Frontier requires additional workarounds in the Spack recipe for VTKm that are specific to the Cray wrappers on Frontier. patch

ZFP

20. ROCm support is under development.