Super Massive Computations in Theoretical Physics

11 Feb 2015, 14:00 → 13 Feb 2015, 15:50 Europe/Rome

Trento

This workshop is organized in the framework of the SUMA-INFN project. SUMA is led by those people of the Istituto Nazionale di Fisica Nucleare (INFN) theoretical community who, although active in different research areas, share the need for massive computational resources. Examples of topics that are of interest for this community are lattice QCD, dynamical systems, classical and ab-initio simulations of bio-systems. Several existing INFN projects have already produced significant results and remarkable technological progresses upon which the new generation of high performance computer systems will be based. SUMA plans to back up the many INFN initiatives aiming at providing efficient tools for the present and future needs of computational physics. The workshop, whose main goal is to promote exchanges of ideas and foster collaborations, will be structured in four sections centred on the following four themes: Lattice Quantum Chromodynamics, Biophysics, Fluid Dynamics and Machine Development. Each one of the four topics will be introduced by a general talk aiming at providing an overview of the state-of-the-art of the field for what concerns scientific achievements and computational aspects. This workshop is open to all the researchers interested in high-performance and large-scale computations in theoretical physics. Registration is free, but compulsory for organizational reasons.

Wednesday, 11 February

15:00 → 16:00 Atomistic Simulation of Single Molecule Experiments: Molecular Machines and a Dynasome.

Proteins are biological nanomachines which operate at many length and time scales. We combined single molecule, x-ray crystallographic, and cryo-EM data with atomistic simulations to elucidate how these functions are performed at the molecular level. Examples include the mechanics of energy conversion in F-ATP synthase and tRNA translocation within the ribosome. We will show that tRNA translocation between A, P, and E sites is rate limiting, and identified dominant interactions. We also show that the so-called L1 stalk actively drives tRNA translocation, and that 'polygamic' interactions dominate the intersubunit interface, thus explaining the detailed interaction free energy balance required to maintain both controlled affinity and fast translation We will further demonstrate how atomistic simulations enable one to mimic, one-to-one, single molecule FRET distance measurements, and thereby to markedly enhance their resolution and accuracy. We will, finally, take a more global view on the 'universe' of protein dynamics motion patterns and demonstrate that a systematic coverage of this 'dynasome' allows one to predict protein function.

Speaker: Helmut Grubmüller

16:00 → 16:30 Coffe break

16:30 → 16:50 Variational Molecular Dynamics

The Variational Molecular Dynamics (VMD) approach is a framework aimed at the efficient computation of the most probable folding pathways connecting denatured protein configurations with the native state. Its high computational efficiency enables the simulation of very slow (up to hours) transitions on large systems (hundreds of residues), with realistic atomistic force fields. The VMD approach can also be used to predict the most probable pathways for a conformational transition between given states of macromolecules. In my talk I will outline the main features of DRP and give an account of our latest results. I will first discuss the folding of a natively knotted protein. I will then talk about the conformational transition of a serpin comprised of 373 residues between its active (metastable) state and its latent state. I will also show how our method allowed us to give some explanation for the working of a pharmaceutical used in connection with the PAI-1 serpin.

Speaker: Silvio Beccara

16:50 → 17:10 Zn induced structural aggregation patterns of β-amyloid peptides by first-principle simulations and XAS measurements

We show in this work that in the presence of Zn ions a peculiar structural aggregation pattern of β-amyloid peptides in which metal ions are sequentially coordinated to either three or four histidines of nearby peptides is favored. To stabilize this configuration a deprotonated imidazole ring from one of the histidines forms a bridge connecting two adjacent Zn ions. Though present in zeolite imidazolate frameworks, remarkably in biological compounds this peculiar Zn–imidazolate–Zn topology is only found in enzymes belonging to the Cu,Zn-superoxide dismutase family in the form of an imidazolate bridging Cu and Zn. The results we present are obtained by combining X-ray absorption spectroscopy experimental data with detailed first-principle molecular dynamics simulations.

Speaker: Velia Minicozzi

Slides Minicozzi_SUMA.pdf

17:10 → 17:30 Ab initio simulations of X-ray Absorption Spectroscopy spectra. The case of Cu(II) ions in water.

The possibilities offered by high performance parallel computing forfirst principle simulations of systems composed by a large number of atoms are opening up the way to new approaches not only for ab initio molecular dynamics simulations, but also for providing improved models for the interpretation of experimental data. In this context, we are developing a strategy to compute the electron density of biologically relevant systems to simulate ab initio X-ray Absorption Spectroscopy (XAS) spectra. The XAS signal is originated by the scattering of the photoelectron with the surrounding atoms and therefore strongly depends on the potential seen by it. The structure of the XAS spectrum is usually computed starting from an approximated potential. Here we develop a strategy to evaluate the potential directly from the knowledge of the exact electron density calculated from first principles. After producing a number of atomic configurations (via classical or first principle MD), our approach consists in calculating the electron density in correspondence of each of these configurations from which the photoelectron potential and the simulated XAS spectrum are computed. Under the ergodicity hypothesis, the average over the computed spectra over many configurations will represent the experimental spectrum as the latter results from the sum of XAS signals from the many thermodynamically accessible configurations of the measured system. We report in this talk the results of an ab initio simulation of the XAS spectra of a system consisting of Cu(II) ions in water (see Figure 1(a)). This system is a good starting point to pave the way to the much more complicated case of computing the XAS spectrum of realistic and biologically relevant systems, such as biomolecules in water in complex with transition metals recently studied in [1]. For the system in Figure 1(a), consisting of one Cu(II) ion dissolved in 29 water molecules, we selected a number of equilibrated configurations along a classical MD trajectory. On these we performed single-point (i.e. fixed nuclei positions) electron density calculations with the help of the QuantumESPRESSO suite [2]. The XAS spectra were finally computed making use of the Xspectra code [3]. In Figure 1(b) we show a comparison between the experimental XAS data of a Cu(II) sulfate water solution (blue line), acquired at the ESRF BM30B beamline, and the theoretical spectrum (red line)resulting from averaging over the simulated spectra of eight system configurations taken along a classical MD trajectory. Our preliminary analysis shows that the ab initio strategy for simulating XAS spectra we have described is quite effective in providing a good description of XAS experimental data.

Speaker: Francesco Stellato (R)

Slides Stellato-SUMA_Trento_11-02-2015.pdf

17:30 → 18:30 First-principles simulations at the nanoscale (and towards the exascale) with Quantum ESPRESSO.

Speaker: Paolo Giannozzi

Slides qe.pdf

18:30 → 19:30 Meeting of the SUMA group.

Thursday, 12 February

09:30 → 10:30 Computing the visible universe via large-scale simulations of QCD.

Quantum Chromodynamics (QCD) is the fundamental theory of the strong interactions that describe, among other phenomena, the binding of nucleons to form nuclei, nuclear fission, supernovae and stellar evolution. Most of the visible matter in the universe is generated via the strong interactions. Understanding the complex phenomena that are due to the strong force is particularly demanding not only for experiments but also for theoretical approaches. The formulation of QCD on a 4-dimensional euclidean lattice, known as lattice QCD, provides a unique approach for studying the strong interactions. Lattice QCD is particularly demanding on the computational power needed for performing these calculations. We will review the status lattice QCD calculations and give future perspectives.

Speaker: Constantia Alexandrou.

Slides Trento_2015.pdf

10:30 → 11:00 Coffee Break

11:00 → 11:20 Chiral symmetry breaking in QCD Lite

We compute the spectral density of the (Hermitean) Dirac operator in Quantum Chromodynamics with two light degenerate quarks near the origin. We use CLS lattices generated with two flavours of O(a)-improved Wilson fermions corresponding to pseudoscalar meson masses down to 190 MeV, and with spacings in the range 0.05--0.08 fm. Thanks to the coverage of parameter space, we can extrapolate our data to the chiral and continuum limits with confidence. The results show that the spectral density at the origin is non-zero because the low modes of the Dirac operator do condense as expected in the Banks--Casher mechanism. Within errors, the spectral density turns out to be a constant function up to eigenvalues of approximatively 80 MeV. Its value agrees with the one extracted from the Gell--Mann--Oakes--Renner relation.

Speaker: Leonardo Giusti (M)

Slides lgiusti_SUMA.pdf

11:20 → 11:40 Confinement and extreme conditions in lattice QCD

We discuss some results obtained in studying the QCD vacuum structure at zero temperature and the QCD phase diagram at finite temperature and baryon density.

Speaker: Leonardo Cosmai (BA)

Slides LCosmai-SUMA-Feb2015.pdf

11:40 → 12:00 Properties of strongly interacting matter in extreme conditions.

We discuss recent results obtained by lattice QCD simulations and regarding the properties of strongly interacting matter at finite temperature and density, or in the presence of external background fields.

Speaker: Francesco Negro (PI)

Slides Negro_SUMA_Trento.pdf

12:00 → 12:20 Non-perturbative renormalization of the energy-momentum tensor in SU(3) Yang-Mills theory.

We present a strategy for a non-perturbative determination of the finite renormalization constants of the energy-momentum tensorin the SU(3) Yang-Mills theory. The computation is performed by imposing on the lattice suitable Ward Identites in a finite box in presence of shifted boundary conditions. We show accurate numerical data for values of the bare coupling g0^2 ranging for 0 to 1.

Speaker: Michele Pepe (MIB)

Slides PEPE.pdf

12:20 → 12:40 Adopting modern hardware for lattice QCD calculations.

The calculation of observables in lattice quantum chromodynamics (QCD) requires to solve many linear equation systems with matrices ofrank up to several millions. We discuss the adoption of modern highly parallel computer hardware, in particular GPUs and Intel MICs, to accelerate this task.

Speakers: Mario Schröck, Silvano Simula (ROMA3)

Slides SUMA15_Silvano+Mario.pdf

12:40 → 13:00 A perturbative study of the Schrödinger Functional in Lattice QCD.

Strong interactions are (so far) correctly described by Quantum Chromodinamics (QCD). This theory explains accurately phenomena from the scale of hadronic physics at low energies to the production of jets in high energy collisions and quark-gluon plasmas. Due to the property of asymptotic freedom, the theory can be studied at high energies by means of perturbation theory. At low energies, a non-perturbative formulation of QCD is required. A widely used possibility is to formulate the theory on a discrete space-time lattice, which yields numerically tractable the study of strong interactions. Here we will describe some results in the context of lattice perturbation theory, which in many occasions is the only way of having analytic control on the theory before embarking on large scale numerical calculations.

Speaker: Pol Vilaseca Mainar

13:00 → 15:00 lunch

15:00 → 16:00 Numerical simulations of fluids at high and at low Reynolds numbers.

In this talk we will review few recent results relative to the direct numerical simulation of turbulent and laminar fluids. We will discuss some of the open challenges in particular in relation to the transport of particulate matter.

Speaker: Federico Toschi (INFN)

16:00 → 16:30 Coffee break

16:30 → 16:50 Magnetic field amplification and the search for Magneto-Rotational-Instability in bar-mode unstable Neutron stars.

Thanks to the recent advances in high performance computing, it is opening the possibility to tackle new problems on the dynamics of magnetic fields inside neutron stars where very-high-resolution three-dimensional simulations in full General Relativity are needed. We present results on the possible roles that magnetic instabilities may have the evolution of neutron star during matter unstable phases. Our main goal was to find and follow the possible onset and growth of magneto-rotational instability (MRI).

Speaker: Roberto De Pietri (PR)

Slides SUMA_trento.pdf

16:50 → 17:10 The Einstein toolkit on SUMA systems.

Recent progress on Numerical Relativity and General Relativistic Magneto Hydrodynamics (GRMHD) are opening new windows on our understanding of possible sources of Gravitational Waves (GWs) that would be detected in the coming years by the INFN gravitational wave observatory VIRGO. Unfortunately the power and memory required for cutting edge simulations with the desired accuracy require ~10PFlops and order of 1TBytes of RAM. The SUMA project aims to study which one of the possible upcoming architecture will be suited to obtain in the near future the desired computational power that would made this simulations achievable. I will discuss scaling tests on the systems available to the SUMA collaboration of one of the leading tool for Numerical Relativity simulations, the Einstein toolkit. This tool implements multithread (OpenMP), multiprocessor (MPI) and mixed MPI+OpenMP parallelization techniques. These results show how does the Einstein toolkit perform on the current generation of supercomputers.

Speaker: Michele Brambilla (INFN)

Slides Brambilla_ETonSUMAmachines.pdf

17:10 → 17:30 Distributed Polychronous Spiking Neural Net (DPSNN) application code for evaluation of off-the-shelf and custom systems dedicated to neural network simulations

In the framework of the EURETILE European project, a natively distributed application has been developed, as a representative of plastic spiking neural network simulators. The DPSNN-STDP (Distributed Simulation of Polychronous Spiking Neural Network with synaptic Spike-Timing Dependent Plasticity) will be used in the context of the CORTICONIC European FET project to produce simulations of cortical slices for the comparison with in-vivo experiments. The application will be also used to drive the development of future parallel/distributed computing systems dedicated to the simulation of plastic spiking networks. The DPSNN-STDP simulator has been designed to generate identical spiking behaviours and network topologies over a varying number of processing nodes, simplifying the quantitative study of scalability on both commodity and custom architectures. Moreover, it can be easily interfaced with standard and custom software/hardware communication interfaces. Being natively distributed and parallel, it should not pose major obstacles against distribution and parallelization on several platforms. During 2015, the DPSNN-STDP application will be further enhanced to enable the description of larger networks, more complex connectomes, and finalize it for the application to biological simulations. The development of the DPSNN-STDP simulator has been funded by the European FET FP7 projects CORTICONIC (grant 600806) and EURETILE (grant 247846), in cooperation with the SUMA project.

Speaker: Elena Pastorelli

Slides 20150212-DPSNN-SUMA.PDF

17:30 → 17:50 NaNet: a network interface card family for GPU-based real-time systems.

The NaNet project aims to deliver a low-latency, high-throughput data transport mechanism for GPU-based real-time systems. The goal is a FPGA-based PCIe Network Interface Card (NIC) design with GPUDirect P2P/RDMA capability featuring a configurable and extensible set of data transmission channels. An ad-hoc network stack protocol offload engine and a data stream processing stage combined with the GPUDirect capability make NaNet suitable for real-time GPU contexts. The design currently supports both standard - GbE (1000BASE-T) and 10GbE (10Base-R) - and custom - 34 Gbps APElink and 2.5 Gbps deterministic latency KM3link - channels, but NaNet architecture modularity allows for a straightforward inclusion of other link technologies. A description of the NaNet architecture and its performances is given, showing two use cases for it: the GPU-based low-level triggerfor the RICH detectorin the NA62 experiment at CERN and the on-/off-shore data transport system for KM3NeT-IT underwater neutrino telescope. This work has been funded by the European FET FP7 project EURETILE (grant 247846).

Speaker: Andrea Biagioni (ROMA1)

Slides NaNet_Suma_Workshop.pdf

17:50 → 18:10 Multi-dimensional Torus networks for current and next generation HPC systems: APEnet+ status and perspectives.

The Apenet family interconnect cards have been out for more than a decade, pursuing the legacy of the APE custom massively parallel computing machines and bringing few concepts (3D-torus network, high bandwidth, low latency) to commodity clusters. Over the years we have been added key features to our custom network, like Remote-DMA programming paradigm or, lately, NVidia peer-to-peer capability for tightly coupling GPUs with our network card. The most strenuous efforts were made on keeping our communication interface IP cores (both for the host side - on PCIe interface, and on remote link side) up to date with current state-of-the-art technology. Apenet+ is the actual production-class interconnect card equipping Quong, the 16-node CPU-GPU hybrid system deployed. Exploring and developing innovative ideas in various research fields, such as fault tolerance, fast address translation or embedded processors, has brought interesting results and significant impact on our system performances. A comprehensive view of these topics will be given in this talk, along with the results achieved at this point. Perspectives of future work will also be given, covering topics such as integrating next-generation ARM System-on-Chips, collective communication optimizations and other planned hardware and software enhancements. This work has been funded by the European FET FP7 project EURETILE (grant 247846) and by the MIUR project SUMA

Speaker: Roberto Ammendola (ROMA2)

Slides ammendola_suma15.pdf

Friday, 13 February

09:30 → 10:30 The European Supercomputer Projects DEEP and DEEP-ER.

Speakers: Norbert Eicker (Julich Supercomputer Centre), Thomas Lippert

Slides DEEP_and_DEEP-ER.pdf

10:30 → 11:00 Coffee break

11:00 → 11:20 Theocluster ZEFIRO.

Attualmente il theocluster denominato 'Zefiro' (cluster per il calcolo di fisica teorica) di nuova installazione finanziato da GR IV e dal progetto SUMA è costituito da 32 macchine ciascuna con 512 Gb di ram e 4 processori (ciascuno da 16 core); conta in totale 2048 core di calcolo AMD Opteron 6380 (2,5GHz) e collegamento Infiniband QDR gestiti da uno switch Mellanox IS5100 a 108 porte. L'utilizzo del cluster previsto è quello del calcolo locale. Gli accessi sono regolati tramite lo scheduler IBM LSF nella versione 9. La sottomissione dei jobs viene fatta tramite code specifiche e indirizzate sul cluster dove i 64 core totali dei singoli nodi sono raggruppati in 2 jobslot. È inotre presente una macchina dedicata alla compilazione dei job cui fa capo una coda specifica e che non è appartenente all'architettura delle 32 macchine di zefiro; è dotata di 2 processori ciascuno con 16 core per un totale quindi pari a 32 core e 256 Gb diram; i core di questa macchina sono statiraggruppati in bunch da 1 core considerato quindi un jobslot. Avremo disponibili un totale di 32 JOBSLOT. Tutti i nodi di calcolo hanno installato SUSE Linux Enterprise Server (SLES), come distribuzione per il funzionamento di base; mentre per l'utilizzo del software di calcolo utilizato per la fisica teorica e per le altre applicazioni di interesse, è prevista una distribuzione Scientific Linux v.6 (SL6) su un'apposita directory di Debug del proprio job su alcune code distinte.

Speaker: Giuseppe Caruso (P)

Slides Presentazione_WorshopSUMA2015.pdf

11:20 → 11:40 Portability, efficiency and maintainability: the case of OpenACC

An increasing number of HPC systems rely on heterogeneous node architectures combining traditional multicore CPUs with power efficient accelerators. Writing efficient applications for these systems could be cumbersome today, since porting may require code rewriting using new programming languages, such CUDA or OpenCL, threatening maintainability, stability and correctness. Several innovative programming environments try to tackle this problem; among them OpenACC offers an high-level approach based on directives: porting applications to heterogeneous architectures ``simply'' requires to annotate existing – C, C++ or Fortran – codes with specific "pragma" clauses to identify regions to offload and run on accelerators. This approach guarantee high portability of codes since support for different accelerators relies on compilers, however one has to carefully assess the relative costs of portability versus computing efficiency. In this presentation we address precisely this issue, using as a test-bench a Lattice Boltzmann code. We describe our experience in implementing and optimizing a multi-GPU Lattice Boltzmann code using OpenACC and OpenMPI, focusing also on overlapping communications and computation to make the code scaling on a large number of accelerators.

Speaker: Dr Enrico Calore (INFN, Sezione di Ferrara)

Slides calore.pdf

11:40 → 13:00 Round Table and Conclusions