Collaborators PERI collaborates both with application teams and with other performance engineering researchers. This page lists the collaborators organized by their role within a particular application engagement or performance tuning effort. At present, there are 3 main collaboration areas, two focusing on direct engagement with computational science projects: 1) Tiger Teams 2) Application code team liaisons and one on community standards and tool interoperability: 3) PERI-DB, performance data interoperability. Applications engagement Tiger Teams Tiger Teams are focused efforts applied to high priority application codes with well documented performance goals or needs. They are composed of performance optimization experts who are typically members of the PERI collaboration, although other performance engineering experts also participate. While the life time of a Tiger Team is indeterminate, they usually last from six to twelve months. At present, PERI has two tiger teams, working with the following codes. S3D S3D solves the full compressible Navier-Stokes equations that describe the conservation of mass, momentum, and energy, and laws of gas behavior, while simultaneously tracking the evolution of reactive species on a rectangular mesh. It uses the message passing interface (MPI) to efficiently distribute the calculation among parallel processors, and is built on a hierarchical, modular structure. The S3D Tiger Team was formed to assist the S3D application team in meeting its Joule requirements. Specific concerns of the application team focused on single node performance as well as how the code would scale on the combined Cray XT3/XT4 system at Oak Ridge National Laboratory (ORNL). The Tiger Team has actively worked with the application team to identify several optimization opportunities and to provide mechanisms to exploit them. Specific changes include development of a simple transformation tool to optimize certain Fortran statements; the modified code is being added to the application team's source base. We have also targeted study of potential optimizations of the exp intrinsic function that will likely be adopted by the application team. Scaling studies on the ORNL system identified load balance issues arising from the XT3 nodes having lower memory bandwidth than the XT4 nodes. The Tiger Team designed a potential solution to this issue as well although the application team has decided not to adopt it since the ORNL machine will only have XT4 nodes in the near future. While the application team has successfully met its Joule requirements, the Tiger Team continues to work with them in order to prepare for full scale runs on the anticipated petascale platform at ORNL. * Tiger Team Lead: Bronis de Supinski (LLNL) * Application Team Contact: Jackie Chen (SNL) * DOE Liaison: Kenneth Roche (ORNL) * PERI Team Members: * David Bailey (LBNL) * Bronis de Supinski (LLNL) * Mike Fagan, John Mellor-Crummey (Rice) * Allan Snavely, Nick Wright (SDSC) * Affiliate Team Members: * Sameer Shende, Alan Morris, Allen Maloney, Kevin Huck (UOregon/ParaTools) * Jeff Larkin (Cray/ORNL) * Application Team Participants * Jackie Chen, David Lignell (SNL) GTC: Gyrokinetic Toroidal Code The Gyrokinetic Toroidal Code (GTC) has been developed to study plasma microturbulence in fusion devices. The GTC Tiger Team was formed to assist with performance optimization and scalability issues related to achieving the Joule goals for GTC of 1) running 50 percent faster on the model problem, and 2) running twice as long in simulation time for a fixed wallclock time. GTC solves the gyro-averaged Vlasov-Poisson system of equations using the particle-in-cell (PIC) approach. In GTC, the main bottleneck is the charge deposition, or scatter operation, and this is also true for most particle codes. The scatter algorithm in GTC is more complex since one is dealing with fast gyrating particles for which motion is described by charged rings being tracked by their guiding center. Hand-tuning techniques such as common subexpression elimination, code movement, loop unrolling, and cache blocking were used to improve performance of the charge deposition routine by around 10 percent. These changes have been incorporated into the production version of the code. New physics in the GTC-S version of the code allows the input of real experimental profiles and magnetic equilibria, introducing the important effects of shaped cross-section physics. The implementation of this new capability required extensive changes to the code, and the goal is now to bring GTC-S to the performance level of the original GTC and beyond, so as to be able to do global simulations of plasmas in the ITER fusion reactor, which will be coming on-line in about five years. Tiger Team participants have used cache modeling techniques to develop further optimizations to the charge deposition and other key GTC-S routines. These code transformations have been sent to the developers and are awaiting their evaluation. Tiger Team participants are also investigating scalability and load imbalance issues with the new GTC-P version of the code that partitions the poloidal plane into radial shells. * Tiger Team Lead: Shirley Moore (UTK) * Application Team Contact: Stephane Ethier (PPPL) * DOE Liaison: Kenneth Roche (ORNL) * PERI Team Members * Haihang You (UTK) * John Mellor-Crummey, Gabriel Marin, Guohua Jin (Rice) * Hongzhang Shan (LBNL) * Affiliate Team Members * Kevin Huck (UOregon) * Eduardo D'Azevedo (ORNL) * Application Team Participants * Stephane Ethier, Weixing Wang, Wei-li Lee (PPPL) * Scott Klasky (ORNL) Application Liaisons In addition to the focused Tiger Team activity, PERI designates members of the PERI collaboration as liaisons with SciDAC science application teams that desire additional help in the performance engineering of their codes. The liaison activity is further broken down into "active" liaisons, in which it is clear how PERI can contribute and in which there is close collaboration between PERI and the application team, and "passive" liaisons, in which PERI provides advice and tracks the progress of the application teams. The status of a PERI liaison activity can change between active and passive at any time, and such changes are expected as the needs of the application projects evolve. As distinct from the Tiger Teams, the liaison activity is a long term engagement with the application teams. PERI has active liaison relationships with the following application teams currently: 1) Advanced Methods for Electronic Structures (http://www.scidac.gov/matchem/petachem.html) Principal Investigator: George Fann Application Team Contact: Rebecca Hartman-Baker (ORNL) PERI Liaison: Philip Roth (ORNL) Other PERI Personnel: Codes: MADNESS 2) Center for Plasma Edge Simulation (http://www.scidac.gov/FES/FES_CPES.html) Principal Investigator: C-S Chang (NYU) Application Team Contact: Julian Cummings (CalTech) PERI Liaison: Patrick Worley (ORNL) Other PERI Personnel: John Mellor-Crummey (Rice) Other Affiliates: Kevin Huck (UOregon) Codes: XGC-1, XGC-0, GTC, M3D, NIMROD, ELITE, DEGAS-2 3) Community Petascale Project for Accelerator Science and Simulation (ComPASS) (http://www.scidac.gov/physics/COMPASS.html) Principal Investigator: Panagiotis Spentzouris (Fermi National Accelerator Laboratory) Application Team Contact: John Cary (Tech-X) PERI Liaison: Boyana Norris (ANL) Other PERI Personnel: ??? Codes: VORPAL 4) Hierarchical Petascale Simulation Framework for Stress and Corrosion Cracking (http://www.scidac.gov/matchem/crack.html) Principal Investigator: Priya Vashishta (USC) Application Team Contact: Aiichiro Nakano (USC) PERI Liaison: Mary Hall (USC) Other PERI Personnel: Jacqueline Chame (USC), Chun Chen (USC), John Mellor-Crummey (Rice) Codes: Quantum chemistry 5) Linear Scale Electronic Structure Calculations for Nanostructures (http://www.nccs.gov/leadership-science/materials/linear-scale-electronic-structure-calculations-for-nanostructures/) Principal Investigator: Lin-Wang Wang (LBNL) Application Team Contact: Lin-Wang Wang (LBNL) PERI Liaison: David Bailey (LBNL) Other PERI Personnel: Dan Gunter (LBNL), Hongzhang Shan (LBNL) Codes: LS3DF 6) Modeling Multiscale-Multiphase-Multicomponent Subsurface Reactive Flows using Advanced Computing (http://www.scidac.gov/groundwater/gwflow.html) Principal Investigator: Peter Lichtner (LANL) Application Team Contact: Glenn Hammond (PNNL), Richard Mills (ORNL) PERI Liaison: G. Mahinthakumar (NCSU) Other PERI Personnel: Sarat Sreepathi (NCSU), Vamsi Sreepathi (NCSU) Codes: PFLOTRAN PERI has also assigned passive liaisons to track the performance needs of the following projects: 1) Hybrid Numerical Methods for Multiscale Simulations of Subsurface Biogeochemical Processes (http://www.scidac.gov/groundwater/gwscaling.html) PERI Liaison: G. Mahinthakumar (NCSU) 2) Framework Application for Core-Edge Transport Simulations (FACETS) (http://www.scidac.gov/fusion/fullscale.html) PERI Liaison: Patrick Worley (ORNL) 3) Multidimensional Simulations of Core-Collapse Supernovas (http://www.nccs.gov/leadership-science/astrophysics/multi-dimensional-simulations-of-core-collapse-supernovae/) PERI Liaison: Robert Fowler (RENCI) 4) National Computational Infrastructure for Lattice Gauge Theory (http://www.scidac.gov/physics/quarks.html) PERI Liaison: Ying Zhang (RENCI) 5) A Scalable and Extensible Earth System Model for Climate Change Science (http://www.scidac.gov/climate/earth.html) PERI Liaison: Patrick Worley (ORNL) 6) Simulations of Turbulent Flows with Strong Shocks and Density Variations (http://www.scidac.gov/physics/shock.html) PERI Liaison: Bronis de Supinski (LLNL)