Author Archives: bernholdtde

SC16 BOF: Software Engineering for Computational Science and Engineering on Supercomputers

Software Engineering for Computational Science and Engineering on Supercomputers

Thursday 17 November 2016, 10:30am-12:00pm, Room 155-A

[SC16 program page]

Description: Software engineering (SWE) for computational science and engineering (CSE) is challenging, with more sophisticated, higher fidelity simulation of larger and more complex problems involving larger data volumes, more domains, and more researchers. Targeting high-end computers multiplies these challenges. We invest a great deal in creating these codes, but we rarely talk about that experience. Instead we focus on the results. Our goal is to raise awareness of SWE for CSE on supercomputers as a major challenge and to begin the development of an international “community of practice” to continue these important discussions outside of annual workshops and other “traditional” venues.

SC16 Tutorial: Testing of HPC Scientific Software

Testing of HPC Scientific Software

Monday 14 November 2016, 1:30pm-5:00pm, Room 255-D

[SC16 program page] [Promotional video]

Description: Testing at various granularities has recently acquired an urgency in high-performance scientific computing communities because of the need for refactoring caused by changing platform architectures. Projects that need to refactor are often lacking in the necessary expertise and resources to acquire such expertise. Testing is also critical for producing credible results and for code maintenance. The IDEAS ( scientific software productivity project aims toward increasing software productivity and sustainability with participants from many projects that define the state of practice in software engineering in the HPC community. We offer a tutorial that distills the combined knowledge of IDEAS team members in the area of scientific software testing. The tutorial will be useful to all projects that recognize the importance of testing in general and will provide tremendous help to projects in need of refactoring their software in particular.


SC16 Workshop: Fourth International Workshop on Software Engineering for High Performance Computing in Computational Science & Engineering (SE-HPCCSE 2016)

Fourth International Workshop on Software Engineering for High Performance Computing in Computational Science & Engineering (SE-HPCCSE 2016)

Sunday 13 November 2016, 2:000pm-5:00pm, Room 251-E

[SC16 program page]

Description: Researchers are increasingly using HPC, including GPGPUs and computing clusters, for computational science and engineering (CSE) applications. Unfortunately, when developing HPC software, developers must solve reliability, availability, and maintainability problems in extreme scales, understand domain specific constraints, deal with uncertainties inherent in scientific exploration, and develop algorithms that use computing resources efficiently. Software engineering (SE) researchers have developed tools and practices to support development tasks, including: validation and verification, design, requirements management, and maintenance. HPC CSE software requires appropriately tailored SE tools/methods. The SE-HPCCSE workshop addresses this need by bringing together members of the SE and HPC CSE communities to share perspectives, present findings from research and practice, and generate an agenda to improve tools and practices for developing HPC CSE software. This workshop builds on the success of the 2013-2015 editions.


Webinar: Software Practices in Computational Science Communities – an Overview

Computational Infrastructure for Geodynamics 2016-2017 webinar series

Software Practices in Computational Science Communities – an Overview

Thursday 13 October 2016, 2:00pm-3:00pm PT

[webinar web page]

Anshu Dubey, Argonne National Laboratory

Scientific code developers typically adopt software processes derived from the mainstream (non-scientific) community when continuing without them becomes impractical. However, many software best practices need modification and/or customization, partly because the codes are used for research and exploration, and partly because of the combined funding and sociological challenges. This webinar will describe the lifecycle of scientific software and important ways in which it differs from other software development.  We will provide a compilation of software engineering best practices that have generally been found to be useful by science communities, and how they are evolving as the needs of their communities grow.

CFP: SE4Science’16 at ICSE’16 – Austin, TX – May 16, 2016


This workshop is concerned with identifying and understanding the unique aspects of software engineering (SE) for the development of scientific software. Specifically, we are interested in:

  • Scientific software applications that solve complex software- or data-intensive research problems. These applications range from large parallel models/simulations of the physical world using HPC systems to smaller scale simulations developed by a single scientist or engineer on a desktop machine or a small cluster.
  • Applications that support scientific research and experiments at scale. Such applications include, but are not limited to, systems for managing and/or manipulating large amounts of data and systems that provide infrastructure for scientific or engineering applications such as libraries or HPC/Cloud software.
  • The process for building, reusing, and publishing software and data used in scientific experiments or engineering innovations. Among others, these process include agile approaches, open source/open data issues, testing scientific software, and managing software or data repositories for publishing goals.

The development of scientific software is significantly different than the development of business information systems, from which many of the SE best practices, tools and techniques have been drawn. Moreover, most conference and journal venues focus either on SE or Scientific Software, but rarely on the intersection of the two domains. Specifically, within the scientific community, there are few places to publish research related to the special SE challenges faced by scientists engaged in software- or data-intensive experiments. The goal of this workshop is to provide a venue for researchers to interact and to support the building of a research agenda to deal with the complex software development issues present in science. Furthermore, the discussion among the researchers will be invaluable in identifying those aspects of SE that should be considered for education programs. This workshop will build upon results from previous Software Engineering for Science workshops ( Similar to the format of the previous workshops, in addition to presentation and discussion of the accepted papers, significant time during the 2016 workshop will be devoted to the continuation of discussions from previous workshops and to general open discussion.

Special foci of this workshop edition
In addition to the traditional goals of this workshop series, the 2016 edition will have two special foci for which we specifically solicit papers.

  • Quality Assurance for scientific software development, and considerations of any particular techniques to improve the adoption of such process.
  • Experience reports (including positive, negative, and neutral) of applying software engineering practices to the development of scientific software. It is as important to understand which SE practices do not work in science contexts as those which do.

Submission Instructions
We encourage participation from members of the SE and scientific software communities. Papers of at most 7 pages are solicited to address issues including but not limited to: 1) Case studies of software development processes used in scientific applications; 2) Design patterns and software architectures for scientific software; 3) SE metrics and tool support for scientific applications; 4) Issues in publishing or reusing scientific research software and data; 5) The use of empirical studies to better understand the environment, tools, languages, and processes used in research application development and how they might be improved; 6) V&V techniques specifically targeted for the scientific domain; 7) SE education for scientific developers. Additionally, practical experience reports are welcome and encouraged, including negative and neutral experiences. In order to increase participation, we will also accept shorter (~3-4 page) position papers. A position paper should include POSITION PAPER in the title.

Accepted papers will appear in the ICSE Companion Proceedings in the ACM Digital Library. Selected papers will also be invited to submit to the Software Engineering track of Computing in Science & Engineering.

Important Dates

  • Submission Deadline: January 22, 2016
  • Notifiation: February 19, 2016
  • Camera-ready Deadline: February 26, 2016

Please observe the following:
1. Full Papers should be at most 7 pages formatted according to the ICSE 2016 guidelines
2. Submit your paper in PDF: EasyChair

For more information, contact Jeffrey Carver ( or Neil Chue Hong (


Computational Science & Engineering Software Sustainability and Productivity Challenges (CSESSP Challenges)

An inter-agency workshop sponsored by the Networking and Information Technology Research and Development (NITRD) / Software Design and Productivity (SDP) Coordinating Group (CG).

October 15th-16th 2015, Washington DC, USA

Extended Deadline: 26th June 2015 (any time of day)

Call for Participation

Software has emerged as a critical technology in all sectors including defense, health systems, banking, transportation, energy, science and engineering, and manufacturing. However, software lifecycle cost is increasingly becoming the dominant fraction of the total information technology investment. Additionally, software activities have been a major factor in large-scale project delays, failures, cost overruns, and productivity bottlenecks. There is a general consensus that current approaches produce software that is difficult to maintain, upgrade, and scale, especially in the face of rapidly changing machine architecture and new system requirements.

The CSESSP Challenges workshop will identify the unique issues around software productivity and sustainability faced by the NITRD computational science and engineering (CSE) communities, bringing together experts from academia, industry, government, and national laboratories. The workshop will focus on general issues and challenges of software systems sustainability and productivity with the aim of making software a first-class issue in the specification, design, cost and lifecycles management of science and engineering infrastructures. In this context, the workshop will discuss technical issues that impact software sustainability, such as software requirements engineering, high-productivity software engineering, reproducibility, software maintenance processes, and scalable, reusable, and portable software system architectures, to name a few.

We invite short (1 or 2 page) papers from computational science software developers, software system engineers, computer system engineers and architects, software managers, experts in related scientific software fields and government agency representatives. In the context of improving CSE software sustainability and productivity, these papers should identify and describe challenges, new approaches and strategies, best practices or experiences in related fields, and non-technical issues such as science policies and economic factors. These papers will be used by the program committee to structure the workshop, provide background material, and contribute to selecting attendees.

Areas of interest for the CSESSP Challenges Workshop include but are not limited to the following:

  • Characterization of the emerging sustainability and productivity crisis from laptops to extreme-scale systems.
    • Software as a critical national infrastructure and as a virtual facility for computational science and engineering.
  • New approaches to scientific software that significantly improve sustainability and productivity, including leveraging software engineering research:
    • Strategies and technologies for minimizing the impacts of rapidly changing architectures and languages on large software systems
    • Facilitating software performance, portability, legacy software factoring, interoperability and reusability.
  • Understanding the economics of CSE software and opportunities and new models of partnership between academia, independent software vendors, the manufacturing industry and government:
    • Understanding licensing and governance issues.
    • Opportunities for economic sustainment of software tools.
  • Encouraging and developing software ecosystems to support and advance sustained scientific innovation and discovery:
    • Supporting and leveraging changes in computing technologies, science methods and algorithms.
    • Understanding how software tools can contribute to sustainability & productivity.
  • Supporting and encouraging CSE user and developer communities; education and building a CSE software workforce.


Submissions of up to two pages should be formatted to be easily readable and submitted as a PDF document using Easychair at

Deadline for Submission:

Extended to 26 June 2015 (any time of day)

Travel Support:

Some limited travel support may be available, please check the workshop web page.

Important Dates:

  • June 26, 2015 – Paper submission deadline (extended)
  • July 24 2015 – Workshop attendees invited
  • September 11, 2015 – Deadline for acceptances
  • October 15-16, 2015 – CSESSP Challenges Workshop


  • Gabrielle Allen, University of Illinois Urbana-Champaign, USA
  • Michael Heroux, Sandia National Laboratories

Steering Committee:

  • Dai Hyun Kim, Office of the Secretary of Defense (OSD)
  • Daniel S. Katz, National Science Foundation
  • James Kirby, Naval Research Laboratory
  • Sol Greenspan, National Science Foundation
  • Steven Drager, Air Force Research Laboratory
  • T. Ndousse-Fetter, Office of Science, US Department of Energy
  • Vivien Bonazzi, NIH/OD/Senior Advisor for Data Science Technologies ADDs Team
  • Walid Keyrouz, National Institute of Standards and Technology

Program Committee:

  • Jeff Carver, U of Alabama
  • Tom Clune, NASA
  • Merle Giles, U of Illinois
  • Lois McInnes, Argonne
  • Manish Parashar, Rutgers University
  • Doug Post, DOD
  • Roldan Pozo, NIST
  • Ethan Coon, LANL

Reading Resources/References

List of workshop reading resources: publications, white papers, workshop reports, websites.

Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2.1@SciPy)

“Sustainable scientific software in Action!”

To be held with SciPy2015, July 10, 1:30 pm to 4:30 pm, AT&T Executive Education and Conference Center at the University of Texas, room 106, Austin, TX.


Group discussions and lightning talks on community management and sustainability practices.  WSSSPE2.1 aims to end with concrete takeaways and “one sentence summaries” of participant experiences to fuel further scientific software sustainability.

To propose a lightning talk about sustainability of scientific software, email Matt Turk <>


  • Matthew Turk, University of Illinois
  • Daniel S. Katz, University of Chicago & Argonne National Laboratory

Third International Workshop on Software Engineering for High Performance Computing in Computational Science and Engineering

Call for Papers

New challenges in computational science & engineering (CSE) continue to push the boundaries of available computing resources. There is a demand to utilize high performance computing (HPC), including GPGPUs and computing clusters, for computational science & engineering (CSE) applications.

However developing HPC software is not an easy task. Developers must solve reliability, availability, and maintainability problems in extreme scales, understand domain specific constraints, deal with uncertainties inherent in scientific exploration, and develop algorithms that use computing resources efficiently.

Software engineering (SE) researchers have developed tools and practices to support various development tasks, including: requirements management, design, validation + verification, deployment, and maintenance. However software development for HPC historically attracted little attention from the SE community. Paradoxically, the HPC CSE community has increasingly been adopting SE techniques and tools. Indeed, the development of CSE software for HPC differs significantly from the development of more traditional business information systems, from which many SE best practices and tools have been drawn. Development of HPC CSE software requires tailoring of SE tools/methods developed for more traditional software applications to fit the requirements of HPC applications.

The SE-HPCCSE workshop addresses this need by bringing together members of the SE and HPC CSE communities to share perspectives, present findings from research and practice, and generate an agenda to improve tools and practices for developing HPC CSE software.

Scope and Aims

This workshop is concerned with identifying the problems faced by those working with HPC Computational Science & Engineering (CSE) applications, and understanding how appropriate software engineering (SE) tools and practices might be applied to support the development of HPC CSE applications. These applications include large parallel models/simulations of the physical world running on HPC systems, and applications that analyze and/or manipulate large amounts of data.

The organizing committee hopes for participation from a broad range of stakeholders from across the SE, CSE, and HPC communities on topics including:

  • Identification of the differences in development of software between business IT environments and research environments like HPC/CSE
  • The challenges of communicating (ideas. common pieces of work, requirements, functionality, practice) between people with SE and CSE backgrounds
  • SE tools and practices which are suited for HPC CSE applications
  • Measuring the impact of SE techniques or tools on “scientific productivity”
  • SE education and training gaps that prevent the development of HPC CSE applications

We invite both full papers (8-page) and shorter position/experience reports (4-page) that will be used to organize panel and group discussion sessions and be published in advance of the workshop to inform all attendees. We especially encourage members of the HPC and CSE communities to submit practical experience papers.

Papers on other related topics are also welcome. Please contact the organizers with any questions about the relevance of particular topics.

A workshop report will be produced which summarizes the workshops findings, and revised papers will be invited to be published in the workshop proceedings following the event.

Important Dates

Submission Deadline: August 15, 2015
Author Notification: September 15, 2015
Camera Ready: October 5, 2015
Workshop Date: November 20, 2015

Submission Instructions

Please observe the following:

  1. Papers should be at most 4 or 8 pages, depending on the type.
  2. Format your paper according to the IEEE Conference Formatting Guidelines
  3. Submit your full papers via EasyChair

3rd Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2) CFP1

First Call for Participation:
3rd Workshop on Sustainable Software for Science Practice and Experiences (WSSSPE3)

September 28-29, 2015, Boulder, CO
(Co-located with 10th Gateway Community Environments (GCE15) Workshop)

Progress in scientific research is dependent on the quality and accessibility of software at all levels and it is now critical to address many new challenges related to the development, deployment, and maintenance of reusable software. In addition, it is essential that scientists, researchers, and students are able to learn and adopt a new set of software-related skills and methodologies. Established researchers are already acquiring some of these skills, and in particular a specialized class of software developers is emerging in academic environments who are an integral and embedded part of successful research teams. WSSSPE provides a forum for discussing these challenges, including presenting both positions and experiences, as well as a forum for the community to assemble and act.

The WSSSPE1 workshop ( engaged the broad scientific community to identify challenges and best practices in areas relevant to sustainable scientific software.  WSSSPE2  ( invited the community to propose and discuss specific mechanisms to move towards an imagined future practice of software development and usage in science and engineering.

WSSSPE3 will organize self-directed teams that will collaborate prior to and during the workshop to create vision documents, proposals, papers, and action plans that will help the scientific software community produce software that is more sustainable, including developing sustainable career paths for community members. These teams are intended to lead into working groups that will be active after the workshop, if appropriate, working collaboratively to achieve their goals, and seeking funding to do so if needed.

The main aim for this first call for participation is to collect additional ideas for teams to work on at WSSSPE3 (via email; see below.)

Initial ideas for these team activities, based on the breakout groups in WSSSPE2, are:

  • Development and Community
    • Writing a white paper/review paper about best practices in developing sustainable software
    • Documenting successful models for funding specialist expertise in software collaborations
    • Creating and curating catalogs for software tools that aid sustainability (perhaps categorized by domain, programming languages, architectures, and/or functions, e.g., for code testing, documentation)
    • Documenting case studies for academia/industry interaction
  • Training
    • Writing a white paper on training for developing sustainable software, and coordinating multiple ongoing training-oriented projects
    • Developing curriculum for software sustainability, and ideas about where such curriculum would be presented, such as a summer training institute
  • Credit
    • Hacking the credit and citation ecosystem (making it work, or work better, for software)
    • Developing a taxonomy of contributorship/guidelines for including software contributions in tenure review
    • Documenting case studies of receiving credit for software contributions
    • Developing a system of awards and recognitions to encourage sustainable software
  • Publishing
    • Developing a categorization of journals that publish software papers (building on existing work), and case studies of alternative publishing mechanisms that have been shown to improve software discoverability/reuse e.g., popular blogs/websites
    • Determining what journals that publish software paper should provide to their reviewers (e.g., guidelines, mechanisms, metadata standards, etc.)
  • Reproducibility and testing
    • Building a toolkit that could allow conference organizers to easily add a reproducibility track
    • Documenting best practices for code testing and code review

Additional community suggestions are welcomed and encouraged!! (via email; see below)

Workshop Format:

  • Opening keynote TBA
  • Lightning talks – submissions welcome (via EasyChair, see below)
  • Team sessions – initial list of possible sessions above; submissions of additional ideas welcome (via email, see below)
  • Team progress report-back to plenary group
  • At the end of the workshop, teams will “pitch” their ideas to the audience, possibly including some funders (who would not committed to funding anything, just providing feedback), including e.g., Moore, Sloan, Digital Science, NSF, NIH.

Call for Participation / Actions:

  1. Save the dates for WSSSPE3: 28-29 September, 2015, Boulder, CO
  2. Suggest additional team actions – Please propose your ideas by email (with subject WSSSPE3) to by 8 July, 2015
  3. Submit lightning talks – submit a 1-page PDF containing the talk title, author names, affiliations, and a short abstract via Easychair,, by 3 August, 2015
  4. Join the WSSSPE mailing list to be sure to get further information on WSSSPE3 – via

Travel Support:

Some limited travel support is likely to be available; please check the workshop web page.

Important Dates:

  • Deadline for suggestions for new team activities: 8 July 2015 (any time of day, no extensions)
  • Initial list of team activities to be posted on WSSSPE3 web page: 22 July 2015
  • Lightning talk submissions: 3 August 2015 (any time of day, no extensions)
  • Workshop: 28-29 September 2015
  • Post-workshop report writing (participation is open to all): 30 September 2015


  • Daniel S. Katz,, University of Chicago & Argonne National Laboratory, USA
  • Gabrielle Allen,, University of Illinois Urbana-Champaign, USA
  • Sou-Cheng (Terrya) Choi,,  NORC at the University of Chicago and Illinois Institute of Technology, USA
  • Neil Chue Hong,, Software Sustainability Institute, University of Edinburgh, UK
  • Sandra Gesing,, University of Notre Dame, USA
  • Lorraine J. Hwang,, University of California, Davis, USA
  • Manish Parashar,, Rutgers University, USA
  • Erin Robinson,, Foundation for Earth Science, USA (local organizer)
  • Matthew Turk,, University of Illinois Urbana-Champaign, USA
  • Colin C. Venters,, University of Huddersfield, UK