News and Announcements
Seventy Years
On July 18, Jack Dongarra celebrated his 70th birthday. As with everything in this pandemic, creative solutions were required to celebrate this milestone. Undeterred, Sue Dongarra and company compiled birthday greetings from around the world and published them in the video above.
Joan and the gang also got the Boss a gift bag full of whiskey, a glass with which to drink the whiskey, and a book of piano ditties to jazz up the atmosphere even further. All of these items were delivered in person—at a social distance. Reports of singing and dancing have also been corroborated.
Jack was originally set to celebrate his 70th in Manchester, and even though we all remain “grounded” this summer, the Linear Algebra group’s weekly meeting was repurposed for a surprise celebration from the Manchester cohort via Zoom.
Happy Birthday, Jack! Here’s to many more.
Employment Opportunities at ICL
ICL is seeking full-time Research Scientists (MS or PhD) to participate in the design, development, and maintenance of numerical software libraries for solving linear algebra problems on large, distributed-memory machines with multi-core processors, hardware accelerators, and performance monitoring capabilities for new and advanced hardware and software technologies.
The prospective researcher will coauthor papers to document research findings, present the team’s work at conferences and workshops, and help lead students and other team members in their research endeavors in ongoing and future projects. Given the nature of the work, there will be opportunities for publication, travel, and high-profile professional networking and collaboration across academia, labs, and industry.
An MS or PhD in computer science, computational sciences, or math is preferred. Background in at least one of the following areas is also preferred: numerical linear algebra, HPC, performance monitoring, machine learning, or data analytics.
For more information check out ICL’s jobs page: http://www.icl.utk.edu/jobs.
Conference Reports
PEARC20
The ACM’s Practice and Experience in Advanced Research Computing conference (PEARC20) went virtual this year to provide a forum for discussing challenges, opportunities, and solutions among a broad range of participants in the research computing community.
ICL’s Stan “the Man” Tomov co-presented a tutorial on How to Build Your Own Deep Neural Network alongside collaborators Kwai Wong, Daniel Nichols, Rocco Febbo, Julian Halloy, and Xianfeng Ma from UT’s Joint Institute for Computational Sciences (JICS).
This tutorial describes how to use the MagmaDNN backend to write a deep neural network software program through a step-by-step illustration—from the mathematical formulation and numerical algebra interpretation to the programming implementation. The MagmaDNN library was developed mostly by the JICS students under the National Science Foundation’s Research Experiences for Undergraduates program with guidance from Kwai Wong and Stan Tomov.
The editor would like to thank Stan Tomov for his contributions to this article.
Recent Releases
New ICL Presentation Templates
Just in time for the ICL annual retreat, new ICL-themed PowerPoint and Keynote templates are now available for download.
MagmaDNN 1.2 Released
MagmaDNN 1.2 is now available. MagmaDNN is a neural network library, written in C++ and using MAGMA as its computational backend, aimed at providing a simple, modularized framework for accelerated deep learning on heterogeneous architectures.
Updates for MagmaDNN 1.2 include:
- bug fixes and performance improvements;
- oneDNN (MKL DNN) support including fully connected, convolutional, and pooling layers;
- CMake build system;
- additional NN examples including CNN, ResNet, AlexNet, LeNet, MNIST, CIFAR interactive, and VGG16;
- additional examples for Tensor Math;
- C++ style formatter;
- modularized distributed optimizer;
- CIFAR10, CIFAR100, and MNIST data loaders;
- additional CUDA streams;
- model summary prinout; and
- Spack package manager installation support.
More information on MagmaDNN 1.2 is provided in this paper and this presentation.
Check out MagmaDNN’s repository on Bitbucket: https://bitbucket.org/icl/magmadnn.
hipMAGMA 2.0.0 Released
hipMAGMA 2.0.0 is now available. This is the second release of hipMAGMA, which adds MAGMA support for AMD GPUs. Updates for this release include:
- merged and generated from the latest MAGMA 2.5.3 release;
- around 250 LAPACK functions and 1,000 routines now available when counting all precisions; these routines depend on BLAS and are ported almost without change;
- BLAS, batched BLAS, and auxiliary routines that were originally written in CUDA have been ported to HIP and included;
- performance improvements through specific tuning for AMD GPUs in existing BLAS and bug fixes;
- improved documentation and installation;
- enabled use of the new hipBLAS routines that were added in the ROCm 3.3 and 3.5 releases (hipMAGMA 2.0.0 requires ROCm 3.5 or older); and
- added and optimized
TRMM,TRMV,GEMV,SYMV,GEAM, and batched BLAS in all precisions.
Check out hipMAGMA’s repository on Bitbucket:
https://bitbucket.org/icl/magma/src/hipMAGMA/.
Interview

Terry Moore
Your first newsletter interview went over fairly well. Are you ready to do another?
Sure.
You mentioned in 2016 that you are originally from Ft. Wayne, Indiana. How did you end up in Knoxville, TN?
I gave one account of that in my 2016 interview, but the shorter answer is “desperation.” In the mid 1980s I was working on my dissertation, had 2 (sometimes 3) part-time jobs, and a family with two young children. The academic job market in Philosophy was terrible at the time; it looked like, even when I finished my PhD, I might have to take a series of 1-year appointments before I could find a tenure-track position, which would mean a lot of moving. That was bad enough. But the thought of trying to put my children through college by working as a gypsy scholar didn’t give me a warm and fuzzy feeling either. Despite the fact that I knew almost nothing about Information and Communication Technology (ICT), some people I trusted (mostly my wife and my brother) thought I would be good at it. So I decided to change careers, even while trying simultaneously to finish my PhD. UNC Chapel Hill turned out to be a very good place to be if you wanted to self-educate in ICT. After a couple of years of hanging out in the campus computing labs (I couldn’t afford a computer of my own at the time), I had learned enough to talk my way into a job doing ICT support for the School of Nursing at UNC-CH. The Associate Dean there, Dr. Joan Uhl, liked my work and my academic credentials. When she got the job as the Dean of the College of Nursing at UTK, she decided to hire me as her “Director of Computing and Communications” to help her bring the College into the Internet Age. And that’s how my family and I came to Knoxville.
Unlike the other “doctors” at ICL, your background is in the humanities. How has your PhD in philosophy shaped your approach to writing and managing the proposal process (among other things) at ICL?
This is a hard question for me to answer, especially if I want to be mercifully brief. I suppose first and foremost, reading and writing Philosophy shaped the way I think and write in general. This comes out partly in my inclination—frequently criticized by my fellow ICLers—for long sentences. But more importantly, Philosophy taught me that a complete unit of thought is an argument, i.e., a paragraph. Second, I did my PhD work on the American philosopher Charles S. Peirce (pronounced “pers”). Peirce spent his working life as a professional scientist, in exactly the same sense of “scientist” (e.g., empirical, quantitative, statistical) that we use today. But his work as a philosopher focused on the Logic of Science in the broadest possible sense of “Logic.” I have always analyzed the “scientific computing” world that ICL inhabits through the lens of Peirce’s Logic of Science. Finally, I did my MS in Political Philosophy. When you’re working on proposals that are going to federal funding agencies, you have to think a bit about the institutional goals and circumstances that are shaping the funders and their funding solicitations. Having some background in Political Philosophy, and a general history of ideas, has been helpful to me in that regard.
Apart from the humanities background, though, the most helpful thing for managing the proposal writing process has been the fact that I played competitive team sports as a kid. Writing proposals at ICL is competitive team writing, and lessons from the former translate to the latter pretty well.
Over the years, you’ve worked on proposals with many members of ICL across all three research groups. Is there one particular research focus that you find more interesting? Speak carefully.
Developing proposals is a somewhat arduous process, but one of its compensatory pleasures is that you get to work with smart people in a very collaborative and creative process. There are people in all three of ICL’s research groups that have a real creative spark, and it’s working with them which has made the process interesting and enjoyable for me. So that spark is something special that is independent of the research focus, but it’s a characteristic of many of the people I have worked with at ICL during my two decades here.
You told me once—during my job interview actually—that we should never stop learning, and that if you stop learning and get comfortable, you should move on. Something about an ascidian eating its own brain. This is not a question, but more so something I thought others should know about your interviewing style.
Moving on. You’ve been with ICL for a couple of decades, and I think you have a pretty good idea of the operation at this point, if I may be so bold. That said, this pandemic has flipped the script on many of us. What has been your biggest challenge working apart from your fellow ICLers?
I’m sure it’s pretty similar to what everyone is experiencing. Our understanding of what’s going on with others is always a guess based on the experiences we have with them. In the last few months, we’ve gone from daily, n-dimensional experiences with real people in the flesh to occasional encounters with flat, 2-dimensional talking heads. It’s like everyone has turned into their own little versions of Max Headroom. So our ability to make our guesses about what our colleagues are thinking and feeling comes from only a very tiny fraction of the experience we used to have with them. As I said above, doing proposals is a kind of competitive team sport. To be effective, you have to have a sense of what your team is thinking and feeling. It’s hard to maintain that sense in Zoomworld.
And even though we’re mostly relegated to Zoom meetings these days, we are fortunate that much of the work and research at ICL can be accomplished remotely. Tell us a little bit about some of the proposals and/or projects you’re involved in at present.
ECP provides us with a solid funding foundation these days, so we’re able to be more selective about the proposals we do. I helped with some AI-related proposals for the DOE that we did in May because AI is rapidly revolutionizing scientific computing, and it’s very important for the long term that we begin to expand our research portfolio in that direction. For similar or related reasons, I worked on a preproposal for NSF Computing Expeditions with Michela Taufer and Micah Beck that was submitted in June. If that effort is given the green light for a full proposal, that will be a big deal, because it is for 15 million dollars over five years and involves four other universities. If we’re selected to submit a full proposal, it would be due in mid-February of 2021. So I guess I’m trying to work on strategic things that might help open up a post-ECP path forward for ICL.
Over the past few years, a lot of the effort within the HPC community to think strategically about cyberinfrastructure for science has been going on in the Big Data and Extreme-scale Computing (BDEC) project. BDEC is a long-running series of international workshops that Jack and Pete Beckman have been leading. At the moment I’m focused on helping to put together a summary report, or a kind of “technology roadmap” document, for that project. Because of the emergence of Deep Learning and the incredible tsunami of data that scientific instruments of all kinds are generating today, this is truly a watershed moment for cyberinfrastructure and for scientific computing generally. So working on the BDEC strategy document is both very challenging and, for someone interested in the history and philosophy of science, like me, extremely interesting. That document will need to be finished this fall, the Covid-19 pandemic notwithstanding.
Finally, tell us what you miss most about working on campus. Do you think we’ll ever get back to (a new) “normal?”
What I’ve said already about the challenges of making ICL work in Zoomworld covers this question to some degree. Briefly put, it’s all about the people. I very much miss what I think of as the social and intellectual metabolism of life at ICL. It’s a special place. I’ve thought that and heard that from people inside and outside of ICL throughout my entire career there. But the pandemic has put us into something like a state of suspended animation. Although we’re keeping the metabolism going, living in Zoomworld has slowed it way down. So as tiring as it could sometimes be, I very much miss the spark and pace of life at ICL, pre-Covid.
As for the “new normal,” I hope and expect to get back to a state of societal homeostasis sometime in the not too distant future, especially if we get an effective vaccine or treatment regimen for Covid reasonably soon. Yet the pandemic, and the poor quality of our society’s response to it, has been a huge shock to our system and its institutions; if and when we do stabilize, it seems obvious that we’re going to have to reestablish our lives in a very different world. And that’s not even to mention all the other forces that are sweeping us along and making the future uncertain. But aside from such lame generalities, I’m sure I don’t know any more than anyone else about where we’re headed.































