The Cosmic Dawn ("CoDa") Project, an international team of astrophysicists, recently reached a new milestone – CoDa III – the first trillion-element simulation of how the universe evolved in its first billion years. This is when galaxies formed and flooded the universe with enough UV starlight to ionize all its atoms and lift the fog that blocked our view. CoDa III is the most detailed and accurate simulation ever produced of this cosmic era, known as the Epoch of Reionization ("EoR"), aligning theoretical and observational data for the first time.
The team described their work in the November Monthly Notices of the Royal Astronomical Society. Science News also reported on the achievement last month.
"The Epoch of Reionization represents the last major transition of the universe in the story of cosmic evolution," says theoretical astrophysicist Paul Shapiro, the Frank N. Edmonds, Jr Regents Professor in Astronomy at the University of Texas at Austin and one of the leaders of the project, "from the phase when all of space was filled with a nearly featureless, homogeneous gas to the phase in which structure emerged, with the first galaxies forming and inside them, stars."
CoDa III solves a long-standing problem between the theory and observations of the EOR. Previous models left the post-reionization universe more transparent to quasar light than is actually observed, allowing it to travel further before being absorbed. This problem vanishes in CoDa III, as the simulation yields predictions that agree with the latest observations for the first time.
Shapiro is the founding Principal Investigator (and one of the Co-PI's now, with current PI Pierre Ocvirk, Observatoire Astronomique de Strasbourg) of The Cosmic Dawn (CoDa) Project, established a decade ago to meet this computational challenge by performing the largest radiation-hydrodynamics simulations of the EoR to date. CoDa III is its third generation.
Caption: This video shows the evolution of Cosmic Dawn ("CoDa") III, the radiation-hydro-N-body simulation of the first billion years of cosmic time, in a subregion 11.8 Megaparsecs across, a zoom-in of its full 94.4 Mpc size (as measured today, following expansion of the universe), just 1/512th of the full volume of the simulation. Before galaxies and stars formed, the universe was filled with a cold, featureless gas of neutral hydrogen and helium atoms. About 100 million years after the Big Bang, slight ripples of extra density with which the smooth universe was imprinted at birth began collapsing gravitationally into a cosmic web of structure, along whose filaments and intersections galaxies formed and, inside them, stars. The most massive stars only lived a few million years before exploding as supernovae, driving shock waves into surrounding interstellar gas, but until their deaths, they released UV starlight that escaped from galaxies and created zones of hot, ionized intergalactic gas, which grew larger and more numerous as more and more galaxies formed. Eventually these ionized patches overlapped and "reionized" the whole universe, a billion years after the Big Bang. Video shows ionizing radiation (blue) which escapes from galaxies as they form in the filaments and knots of the cosmic web, revealed by gas density (white), while supernova explosions blow out like bubbles into the intergalactic medium, their shock-waves heating gas to even higher temperature (red). Time goes forward as indicated by the count-down of redshift z (high z corresponds to early times). [CREDIT: Pierre Ocvirk (Observatoire Astronomique de Strasbourg), Paul Shapiro (The University of Texas at Austin), The Cosmic Dawn ("CoDa") and CLUES collaborations; Summit/Oak Ridge Leadership Computing Facility]
Observing the distant sources of reionization directly is challenging, and detections are so far limited to the brightest galaxies -- the tip of the iceberg. Astronomers also look for signs of the spread of reionization thru the universe, as it heated and ionized ever-larger patches of the intergalactic medium (IGM), in cosmic radiation backgrounds at different wavelengths, such as that from the 21cm transition of neutral hydrogen atoms in neutral patches before they were ionized, redshifted into the 100 MHz range enroute to Earth, or the cosmic microwave background emitted before the EoR, scattered by free electrons in ionized patches as it passed thru them. They also probe the intervening gas along the lines of sight to the brightest sources they can find at such early times, quasars, by their absorption spectra. Such observations, in space and on the ground, are the latest frontier of astronomy, as technology for detecting ever weaker signals from ever greater distance advances to make this possible.
Interpreting and predicting these observations, however, requires theoretical modelling. For this, astrophysicists use computer simulations to recreate the rich physics of the EoR, to understand it and predict what observers will see, thereby testing the current theoretical paradigm of galaxy formation in a universe where atomic matter is just a small component, while most is an unknown relic of the Big Bang called "dark matter."
Simulating the EoR with CoDa III required heavy computational lifting. The simulation ran for 10 days on 131,072 processors coupled to 24,576 graphic processing units on one of the world's biggest, fastest, massively-parallel supercomputers, Summit, located at Oak Ridge National Laboratory in Tennessee, generating 20 Petabytes of data.
Size isn't the only remarkable feature of the CoDa III simulation, says Shapiro. Tracking the evolution of galaxy formation and reionization requires accounting for a mutual feedback process: ionizing radiation that leaked out of galaxies heated the intergalactic medium. That additional heat, in turn, pressurized gas enough to resist the gravitational pull of nearby galaxies. Since this gas would otherwise have fueled their star formation, the net result is to stymie new stars, thereby throttling back on reionization. Adding this radiative transfer to cosmological simulations of hydrodynamics and gravity, however, greatly increases their complexity and computational challenge.
Without radiative transfer, time in the evolutionary model would have to be divided into steps small enough to represent the changing densities of gas and stars and dark matter. The addition of this feedback loop, however, means steps must be hundreds of times smaller to capture the high speed of the "surfaces of ionization"—rapidly expanding ionizing bubbles racing outward from newly-formed galaxies and sweeping across the universe. As such, conventional approaches which add radiative transfer to cosmological simulations require hundreds of times more steps to finish – and correspondingly larger amounts of computer time – than those without it, making it computationally intractable.
CoDa III overcomes this barrier by a hybrid approach which requires a massively-paralleled supercomputer with thousands of nodes, each containing dozens of high-speed processor cores (or CPUs), linked to several, much-faster GPUs. This allows it to perform a hundred radiative transfer substeps on the GPUs for every gas-dynamical time step performed simultaneously on their partner CPUs. The linked processors and GPUs at the Summit supercomputer, Shapiro says, thereby made it possible to solve these equations almost as quickly as if the model did not include radiation.
Notably, Shapiro says, CoDa III solves a long-standing problem between the theory and observations of the EoR; namely, that previous predictions do not line up with observations of distant quasars whose light passes through gas at the end of the EoR and after. Previous models left the post-reionization universe "over-ionized" when it ended, making it too transparent to quasar light, allowing it to travel further before being absorbed than is actually observed. It was as if these models were incapable of "putting on the brakes" as they reached the end of the EoR. Something was missing from the models, either small-scale gas clumping to soak up excess ionizing radiation, but too fine-grained for previous simulations to resolve, or something to inhibit the sources of that radiation before they released too much.
This problem vanishes in CoDa III, as the simulation yields predictions that agree with the latest observations for the first time. It does this both by resolving fine-grained density variations better and by incorporating radiative transfer into the hydrodynamics, necessary to account for the feedback of reionization on galaxies and their intergalactic fuel supply, which causes star formation in low-mass galaxies to be suppressed once their neighborhoods were flooded by ionizing radiation.
Shapiro predicts that the study of the EoR is poised to undergo its own rapid expansion in coming years. Space-based observatories like NASA's JWST, launched in December 2021, and the Nancy Grace Roman Space Telescope, scheduled to launch in 2027, along with ground-based projects like the Extremely Large Telescope and the Giant Magellan Telescope (in which UT Austin is a leading partner) will improve astronomers' ability to observe the far-flung drivers of reionization. Present and upcoming radio telescopes, like the Square Kilometer Array, will also help researchers better constrain the clumpy, inhomogeneous way the IGM became ionized.
Simulations like Cosmic Dawn III, says Shapiro, provide a theoretical foundation for what these sophisticated telescopes will see.
"Apart from matching the existing spectrum of observations and predicting new ones," he says, "it provides critical insight into the nature of the physical processes that took place."
This post is adapted from a press release by the American Physical Society.
CitationsJ.S.W. Lewis et al. The short ionizing photon mean free path at z = 6 in Cosmic Dawn III, a new fully coupled radiation-hydrodynamical simulation of the Epoch of Reionization. Monthly Notices of the Royal Astronomical Society. Vol. 516, November 2022, p. 3389. doi: 10.1093/mnras/stac2383.
Comments