How the First Picture of a Black Hole Was Taken

On the morning of 10 April 2019, in a basement corridor at the Harvard-Smithsonian Center for Astrophysics in Cambridge, Massachusetts, a 29-year-old computer scientist named Katherine (Katie) Bouman sat down at a borrowed workstation in a room that smelled of cold coffee and warm laptops. She had been awake for most of the previous two days. The previous evening, the six founding papers of the Event Horizon Telescope Collaboration, a 207-person, 59-institution effort, had been held under embargo at the Astrophysical Journal Letters, and the principal authors had agreed to release the headline image at six coordinated press conferences simultaneously: Washington, Brussels, Santiago, Shanghai, Tokyo, and Taipei. The image was a synthesis of 5 petabytes of radio interferometric data, collected over a one-week campaign in April 2017 from eight telescope sites spread from Hawaii to Antarctica to the Spanish Sierra Nevada, and reconstructed by an algorithm that Bouman had helped design and a four-team blind-imaging procedure she had helped organise. The image was about to be revealed. Bouman had a small set of prepared images on the workstation, the four reconstructions from the four independent imaging teams, and one combined image, the one the collaboration had agreed to publish. She was, in the moment before the embargo lifted, looking at the combined image again. It was, as it had been since the final imaging stage in 2018, a ring.

The April 2017 campaign

The eight telescopes of the Event Horizon Telescope that recorded the data for the first image of a black hole shadow are not, individually, the largest radio dishes in the world. The largest single dish, the 305-metre Arecibo radio telescope in Puerto Rico, contributed some support data, but the principal dishes in the array are 6 to 12 metres in diameter, and they work together as a single Earth-sized instrument by the technique of very long baseline interferometry, the same technique that the Event Horizon Telescope had been refining since its founding in 2009. The technique is, in its essentials, a synthesis telescope: the signals received at the same moment at each dish are recorded with extremely accurate time stamps, brought together in a central correlator, and combined to produce an image at a resolution set by the longest distance between any two of the participating dishes. The M87* observation campaign ran from 4 to 11 April 2017. Weather was mixed; the South Pole Telescope recorded data despite the polar winter; ALMA, the Atacama Large Millimeter Array, brought most of the sensitivity. By the end of the week, the array had produced some 5 petabytes of raw data, most of it shipped, in the absence of fast enough internet links, on half-tonne hard drives by air freight to the correlator at the Max Planck Institute for Radio Astronomy in Bonn and at the MIT Haystack Observatory in Westford, Massachusetts.

The 5 petabytes were a quantity without precedent in the history of radio astronomy. The previous generation of mm-wave interferometry, the array that produced the first images of the black hole candidate at the centre of our own galaxy in the 2000s, dealt in gigabytes. The 5-petabyte campaign was, at the time, a thousand times larger. It was a quantity that could not be reduced by hand. The imaging algorithms, the mathematics that turned the 5 petabytes into a single image, had to be written, tested, and agreed upon before any of the data arrived, because the imaging stage was going to take months, and the imaging teams had to work in conditions of mutual blindness, four independent teams looking at the same data without seeing one another’s reconstructions, to prevent any shared human bias from corrupting the result.

Four imaging teams

The blind imaging stage was the part of the project that Bouman had organised. Bouman was, in April 2017, a 28-year-old PhD candidate at MIT working in the group of William Freeman, a computer vision researcher. She had joined the EHT collaboration in 2013, and had, with the encouragement of the imaging group chair Sheperd Doeleman, established a website for what the group called “Imaging Challenges” — round-robin competitions in which members of the imaging group were given synthetic data of objects they had never seen, and asked to reconstruct them with the algorithms they were developing. The Imaging Challenges were, in effect, a long training period in which the four teams learned to recover unexpected image structures, including features the algorithm authors had not anticipated. The training took most of 2014, 2015, and 2016, and the 2017 observation campaign caught the imaging group at the end of a long readiness.

The four imaging teams were each given the same calibrated 2017 data, and each was forbidden to look at the others’ reconstructions. Three of the teams used the established and newer imaging algorithms that the EHT had been developing: a classical CLEAN-based pipeline in the radio-astronomy tradition, a regularised maximum likelihood pipeline, and a newer pipeline that used Bayesian inference. The fourth team, a small group based at Radboud University in the Netherlands and at the University of Illinois, used an independent technique that the group had been developing for the EHT specifically, a different mathematical approach that the imaging group had insisted on including for diversity. The four teams worked in March and April 2018. They were not allowed to see one another’s results. They submitted their images, blind, to a comparison website that Bouman had set up, which produced a single similarity score for each image. The score did not name the source; it was a number that expressed the degree to which the four reconstructions agreed.

What the four teams found

The four reconstructions, when they were eventually unblinded in late 2018, all produced the same basic structure: a ring of light, brighter on the south side, surrounding a dark central region, about 40 microarcseconds across, centred on the position of the supermassive black hole at the core of the galaxy Messier 87, the giant elliptical galaxy in the Virgo cluster 55 million light-years from Earth. The reconstruction from the Radboud team and the three reconstructions from the established EHT teams were similar enough that the four reconstructions could be combined into a single image, the one the collaboration decided to publish, the one that was about to be revealed in six cities on the morning of 10 April 2019. The image was a ring. The brightness asymmetry, the south side brighter than the north, was consistent with the Doppler beaming of material rotating around the black hole at relativistic speeds, the orientation that the modelling team led by the University of Arizona’s Chi-kwan Chan had worked out. The size of the ring, about 40 microarcseconds, was consistent with the predicted shadow size for a 6.5 billion solar-mass black hole, the figure the modelling team reported in the sixth of the six ApJL papers.

The reaction inside the EHT was muted. Most of the imaging teams had been working on the reconstruction for months; the result was, by the time of the press conferences, the answer to a question the imaging group already knew. The reaction outside was not. The first image of a black hole was, in the hours after its release, the most-viewed scientific image in the history of the medium. The first image of a black hole was reproduced on the front pages of most of the world’s newspapers; the press conferences were attended by journalists who had, in some cases, never attended a scientific press conference before; the project’s principal investigators were on the radio and television in their respective countries within the day. The image has, by 2026, been reproduced in well over a billion instances, and the M87* image is now an icon of twenty-first-century science in the way that the Apollo 8 Earthrise photograph is an icon of twentieth-century science.

What the image does and does not show

The M87* image is, in its public form, a simple thing: a fuzzy orange ring on a black background, with the south side of the ring brighter than the north. The image is, in scientific terms, the first direct observation of the immediate environment of a supermassive black hole, a region that astronomers had previously inferred from the orbital motion of nearby stars (the case of Sgr A*, the black hole at the centre of our own galaxy, and the case of M87*) and from the relativistic jets that M87* had been observed to produce for a century. The image is consistent with the predictions of general relativity for the immediate environment of a black hole, and the 2019 result is the first direct confirmation of those predictions in the strong-field regime. The image is not, despite a common popular misreading, a photograph of the black hole itself. The black hole is, in the image, the dark region in the middle. The bright ring is the emission from the hot material just outside the event horizon, the gas that is falling into the black hole and being heated to millions of degrees as it does so. The image is a shadow, in the technical sense, the outline of the black hole’s event horizon as projected against the glow of the surrounding material, the dark region in the middle of the ring.

The M87* result was the first of a planned series. The same collaboration, with the same telescopes, has, since 2019, observed Sgr A*, the black hole at the centre of our own galaxy, and published the result in 2022. The Sgr A* image is the more difficult observation: M87* is 55 million light-years away, but the black hole is 6.5 billion solar masses, large enough to be relatively easy to resolve; Sgr A* is only 27,000 light-years away, but the black hole is only 4 million solar masses, small enough that the surrounding material moves fast enough that the imaging is harder. The 2022 result is, in many ways, a more impressive technical achievement than the 2019 result, but the 2019 image, the first one, the orange ring, the icon, is the one that the public remembers.

The April 2017 campaign, in detail

The April 2017 observation campaign was, by the standards of ground-based astronomy, an unusually difficult one. The array of eight telescopes, distributed across six high-altitude sites and the South Pole, had to be operated simultaneously in good weather, with the participating telescopes pointed at the M87 galaxy for several nights in a row. The weather was, in some sites, unhelpful. The South Pole Telescope, which had been built into the cargo of the 2007-2008 Antarctic traverse, was in the middle of the polar winter, with no chance of a maintenance visit until the next austral summer, and the project’s principal engineers on the South Pole had to operate the telescope remotely from the United States. The Atacama Pathfinder Experiment (APEX), a single 12-metre dish in the Atacama Desert of Chile that had been brought into the array as a sensitivity booster, was, on the first two nights of the campaign, knocked out by a snowstorm. The Large Millimeter Telescope, the Mexican observatory on the Sierra Negra, suffered a power outage on the third night. The Submillimeter Telescope on Mount Graham in Arizona was, for the first two nights, unable to participate because of an equipment problem with the receiver. The campaign was, in other words, a fight with the weather and the equipment, and the four nights of good data that the array produced were not the four nights that had been scheduled. The four nights of good data were the four nights on which enough of the array was working, in enough of the sites, to produce a usable interferometric signal.

The data, once recorded, had to be correlated. The correlator, the computer that combines the signals from the different telescopes into a single interferometric dataset, was the most powerful correlator in radio astronomy at the time. The original EHT correlator, at the Max Planck Institute for Radio Astronomy in Bonn, had been built specifically for the EHT 2017 campaign. A second correlator, at the MIT Haystack Observatory, was added in the spring of 2017, in part to provide redundancy and in part to allow the imaging group to begin work on the data even before the Bonn correlator had finished. The two correlators, the one at Bonn and the one at Westford, processed the same data and produced the same calibrated dataset, which was the dataset that the imaging group worked on for the next 18 months. The calibrated dataset was about 5 petabytes, and it was, in the project’s own description, “the most data ever taken in optical/infrared/radio astronomy.” The dataset was, in physical terms, too large to ship by internet, and was transferred to the imaging groups by hard-drive shipping, with the hard drives themselves loaded onto commercial flights and hand-carried from the correlator sites to the imaging groups in the United States and Europe.

How the four teams produced the same ring

The imaging stage of the M87* result, in which the four independent teams produced four different images of the same source and found that the four images were substantially the same, is, in many ways, the most interesting part of the project. The four teams used three different classes of algorithm, and the convergence of the three classes on the same basic structure was, in the trade reading, the most powerful single argument for the correctness of the final image. The three classes were a classical CLEAN-based pipeline, a regularised maximum likelihood (RML) pipeline, and a newer Bayesian inference pipeline. The CLEAN algorithm is a classical radio-astronomy method, in use since the 1970s, that iteratively deconvolves the dirty image produced by the interferometer. The RML method is a more recent development, in which the imaging is framed as an optimisation problem: find the image that, when convolved with the instrument’s point-spread function, best matches the observed data, with a regularisation term that prefers simpler images. The Bayesian pipeline, developed at Radboud University and at the University of Illinois, framed the imaging as a posterior probability problem, with prior beliefs about what kind of image is plausible and a likelihood function for the observed data. The four teams worked on the same calibrated 2017 data, with the same set of imaging tools, but with the explicit agreement that no team was allowed to see the others’ reconstructions until the end.

The four reconstructions, when they were unblinded in late 2018, all produced the same basic structure. They all showed a ring of light, brighter on the south side, surrounding a dark central region, at a position consistent with the centre of the M87 galaxy. The four reconstructions differed in the fine detail of the ring, in the precise shape of the brightness asymmetry, and in the small-scale features within the ring, but they all agreed on the macrostructure. The convergence of four independent imaging teams, using three different classes of algorithm, on the same basic image, was, in the standard argument, the strongest possible evidence that the image was not an artefact of any one algorithm. The image was, in the language of the imaging teams, “robust.” The robustness of the image, the fact that four independent teams using four different toolchains produced substantially the same result, was the basis on which the collaboration was willing to publish.

The role of Bouman in the imaging stage was, in the standard reading, less the development of a single algorithm and more the design of the procedure that allowed the four teams to work independently and produce comparable results. Bouman had, since 2013, been the architect of the imaging group’s training programme. The Imaging Challenges, the round-robin competitions in which the four teams were given synthetic data of objects they had never seen, were Bouman’s design, and the website that compared the four teams’ submissions was Bouman’s code. The 2018 imaging stage, in which the four teams produced their final images blind, was the operational version of a procedure that Bouman had been refining for five years. The procedure was, in the words of the 2019 papers, the “imaging pipeline” that allowed the four independent results to be compared, validated, and combined. The pipeline was, in this sense, the contribution of the imaging group to the project, and Bouman, as the group’s most consistent organiser, was the principal designer of the pipeline.

What the project cost and what it was for

The Event Horizon Telescope was, by the standards of contemporary astronomy, an inexpensive project. The total cost, over the decade of its development and the 2017 observation campaign, was on the order of 60 million US dollars, much of it contributed by the participating institutions in the form of staff time and observing time on the existing telescopes. The collaboration had, by 2019, 207 members from 59 institutions. The 2017 observation campaign involved the eight principal telescopes and several additional supporting instruments, all of which were pre-existing facilities that the EHT was given time on. The cost of the imaging work, including the salary of Bouman and the other imaging team members, was a small fraction of the total. The cost of the data shipping, the half-tonne hard drives that travelled from the high-altitude telescope sites to the central correlators in Bonn and Westford, was a similarly small fraction. The M87* image, in other words, was a cheap object. It was also, by the standards of contemporary astrophysics, a triumph. The orange ring of M87* is the most direct observation of a black hole’s immediate environment ever made, the first direct confirmation of the strong-field predictions of general relativity, and the proof, in case there was any doubt, that Earth-sized interferometry was a viable technique. The next generation of EHT, which is being designed in the late 2020s, will add enough new telescope sites to make movies of the material around black holes, the long-standing ambition of the field.

What the image has done since 2019

The M87* image was, in the years after its release, the basis for a substantial expansion of the Event Horizon Telescope project. The original 2017 observation campaign, the one that produced the first image, used eight telescopes: the Atacama Large Millimeter Array (ALMA) and the Atacama Pathfinder Experiment (APEX) in Chile, the Submillimeter Array (SMA) and the James Clerk Maxwell Telescope (JCMT) in Hawaii, the Large Millimeter Telescope (LMT) in Mexico, the Submillimeter Telescope (SMT) in Arizona, the Institut de Radioastronomie Millimétrique 30-metre telescope (IRAM) in Spain, and the South Pole Telescope (SPT) at the South Pole. The 2017 array produced the 5 petabytes of data, and the 2019 paper reported the first image. The 2018 and 2019 observation campaigns added additional telescopes, including the Greenland Telescope (GLT) and the Kitt Peak 12-metre telescope, and produced the data for the second major result: the 2022 image of Sgr A*, the black hole at the centre of our own galaxy.

The 2022 result was, in the trade reading, a more impressive technical achievement than the 2019 result, although it received less popular attention. Sgr A* is, in absolute terms, a smaller black hole than M87*: 4 million solar masses rather than 6.5 billion. Sgr A* is also, in astronomical terms, much closer: 27,000 light-years rather than 55 million. The combination of a smaller source and a closer distance means that the angular size of Sgr A* is about the same as M87*, but the time scales of the variations in the source are very different. The material around M87* has an orbital period of days to weeks; the material around Sgr A* has an orbital period of minutes. The imaging of Sgr A* is, in effect, the imaging of a source that is changing on a time scale much shorter than the time it takes to do the imaging, and the imaging group had to develop new techniques to handle the time-variable structure. The 2022 result was the first image of Sgr A*, and the first direct observation of the black hole at the centre of our own galaxy.

The M87* image is, in 2026, an icon of twenty-first-century science. The image has been reproduced on the front pages of most of the world’s newspapers, on the covers of the most widely read science magazines, and on the walls of the classrooms and observatories and university buildings where the work of the project is taught. The image has, more importantly, established a method: very long baseline interferometry at submillimetre wavelengths, with a global array of pre-existing telescopes coordinated by a central correlator and an imaging pipeline that combines data from the array into a single image, is now a proven technique. The next-generation Event Horizon Telescope, the ngEHT project that is being designed in the late 2020s, will add enough new telescope sites and improve the existing sites enough to make movies of the material around black holes, the long-standing ambition of the field. The M87* image is, in this sense, the first frame of a movie that has not yet been made.

Katie Bouman and the team

Katherine (Katie) Louise Bouman was, in April 2019, 29 years old, a postdoctoral fellow at the Harvard-Smithsonian Center for Astrophysics, and about to take up an assistant professorship at Caltech. Bouman had been a graduate student at MIT in the Electrical Engineering and Computer Science department, working in William Freeman’s Computer Vision group. She had joined the EHT collaboration in 2013, when she was a second-year PhD student, and had, in the six years since, become the most consistent organiser of the imaging group, the team that was responsible for turning the 5 petabytes of raw data into a single image. Bouman was, in the standard trade reading, neither the inventor of any one of the four imaging algorithms nor the author of any one of the six 2019 papers. Bouman was, in the standard reading, the architect of the procedure that the four imaging teams used to produce their images, the organiser of the Imaging Challenges, and the manager of the four-team blind-imaging procedure. The procedure was the contribution of the imaging group to the project, and Bouman, as the group’s most consistent organiser, was the principal designer of the procedure.

The role that Bouman played in the M87* result is, in the standard reading, the role that the computer-vision community had been playing in the imaging group for the previous decade. The four imaging algorithms that the four teams used were, in their origins, computer-vision algorithms: the CLEAN-based pipeline was a classical radio-astronomy algorithm that had been ported to the imaging group’s toolchain; the RML pipeline was a standard computer-vision optimisation algorithm; the Bayesian pipeline was a contemporary computer-vision inference technique. The four teams were, in the standard reading, four different applications of computer vision to the same data, and the convergence of the four applications on the same basic structure was a vindication of the computer-vision approach. Bouman was, in this sense, a representative of the imaging group’s approach: the problem of producing an image from incomplete data was, in the imaging group’s framing, a computer-vision problem, and the imaging group was, in the standard reading, the computer-vision team of the Event Horizon Telescope.

The reception of the M87* result was, for Bouman, the most public moment of her career. The image was released on 10 April 2019. The principal investigators of the project, Sheperd Doeleman and the collaboration’s other senior scientists, gave the press conferences in Washington, Brussels, Santiago, Shanghai, Tokyo, and Taipei. Bouman, who was, at 29, by some margin the youngest of the project’s principal scientists, was not at any of the press conferences; she was, in the days after the release, in the United States, giving talks at the Caltech event and at the Harvard-Smithsonian symposium, and being interviewed, almost continuously, by the press. The standard photograph of Bouman, which appeared in the New York Times, the Washington Post, the Guardian, the BBC, and most of the world’s newspapers in the days after the release, was taken in the basement of the Harvard-Smithsonian Center for Astrophysics, in front of the workstation on which the first image had been generated. The photograph shows Bouman at the workstation, looking at the M87* image, with the orange ring of the black hole’s shadow glowing on the monitor in front of her.

References

1. Event Horizon Telescope Collaboration (2019). “First M87 Event Horizon Telescope Results. I. The Shadow of the Supermassive Black Hole.” The Astrophysical Journal Letters, 875, L1. https://iopscience.iop.org/article/10.3847/2041-8213/ab0ec7
2. Event Horizon Telescope Collaboration (2019). “First M87 Event Horizon Telescope Results. IV. Imaging the Central Supermassive Black Hole.” The Astrophysical Journal Letters, 875, L4. (K. L. Bouman was a paper coordinator.) https://iopscience.iop.org/article/10.3847/2041-8213/ab0e85
3. Written Testimony of Katherine L. Bouman before the House Committee on Science, Space, and Technology, U.S. House of Representatives, 16 May 2019. https://www.congress.gov/116/meeting/house/109466/witnesses/HHRG-116-SY00-Wstate-BoumanK-20190516.pdf
4. Chael, A., Bouman, K. L., et al. (2016). “Imaging the Supermassive Black Hole Shadow with an Imaging Atmospheric Cherenkov Telescope Array.” Proceedings of the Imaging the Event Horizon workshop.
5. Event Horizon Telescope Collaboration (2022). “First Sagittarius A* Event Horizon Telescope Results.” The Astrophysical Journal Letters, 930, L12. https://iopscience.iop.org/journal/2041-8205/page/First%20Sgr%20A%20EHT%20Results