Can AI Help Our Cities Beat the Heat? Inside the University of Michigan’s AI for Urban Heat Resilience Hackathon

Extreme heat is now the leading weather-related cause of death in the United States. In 2023, heat claimed over 2,300 lives, the highest number in 45 years of records, according to an Associated Press analysis of CDC data. Heat-related deaths have risen 117 percent since 1999 (JAMA), and the economic toll reaches an estimated $220 billion annually in lost U.S. productivity (Duke University Nicholas Institute). The burden falls disproportionately on low-income communities, older adults, outdoor workers, and residents of dense urban environments with little tree canopy or green space.

Yet urban heat remains one of the most data-poor problems in public health. Publicly available satellite thermal imagery — the most widely accessible — have low spatial and temporal resolution. For example, the harmonization of surface reflectance data from NASA’s Landsat and European Space Agency’s Sentinel-2 satellites produces a spatial resolution of about 30m and provides a temporal resolution of ~1.6 days. That’s useful regionally, but far too coarse and sparse for the street-level planning decisions that actually shape heat exposure in neighborhoods. Though commercial satellite systems such as SatVu have thermal sensors that can achieve up to 3.5m resolution and with multiple scans (10-20 revisits per day) of the earth’s surface, this comes at high cost.

This spring, AWS partnered with the University of Michigan’s Center for Global Health Equity (CGHE), the Michigan Institute for Data & AI in Society (MIDAS), SmithGroup, and Ecosystems, Finance and Health (EFH) to co-sponsor the AI for Urban Heat Resilience Hackathon — a two-day event built around a deceptively simple question: Can machine learning generate thermal imagery from standard RGB photographs?

If yes, urban heat mapping becomes feasible at a scale and cost currently out of reach for most cities and research groups worldwide.

Why We Got Involved

As an AWS team supporting higher education, we see firsthand how universities are tackling society’s hardest problems — but often face infrastructure constraints that slow the pace of discovery. When the University of Michigan approached us about this hackathon, the opportunity was clear: a real-world public health challenge, a strong AI/ML use case, and a chance to put cloud-scale compute directly in the hands of researchers and students working on something that matters. This is exactly the kind of collaboration we believe in.

The Urban Heat Island Effect

The urban heat island (UHI) effect is a predictable consequence of how cities are built. Low-albedo surfaces, anthropogenic heat from HVAC and traffic, and the thermal mass of concrete and asphalt absorb heat by day and release it after sunset. The result: urban areas can be 1–7°F warmer than surrounding areas during the day, and up to 22°F warmer at night (EPA Heat Island Resource Center).

As Verrah Otiende, Eric and Wendy Schmidt AI in Science African Faculty Fellow at the University of Michigan and hackathon co-lead, put it:

“The communities most exposed to urban heat are often the least represented in the data we use to study it. This hackathon pushed us to build tools that are technically rigorous and genuinely useful to the people who need them most.”

For lower-income countries, the picture is grimmer — higher vulnerability, fewer protections, and a heat burden that already outstrips existing resources. Mapping that risk at a useful scale, neighborhood by neighborhood, remains technically difficult. That’s where AI and cloud computing come in.

The Challenge: RGB to Thermal at Street-Level Resolution

The hackathon was led by Geoffrey Siwo (Research Assistant Professor, Department of Learning Health Sciences and Pharmacology; AI Lead CGHE) and Verrah Otiende, who shaped both its technical direction and its commitment to equitable design. Siwo framed the core insight: RGB cameras are universal. If a model can reliably infer thermal information from standard color imagery, the barrier to high-resolution urban heat mapping drops dramatically.

“Urban heat is a public health crisis that we can now address with the tools of modern AI — but only if we build those tools with the communities most at risk in mind.” — Geoffrey Siwo

The dataset included 468 paired RGB and thermal drone images from Ann Arbor, MI, supplemented with satellite embeddings, GPS coordinates, timestamps, and weather variables. Teams were evaluated on a held-out test set of 202 images using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSI), and Learned Perceptual Patch Similarity (LPIPS).

Keenan Gibbons, a landscape architect at SmithGroup and University of Michigan faculty member, provided the applied research context. His work uses UAV-mounted thermal cameras to map streetscapes at sub-centimeter resolution — orders of magnitude finer than satellite data.

“High-resolution thermal data enables an evidence-based approach to urban design — one where the thermal performance of a proposed intervention can be modeled before construction begins.” — Keenan Gibbons

How AWS Powered the Research

AWS provided investment, technical guidance, and cloud infrastructure to support hackathon teams. Participants used Amazon SageMaker for model training and experimentation and Amazon Bedrock for generative AI capabilities — giving researchers access to the same enterprise-grade tools used by organizations worldwide to build, train, and deploy ML models at scale.

By removing infrastructure barriers, researchers could focus on the science — iterating faster on the complex modeling challenges that problems like urban heat demand.

What the Teams Built — and What They Found

Teams pursued three broad technical strategies:

Conditional U-Net — Encoder-decoder architectures with auxiliary conditioning inputs (land cover, temperature, drone metadata) injected at the bottleneck to improve spatial feature learning.

Pix2Pix (Conditional GAN) — Image-to-image translation using a conditional generative adversarial network, with a discriminator supervising output realism alongside reconstruction loss.

Cross-Sensor Alignment — Preprocessing pipelines addressing the modality gap between RGB and thermal sensors through edge-detection-based feature matching, scale correction, and viewpoint alignment.

Key findings:

The most consistent challenge was spatial structure preservation — predicted thermal maps frequently lost structural correspondence with ground-truth scans, blurring boundaries between impervious and vegetated surfaces even in well-tuned architectures.

Performance varied with land cover composition: scenes with strong material contrast (impervious vs. vegetated) scored better than structurally homogeneous scenes — an important insight for real-world deployment across diverse urban environments.

Results suggest the problem is partially inherent to the signal mismatch between RGB and thermal sensors, pointing toward multi-modal fusion approaches as a promising next direction.

These findings don’t represent a dead end — they represent a research frontier. The hackathon demonstrated that RGB-to-thermal translation is achievable, while clearly mapping the technical challenges that remain.

A Model for University-Industry Collaboration

This hackathon demonstrates what becomes possible when research universities and AWS work together on problems that matter. The University of Michigan brought domain expertise, research infrastructure, and a commitment to equitable design. AWS brought scalable AI/ML services, technical guidance, and investment to help researchers push boundaries.

What’s Next

The research doesn’t stop here. The University of Michigan team is exploring expanded datasets across additional cities and climates, model architectures that better preserve spatial structure, and pathways to make these tools accessible to urban planners and public health practitioners globally. AWS remains committed to supporting this work as it moves from hackathon prototype toward real-world application.

Urban heat is a solvable problem. The data gap is real, but it is closing — and AI, powered by cloud infrastructure, is accelerating that progress.

Figure 1: Photo of the best performing team – GeoHeat-AI presenting the results of their approach.