Resilience Data Analytics Tool and the Cloud Help Humans Survive and Thrive

On the topic of resilience—the ability to withstand, respond and adjust to chronic or acute stressors— there are a lot of data sets out there on social ecological systems, human environment, stressors, shocks, natural disasters, and conflict. The challenge is these data sets are often stored in silos or confined to the academic community.

However, if we could analyze existing data to give insights into what kind of investment and intervention countries should make, then more people could become resilient.

Resilience Atlas brings together 60 data sets for governments and scientists: 12TB of data now available

We had the opportunity to talk with Alex Zvoleff, Director of Data Science, and Sandy Andelman, Chief Scientist at Conservation International.

Conservation International, with funding from the Rockefeller Foundation, released a new online tool, the Resilience Atlas. The Atlas is designed to build understanding of the extent and severity of stresses and disasters affecting rural livelihoods, production systems, and ecosystems and how different types of assets, from natural capital, to financial capital and social networks affect their ability to thrive and even transform in the face of adversity.

For the first time, data from satellites, ground-based biophysical measurements and household surveys – from more than 60 of the best available data sets (including the NASA NEX data set) totaling over 12 terabytes – have been integrated, analyzed and made available in an easy-to-use map interface. By integrating these disparate data sets, the Atlas connects themes and perspectives so that people making important investment, development and security decisions can easily see the full picture.

What is the challenge you are trying to solve with the Resilience Atlas?

Sandy: In order to thrive, societies need to exhibit resilience. Evidence-based decision-making is a huge challenge in areas where data are inaccessible, and the Resilience Atlas hopes to help make essential information available in a digestible form to governments, communities, donors and businesses who are struggling to manage the risks and uncertainties associated with climate change, conflict, population growth, and other stressors.  It can provide them with insights on the magnitude of the challenge and on which kinds of interventions and investments will make a difference. By creating a system, instead of just providing the answers, people can reach their own insights with the publicly provided data.

How was the map created?

Alex: Faced with large amounts of data volume (12 TB) during work on the Atlas, we had to be able to handle the large volume, the intense computation, and be able to access the data on demand. Prior to considering the cloud to host the Atlas, a lot of the data sets were held by individual researchers. Though a few were made available in publications, they were not easy to access, not easy to crop or download, and in general a lot of the data was not publicly available or it needed specialized knowledge to decipher. Amazon Elastic Compute Cloud (Amazon EC2) spot instances were used for processing and bringing up a large fleet of servers at one time; 120 servers were used in parallel. Amazon EC2 made it possible to do in only a few days what would have taken over a month. Additionally, we have the ability to automate processes as more data sets covering different trends become of interest to the community. All of this is made possible by the cloud computing infrastructure of Amazon Web Services (AWS).

How does the map impact human life?

Sandy: The hope is that this map provides insights to how extensive and how severe the different kinds of shocks people face are in more than forty countries in the Sahel, Horn of Africa, and South and Southeast Asia. For example, with increasing climate variability or financial market shocks, the atlas can give insights into the particular systems and then to try to shift decision making to a more evidence-based approach pulling together the best data to get the complete picture. By mining data, users can understand which kinds of interventions and investments have actual evidence for their effectiveness. Another example is the Journey tool. One focuses on Ethiopian pastoralists, guiding users first to a map showing where they live and then exploring stressors they face, such as changing rainfall patterns that threaten the viability of pastoralism as a livelihood and the lack of investments in human capital, such as literacy and access information, which can hinder them to adapt or transform.

What is the goal of the Atlas in relation to governments, communities, donors and businesses?

Sandy: We can work with governments to use the Atlas as a planning tool. Officials can use the data already in the atlas or they can work with us to put their finer-scale data into the Atlas as a tool to assess resilience and to inform better investment decisions. The open access to this data gives a better understanding of important issues like climate change, flooding, droughts to more people. What was missing before was the integrated picture and more and more in the world we live in today we need a system perspective, because decisions about poverty alleviation are not independent from decisions about conservation and what kind of agriculture to invest in. The Atlas is unique because it pulls together all the different data sets that people might be familiar individually with and puts them together to give the community the full picture to make their important decisions. Determining cause and effect is complex, hence why it requires multiple disciplines and experts to look at problems from multiple angles.

What is the user experience like with the Atlas?

Alex: With the “Journeys” feature, the Atlas guides users on how to tell stories with data, enabling users to explore the specific data that are relevant to the questions they want to answer. Instead of telling them the answers, the Atlas helps them to discover the answers for themselves. It has a simple three-step approach:

1. Select geography and system of interest to produce a map of how it is distributed.
2. Identify the stressors and shocks affecting system, and their extent and severity.
3. Explore what kinds of assets (such as natural, human, social, financial, and manufactured capital) might increase resilience.

Users can share their insights by sharing map links via social media or by embedding Atlas data within their own webpages. All of this data is openly available without a fee and the site supports API access for private industries, allowing a broader audience to work with the data to build other tools.

Go ahead and explore the map here.
AWS also provides public data sets, visit www.aws.amazon.com/opendata to learn more.