Evaluated GIFs & metadata

We used 20 data-GIFs as stimuli in our work for conducting a qualitative study through interviews (Section 4) and an online study (Section 5). This document presents these 20 evaluated GIFs and their associated metadata, including the titles, the sources, how they were encoded in our proposed design space, and the comprehension questions & answers posed to the participants. The correct answer for each question is colored in red. The questions for each GIF and the answer options for each question are shuffled in the online survey. The appearance times of each design choice in 20 samples are shown in Fig. 5 in the paper.

fig2a-temporal.gif

Title: How the Coronavirus spread across the U.S.？

Source: https://twitter.com/nytimes/status/1241387140933652481 

Number of valid responses: 22

Average accuracy of the GIF: 83.3% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 90.9%)

1. It shows the increase of total cases across the US over time
2. It compares the number of total cases in different states
3. It narrates a story about the chart with annotation.
4. It animates the creation of a visualization
5. I don’t know.

2. What is the meaning of the size of the red circle in the map? (accuracy: 77.3%)

1. Total deaths per region
2. Aid raised per region
3. Total cases per region
4. Population per region
5. I don’t know

3. What is the meaning of the line chart? (accuracy: 81.8%)

1. Show the number of the total deaths over time
2. Show the number of the total aid raised over time
3. Show the number of the the total cases over time
4. Show the number of the total population over time
5. I don’t know

fig2b-faceting.gif

Title: Which countries have flattened the curve for the coronavirus?

Source: https://twitter.com/nytimes/status/1240790201414438912 

Number of valid responses: 23

Average accuracy of the GIF: 68.1% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 100%)

1. It shows the curve for the coronavirus in the U.S. over time
2. It shows the curve for the coronavirus of different countries.
3. It narrates a story about the chart with annotation.
4. It animates the creation of a visualization
5. I don’t know.

2. What is the meaning of the line chart? (accuracy: 8.7%)

1. The change in the number of the new cases over time
2. The change of the number of the 7-day average of new cases over time
3. The change of the number of the deaths over time
4. The change of the number of the cured over time
5. I don’t know

3. Which country has flattened the curve?  (accuracy: 95.7%)

1. Mainland China
2. Italy
3. Iran
4. United States
5. I don’t know

fig2c-narrative.gif

Title: Detroit's Debt and Revenue  

Source: http://lenagroeger.com/datagifs/#/30 

Number of valid responses: 26

Average accuracy of the GIF: 65.4% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 23.1%)

1. It shows the increase of the revenue and debt of Detroit over time
2. It changes between the revenue and debt
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization
5. I don’t know

2. What is the meaning of the line chart? (accuracy: 88.5%)

1. It shows the revenue (blue) and debt (red) in a time period
2. It shows the revenue (red) and debt (blue) in a time period
3. It shows the revenue of Detroit (blue) and Wall Street (red) in a time period
4. It shows the revenue of Detroit (red) and Wall Street (blue) in a time period
5. I don’t know

3. What is the added text used for? (accuracy: 84.6%)

1. It complements information about the designer’s idea.
2. It complements information about the critical timestamps.
3. It complements information about the comparison between Detroit and Wall Street.
4. It is just used for enriching the visual.
5. I don’t know.

fig2d-setup.gif

Title: The number of total cases in multiple countries

Source: https://wap.peopleapp.com/article/5210197/5111953 

Number of valid responses: 24

Average accuracy of the GIF: 51.4% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 20.8%)

1. It shows the increase of total cases over time
2. It changes among each country with the number of total cases
3. It narrates a story about the chart with annotation.
4. It animates the creation of a visualization
5. I don’t know.

2. What is the meaning of the bar length? (accuracy: 87.5%)

1. The number of total cases
2. The number of deaths
3. The number of the cured
4. No meaning
5. I don’t know

3. What does the growing speed of the bars mean? (accuracy: 45.8%)

1. The increasing speed of the cases
2. The increasing speed of the deaths
3. The increasing speed of the cured
4. No meaning.
5. I don’t know.

fig3a-baseline.gif

Title: The Shape-shifting US Income Distribution

Source: http://lenagroeger.com/datagifs/#/15

Number of valid responses: 27

Average accuracy of the GIF: 79.0% 

Encoding:

1.What is the animation showing with the GIF playing? (accuracy: 81.5%)

1. It shows the income distribution of different people over time
2. It shows people at different income
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization
5. I don’t know

2. What is the meaning of the bar chart? (accuracy: 85.2%)

1. It shows income slices (x-axis) and their population percent (y-axis) in a year
2. It shows the population percent (x-axis) and their income (y-axis) in a year
3. It shows the years (x-axis) and the population with middle income (y-axis).
4. It shows the years (x-axis) and the income (y-axis).  
5. I don’t know

3. What is the meaning of the blue line? (accuracy: 70.4%)

1. It shows the income distribution of the first year.
2. It shows the income of the middle class.
3. It shows the income distribution of the last year
4. No meaning
5. I don’t knows

fig3b-trails.gif

Title: Watch 118 Bird Species Migrate Across A Map Of The Western Hemisphere

Sources: https://www.allaboutbirds.org/mesmerizing-migration-watch-118-bird-species-migrate-across-a-map-of-the-western-hemisphere/ 

Number of valid responses: 24

Average accuracy of the GIF: 75.0% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 87.5%)

1. It shows the bird migration over a year.
2. It shows the migration of different bird species.
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know.

2. What do the gray trails of the points show? (accuracy: 45.8%)

1. The speed of the bird movement
2. Time
3. The number of birds
4. No meaning
5. I don’t know

3. What does the color of the points represent? (accuracy: 91.7%)

1. Bird species
2. Time
3. Number of birds
4. No meaning
5. I don’t know

fig3c-overview.gif

Title: The Fall And Rise Of U.S. Income Inequality

Sources: http://lenagroeger.com/datagifs/#/16 

Number of valid responses: 29

Average accuracy of the GIF: 46.0% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 69.0%)

1. It shows income of different people over time
2. It shows people at different income averages
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization
5. I don’t know

2. What is the meaning of the point in the chart? (accuracy: 44.8%)

1. It shows the average income for the top 1% (x-axis) and for the bottom 90%(y-axis) in a year.
2. It shows the average income of the top 1% (x-axis) and their population(y-axis) in a year.
3. It shows the average income of the bottom 90% (x-axis) and their population(y-axis) in a year.
4. It shows a group of people with a certain income.
5. I don’t know

3. What are the gray points used for? (accuracy: 24.1%)

1. They indicate the income situations for the upcoming years.
2. They provide the average income of other people.
3. They are used as disturbing factors.  
4. No functions.
5. I don’t know

fig3d-long-exposure.gif

Title: Global temperature change (1850-2016)

Sources: http://lenagroeger.com/datagifs/#/2 

Number of valid responses: 22

Average accuracy of the GIF: 54.5% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 95.5%)

1. It shows the temperature change over time
2. It shows timestamps in different temperature changes
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization
5. I don’t know.

2. What does the radius mean? (accuracy: 45.5%)

1. The value of the temperature change
2. Time
3. The temperature
4. No meaning
5. I don’t know

3. What does the color of the growing line represent? (accuracy: 22.7%)

1. The value of the temperature change
2. The temperature
3. Time
4. No meaning
5. I don’t know

fig4a-loop.gif

Title: The difference between men and women populations by age

Sources: https://i2.wp.com/flowingdata.com/wp-content/uploads/2018/10/male-female-bivariate-zeroed.gif?resize=670%2C420&ssl=1 

Number of valid responses: 23

Average accuracy of the GIF: 39.1% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 87.0%)

1. It shows the population differences of men and women by age over time.
2. It shows the population of men and women at each age.  
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know.

2. What is the meaning of the orange area? (accuracy: 8.7%)

1. It shows the number of more men than women in a year.
2. It shows the number of more women than men in a year.
3. It shows the population of men by age in a year.
4. It shows the population of women by age in a year.
5. I don’t know.

3. What does the transparent area mean? (accuracy: 21.7%)

1. It shows the situation of the first year.
2. It shows the ideal situation of the population difference.
3. It shows the average population difference of the current frame.
4. No meaning
5. I don’t know.

fig4b-pause.gif

Title: The proliferation of Walmart

Source: https://twitter.com/toddrjones/status/1167112782124179456

Number of valid responses: 25

Average accuracy of the GIF: 69.3%

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 88.0%)

1. It shows the distribution of Walmart  over time.
2. It compares the number of Walmat in each region.  
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know

2. What does the blue circle show? (accuracy: 64.0%)

1. It shows the ideal locations to open Walmart in that frame.
2. It shows the locations to open Walmart in that frame.
3. It shows the historical locations of opened Walmart.
4. No meaning
5. I don’t know.

3. What does the yellow circle represent? (accuracy: 56.0%)

1. It shows the number of opened Walmart in that region at that frame.
2. It shows the locations to open Walmart at that frame.
3. It shows the historical locations of Walmart opened.
4. No meaning
5. I don’t know.

fig4c-bounce.gif

Title: How did BMI among adults change over the years?

Source: https://flowingdata.com/2016/06/14/growing-to-obesity/ 

Number of valid responses: 20

Average accuracy of the GIF: 60.0% 

Encoding:

Note (show with the questions): Body Mass Index (BMI) indicates the weight. The larger your BMI is, the heavier you are.

1. What is the animation showing with the GIF playing? (accuracy: 55.0%)

1. It shows the BMI distribution over time
2. It changes the BMI distribution between females and males
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization
5. I don’t know

2. What is the meaning of the width of the bar? (accuracy: 80.0%)

1. The Body Mass Index (BMI) value
2. The percentage of males/females at a BMI value
3. Time
4. The age of males/females
5. I don’t know

3. What is the overall change of weight among people?  (accuracy: 45.0%)

1. First increase and then decrease
2. First decrease and then increase
3. Generally increase
4. Generally decrease
5. I don’t know

GIF-12.gif

Title: Flows of development assistance for health

Source: https://medicalxpress.com/news/2017-04-widely-disparate-health.html 

Number of valid responses: 17

Average accuracy of the GIF: 80.4% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 94.1%)

1. It shows the total funds for different health areas over time
2. It shows the flow from different sources to different health areas
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization
5. I don’t know

2. What is the meaning of the chart? (accuracy: 94.1%)

1. It shows the flows of the funds from health areas, to channels, and to sources.
2. It shows the flow of the funds from sources to channels to health areas.
3. It shows the total funds of different channels.
4. It shows the total funds for different health areas.
5. I don’t know.

3. What do the colors of the flows represent? (accuracy: 52.9%)

1. Different channels
2. The amount of funding
3. The number of flows from sources to this channel
4. No meaning
5. I don’t know

GIF-13.gif

Title: Share of Google search interest for Batman and Superman in a period

Source: https://www.blog.google/outreach-initiatives/google-news-initiative/make-your-own-data-gifs-our-new-too/ 

Number of valid responses: 18

Average accuracy of the GIF: 81.5% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 94.4%)

1. It shows the search interests of two competing topics over time.
2. It switches between two competing topics.  
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know.

2. What does the blue bar width mean? (accuracy: 83.3%)

1. The share of search interest in Batman
2. The share of search interest in Superman
3. The number of fans of Batman
4. The number of fans of Superman
5. I don’t know.

3. What is the overall trend in the GIF? (accuracy: 66.7%)

1. Batman generally had more searches than Superman, but there was a time Superman caught up and had more searches than Batman.
2. Batman generally had more searches than Superman, but there was a time Superman caught up and still had fewer searches than Batman.
3. Batman generally had more fans than Superman, but there was a time Superman caught up and had more fans than Batman.
4. Batman generally had more fans than Superman, but there was a time Superman caught up and still had fewer fans than Batman.
5. I don’t know

GIF-14.gif

Title: Flight Patterns after Brussels Attacks

Sources: http://lenagroeger.com/datagifs/#/9 

Number of valid responses: 27

Average accuracy of the GIF: 72.8% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 88.9%)

1. It shows the flight patterns in Belgium over time.
2. It shows the flight patterns of each city in Belgium .  
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know

2. What happened in the GIF? (accuracy: 51.9%)

1. It shows the flight patterns on a regular day.
2. It shows there are more flights from a certain time.
3. It shows that many flights went to Belgium from a certain time.
4. It shows many flights rerouted from a certain time.
5. I don’t know.

3. What does the number on the top-left corner show? (accuracy: 77.8%)

1. It shows the time and does not change color.
2. It shows the time and changes to red when an event  happened.
3. It shows the number of flights and does not change color.
4. It shows the number of flights and changes to red when an event happened.
5. I don’t know.

GIF-15.gif

Title: The 150 largest Cities in the World

Sources: http://bl.ocks.org/nbremer/8df57868090f11e59175804e2062b2aa
Number of valid responses: 23

Average accuracy of the GIF: 34.8% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 13.0%)

1. It shows the evolution of the largest cities in the world over time.
2. It compares the largest cities in each country.  
3. It narrates a story about the largest cities.
4. It animates the creation of a static visualization.
5. I don’t know

2. What does the size of the small circle on the map mean? (accuracy: 47.8%)

1. The area of the city
2. The number of the cities in the area
3. The population of the city
4. No meaning
5. I don’t know

3. What does the big circle at the beginning show? (accuracy: 43.5%)

1. The area of all the largest cities
2. The number of all the largest cities
3. The total population of all the largest cities
4. The origin of all the largest cities.
5. I don’t know

GIF-16.gif

Title: The Population Pulse of a Manhattan Week

Sources: https://untappedcities.com/2018/05/15/fun-maps-see-the-population-of-manhattan-hour-by-hour/ 

Number of valid responses: 25

Average accuracy of the GIF: 84.0% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 92.0%)

1. It shows the population distribution in Manhattan in a week
2. It shows each region with the population
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know

2. What is the meaning of the bar height? (accuracy: 88.0%)

1. The region area
2. The population
3. Time
4. No meaning
5. I don’t know

3. What does the color of the bars mean? (accuracy: 72.0%)

1. The region area
2. The population
3. Time
4. No meaning
5. I don’t know

GIF-17.gif

Title: Discover new market

Sources: https://d31ogkc41ydx1q.cloudfront.net/wp-content/uploads/2019/06/2-discover-new-markets.gif 

Number of valid responses: 32

Average accuracy of the GIF: 46.9% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 28.1%)

1. It shows the distribution of markets over time.
2. It shows the markets at different positions.
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know.

2. What is the meaning of the red location icons? (accuracy: 75.0%)

1. Markets with best customers
2. Addressable markets
3. New markets it targets
4. No meaning
5. I don’t know

3. What is the meaning of the transparent circles? (accuracy: 37.5%)

1. Markets with best customers
2. Addressable markets
3. New markets it targets
4. No meaning
5. I don’t know

GIF-18.gif

Title: Balance of power

Sources: https://aemstatic-ww2.azureedge.net/content/dam/pei/online-articles/2017/07/ElecConsumption_BackForth2a%20(1).gif 

Number of valid responses: 26

Average accuracy of the GIF: 57.7%

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 65.4%)

1. It shows the power consumption of 20 countries over time.
2. It shows the power consumption and territories of 20 countries
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know.

2. What does the scaled world map represent ? (accuracy: 76.9%)

1. It scales to the territory of each country.
2. It scales to the energy consumption of each country.
3. It scales to the energy population of each country.
4. It scales to the population of each country.
5. I don’t know

3. What does the color mean? (accuracy: 30.8%)

1. The energy consumption of each country
2. The energy generation of each country
3. The territory of each country
4. The population of each country
5. I don’t know

GIF-19.gif

Title: Here’s How Far the World Is From Meeting Its Climate Goals

Sources: https://twitter.com/nytgraphics/status/927516354982371328 

Number of valid responses: 23

Average accuracy of the GIF: 37.7% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 47.8%)

1. It shows the emissions of CO2 over time.
2. It shows the emissions of CO2 under different conditions.  
3. It narrates a story about the chart with annotation.
4. It animates the creation of a static visualization.
5. I don’t know

2. What does the line chart show? (accuracy: 43,5%)

1. It shows the historical emission of CO2.
2. It shows the future emission of CO2.
3. It shows the historical and future emission of CO2.
4. It shows the relation between the emission of CO2 and the temperature.  
5. I don’t know

3. Which requires the lowest CO2 emission? (accuracy: 21.7%)

1. Current emission
2. Paris agreement
3. Below 2℃ of the warming
4. All the above three.
5. I don’t know.

GIF-20.gif

Title: Why racial segregation has persisted in multiple cities?

Sources: https://twitter.com/chingos/status/1067823844864471045

Number of valid responses: 20

Average accuracy of the GIF: 15.0% 

Encoding:

1. What is the animation showing with the GIF playing? (accuracy: 5.0%)

1. It shows the evolution of racial segregation over time.
2. It compares the cities with different segregation.  
3. It narrates a story about the chart with annotation.
4. It animates the creation of the final bubble chart.
5. I don’t know

2. What does the  45 degree line in the chart show? (accuracy: 35.0%)

1. It shows the cities where schools are more segregated than neighborhoods in races.
2. It shows the cities where neighborhoods are more segregated than schools in races.
3. It shows the cities where the segregation of schools equals that of neighborhoods in races.
4. It shows the equal number of schools and neighborhoods.
5. I don’t know

3. What is the overall distribution in the GIF? (accuracy: 5.0%)

1. In most cities, schools are more segregated compared to neighborhoods.
2. In most cities, neighborhoods are more segregated compared to schools.
3. In most cities, the segregation of schools equals the neighborhoods.
4. In most cities, there is no segregation in schools and neighborhoods.
5. I don’t know.