CHI2022 WrapUp

After a few weeks, we take the opportunity to look back and reflect on the CHI experience. 

CHI and COVID

More people reported cases of COVID infections than we had hoped. We always knew COVID infection was a risk at the conference, and took the steps we could – including having a strong online option for the conference.

As of writing, 127 people reported positive cases to the ACM. Of course, that number is likely higher – given that people may not have reported. Without asking people to report negative tests as well, we can’t really even generate a response rate. At the end of the day, it doesn’t matter if we’re talking about a 5% or 10% infection rate. Anyone getting sick is bad, and some realized their worst fears about in-person conferences. For everyone, we’ve reflected deeply on every cough, every moment of anxiety, and every hour stolen from already absent loved ones.

We could go back and forth about the steps we took to prevent COVID at the conference, or where the infections stemmed from. We’re not epidemiologists, so we’d be bad at that. There are really only two questions that matter: Should we have had an in-person conference? What should the next in-person conference do?

In terms of whether we should have had an in-person conference, we stand by our decision to have an in-person version of the conference. Everyone should be able to make the choices to protect their health without sacrificing their careers, so we need to put more thought into meaningful online experiences.  For advice for future conferences, it’s important to think about that hybrid experience and how the online experience creates value for people. In terms of future conferences meeting in person, we hope they have less infection than we saw. However, this may be the world we live in for a while – where we have to do our best to protect the safety of people in person, while realizing we have limited abilities to control outcomes.

How Hybrid Went

Our goal this year was to privilege a form of hybrid that emphasized synchronous presentation and interaction. We also wanted to allow both presenters and audience maximum flexibility, as we realized people would be deciding often at the last minute whether they would be in person or not. As with all decisions, we also prioritized accessibility as one of the major criteria that our system should have.

Frankly speaking, nothing worked as well as we hoped. The system was too clunky, and the flexibility we hoped for from AV turned mostly into points of potential failure. What worked and what didn’t? 

• WORKED: The SIGCHI Progressive Web App. This app worked consistently and well. With some minor additions, it really could be a center for a conference’s hybrid strategy.
• DID NOT WORK: Online platforms. People often got confused if they were in Hubb, or Zoom, or what. We had hoped we had found one vendor that would create a single online experience, but when that didn’t work, we had to pivot to other solutions in the moment to make our interaction goals work.
• MIXED BAG: The AV set up. The AV industry has been hit hard by the pandemic. In addition, to save money we had hoped to use less staff and more volunteers. Overall, we spent 3x more on AV than any previous CHI – which is necessary because of the synchronous hybrid goal. Things got better quickly, but it was a rocky start.
• WORKED: Discord. While it was “yet another platform”, it was free and worked consistently well.

There are teams at both the CHI Steering Committee and the SIGCHI Executive Committee post-gaming hybrid experiences and trying to make recommendations for the future. The scale of CHI is a challenge on many levels, but there are some consistent issues all conferences are facing.

Comportment

This is an issue that surprised us – breakdowns in how we treated each other.

It’s interesting to think why this is. To some extent, norms have changed over the pandemic, often for the better. For example, people are more willing to say something if they feel uncomfortable. People have a better sense of what harassment is and feel more like they can report when conversations cross a line. That’s all good.

Another effect might be the general anxiety we feel from the ongoing pandemic expressing itself on underserving targets. The student volunteers were often the targets of bad behavior in the rooms they were helping to support. That’s never acceptable. Our hope is that this was unusual behavior shaped by stress, because we will not ever condone bad treatment of our SVs.

While some norms have seen positive change over time, some have changed for the worse. One of those we saw was the tendency to state things in sensationalist or absolutist terms in order to receive approbation. We see this all the time on social media, where signals of attention are poor so some people post the most extreme version of their message in order to get a reaction. 

Everyone deserves to feel safe at the CHI conference. No one should feel demeaned because of their country of origin, their research area, or their institutional affiliation among other things. While it’s a good thing that we raise critical points about issues in the tech industry, it’s possible to do so without demonizing or trying to score points. Whatever issues one has with any institution, the people at our conference who are part of that institution still deserve to feel safe and not demeaned. We will work better in the future to help our keynotes understand these values.

We continue to grow and learn to act better. This year tested and executed on our anti-harassment policies, but in that test we saw positive outcomes. Justice and safety for our members is a process, but we are encouraged by what parts of that process we saw.

Survey Results

A total of 749 people responded to the request to fill out the post-conference survey. 582 reported attending in person, 148 virtual, and 19 didn’t report their mode of attendance. This means that response rates for the virtual mode of the conference (~8%) are quite a bit worse than for those who attended the in-person mode (~29%). We don’t know why the response rates are so different.

One of the most essential questions we ask, and really the essential dependent variable we ask year-to-year is “How valuable was your CHI experience?” This is measured on a 5 point Likert scale where 1 is “Not at all valuable” and 5 is “Very Valuable”. When we collapse the modes of the conference, overall the ratings were fairly high. The median score is 4, and the mean is 4.27. The bar chart below shows the overall pattern.

The numbers look different when we compare the virtual and in-person modes. 

In-person:

Virtual: 

As usual with CHI, people value different parts of the conference. The chart below shows different ratings by different parts of CHI. It’s impossible to experience ALL of CHi, and in general people seemed pretty happy with the parts in which they participated.

We asked people about how CHI2022 dealt with COVID-19. In general, people felt like the organizers did a decent job with the measures in place.

Q9. Do you agree or disagree that organizers adequately considered COVID-19 protocols at the convention center?

In the open-ended responses, two possible different measures were mentioned. One was to encourage daily testing by participants. The second was to not offer food at breaks and receptions, given that sharing food obviated to some extent the benefits of masking. Participants also felt like they weren’t as careful as they could have been.

Q11. How informed did you feel regarding the conference’s COVID-19 protocols before attending the conference?

There’s a lot more to the survey, all of which is being considered by the CHI Steering Committee and SIGCHI Executive Committee. For instance, having a cash bar was controversial. Some people loved it, some people felt like it didn’t go far enough in removing alcohol from the conference, and some people hated the move as punishing all attendees while not really preventing harassment. 

HECKUVA RIDE

In summary, there are some things we’d like to change about how CHI2022 happened, given the clarity of being after the fact. However, we are also proud of the event that happened given the uncertainty, complexity and sheer difficulty of putting the event together during the pandemic. We want to end by appreciating the incredibly hard work of all of the volunteers who put time into the conference. This was very much like organizing 2.5 conferences, and it could not have happened without extraordinary effort by our organizing team.