User Experience Insights from Open Data Customers
At Socrata we love data, not just the data that lives in our products but also the data about our products and the data about the people who use our products. If you're interested in this data and our user experience research efforts at Socrata, read on.
Study Details
In September 2015, we sent out a survey to our customers with the goal of systematically assessing their goals, their usage, and their feedback on our products. We were hoping to hear from 30-50 customers, especially with a variety of needs and behaviors. We received a total of 49 responses. Thank you to every customer who took the time and effort to share your responses with us!
One of the goals of this survey was to make sure we heard from our less active and less vocal customers. When we design and build our products we want to ensure that all of our customers and users are represented, not just the ones who are actively providing feedback (but we do love that feedback!). And we met that goal: About half of the respondents of this survey use our products less than once a week, and about half of them contact Socrata less than once a month.
A quick note about the methodology: In order to conduct valid and representative UX research we apply standardized qualitative research methods, we represent values as approximations, and we triangulate our insights across various methods and sources. More details here.
Insights on the Customer Usage
We tend to think of our customers as the ones who do the most publishing and analysis. While all of the survey respondents said that they do publish data and analyze data, it turns out that publishing data is one of the least common activities, relative to the other activities we asked about. Customers are more frequently browsing and organizing their data. This finding is a great reminder that even customers (not just end users) want seamless consumption experiences.
Question: How often do you conduct the following activities?
Customer Goals and Outcomes
We asked customers to rank a set of predetermined goals based on their own organizational responsibilities. The top goal was supporting data driven decisions, followed by increasing operational effectiveness and increasing transparency. The following three were about impacting citizens.
Question: When thinking about your responsibilities, which of the following goals do you think you should target? Drag the goals into the bucket and order them by priority
Customer Success Stories
We also asked customers to provide stories about the value that they’ve experienced from open data. Here are some representative responses from each of the categories we uncovered:
Increasing operational efficiency
“In some cases, staff time necessary to respond to Freedom of Information Act requests has been significantly reduced, while the people making the requests actually get their information faster.”
Increasing transparency
“Increased transparency in govt operations has resulted in increased public trust and acceptance of the challenges faced by govt in providing services. Issues that have come into the public eye have been reported much more fairly, with acknowledgement of contributing factors to the situation.”
Enhancing citizen experiences
“Drivers are informed about conditions on state highways thanks to public data improved with private ingenuity.”
Supporting data driven decisions and storytelling
“Average earnings by college graduates by degree and level of education and Medicare cost data have been valuable to citizens. They are able to make more informed decisions about their education and health care investments because this data historically has been very difficult to obtain.”
“We can now begin to share our story through the use of data with citizens.”
Providing metrics
“Anyone (citizens, policymakers and others) can get unfiltered data from the city. We use this open, unfiltered data in our presentations and discussions about the city business improving credibility of the city and the information itself.”
Providing economic value
“Having an Open Data portal has allowed us to forge relationships with private businesses... This in turn gives us the opportunity to find out, from their point of view, what data is useful.”
Comparing Goals with Outcomes
We conducted a qualitative analysis of the success stories by coding and theming the responses. You'll see those listed on the right side of the image below, along with a number representing the number of times that this category occurred across all success stories. Participants could provide up to 3 stories, which is why the value of total N is more than the total number of participants.
The first insight is that our top customers goals are represented by the most common customer goals. We also saw two interesting shifts in the data.
One, is that data driven decisions was not one of the most common stories. There’s several possible reasons for this – one might be that our customers lack a feedback loop, so they’re not aware of the outcomes. A second might be that data driven decision making doesn’t make as great of a story, because it’s actually a means to an end (a great end!), that's represented by other values here like saving time or money, or providing value for citizens.
The second shift is around operational efficiency. We were amazed and excited to hear such an incredibly large number of success stories around this. It’s something that we’ve underestimated and we’re excited to elevate it in our own storytelling. The degree to which we are helping our customers increase operational efficiency is an exciting outcome from this research.
Constructive Feedback and Takeaways
That's all great, but how can we improve the open data experiences for our customers and their stakeholders? We asked customers to provide feedback through a set of four different questions:
- What are your biggest challenges in working with open data?
- What are your biggest usability challenges in working with Socrata products?
- What are some of the most common questions that your stakeholders have of the data?
- If you could make three wishes for open data what would they be?
The responses to these four questions help us understand both the challenges and the unmet needs of customers and stakeholders. Here's how we'd summarize the top three categories of responses.
Improve the quality of my data
This represents the 3rd most common category of feedback. In particular, customers want us to provide tools to improve the quality and standardization of data, improve the ease of use of charting experiences, and improve the ease of use of mapping experiences. This is solid customer feedback - standardization practices and visualization tools are hot on our list as well.
Save me time
This represents the 2nd most common category of feedback. In particular, customers want us to improve performance, provide more seamless data editing and updating experiences, improve the onboarding experiences, and provide more support resources. This is also important customer feedback to us. We're excited that our new backend, among other new investments, will save our users a ton of time.
Help me get others on board the open data train
Internal adoption across departments and agencies represents our customer's biggest barrier to success. Here are some of the comments we heard [emphasis added].
“It’s a priority to talk about it and say we are doing it, to make noises that sound like open data--but it’s not a priority to actually do something that people care about…"
“It’s not part of our regular routines. It’s like if you had to use a separate keyboard for vowels, and that keyboard was in a vault in another building, and you had to speak Norwegian to get in.”
"[Others tell us...] "If we share that data and it becomes popular, we don’t have the staff to support it. It is important that we make it challenging to access so we can keep use of it to a manageable level."”
“When we try to do anything big, like budget data or something like that, people get worried about the implications of real transparency.”
“[Others tell us...] "We don’t want to share that data because we don’t know what people will do with it. They could use it to misrepresent us. They could use it to draw the wrong conclusions. We need to control the conclusions and not share the underlying data."”
It needs to be a priority in practice (not just in speech), it needs to be a part of everyday jobs and incredibly efficient, and we need tools to help guide outcomes and conclusions from the data.
Summary and Takeaways
We were very excited to hear all the incredible success stories from our customers, and delighted by the number of stories representing gains in operational efficiency.
We also very much value the customer feedback to improve the quality of the data and the efficiency of the experiences. These insights will help us prioritize future investments. As for the challenges on internal adoption, this isn't just a marketing issue. Within the Socrata Product team we're investigating:
- How can data publishing be incorporated into everyday tasks?
- Which tasks can be (more) automated?
- Can we share rewarding and valuable feedback with publishers?
- Can the uses of the data be more transparent such that the process is not feared?
We are eager to create solutions to better meet the needs of our customers and their stakeholders.
Tell us what surprised you! Send us your questions. And collaborate with us in future research. We look forward to learning more from a broader range of customers and a broader range of end users. You can reach me at Tira.Schwartz@Socrata.com.
-Tira Schwartz, Principal UX Researcher