I believe this is a great piece of work, so I asked Elizabeth and Peter if they would be willing to expand a bit on the topic here at the Subculture. Really, we could all use a little more discipline when it comes to decisions (like why am I drinking Dr. Pepper at 7:30 AM...whazeva ....don't judge me :D) Also, CalSAE's IDEAL(TM) program on June 24 will feature a call in from Elizabeth and we are looking so forward to that!
So with no further ado:
Why now? Was there something specific that caught your attention that led you both to write and release this paper?
It's a topic that has interested both of us for some time (Peter, for instance, presents on the topic of associations' data frequently at conferences). The impetus to do it NOW came from the fact that we recognized that associations are running well behind business when it comes to evidence-based decision-making, and we wanted to bring attention to the topic to see if we could help fix that.
What do you think is the most pressing issue for associations to address in regards to how they go about evaluating data and making decisions?
The MOST important thing is to ask better questions. As we detail in the whitepaper, too often, we choose the questions we ask by what we can easily answer given the data at hand, rather than focusing on the real, underlying issue. As a result, we manage to what we can measure (easily) rsather than to what we should be measuring. A lot of this is because we're reluctant to upgrade outdated legacy software or to consolidate fragmented data systems.
Can a strong "decision making process" overcome volunteer resistance to making better decisions, faster? If not, what can?
That is definitely a problem, not least of which because our Boards tend to be representative rather than competency based, and because we tend to fall prey to the narrative fallacy, where we feel compelled to construct stories around data and then force any additional data into that story, whether or not it fits. Choosing to decide based on evidence requires equal commitment from volunteer leaders and senior staff members to examine data, to hold out for more potential explanations of that data, and to test the hypotheses that result. It's a more intensive process than just going with what feels right based on limited information and time, but it does also bring about better outcomes.
In terms of tools, have you seen any AMS providers in the space embracing an expanded data collection and analysis role or are there outside tools we should be using?
We would suggest that systems integration, rather than any specific tool, should be the priority. The best tools are functionally useless if they don't talk to each other gracefully. As one of our case studies, Guillermo Ortiz De Zarate from NCARB noted, NCARB will not consider any new systems that do not include an API (application programming interface) that integrates with their existing systems.
I noted you don't discount intuition as part of the decision making process. Can you expand a little bit on the role intuition plays in a robust data environment?
It's the "a-ha!" moment that we've all experienced. And there's a pattern to it: you take in information, data, evidence, research, etc. Then you let your brain process all that at a subconscious level. All of a sudden, the picture snaps into focus. This is the source of, "I get my best ideas in the shower," or "while running," or "maybe I better sleep on this." As we note in the whitepaper, this is definitely a skill that develops, and the best way to improve it is to get more inputs (in other words, use more data!), to practice (in other words, make decisions), and get feedback (either from a mentor or by systematically tracking what happens with your decision-making processes and outcomes). Using intuition is not about abandoning data, but rather about acknowledging the symbiotic power of a data/intuition (or experience) partnership.
How can we get better at pattern-recognition? Are there ways to visualize our data that will lead to better breakthroughs than pie charts and numbers a' la "The Visual Miscellaneum?"
Software products such as Tableau and Qlikview make data patterns more obvious, but, like any filter, have the potential to create false positives (or negatives). Effective pattern recognition requires the diligence to confirm the validity of the underlying data (i.e., does the emerging picture make sense) and to accurately distinguish correlation from cause.
Big data will only become more important. What are the top three issues you believe big data will help us grapple with?
- Minimizing the impact of the narrative fallacy explained above
- Broadening our understanding of the impact and interaction of multiple variables in any given situation
- Helping us realize there is no such thing as the perfect decision
In spite of our best efforts, we can still make mistakes. How do good decision making processes help us minimize or recover from mis-steps?
The nature of evidence-based decisions is that there's a clearly defined rationale that the decision-makers articulate and share (i.e., a "paper trail"). When things do go awry, it is much easier to identify and address the specific error(s) in a rigorous way, rather than engaging in the type of finger-pointing and obfuscating that often results from purely gut-based strategies.
Do you have any advice for executives who want to move from a "shoot from the hip" volunteer culture to a more rigorous, data-driven one?
Those of us who've gone through the CAE process remember the mantra, "volunteer to volunteer, staff to staff, member to member." As we noted above, the only way this will work is with firm commitment to evidence-based decision making at both the volunteer leader (i.e., your Board chair) and senior staff, (i.e., your CEO) levels. Your association must establish a comprehensive, systems-based approach to data management and provide the appropriate tools and access to those responsible for making decisions at all levels. No silos, no territorialism, no information hoarding - everyone needs transparent access to key association data and the training and management/leadership support to follow where it leads, even if it takes you far from the way you've always done things.
Download your free copy of "Getting to the Good Stuff: Evidence Based Decision Making for Associations".