What are some common pitfalls or biases in data interpretation?

You need to expose precision and accuracy to downstream consumers, gaps in data, anomalies and so on in order not to give the sense of false precision.

The importance of data quality can vary based off the project. Precise spatial accuracy may not be important in the case of reference maps or visualizing thematic data. But it becomes super relevant when you don’t want to break the water main with your excavator. I think we need to ask the question “how accurate does this need to be?” because we could lose “useful and on time” in pursuit of perfect.

It is crucial to recognise the unique challenges that come with collecting qualitative data. Effectively engaging the communities that you’ll collect the data from, will be critical in ensuring that you minimise participant biases such as habituation bias and social desirability bias. If possible, consider training community research assistants to assist with data collection and to minimise sponsor bias.

To be more specific, depending on the type of research being conducted, you will also need to ensure the accuracy of the measures you use to collect data. You cannot collect high quality, accurate data if the methods you use are flawed. With respect to measures, it is important to collect data using established measures that have established their reliability and validity. A lot goes into designing the right types of questions in the right ways to accurately discriminate between response types. Doing a survey monkey or using an obscure measure or one that you create without testing simply won't cut it in research.

In data analysis, the phrase "your data is only as good as the methods that produced it" is my guiding principle. It can be difficult to fully document and understand the requirements stakeholders need for a data collection program, but it is essential for success that business operations understand the data being produced. That is where we come in as data experts. We find the right equations and algorithms, create the right analytics environment, and align reporting and feed run times. Data experts need to know the data and the stack they are working in. We distill this information and use our knowledge of the business to develop a deep understanding of the data we produce. Being a data expert and a business partner is our superpower.

Qualitative data collection requires acknowledging the roles and principles of Community-Based Participatory Research (CBPR). Whether you’re conducting focus group discussions or interviews, you’ll need to be aware of participation fatigue and sometimes, focus on stopping collection when you achieve data saturation. Quantity in qualitative data collection , therefore, is not a “one-size-fits-all”. Regardless of the quantity, consider stopping when you’ve had enough data with responses that are similar and seemingly confirmatory.

Whether using modern data structures (like sketches and HyperLogLog) or sampling or other methods, you should always inflect the data products that are built with these datas to inform users as to the caveats based on how you could not expose the full row-level dataset.

You’d want to have enough data volume for representation of the different groups and to be able to execute accurate pattern recognition. However, to eliminate the random bias the dataset must be well balanced in its content, especially when it comes to ML training, your model will be as good as the data you use to train it.

Data sets that are too large can cause hyper sensitivity, in that the sample size can be so large that it will be likely to pick up on any small variation and resulting analyses will likely find that to be a statistically significant result due to the size of the data set, rather than this reflecting a true difference.

Adequate sample size, representative data, and careful consideration of dimensionality are very vital for meaningful insights. Efficient utilization of computational resources and appropriate tools ensures effective processing.

There are several approaches you can employ in analysing your qualitative data. Whether it’s thematic content analysis or framework analysis, you should prioritise methods that are conducive to your research aims and objectives.

One of the key aspects of thorough data analysis is defining the hypothesis accurately and correctly before deep diving into the data. Some hypothesis maybe confirmatory, which could be a pain point or an idea you’d want to confirm if it exists in the data. Second is exploratory, which represents finding patterns that you were not aware of today. This requires generating specific KPIs and information for different user segments and groups. Finally, benchmarking, comparing your correlation and causation with the similar datasets in your field to develop an understanding of the gaps and differences in nuances.

Charts don't exist in isolation to answer a single question, they develop the culture of an organization and factor into the design of metrics and the evaluation of data in the future. Additionally, chart bias toward familiar charts can lock you into approaches and limit innovation because familiar charts typically only present numerical precision and miss patterns that only exist in more complicated structures (like topological, hierarchical or geographical patterns).

By taking measures of community engagement to minimise biases such as social desirability bias, habituation bias and sponsor bias, you can improve your data quality. Data interpretation might also involve reviewing your field notes, and participant observations to note any confirmatory themes. Regardless of the approaches you use, be sure to communicate your data and it’s impact on policy to the members of the communities that you conducted your qualitative data collection. This in accordance with the principles of Community-Based Participatory Research (CBPR).

Maps can be bent to show or obscure information just like any other communicative medium. Scales, symbology, classification schemes, labels, font, colors can all be used to paint a picture of a particular narrative. People tend to trust maps. It is just as important to have a healthy skepticism of data shown on a map as it is to be think critically of the media you watch and listen to. An example of this is the Modifiable Areal Unit Problem or (MAUP) it refers to the issue of using different geographic units or scales in the analysis of spatial data. The choice of geographic unit can affect the statistical analysis and lead to different conclusions being drawn. Geographies can be modified to have the map reader draw different conclusions.

When summarising and interpreting your results, it is important in the final stages to zoom out by putting it into context of your research area. This is where we come back to situating our results in context of the research that has already been done in the area, the contribution the current research makes, along with its limitations, and suggestions for next steps in a future piece of research in the area.