Insights from Data

Photo credit: Vojtech Okenka from Pexels

Having worked as an academic, as a Data Scientist, as a Business Intelligence Specialist as well as doing a lot of consulting I have learnt a lot about the way we use data, and the way people perceive “statistics” (whatever that word really means).

Sometimes, a problem is so straightforward it only requires technical input (data processing) and it’s solved. More often though, a lot more interpretation is needed. Often, the problem is not so much about the “statistics” but about the study design. But plenty of times, people have seen some data and the problem is one of interpretation. People tend to prefer concise answers with no hedging. However, that is rarely possible. In part this is cultural; we have to accept that decisions need to be made in the absence of perfect knowledge. All we can do is to challenge some of the most egregious interpretative errors (and that is hard enough). Some of these fallacies are actually hard baked teachings in a lot of statistics text books and courses and it is so much harder to correct them after the event. Sometimes people approach me with an open mind; sometimes they have been so misleady about the nature of statistics that there is nothing I can do to help. However, sometimes, I found good answers to the problems and questions that got thrown my way. Sometimes, I got asked good questions I couldn’t answer. I have set up this blog as a reflective effort to try and summarise that experience and see if I can find any consistent themes.