The week after Thanksgiving we spent our time in class discussing trustworthiness in research because as our structured abstract at the end of class notes - trustworthiness is a big, important, problematic, critical topic.
I think this was the perfect targeted investigation to work on because my thoughts throughout this week and last have been - yeah I am at the point now where I have been introduced to the tools necessary to evaluate research papers and as I navigate learning how to read research for me one of the major questions I have after reading a paper is do I trust it?
- Do I trust the work to share the design implications with my mom?
- Do I trust the work to share it with a friend?
- Do I trust the methodology to follow a similar methodology in my research?
- Do I trust the researcher's ethics to trust their work?
- Do I trust the work that if I were peer-reviewing a paper before publication I believe it deserves to be shared with the world?
- Do I trust the arguments the author's present given the data they present?
- Do I trust their research quality?
In order to begin to answer these questions, we needed to learn how to evaluate trustworthiness. Is it a gut feeling? Definitely not completely, but probably somehow influenced. There are many factors that influence your analysis of trust and quality in the research. Compared to prior investigations, this week I feel more at ease with the ambiguity around evaluating trustworthiness because I feel as if I have the tools to approach this process.
As I begin to publish my own words, thoughts, and research it is critical that I ask myself the type of questions Walther et al suggest. It is critical that as I go through my research project rotations I am asking myself along the way - do I trust my process? If I explaining myself to myself would I trust the work? the methods? the analysis? Am I letting the data drive the results? How are my researcher biases coming into play.
Although recognizing no work is perfect is crucial so that you do not overwhelm yourself, setting a high standard for quality and trust in research is important for all involved in the research, the researcher, the consumers of the paper, those affected by design implications that are taken up, and the humans affected by your design, work, and research study.
The concept of a process notebook is something that I want to take up and check with myself after this lesson that I am taking rigorous notes and reflecting as a researcher. Walther et al. note the importance of taking notes of what could be influencing the data collection process,what is the researcher's relationship to the participants in the study, and how do you acknowledge this limitation in interpretive engineering education research.
I'll end with a couple tweets that come out of my notes from class during the past week and a half of our targeted investigation into trustworthiness.
Evaluating quality is an instinctive process rather than a prescriptive process
Use a monster game to evaluate validity in your work!
What is the limit to how much you can inform about the research process of a paper?
Understanding what counts as a rigorous systems paper motivation section is messy
I can ask myself "Why do I trust this research contribution?"