Iota Reliability Concept of the Second Generation : Measures for Content Analysis Done by Humans or Artificial Intelligences / / Florian Berding and Julia Pargmann

• Autore: Berding Florian
• Pubblicazione: Berlin, Germany : , : Logos Verlag Berlin GmbH, , 2022
• Descrizione fisica: 1 online resource (ii, 143 pages) : illustrations
• Disciplina: 006.3
• Soggetto topico: Artificial intelligence; Content analysis (Communication)
• Persona (resp. second.): PargmannJulia
• Nota di contenuto: 1 Introduction. 1 -- 2 Summarizing the Iota Concept of the First Generation. 7 -- 3 Introducing Iota Concept of the Second Generation. 11 -- 3.1 Refining the Assignment Error Matrix, the Alpha and the Beta Elements11 -- 3.2 Introducing the Chance-Correction for Alpha and Beta Elements13 -- 3.3 Refining Iota 15 -- 3.4 Reliability on the Scale Level18 -- 3.5 Assumption of Weak Superiority .20 -- 3.6 Comparing the Iota Concepts 21 -- 4 Estimation and Log-Likelihood . 25 -- 5 Simulation Study I 31 -- 5.1 Hypotheses and Design of Simulation Study I31 -- 5.2 Results of Simulation Study I .34 -- 5.2.1 Data Description and Preparation 34 -- 5.2.2 Accuracy of the Estimated Assignment Error Matrix and Categorical Sizes.35 -- 5.2.3 Accuracy of the Derived Reliability Measures .40 -- 5.3 Summary of Simulation Study I47 -- 6 Simulation Study II. 49 -- 6.1 Research Questions and Design of Simulation Study II 49 -- 6.2 Results of Simulation Study II 56 -- 6.2.1 Overview.56 -- 6.2.2 Analyses of the Deviation Between True and Estimated Sample Association/Correlation.62 -- 6.2.3 Analyses of Type I Errors.68 -- 6.2.4 Analyses of Type II Errors72 -- 6.2.5 Analysis of Correct Classification of Effect Sizes74 -- 6.3 Summary of Simulation Study II.78 -- Iota Reliability Concept of the Second Generation 7 Simulation Study III 85 -- 7.1 Design of the Study85 -- 7.2 Results of Simulation Study III .86 -- 7.2.1 Overview.86 -- 7.2.2 Potential Cut-off Values and Certainty of Reliability Effects for Deviation.89 -- 7.2.3 Potential Cut-off Values and Certainty of Reliability Effects for Type I Errors.92 -- 7.2.4 Potential Cut-off Values and Certainty of Reliability Effects for Classifying Effect Sizes 95 7.3 Summary of Simulation Study III97 -- 8 Discussion 101 -- 8.1 Conclusions101 -- 8.2 Examples for Practical Applications of iotarelr.105 -- 8.2.1 Overview.105 -- 8.2.2 Checking the Quality of Codings of New Raters 106 -- 8.2.3 Checking for Bias and Different Guidance of a Coding Scheme .110 -- 8.2.4 Improving the Quality of Codings112 -- 8.3 Limitations and Further Directions 113 -- References. 114 -- Appendix A - Confidence Intervals -- Appendix B - Illustrations of the Relationship Between Reliability and the Deviation Between the True and Estimated Association/Correlation -- Appendix C - Global Indices of Model Fit in Simulation Study II -- Appendix D - Global Indices of Model Fit in Simulation Study III.
• Sommario/riassunto: In educational settings, analyzing textual data via content analysis is a popular research method. The data is a valuable source of information as it offers deep insights into learning and learning outcomes. In practice, it can be used to improve classroom diagnostics and instruction. Nowadays, technology such as learning analytics can be used for the same cause. For both purposes, reliable research instruments are needed. Content analysis, often the measure of choice, is required to meet quality criteria such as objectivity, reliability and validity. However, some of the reliability measures most frequently used have lately been discussed controversially, indicating that there is room for improvement. The first generation of the Iota concept caters to the idea of improved reliability measures for content analysis done by humans or artificial intelligences. In this book, the authors introduce a refined measure: The Iota concept of the second generation. In contrast to pre-existing measures, second generation Iota can for example a. provide insights into the reliability of every single category of a scale and how a coding scheme may produce bias, b. provide rules of thumb for evaluating content analysis and c. provide possibilities for data replication and error-corrected data. This book is structured as a guide for researchers that want to learn more about the mechanics and details of the Iota concept or use it as the reliability measure of choice in their research.
• Altri titoli varianti: Iota Reliability Concept of the Second Generation
• Titolo autorizzato: Iota Reliability Concept of the Second Generation
• ISBN: 3-8325-5581-1
• Formato: Materiale a stampa
• Livello bibliografico: Monografia
• Lingua di pubblicazione: Inglese
• Record Nr.: 9910647264503321
• Lo trovi qui: Univ. Federico II
• Serie: Berufs- und Wirtschaftspädagogik.