LEADER 05332nam 2200421 450 001 9910647264503321 005 20230324073043.0 010 $a3-8325-5581-1 035 $a(CKB)5580000000508725 035 $a(NjHacI)995580000000508725 035 $a(EXLCZ)995580000000508725 100 $a20230324d2022 uy 0 101 0 $aeng 135 $aur||||||||||| 181 $ctxt$2rdacontent 182 $cc$2rdamedia 183 $acr$2rdacarrier 200 10$aIota Reliability Concept of the Second Generation $eMeasures for Content Analysis Done by Humans or Artificial Intelligences /$fFlorian Berding and Julia Pargmann 210 1$aBerlin, Germany :$cLogos Verlag Berlin GmbH,$d2022. 215 $a1 online resource (ii, 143 pages) $cillustrations 225 1 $aBerufs- und Wirtschaftspa?dagogik 327 $a1 Introduction. 1 -- 2 Summarizing the Iota Concept of the First Generation. 7 -- 3 Introducing Iota Concept of the Second Generation. 11 -- 3.1 Refining the Assignment Error Matrix, the Alpha and the Beta Elements11 -- 3.2 Introducing the Chance-Correction for Alpha and Beta Elements13 -- 3.3 Refining Iota 15 -- 3.4 Reliability on the Scale Level18 -- 3.5 Assumption of Weak Superiority .20 -- 3.6 Comparing the Iota Concepts 21 -- 4 Estimation and Log-Likelihood . 25 -- 5 Simulation Study I 31 -- 5.1 Hypotheses and Design of Simulation Study I31 -- 5.2 Results of Simulation Study I .34 -- 5.2.1 Data Description and Preparation 34 -- 5.2.2 Accuracy of the Estimated Assignment Error Matrix and Categorical Sizes.35 -- 5.2.3 Accuracy of the Derived Reliability Measures .40 -- 5.3 Summary of Simulation Study I47 -- 6 Simulation Study II. 49 -- 6.1 Research Questions and Design of Simulation Study II 49 -- 6.2 Results of Simulation Study II 56 -- 6.2.1 Overview.56 -- 6.2.2 Analyses of the Deviation Between True and Estimated Sample Association/Correlation.62 -- 6.2.3 Analyses of Type I Errors.68 -- 6.2.4 Analyses of Type II Errors72 -- 6.2.5 Analysis of Correct Classification of Effect Sizes74 -- 6.3 Summary of Simulation Study II.78 -- Iota Reliability Concept of the Second Generation 7 Simulation Study III 85 -- 7.1 Design of the Study85 -- 7.2 Results of Simulation Study III .86 -- 7.2.1 Overview.86 -- 7.2.2 Potential Cut-off Values and Certainty of Reliability Effects for Deviation.89 -- 7.2.3 Potential Cut-off Values and Certainty of Reliability Effects for Type I Errors.92 -- 7.2.4 Potential Cut-off Values and Certainty of Reliability Effects for Classifying Effect Sizes 95 7.3 Summary of Simulation Study III97 -- 8 Discussion 101 -- 8.1 Conclusions101 -- 8.2 Examples for Practical Applications of iotarelr.105 -- 8.2.1 Overview.105 -- 8.2.2 Checking the Quality of Codings of New Raters 106 -- 8.2.3 Checking for Bias and Different Guidance of a Coding Scheme .110 -- 8.2.4 Improving the Quality of Codings112 -- 8.3 Limitations and Further Directions 113 -- References. 114 -- Appendix A - Confidence Intervals -- Appendix B - Illustrations of the Relationship Between Reliability and the Deviation Between the True and Estimated Association/Correlation -- Appendix C - Global Indices of Model Fit in Simulation Study II -- Appendix D - Global Indices of Model Fit in Simulation Study III. 330 $aIn educational settings, analyzing textual data via content analysis is a popular research method. The data is a valuable source of information as it offers deep insights into learning and learning outcomes. In practice, it can be used to improve classroom diagnostics and instruction. Nowadays, technology such as learning analytics can be used for the same cause. For both purposes, reliable research instruments are needed. Content analysis, often the measure of choice, is required to meet quality criteria such as objectivity, reliability and validity. However, some of the reliability measures most frequently used have lately been discussed controversially, indicating that there is room for improvement. The first generation of the Iota concept caters to the idea of improved reliability measures for content analysis done by humans or artificial intelligences. In this book, the authors introduce a refined measure: The Iota concept of the second generation. In contrast to pre-existing measures, second generation Iota can for example a. provide insights into the reliability of every single category of a scale and how a coding scheme may produce bias, b. provide rules of thumb for evaluating content analysis and c. provide possibilities for data replication and error-corrected data. This book is structured as a guide for researchers that want to learn more about the mechanics and details of the Iota concept or use it as the reliability measure of choice in their research. 410 0$aBerufs- und Wirtschaftspa?dagogik. 517 $aIota Reliability Concept of the Second Generation 606 $aArtificial intelligence 606 $aContent analysis (Communication) 615 0$aArtificial intelligence. 615 0$aContent analysis (Communication) 676 $a006.3 700 $aBerding$b Florian$01347746 702 $aPargmann$b Julia 801 0$bNjHacI 801 1$bNjHacl 906 $aBOOK 912 $a9910647264503321 996 $aIota Reliability Concept of the Second Generation$93084484 997 $aUNINA