Análise de Dados Ambientais
Análise de Dados Ambientais
Estatística aplicada, correlação, regressão, ANOVA, GLM/GEE, análise fatorial e modelagem ambiental para pós-graduação.
Introdução e aplicação de técnicas avançadas de análise estatística em estudos ambientais. Discussão e aprofundamento teórico-prático sobre a análise e interpretação de dados ambientais, incluindo métodos de análises descritivas, inferenciais, bi e multivariadas. Exploração de modelos estatísticos generalizados e suas aplicações em questões ambientais, abordando técnicas de moderação, mediação e análise de redes ecológicas. Utilização de métodos de regressão avançados, como regressão de Poisson e binomial negativa, para modelagem de eventos ambientais. Análise de sobrevida aplicada a estudos de longevidade de espécies e ecossistemas. Desenvolvimento de competências na criação e análise de bancos de dados ambientais, incluindo dados secundários, temporais e correlacionais. Aplicação de análise fatorial exploratória e confirmatória e modelagem por equações estruturais em dados ambientais, com enfoque no uso de ferramentas como JASP e FACTOR e Mplus. Capacitação para a interpretação, comunicação e publicação de resultados em revistas científicas, contribuindo para a tomada de decisões fundamentadas em conservação, gestão ambiental e políticas públicas.
Objetivo: Aprofundar o conhecimento sobre técnicas estatísticas avançadas aplicadas à análise e interpretação de dados ambientais. Desenvolver a capacidade de aplicar análises descritivas, inferenciais, bi e multivariadas em estudos ambientais, com foco em questões relacionadas à biodiversidade, qualidade do solo e mudanças climáticas. Analisar criticamente os modelos estatísticos generalizados e suas aplicações em estudos ambientais complexos, explorando as técnicas de moderação, mediação e análise de redes ecológicas.
Habilidades e Competências: Capacitar os alunos para a utilização de regressões avançadas e análise de sobrevida em contextos ambientais, com ênfase em modelagem de eventos e longevidade de espécies. Facilitar o desenvolvimento de competências para a criação e análise de bancos de dados ambientais, incluindo o uso de dados secundários, temporais e correlacionais. Promover a habilidade de aplicar análise fatorial exploratória e confirmatória, além de modelagem por equações estruturais, utilizando ferramentas como JASP e Mplus, para entender e explicar relações complexas entre variáveis ambientais. Preparar os alunos para a comunicação eficaz e publicação de resultados de pesquisas ambientais, contribuindo para a tomada de decisões informadas em conservação, gestão ambiental e sustentabilidade.
| Semana | Aulas | Assuntos Previstos | Slides |
|---|---|---|---|
| 1ª | 01–02 | Introdução e Etimologia Estatística; Taxonomia de Bloom. Paradigmas frequentista e bayesiano; níveis cognitivos aplicados à análise de dados. | 01, 02 |
| 2ª | 03–04 | Estatística Descritiva e Inferencial. Tendência central, dispersão, escore Z; normalidade (Shapiro-Wilk, K-S), Q-Q plots, transformações. | 03, 04 |
| 3ª | 05–06 | Bootstrapping e Bootstrap Avançado. Intervalos de confiança via reamostragem; técnicas avançadas e aplicações ambientais. | 05, 06 |
| 4ª | 07–08 | Correlação e Teste t. Pearson, Spearman, ponto-bisserial, parcial; H₀/H₁, erro tipo I/II, testes t para 1 e 2 amostras. | 07, 08 |
| 5ª | 09–10 | ANOVA de Uma Via e ANOVA Fatorial. Particionamento da variância; comparações múltiplas; efeitos principais e interações. | 09, 10 |
| 6ª | 11–12 | ANCOVA e ANOVA de Medidas Repetidas. Covariáveis; esfericidade; correções e interpretação dos resultados. | 11, 12 |
| 7ª | 13 | ANOVA — Visão Completa e MANOVA. Fatorial de medidas repetidas; testes multivariados e aplicações em conservação. | 13 |
| 8ª | 14–15 | Regressão e Qui-Quadrado. Regressão simples/múltipla; diagnóstico de resíduos; qui-quadrado 2×2, n×k, McNemar, Q de Cochran. | 14, 15 |
| 9ª | 16–17 | Associação e Dependência e Testes Não Paramétricos. Odds ratio, log-linear, correspondência; Mann-Whitney, Kruskal-Wallis, Friedman. | 16, 17 |
| 10ª | 18–19 | Escolha do Teste Estatístico e GLM/GEE. Fluxogramas de decisão; família exponencial; GEE para dados longitudinais e repetidos. | 18, 19 |
| 11ª | 20–21 | Detecção de Anomalias e Análise Bi e Multivariada. Séries temporais, outliers; PCA, redes ecológicas, centralidade, modularidade. | 20, 21 |
| 12ª | 22–23 | Análise Fatorial Exploratória e Construção de Instrumentos. Hull, número de fatores, rotação; elaboração de escalas e itens. | 22, 23 |
| 13ª | 24–25 | Adaptação e Validade de Instrumentos. Tradução e adaptação transcultural; validade de conteúdo, construto e critério. | 24, 25 |
| 14ª | 26–27 | Evidências de Validade e Introdução à TRI. AFC, invariância, MEE; Teoria de Resposta ao Item: conceitos e modelos. | 26, 27 |
| 15ª | 28–29 | Modelo de Rasch e Metanálise. Ajuste de itens, mapa de Wright; efeito combinado, forest plot, heterogeneidade, sobrevida. | 28, 29 |
Observação: Aulas às segundas-feiras, 08h–12h (4h/aula). Período: março a junho.
O plano completo (ementa, metodologia, avaliações, cronograma detalhado e referências) está disponível em:
Apresentação da disciplina, bases epistemológicas e paradigmas frequentista e bayesiano.
Níveis cognitivos e objetivos de aprendizagem aplicados à análise de dados.
Medidas de tendência central, dispersão, assimetria e curtose; escore Z.
Shapiro-Wilk, Kolmogorov-Smirnov, Q-Q plots, transformações e estratégias para não normalidade.
Intervalos de confiança, estimação e reamostragem para inferência robusta.
Técnicas avançadas de reamostragem e aplicações em dados ambientais.
Pearson, Spearman, ponto-bisserial, parcial, R-to-Z e distinção correlação–causalidade.
H₀/H₁, erro tipo I/II, poder do teste; teste t para 1 e 2 amostras; testes pareados.
Introdução à ANOVA one-way, solicitação da análise e descrição dos resultados.
Teoria, seleção da análise e descrição dos resultados da ANOVA fatorial.
Análise de covariância: introdução, prática e interpretação dos resultados.
Introdução, realização da análise e descrição dos resultados.
ANOVA fatorial de medidas repetidas, MANOVA e visão integrada.
Regressão linear simples e múltipla; diagnóstico de resíduos, seleção de variáveis e dummy.
Qui-quadrado 2×2, n×k, McNemar e Q de Cochran.
Phi, V de Cramér, odds ratio, risco relativo, log-linear e análise de correspondência.
Mann-Whitney, Kruskal-Wallis, Wilcoxon e ANOVA de Friedman.
Fluxogramas de decisão e critérios para selecionar o teste adequado.
Família exponencial, funções de ligação; GEE para dados longitudinais e medidas repetidas.
Teoria, pressupostos, equidispersão, prática (SPSS/R), interpretação do IRR e descrição.
Modelo superdisperso, regressão binomial negativa, comparação de modelos (AIC) e descrição.
Razão de prevalência vs. odds ratio, variância robusta, prática (SPSS/R) e descrição.
Indicadores de desmatamento, tendência, sazonalidade e detecção de outliers temporais.
PCA, análise de redes ecológicas, centralidade, modularidade e conectância.
Método Hull, número de fatores, rotação e interpretação.
Elaboração de escalas, itens e procedimentos de validação.
Tradução, adaptação transcultural e equivalência de medidas.
Validade de conteúdo, construto e critério; confiabilidade.
AFC, invariância de medida e modelagem por equações estruturais.
Teoria de Resposta ao Item: conceitos, modelos e aplicações.
Modelo de Rasch, ajuste de itens, mapa de Wright e aplicações.
Efeito combinado, forest plot, heterogeneidade e análise de sobrevida.
Introdução, censura, Kaplan-Meier, tábua de sobrevida, teste log-rank e como reportar.
Modelo de riscos proporcionais, Hazard Ratio, análise uni e multivariada, forest plot e descrição.
Planejando e conduzindo sua pesquisa
Althouse, A. D. (2016). Adjust for Multiple Comparisons? It’s Not That Simple. The Annals of Thoracic Surgery, 101(5), 1644–1645. doi:10.1016/j.athoracsur.2015.11.024
Gelman, A., Hill, J., & Yajima, M. (2012). Why We (Usually) Don’t Have to Worry About Multiple Comparisons. Journal of Research on Educational Effectiveness, 5(2), 189–211. doi:10.1080/19345747.2011.618213
Albers, C., & Lakens, D. (2018). When power analyses based on pilot data are biased: Inaccurate effect size estimators and follow-up bias. Journal of Experimental Social Psychology, 74, 187–195. doi:10.1016/j.jesp.2017.09.004
Anvari, F., & Lakens, D. (2021). Using anchor-based methods to determine the smallest effect size of interest. Journal of Experimental Social Psychology, 96, 104159. doi:10.1016/j.jesp.2021.104159
Brysbaert, M., & Stevens, M. (2018). Power Analysis and Effect Size in Mixed Effects Models: A Tutorial. Journal of Cognition, 1(1), Art. 1. doi:10.5334/joc.10
Kovacs, M., Ravenzwaaij, D. van, Hoekstra, R., & Aczel, B. (2021). SampleSizePlanner: A Tool to Estimate and Justify Sample Size for Two-Group Studies. MetaArXiv. doi:10.31222/osf.io/rm9dn
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4. doi:10.3389/fpsyg.2013.00863
Lakens, D. (2022). Sample Size Justification. Collabra: Psychology, 8(1), 33267. doi:10.1525/collabra.33267
Lakens, D., & Caldwell, A. R. (2021). Simulation-Based Power Analysis for Factorial Analysis of Variance Designs. Advances in Methods and Practices in Psychological Science, 4(1), 2515245920951503. doi:10.1177/2515245920951503
Lakens, D., Scheel, A. M., & Isager, P. M. (2018). Equivalence Testing for Psychological Research: A Tutorial. Advances in Methods and Practices in Psychological Science, 1(2), 259–269. doi:10.1177/2515245918770963
Maier, M., & Lakens, D. (2021). Justify Your Alpha: A Primer on Two Practical Approaches. PsyArXiv. doi:10.31234/osf.io/ts4r6
Perugini, M., Gallucci, M., & Costantini, G. (2018). A Practical Primer To Power Analysis for Simple Experimental Designs. International Review of Social Psychology, 31(1), Art. 1. doi:10.5334/irsp.181
Piaggio, G., Elbourne, D. R., Altman, D. G., Pocock, S. J., Evans, S. J. W., & the CONSORT Group. (2006). Reporting of noninferiority and equivalence randomized trials: An extension of the CONSORT Statement. JAMA, 295(1), 1152–1161. doi:10.1001/jama.295.10.1152
Shieh, G. (2017). Power and Sample Size Calculations for Contrast Analysis in ANCOVA. Multivariate Behavioral Research, 52(1), 1–11. doi:10.1080/00273171.2016.1219841
Sommet, N., Weissman, D., Cheutin, N., & Elliot, A. J. (2022). How many participants do I need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction. OSF Preprints. doi:10.31219/osf.io/xhe3u
Szucs, D., & Ioannidis, J. P. A. (2020). Sample size evolution in neuroimaging research: An evaluation of highly-cited studies (1990–2012) and of latest practices (2017–2018) in high-impact journals. NeuroImage, 221, 117164. doi:10.1016/j.neuroimage.2020.117164
Epskamp, S. (2019). Reproducibility and Replicability in a Fast-Paced Methodological World. Advances in Methods and Practices in Psychological Science, 2(2), 145–155. doi:10.1177/2515245919847421
Haroz, S. (2022). Comparison of Preregistration Platforms. MetaArXiv. doi:10.31222/osf.io/zry2u
Isager, P. M., Aert, R. C. M. van, Bahník, Š., et al. (2020). Deciding what to replicate: A formal definition of “replication value” and a decision model for replication study selection. MetaArXiv. doi:10.31222/osf.io/2gurz
Kathawalla, U.-K., Silverstein, P., & Syed, M. (2021). Easing Into Open Science: A Guide for Graduate Students and Their Advisors. Collabra: Psychology, 7(1). doi:10/ghv2jx
Klein, O., Hardwicke, T. E., Aust, F., et al. (2018). A Practical Guide for Transparency in Psychological Science. Collabra: Psychology, 4(1), 20. doi:10.1525/collabra.158
Kowalczyk, O., Lautarescu, A., Blok, E., Dall’Aglio, L., & Westwood, S. (2020). What senior academics can do to support reproducible and open research: A short, three-step guide. PsyArXiv. doi:10.31234/osf.io/jyfr7
Lakens, D. (2019). The value of preregistration for psychological science: A conceptual analysis. Japanese Psychological Review, 62(3), 221–230. doi:10.24602/sjpr.62.3_221
Moshontz, H., Binion, G., Walton, H., Brown, B. T., & Syed, M. (2021). A Guide to Posting and Managing Preprints. Advances in Methods and Practices in Psychological Science, 4(2), 25152459211019948. doi:10.1177/25152459211019948
Peikert, A., & Brandmaier, A. M. (2019). A Reproducible Data Analysis Workflow with R Markdown, Git, Make, and Docker. PsyArXiv. doi:10.31234/osf.io/8xzqy
Peikert, A., Lissa, C. J. van, & Brandmaier, A. M. (2021). Reproducible Research in R: A tutorial on how to do the same thing more than once. PsyArXiv. doi:10.31234/osf.io/fwxs4
Perez-Riverol, Y., Gatto, L., Wang, R., et al. (2016). Ten Simple Rules for Taking Advantage of Git and GitHub. PLOS Computational Biology, 12(7), e1004947. doi:10.1371/journal.pcbi.1004947
Pownall, M., Azevedo, F., Aldoh, A., et al. (2021). Embedding open and reproducible science into teaching: A bank of lesson plans and resources. PsyArXiv. doi:10.31234/osf.io/fgv79
Pownall, M., Talbot, C. V., Henschel, A., et al. (2020). Navigating Open Science as Early Career Feminist Researchers. PsyArXiv. doi:10.31234/osf.io/f9m47
Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing Transparency Through a Multiverse Analysis. Perspectives on Psychological Science, 11(5), 702–712. doi:10.1177/1745691616658637
Weston, S. J., Ritchie, S. J., Rohrer, J. M., & Przybylski, A. K. (2019). Recommendations for Increasing the Transparency of Analysis of Preexisting Data Sets. Advances in Methods and Practices in Psychological Science, 2(3), 214–227. doi:10.1177/2515245919848684
Wiebels, K., & Moreau, D. (2021). Leveraging Containers for Reproducible Psychological Research. Advances in Methods and Practices in Psychological Science, 4(2), 25152459211017852. doi:10.1177/25152459211017853
Broman, K. W., & Woo, K. H. (2018). Data Organization in Spreadsheets. The American Statistician, 72(1), 2–10. doi:10/gdz6cm
Lewis, C. (2022). Data Management in Large-Scale Education Research. cghlewis.github.io/mpsi-training3
Logan, J. (2019). Data Management and Data Management Plans. figshare. Presentation. doi:10.6084/m9.figshare.7890827.v1
Michener, W. K. (2015). Ten Simple Rules for Creating a Good Data Management Plan. PLOS Computational Biology, 11(10), e1004525. doi:10.1371/journal.pcbi.1004525
- Vanhove, J. (2018). Checking the Assumptions of Your Statistical Model Without Getting Paranoid. PsyArXiv. doi:10.31234/osf.io/zvawb
Análise de variância (ANOVA)
Haverkamp, N., & Beauducel, A. (2017). Violation of the sphericity assumption and its effect on type-I error rates in repeated measures ANOVA and multi-level linear models (MLM). Frontiers in Psychology, 8, 1841. doi:10.3389/fpsyg.2017.01841
Lane, D. (2016). The assumption of sphericity in repeated-measures designs: What it means and what to do when it is violated. Quantitative Methods for Psychology, 12, 114–122. doi:10.20982/tqmp.12.2.p114
Bono, R., Arnau, J., Blanca, M. J., & Alarcón, R. (2016). Sphericity estimation bias for repeated measures designs in simulation studies. Behavior Research Methods, 48(4), 1621–1630. doi:10.3758/s13428-015-0673-1
Dwivedi, A. K., Mallawaarachchi, I., & Alvarado, L. A. (2017). Analysis of small sample size studies using nonparametric bootstrap test with pooled resampling method. Statistics in Medicine, 36(14), 2187–2205. doi:10.1002/sim.7263
Konietschke, F., & Pauly, M. (2014). Bootstrapping and permuting paired t-test type statistics. Statistics and Computing, 24(3), 283–296. doi:10.1007/s11222-012-9370-4
Haukoos, J. S., & Lewis, R. J. (2005). Advanced statistics: Bootstrapping confidence intervals for statistics with “difficult” distributions. Academic Emergency Medicine, 12(4), 360–365. doi:10.1197/j.aem.2004.11.018
Ialongo, C. (2016). Understanding the effect size and its measures. Biochemia Medica, 26(2), 150–163. doi:10.11613/BM.2016.015
Okada, K. (2013). Is omega squared less biased? A comparison of three major effect size indices in one-way ANOVA. Behaviormetrika, 40(2), 129–147. doi:10.2333/bhmk.40.129
Skidmore, S. T., & Thompson, B. (2013). Bias and precision of some classical ANOVA effect sizes when assumptions are violated. Behavior Research Methods, 45(2), 536–546. doi:10.3758/s13428-012-0257-2
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4, 863. doi:10.3389/fpsyg.2013.00863
Ferguson, C. J. (2016). An effect size primer: A guide for clinicians and researchers. In A. E. Kazdin (Ed.), Methodological Issues and Strategies in Clinical Research (4th ed.). American Psychological Association. doi:10.1037/14805-020
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect size estimates: Current use, calculations, and interpretation. Journal of Experimental Psychology: General, 141(1), 2–18. doi:10.1037/a0024338
Homack, S. R. (2001). Understanding What ANOVA Post Hoc Tests Are, Really. ERIC Document, ED449222. eric.ed.gov/?id=ED449222
Hilton, A., & Armstrong, R. A. (2006). Statnote 6: Post-hoc ANOVA tests. Microbiologist, 2006, 34–36. PDF
McHugh, M. L. (2011). Multiple comparison analysis testing in ANOVA. Biochemia Medica, 21(3), 203–209. doi:10.11613/BM.2011.029
Astivia, O. L. O., & Zumbo, B. D. (2019). Heteroskedasticity in Multiple Regression Analysis: What it is, How to Detect it and How to Solve it with Applications in R and SPSS. Practical Assessment, Research, and Evaluation, 24(1). doi:10.7275/q5xr-fr95
Delacre, M., Lakens, D., & Leys, C. (2017). Why psychologists should by default use Welch’s t-test instead of Student’s t-test. International Review of Social Psychology, 30(1). doi:10.5334/irsp.82
Derrick, B., Toher, D., & White, P. (2016). Why Welch’s test is Type I error robust. The Quantitative Methods in Psychology, 12(1). doi:10.20982/tqmp.12.1.p030
Vanbrabant, L., Van De Schoot, R., & Rosseel, Y. (2015). Constrained statistical inference: Sample-size tables for ANOVA and regression. Frontiers in Psychology, 5, 1565. doi:10.3389/fpsyg.2014.01565
Shaw, R. G., & Mitchell-Olds, T. (1993). ANOVA for unbalanced data: An overview. Ecology, 74(6), 1638–1645. doi:10.2307/1939922
Martin, C. G., & Games, P. A. (1977). ANOVA tests for homogeneity of variance: Nonnormality and unequal samples. Journal of Educational Statistics, 2(3), 187–206. doi:10.3102/10769986002003187
Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110(1), 40–48. doi:10.1037/0021-843X.110.1.40
Owen, S. V., & Froman, R. D. (1998). Uses and abuses of the analysis of covariance. Research in Nursing & Health, 21(6), 557–562. doi:10.1002/(SICI)1098-240X(199812)21:6<557::AID-NUR9>3.0.CO;2-Z
Huitema, B. E. (2011). The Analysis of Covariance and Alternatives: Statistical Methods for Experiments, Quasi-Experiments, and Single-Case Studies (2nd ed.). Wiley.
Keppel, G., & Wickens, T. D. (2004). Design and Analysis: A Researcher’s Handbook (4th ed.). Pearson.
Tabachnick, B. G., & Fidell, L. S. (2013). Using Multivariate Statistics (6th ed.). Pearson.
Shieh, G. (2020). Power analysis and sample size planning in ANCOVA designs. Psychometrika, 85(1), 101–120. doi:10.1007/s11336-019-09692-3
Análise Fatorial Confirmatória (AFC)
Chen, F., Curran, P. J., Bollen, K. A., Kirby, J., & Paxton, P. (2008). An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociological Methods & Research, 36(4), 462–494. doi:10.1177/0049124108314720
Goretzko, D., Siemund, K., & Sterner, P. (2024). Evaluating Model Fit of Measurement Models in Confirmatory Factor Analysis. Educational and Psychological Measurement, 84(1), 123–144. doi:10.1177/00131644231163813
Groskurth, K., Bluemke, M., & Lechner, C. M. (2023). Why we need to abandon fixed cutoffs for goodness-of-fit indices: An extensive simulation and possible solutions. Behavior Research Methods. doi:10.3758/s13428-023-02193-3
Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2015). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44(3), 486–507. doi:10.1177/0049124114543236
McNeish, D., & Wolf, M. G. (2021). Dynamic fit index cutoffs for confirmatory factor analysis models. Psychological Methods. doi:10/gm79hm
Nye, C. D. (2023). Reviewer Resources: Confirmatory Factor Analysis. Organizational Research Methods, 26(4), 608–628. doi:10.1177/10944281221120541
Shi, D., Maydeu-Olivares, A., & Rosseel, Y. (2020). Assessing fit in ordinal factor analysis models: SRMR vs. RMSEA. Structural Equation Modeling, 27(1), 1–15. doi:10.1080/10705511.2019.1611434
Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. doi:10.3758/s13428-018-1055-2
Lai, K., & Green, S. B. (2016). The problem with having two watches: Assessment of fit when RMSEA and CFI disagree. Multivariate Behavioral Research, 51(2–3), 220–239. doi:10.1080/00273171.2015.1134306
Moshagen, M., & Auerswald, M. (2018). On congruence and incongruence of measures of fit in structural equation modeling. Psychological Methods, 23(2), 318–336. doi:10.1037/met0000122
Bornovalova, M. A., Choate, A. M., Fatimah, H., Petersen, K. J., & Wiernik, B. M. (2020). Appropriate Use of Bifactor Analysis in Psychopathology Research: Appreciating Benefits and Limitations. Biological Psychiatry, 88(1), 18–27. doi:10.1016/j.biopsych.2020.01.013
Jennrich, R. I., & Bentler, P. M. (2011). Exploratory Bi-Factor Analysis. Psychometrika, 76, 537–549. doi:10.1007/s11336-011-9218-4
DeMars, C. E. (2013). A tutorial on interpreting bifactor model scores. International Journal of Testing, 13(4), 354–378. doi:10.1080/15305058.2013.799067
Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16(S1), 19–31. doi:10.1007/s11136-007-9183-7
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. doi:10.1080/00273171.2012.715555
Reise, S. P., Bonifay, W., & Haviland, M. G. (2018). Bifactor modeling and the evaluation of scale scores. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley Handbook of Psychometric Testing (pp. 677–707). Wiley Blackwell. doi:10.1002/9781118489772.ch22
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137. doi:10.1037/met0000045
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98(3), 223–237. doi:10.1080/00223891.2015.1089249
Flora, D. B., & Flake, J. K. (2017). The purpose and practice of exploratory and confirmatory factor analysis in psychological research: Decisions for scale development and validation. Canadian Journal of Behavioural Science, 49(2), 78. doi:10.1037/cbs0000069
Coltman, T., Devinney, T. M., Midgley, D. F., & Venaik, S. (2008). Formative versus reflective measurement models: Two applications of formative measurement. Journal of Business Research, 61(12), 1250–1262. doi:10.1016/j.jbusres.2008.01.013
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323–338. doi:10.3200/JOER.99.6.323-338
- Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323–338. doi:10.3200/JOER.99.6.323-338
Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper Solutions in Structural Equation Models. Sociological Methods & Research, 29(4), 468–508. doi:10.1177/0049124101029004003
Cooperman, A. W., & Waller, N. G. (2022). Heywood you go away! Examining causes, effects, and treatments for Heywood cases in exploratory factor analysis. Psychological Methods, 27(2), 156–176. doi:10.1037/met0000384
Kolenikov, S., & Bollen, K. A. (2012). Testing negative error variances: Is a Heywood case a symptom of misspecification? Sociological Methods & Research, 41(1), 124–167. doi:10.1177/0049124112442138
Dillon, W. R., Kumar, A., & Mulani, N. (1987). Offending estimates in covariance structure analysis: Comments on the causes of and solutions to Heywood cases. Psychological Bulletin, 101(1), 126. doi:10.1037/0033-2909.101.1.126
Wang, S., De Boeck, P., & Yotebieng, M. (2023). Heywood Cases in Unidimensional Factor Models and Item Response Models for Binary Data. Applied Psychological Measurement, 47(2), 141–154. doi:10.1177/01466216231151701
Kyriazos, T. A. (2018). Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology, 9(08), 2207. doi:10.4236/psych.2018.98126
Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73(6), 913–934. doi:10.1177/0013164413495237
Brauer, K., Ranger, J., & Ziegler, M. (2023). Confirmatory Factor Analyses in Psychological Test Adaptation and Development. Psychological Test Adaptation and Development, 4(1), 4–12. doi:10.1027/2698-1866/a000034
DiStefano, C., McDaniel, H. L., Zhang, L., Shi, D., & Jiang, Z. (2019). Fitting Large Factor Analysis Models With Ordinal Data. Educational and Psychological Measurement, 79(3), 417–436. doi:10.1177/0013164418818242
Flora, D. B., & Curran, P. J. (2004). An Empirical Evaluation of Alternative Methods of Estimation for Confirmatory Factor Analysis With Ordinal Data. Psychological Methods, 9(4), 466–491. doi:10.1037/1082-989X.9.4.466
Lei, P.-W., & Shiverdecker, L. K. (2020). Performance of Estimators for Confirmatory Factor Analysis of Ordinal Variables with Missing Data. Structural Equation Modeling, 27(4), 584–601. doi:10.1080/10705511.2019.1680292
Li, C.-H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369–387. doi:10.1037/met0000093
Li, C.-H. (2016). Confirmatory factor analysis with ordinal data: Comparing robust maximum likelihood and diagonally weighted least squares. Behavior Research Methods, 48(3), 936–949. doi:10.3758/s13428-015-0619-7
Schumacker, R. E., & Beyerlein, S. T. (2000). Confirmatory Factor Analysis With Different Correlation Types and Estimation Methods. Structural Equation Modeling, 7(4), 629–636. doi:10.1207/S15328007SEM0704_6
Shi, D., & Maydeu-Olivares, A. (2020). The Effect of Estimation Methods on SEM Fit Indices. Educational and Psychological Measurement, 80(3), 421–445. doi:10.1177/0013164419885164
Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. doi:10.3758/s13428-018-1055-2
Damásio, B. F. (2013). Contribuições da Análise Fatorial Confirmatória Multigrupo (AFCMG) na avaliação de invariância de instrumentos psicométricos. Psico-USF, 18, 211–220. doi:10.1590/S1413-82712013000200005
Luong, R., & Flake, J. K. (2022). Measurement invariance testing using confirmatory factor analysis and alignment optimization: A tutorial for transparent analysis planning and reporting. Psychological Methods. doi:10.1037/met0000441
Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543. doi:10.1007/BF02294825
Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. doi:10.1016/j.dr.2016.06.004
De Roover, K. (2021). Finding clusters of groups with measurement invariance: Unraveling intercept non-invariance with mixture multigroup factor analysis. Structural Equation Modeling, 28(5), 663–683. doi:10.1080/10705511.2020.1866577
Van De Schoot, R., Schmidt, P., De Beuckelaer, A., Lek, K., & Zondervan-Zwijnenburg, M. (2015). Editorial: Measurement Invariance. Frontiers in Psychology, 6. doi:10.3389/fpsyg.2015.01064
Wicherts, J. M., & Dolan, C. V. (2010). Measurement Invariance in Confirmatory Factor Analysis: An Illustration Using IQ Test Performance of Minorities. Educational Measurement: Issues and Practice, 29(3), 39–47. doi:10.1111/j.1745-3992.2010.00182.x
Análise Fatorial Exploratória (AFE)
Nájera, P., Abad, F. J., & Sorrel, M. A. (2023). Is exploratory factor analysis always to be preferred? A systematic comparison of factor analytic techniques throughout the confirmatory–exploratory continuum. Psychological Methods. Advance online publication. doi:10.1037/met0000579
Montoya, A. K., & Edwards, M. C. (2021). The Poor Fit of Model Fit for Selecting Number of Factors in Exploratory Factor Analysis for Scale Evaluation. Educational and Psychological Measurement, 81(3), 413–440. doi:10.1177/0013164420942899
Finch, W. H. (2020). Using Fit Statistic Differences to Determine the Optimal Number of Factors to Retain in an Exploratory Factor Analysis. Educational and Psychological Measurement, 80(2), 217–241. doi:10.1177/0013164419865769
Goretzko, D., & Bühner, M. (2022). Robustness of factor solutions in exploratory factor analysis. Behaviormetrika, 49, 131–148. doi:10.1007/s41237-021-00152-w
Goretzko, D., & Bühner, M. (2020). One model to rule them all? Using machine learning algorithms to determine the number of factors in exploratory factor analysis. Psychological Methods, 25(6), 776–786. doi:10.1037/met0000262
Goretzko, D., Pham, T. T. H., & Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments and recommendations for good practice. Current Psychology, 40, 3510–3521. doi:10.1007/s12144-019-00300-2
Schreiber, J. B. (2021). Issues and recommendations for exploratory factor analysis and principal component analysis. Research in Social and Administrative Pharmacy, 17(5), 1004–1011. doi:10.1016/j.sapharm.2020.07.027
Lorenzo-Seva, U., Timmerman, M. E., & Kiers, H. A. (2011). The Hull method for selecting the number of common factors. Multivariate Behavioral Research, 46(2), 340–364. doi:10.1080/00273171.2011.564527
Lorenzo-Seva, U. (2013). How to determine the number of common factors using the Hull method. Tarragona: Universitat Rovira i Virgili. PDF
- Orcan, F. (2018). Exploratory and confirmatory factor analysis: Which one to use first. Journal of Measurement and Evaluation in Education and Psychology, 9(4), 414–421. PDF
Auerswald, M., & Moshagen, M. (2019). How to determine the number of factors to retain in exploratory factor analysis: A comparison of extraction methods under realistic conditions. Psychological Methods, 24(4), 468. doi:10.1037/met0000200
Cukadar, I. (2019). Evaluation of Four Methods for Determining the Number of Factors Underlying Measurement Indicators under the Presence of Guessing Effects. Florida State University. Link
Çokluk, Ö., & Koçak, D. (2016). Using Horn’s Parallel Analysis Method in Exploratory Factor Analysis for Determining the Number of Factors. Educational Sciences: Theory and Practice, 16(2), 537–551. ERIC
Watkins, M. W. (2018). Exploratory Factor Analysis: A Guide to Best Practice. Journal of Black Psychology, 44(3), 219–246. doi:10.1177/0095798418771807
Yang, Y., & Xia, Y. (2015). On the number of factors to retain in exploratory factor analysis for ordered categorical data. Behavior Research Methods, 47(3), 756–772. doi:10.3758/s13428-014-0499-2
- Lorenzo-Seva, U., & Ferrando, P. J. (2021). Not Positive Definite Correlation Matrices in Exploratory Item Factor Analysis: Causes, Consequences and a Proposed Solution. Structural Equation Modeling, 28(1), 138–147. doi:10.1080/10705511.2020.1735393
Gomila, R., & Clark, C. S. (2022). Missing data in experiments: Challenges and solutions. Psychological Methods, 27(2), 143–155.
Enders, C. K. (2023). Missing data: An update on the state of the art. Psychological Methods. Advance online publication.
Berk, R., Brown, L., & Zhao, L. (2010). Statistical Inference After Model Selection. Journal of Quantitative Criminology, 26(2), 217–236. doi:10.1007/s10940-009-9077-7
Carmo, M. (2022, April 7). Simulating post selection inference. marwincarmo.github.io/posts/2021-12-20-model-selection-bias
Leeb, H., & Pötscher, B. M. (2005). Model Selection and Inference: Facts and Fiction. Econometric Theory, 21(01). doi:10.1017/S0266466605050036
Lukacs, P. M., Burnham, K. P., & Anderson, D. R. (2009). Model selection bias and Freedman’s paradox. Annals of the Institute of Statistical Mathematics, 62(1), 117. doi:10.1007/s10463-009-0234-4
Smith, G. (2018). Step away from stepwise. Journal of Big Data, 5(1), 32. doi:10.1186/s40537-018-0143-6
Análises de Concordância
Alexandre, N. M. C., & Coluci, M. Z. O. (2011). Validade de conteúdo nos processos de construção e adaptação de instrumentos de medidas. Ciência & Saúde Coletiva, 16, 3061–3068. scielo.br/pdf/csc/v16n7/06.pdf
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. doi:10.1016/j.sapharm.2018.03.066
Ayre, C., & Scally, A. J. (2014). Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Measurement and Evaluation in Counseling and Development, 47(1), 79–86. doi:10.1177/0748175613513808
Hernandez-Nieto, R. (2002). Contributions to Statistical Analysis: The Coefficients of Proportional Variance, Content Validity and Kappa. CreateSpace Independent Publishing Platform.
Souza, A. C. D., Alexandre, N. M. C., & Guirardello, E. D. B. (2017). Propriedades psicométricas na avaliação de instrumentos: avaliação da confiabilidade e da validade. Epidemiologia e Serviços de Saúde, 26, 649–659. scielo.iec.gov.br
Zapf, A., Castell, S., Morawietz, L., & Karch, A. (2016). Measuring inter-rater reliability for nominal data – which coefficients and confidence intervals are appropriate? BMC Medical Research Methodology, 16(1), 1–10. doi:10.1186/s12874-016-0200-9
- Sertdemir, Y., Burgut, H. R., Alparslan, Z. N., Unal, I., & Gunasti, S. (2013). Comparing the methods of measuring multi-rater agreement on an ordinal rating scale: a simulation study with an application to real data. Journal of Applied Statistics, 40(7), 1506–1519. doi:10.1080/02664763.2013.788617
Arifin, W. N. (2024). Sample size calculator (web). Retrieved from wnarifin.github.io
Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33(3), 613–619. doi:10.1177/001316447303300309
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. doi:10.1016/j.jcm.2016.02.012
Cicchetti, D. V., & Feinstein, A. R. (1990). High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology, 43(6), 551–558. jclinepi.com
Brenner, H., & Kliebsch, U. (1996). Dependence of weighted kappa coefficients on the number of categories. Epidemiology, 7(2), 199–202. journals.lww.com
Fleiss, J. L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33(3), 613–619. doi:10.1177/001316447303300309
Cohen, J. (1968). Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213. doi:10.1037/h0026256
Sim, J., & Wright, C. C. (2005). The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Physical Therapy, 85(3), 257–268. doi:10.1093/ptj/85.3.257
Vanbelle, S. (2016). A new interpretation of the weighted kappa coefficients. Psychometrika, 81(2), 399–410. doi:10.1007/s11336-014-9439-4
Análises de Correlação
- Havlicek, L. L., & Peterson, N. L. (1976). Robustness of the Pearson correlation against violations of assumptions. Perceptual and Motor Skills, 43(3_suppl), 1319–1334. doi:10.2466/pms.1976.43.3f.1319
Xu, W., Hou, Y., Hung, Y. S., & Zou, Y. (2013). A comparative analysis of Spearman’s rho and Kendall’s tau in normal and contaminated normal models. Signal Processing, 93(1), 261–276. doi:10.1016/j.sigpro.2012.08.005
Myers, L., & Sirois, M. J. (2006). Spearman correlation coefficients, differences between. Encyclopedia of Statistical Sciences, 12. doi:10.1002/0471667196.ess5050.pub2
Schönbrodt, F. D., & Perugini, M. (2013). At what sample size do correlations stabilize? Journal of Research in Personality, 47(5), 609–612. doi:10.1016/j.jrp.2013.05.009
Schönbrodt, F. D., & Perugini, M. (2018). “At what sample size do correlations stabilize?”: Corrigendum. Journal of Research in Personality, 74, 194. doi:10.1016/j.jrp.2018.02.010
Bonett, D. G., & Wright, T. A. (2000). Sample size requirements for estimating Pearson, Kendall and Spearman correlations. Psychometrika, 65(1), 23–28. doi:10.1007/BF02294183
Análises de Rede
Bringmann, L. F., Pe, M. L., Vissers, N., Ceulemans, E., Borsboom, D., Vanpaemel, W., Tuerlinckx, F., & Kuppens, P. (2016). Assessing temporal emotion dynamics using networks. Assessment, 23(4), 425–435. doi:10.1177/1073191116645909
Beard, C., Millner, A. J., Forgeard, M. J. C., Fried, E. I., Hsu, K. J., Treadway, M. T., Leonard, C. V., Kertz, S. J., & Björgvinsson, T. (2016). Network analysis of depression and anxiety symptom relationships in a psychiatric sample. Psychological Medicine, 46(16), 3359–3369. doi:10.1017/S0033291716002300
Blanken, T. F., Deserno, M. K., Dalege, J., Borsboom, D., Blanken, P., Kerkhof, G. A., & Cramer, A. O. J. (2018). The role of stabilizing and communicating symptoms given overlapping communities in psychopathology networks. Scientific Reports, 8(1), 5854. doi:10.1038/s41598-018-24224-2
Cramer, A. O. J., van Borkulo, C. D., Giltay, E. J., van der Maas, H. L. J., Kendler, K. S., Scheffer, M., & Borsboom, D. (2016). Major depression as a complex dynamic system. PLOS ONE, 11(12), e0167490. doi:10.1371/journal.pone.0167490
Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: an integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9(1), 91–121. pubmed:23537483
Bringmann, L. F., Vissers, N., Wichers, M., Geschwind, N., Kuppens, P., Peeters, F., Borsboom, D., & Tuerlinckx, F. (2013). A network approach to psychopathology: new insights into clinical longitudinal data. PLoS ONE, 8(4), e60188. doi:10.1371/journal.pone.0060188
Cramer, A. O. J., Waldorp, L. J., van der Maas, H. L. J., & Borsboom, D. (2010). Comorbidity: a network perspective. Behavioral and Brain Sciences, 33(2–3), 137–150. doi:10.1017/S0140525X09991567
Costantini, G., Epskamp, S., Borsboom, D., Perugini, M., Mõttus, R., Waldorp, L. J., & Cramer, A. O. J. (2015). State of the aRt personality research: a tutorial on network analysis of personality data in R. Journal of Research in Personality, 54, 13–29. doi:10.1016/j.jrp.2014.07.003
Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L. J., van der Maas, H. L. J., & Maris, G. (2018). An introduction to network psychometrics: relating Ising network models to item response theory models. Multivariate Behavioral Research, 53(1), 15–35. doi:10.1080/00273171.2017.1379379
Hevey, D. (2018). Network analysis: a brief overview and tutorial. Health Psychology and Behavioral Medicine, 6(1), 301–328. doi:10.1080/21642850.2018.1521283
Jones, P. J., Ma, R., & McNally, R. J. (2021). Bridge centrality: a network approach to understanding comorbidity. Multivariate Behavioral Research, 56(2), 353–367. doi:10.1080/00273171.2019.1614898
Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: a network perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43–53. doi:10.1016/j.newideapsych.2011.02.007
Curva ROC
- Flach, P. A. (2016). ROC analysis. In Encyclopedia of Machine Learning and Data Mining (pp. 1–8). Springer. research-information.bris.ac.uk
Sachs, M. C. (2017). plotROC: a tool for plotting ROC curves. Journal of Statistical Software, 79. doi:10.18637/jss.v079.c02
Goksuluk, D., Korkmaz, S., Zararsiz, G., & Karaagaoglu, A. E. (2016). easyROC: an interactive web-tool for ROC curve analysis using R language environment. R Journal, 8(2), 213.
Negida, A., Fahim, N. K., & Negida, Y. (2019). Sample size calculation guide – part 4: how to calculate the sample size for a diagnostic test accuracy study based on sensitivity, specificity, and the area under the ROC curve. Frontiers in Emergency Medicine, 3(3), e33. fem.tums.ac.ir
Obuchowski, N. A., & Bullen, J. A. (2018). Receiver operating characteristic (ROC) curves: review of methods with applications in diagnostic medicine. Physics in Medicine & Biology, 63(7), 07TR01. doi:10.1088/1361-6560/aab4b1
GLzM, GEE, GMM, HGMM
- McNeish, D., Stapleton, L. M., & Silverman, R. D. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114. doi:10.1037/met0000078
- Bates, D., Kliegl, R., Vasishth, S., & Baayen, H. (2015). Parsimonious mixed models. arXiv preprint arXiv:1506.04967. arxiv.org/abs/1506.04967
Muth, C., Bales, K. L., Hinde, K., Maninger, N., Mendoza, S. P., & Ferrer, E. (2016). Alternative models for small samples in psychological research: applying linear mixed effects models and generalized estimating equations to repeated measures data. Educational and Psychological Measurement, 76(1), 64–87. doi:10.1177/0013164415580432
Guimarães, L. S. P., & Hirakata, V. N. (2012). Uso do modelo de equações de estimações generalizadas na análise de dados longitudinais. Revista HCPA, 32(4), 503–511. lume.ufrgs.br
Lipsitz, S. R., Fitzmaurice, G. M., Orav, E. J., & Laird, N. M. (1994). Performance of generalized estimating equations in practical situations. Biometrics, 50(1), 270–278. jstor.org/stable/2533218
Amatya, A., & Bhaumik, D. K. (2018). Sample size determination for multilevel hierarchical designs using generalized linear mixed models. Biometrics, 74(2), 673–684. doi:10.1111/biom.12764
Cundill, B., & Alexander, N. D. (2015). Sample size calculations for skewed distributions. BMC Medical Research Methodology, 15(1), 1–9. doi:10.1186/s12874-015-0023-0
Li, Z., & McKeague, I. W. (2013). Power and sample size calculations for generalized estimating equations via local asymptotics. Statistica Sinica, 23(1), 231. ncbi.nlm.nih.gov/pmc/articles/PMC3903421
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852. doi:10.1177/1094428106296642
Tosteson, T. D., Buzas, J. S., Demidenko, E., & Karagas, M. (2003). Power and sample size calculations for generalized regression models with covariate measurement error. Statistics in Medicine, 22(7), 1069–1082. doi:10.1002/sim.1388
- McCulloch, C. E., & Neuhaus, J. M. (2005). Generalized linear mixed models. Encyclopedia of Biostatistics, 4. doi:10.1002/0470013192.bsa251
Kincaid, C. (2005, April). Guidelines for selecting the covariance structure in mixed model analysis. In Proceedings of the Thirtieth Annual SAS Users Group International Conference (Vol. 30, pp. 198–130). SAS Institute Inc. support.sas.com
Littell, R. C., Pendergast, J., & Natarajan, R. (2000). Modelling covariance structure in the analysis of repeated measures data. Statistics in Medicine, 19(13), 1793–1819. doi:10.1002/1097-0258(20000715)19:13<1793::AID-SIM482>3.0.CO;2-Q
Interpretando e apresentando os resultados
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M. (2018). Journal article reporting standards for quantitative research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 3–25. doi:10.1037/amp0000191
Kasza, J., & Wolfe, R. (2014). Interpretation of commonly used statistical regression models. Respirology, 19(1), 14–21. doi:10/gmprxp
Levitt, H. M., Bamberg, M., Creswell, J. W., Frost, D. M., Josselson, R., & Suárez-Orozco, C. (2018). Journal article reporting standards for qualitative primary, qualitative meta-analytic, and mixed methods research in psychology: The APA Publications and Communications Board task force report. American Psychologist, 73(1), 26–46. doi:10.1037/amp0000151
| Recurso | Descrição | Link |
|---|---|---|
| Repositório da Disciplina | Materiais extras | extras |