--- title: 'writeAlizer: Scoring Model Development' author: Sterett H. Mercer output: rmarkdown::html_vignette: toc: true toc_depth: 2 bibliography: references.bib csl: apa.csl vignette: > %\VignetteIndexEntry{writeAlizer: Scoring Model Development} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- > This vignette provides details on the scoring models included in writeAlizer. # Recommended Models for Use [ReaderBench-Model-3](#readerbench-model-3) and [Coh-Metrix-Model-3](#cohmetrix-model-3) are the best models for generated predicted writing quality scores, and [aWE-CBM-Model-1](#awecbm-model-1) is the best available model for generating automated written expression curriculum-based measurement scores. # Scoring Model Development {#scoring-model-development} The general process used to generate all scoring models is presented below. ## Predictive Algorithms and R Packages Used The [`caret`](https://CRAN.R-project.org/package=caret) and [`caretEnsemble`](https://CRAN.R-project.org/package=caretEnsemble) packages were used as wrappers for the following predictive algorithms: * Random forest regression (package [`randomForest`](https://CRAN.R-project.org/package=randomForest)) * Cubist regression (package [`Cubist`](https://cran.r-project.org/package=Cubist)) * Support vector machines with a radial kernel (package [`kernlab`](https://cran.r-project.org/package=kernlab)) * Bagged multivariate adaptive regression splines (package [`earth`](https://cran.r-project.org/package=earth)) * Stochastic gradient boosted trees (package [`gbm`](https://cran.r-project.org/package=gbm)) * Partial least squares regression (package [`pls`](https://cran.r-project.org/package=pls)) * Elasticnet regression (package [`elasticnet`](https://cran.r-project.org/package=elasticnet)) These algorithms are described in detail in the following references (among others): * Hastie, T., Tibshirani, R., & Friedman, J. (2009). _The elements of statistical learning: Data mining, inference, and prediction_ (2nd ed.). Springer. * Kuhn, M., & Johnson, K. (2013). _Applied predictive modeling._ Springer. ## Steps The following flowchart provides an overview of the scoring model development workflow, with more details on some steps provided below. ![Figure 1. Model Development Process.](model_flow_chart.png) ### 1. Import Data Depending on the specific scoring model, ReaderBench, Coh-Metrix, and/or GAMET output files were imported into R using functions similar to the import_XXXX.R functions in writeAlizer (see ) ### 2. Pre-Process Data Automated data pre-processing were done using the [`preProcess()`](https://www.rdocumentation.org/packages/caret/versions/6.0-86/topics/preProcess) function in `caret`: * Predictors from the output file with near zero variance (defined based on defaults in the [`nearZeroVar()`](https://www.rdocumentation.org/packages/caret/versions/6.0-76/topics/nearZeroVar) function) were removed, and the remaining predictors were standardized. * Highly correlated | _r_ > .90 | predictors were identified, with the predictor that had the highest mean correlation with all of the other predictors removed. * The reduced set of predictors was submitted to the next step of the analysis. ### 3. Determine Optimal Tuning Parameters The following tuning hyperparameters were optimized based on resampling (repeated 10 fold) in `caret`. Each algorithm was tuned separately. Full descriptions of the tuning parameters are available in each package's documentation. * Random forest regression: `mtry` * Cubist regression: `committees`, `neighbors` * Support vector machines: `sigma`, `C` * Bagged multivariate adaptive regression splines: `nprune`, `degree` * Stochastic gradient boosted trees: `n.trees`, `interaction.depth`, `shrinkage`, `n.minobsinnode` * Partial least squares regression: `ncomp` * Elastic net regression: `fraction`, `lambda` ### 4. Final/Optimal Model for each Algorithm A model for each algorithm was fit with the hyperparameters set to the optimal values found in Step 3, with bootstrapped (1000 samples) resampling-based cross-validation so that an ensemble model (weighting each algorithm) could be built based on the resamples. This step was done with the [`caretList()`](https://zachmayer.github.io/caretEnsemble/reference/caretList.html) function of the `caretEnsemble` package. This process is illustrated in more detail in the `caretEnsemble` vignette: ### 5. Estimate an Ensemble Model to Combine the Algorithms The [`caretEnsemble()`](https://zachmayer.github.io/caretEnsemble/reference/caretEnsemble.html) function was used to determine the optimal linear weighting of the algorithms that minimized RMSE (i.e., discrepancy between actual writing quality scores and predicted quality scores) in the resamples from Step 4. Algorithms with near zero or negative weights were removed from the ensemble models. The [`varImp()`](https://topepo.github.io/caret/variable-importance.html) function of `caretEnsemble` was used to generate estimates of relative predictor importance for the overall ensemble model and for each individual algorithm. ### 6. Generate Predicted Quality Scores from each Ensemble The [`predict()`](https://zachmayer.github.io/caretEnsemble/reference/predict.caretList.html) function of `caretEnsemble` was used to generate/store predicted quality scores for the ensemble models. ### 7. Average Scores to get Final Predicted Quality Scores The predicted scores from each ensemble were averaged to produce the final predictions. --- # ReaderBench Model 1 {#readerbench-model-1} ## General Description Model 1 has been replaced by the greatly simplified [Model 2](#readerbench-model-2) that better handles multi-paragraph compositions. Model 2 is recommended over Model 1. Model 1 is an ensemble (formed by averaging predicted quality scores) of the six sub-models described below. All of these sub-models used `ReaderBench` scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019]. Highly correlated ReaderBench metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). This scoring model was evaluated in the following publications: [@Matta2022; @Mercer2022; @Keller-Margulis2021] ## ReaderBench Model 1a This model was trained on fall data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:--------|:-------|:--------|:--------|:-------|:-------|:-------| | -3.9077 | -0.1323 | 0.4789 | -0.0963 | -0.0361 | 0.3985 | 0.1297 | 0.3442 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------------------------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | WdEnt | 14.61 | 40.66 | 2.49 | 1.93 | 13.17 | 3.97 | 62.12 | 19.56 | | LxcDiv | 3.02 | 4.3 | 2.2 | 1.52 | 2.69 | 2.77 | 0 | 5.55 | | AvgUnqPrepositionBl | 2.88 | 3.21 | 2.18 | 1.42 | 6.67 | 1.98 | 0 | 5.84 | | AvgNmdEntBl | 2.71 | 0.47 | 1.13 | 0.4 | 0.79 | 0.89 | 18.1 | 2.92 | | AvgUnqVerbBl | 2.4 | 2.36 | 2.21 | 1.48 | 6.53 | 2.33 | 0 | 3.5 | | WdDiffLemmaStem | 2.07 | 1.07 | 0.82 | 0.91 | 0 | 1.35 | 13 | 1.46 | | RdbltyFlesch | 1.91 | 0.7 | 0.41 | 0.41 | 4.1 | 1.04 | 6.51 | 3.94 | | AvgChainSpan | 1.84 | 3.5 | 1.77 | 1.16 | 0 | 2.11 | 0 | 2.04 | | AvgDepsBl_det | 1.64 | 2.01 | 1.63 | 0.79 | 3.96 | 1.6 | 0 | 2.19 | | AvgBlScore | 1.55 | 0.87 | 2.01 | 1.32 | 0 | 1.76 | 0 | 1.75 | | AvgPronBl_first_person | 1.52 | 0.7 | 1.56 | 0.75 | 2.38 | 1.39 | 0 | 2.63 | | AvgDepsBl_nsubj | 1.5 | 1.34 | 2.18 | 1.45 | 0 | 1.9 | 0 | 0.88 | | AvgUnqAdverbBl | 1.48 | 1.86 | 1.88 | 1.06 | 1.3 | 2.12 | 0 | 0.73 | | AvgDepsBl_punct | 1.45 | 1.45 | 1.77 | 0.93 | 1.9 | 2.1 | 0 | 0.88 | | WdDiffWdStem | 1.43 | 2.02 | 1.47 | 0.84 | 1.36 | 1.04 | 0 | 2.34 | | AvgDepsSen_punct | 1.41 | 0.64 | 1.33 | 0.65 | 4.76 | 1.04 | 0 | 2.63 | | AvgDepsSen_dep | 1.34 | 0.04 | 0.65 | 0.33 | 5.78 | 0.53 | 0 | 4.09 | | AvgSenScore | 1.28 | 0.39 | 0.27 | 0.22 | 0 | 0.53 | 0 | 4.82 | | AvgPronounBl | 1.23 | 0.2 | 1.8 | 1.08 | 0 | 1.38 | 0 | 1.31 | | TCorefChainDoc | 1.2 | 0.41 | 1.59 | 0.77 | 0 | 0.88 | 0 | 2.04 | | AvgDepsBl_nmod | 1.12 | 0.52 | 1.9 | 1.09 | 0 | 1.57 | 0 | 0.29 | | AvgAOASen_Shock | 1.09 | 1.04 | 0.77 | 0.52 | 3.75 | 1.04 | 0 | 1.9 | | AvgSenBl | 1.03 | 0.42 | 1.51 | 0.68 | 0 | 1.29 | 0 | 0.88 | | WdLettStdDev | 1.01 | 0.85 | 1.27 | 0.8 | 0 | 0.84 | 0 | 1.46 | | AvgWdLen | 0.97 | 1.33 | 1.4 | 0.75 | 0 | 1.26 | 0 | 0.44 | | AvgDepsBl_nummod | 0.95 | 0.75 | 1.16 | 0.42 | 0.04 | 1.23 | 0 | 1.02 | | AvgCorefChain | 0.94 | 0.56 | 1.01 | 0.34 | 2.87 | 0.32 | 0 | 2.04 | | TActCorefChainWd | 0.93 | 0.78 | 0.83 | 0.55 | 0.99 | 1.15 | 0 | 1.31 | | RdbltyDaleChall | 0.92 | 0.93 | 0.95 | 0.38 | 0 | 1.07 | 0.27 | 1.17 | | AvgDepsBl_advmod | 0.89 | 0.2 | 1.65 | 0.84 | 1 | 1.26 | 0 | 0 | | AvgAOABl_Bristol | 0.86 | 0.73 | 0.93 | 0.41 | 0 | 0.52 | 0 | 1.75 | | AvgUnqNoundBl | 0.85 | 0.28 | 1.64 | 0.82 | 0.65 | 0.88 | 0 | 0.29 | | AvgDepsBl_dobj | 0.85 | 0.19 | 1.49 | 0.68 | 1.5 | 0.93 | 0 | 0.44 | | AvgAOABl_Shock | 0.83 | 2.04 | 1.3 | 0.63 | 0.12 | 0.96 | 0 | 0 | | SenStdDevWd | 0.77 | 0.67 | 1.07 | 0.56 | 0.83 | 1.42 | 0 | 0 | | AvgDepsBl_mark | 0.76 | 0.15 | 1.31 | 0.53 | 1.23 | 0.7 | 0 | 0.58 | | LexChainMaxSp | 0.75 | 0.31 | 1.57 | 0.75 | 0 | 0.85 | 0 | 0 | | WdAvgDpthHypernymTree | 0.75 | 0.42 | 0.63 | 0.36 | 0.57 | 0.6 | 0 | 1.61 | | AvgPronBl_indefinite | 0.72 | 0.12 | 1.44 | 0.64 | 0 | 0.62 | 0 | 0.44 | | AvgSenAdjCoh_Path | 0.69 | 0.35 | 0.83 | 0.49 | 0.57 | 0.77 | 0 | 0.88 | | AvgDepsBl_cop | 0.68 | 0.15 | 1.2 | 0.44 | 2.55 | 0.92 | 0 | 0 | | CharEnt | 0.68 | 0.26 | 1.44 | 0.69 | 0.42 | 0.73 | 0 | 0 | | AvgConnBl_simp_subords | 0.66 | 0.11 | 1.2 | 0.45 | 0.31 | 1.05 | 0 | 0 | | LexChainAvgSpan | 0.66 | 0.49 | 1.17 | 0.7 | 0 | 0.94 | 0 | 0 | | AvgConnBl_reas_purp | 0.66 | 0.11 | 1.26 | 0.5 | 0 | 1.02 | 0 | 0 | | AvgDepsBl_advcl | 0.64 | 0.05 | 1.35 | 0.56 | 1.13 | 0.7 | 0 | 0 | | TCorefChainBigSpan | 0.63 | 0.01 | 1.16 | 0.42 | 1.31 | 0.54 | 0 | 0.44 | | AvgUnqAdjectiveBl | 0.63 | 0.48 | 1.37 | 0.58 | 0 | 0.61 | 0 | 0 | | AvgDepsBl_amod | 0.63 | 0.05 | 1.28 | 0.51 | 0.32 | 0.85 | 0 | 0 | | AvgDepsBl_aux | 0.62 | 0.57 | 0.98 | 0.3 | 0 | 1.07 | 0 | 0 | | WdPathCntHypernymTree | 0.62 | 0.99 | 0.79 | 0.44 | 3.29 | 0.7 | 0 | 0.15 | | WdSylCnt | 0.62 | 0.37 | 0.8 | 0.56 | 0 | 1.3 | 0 | 0 | | AvgUnqPronounBl | 0.61 | 0.21 | 1.3 | 0.53 | 0.62 | 0.65 | 0 | 0 | | FrqRhythmId | 0.59 | 0.73 | 0.9 | 0.44 | 0 | 0.94 | 0 | 0 | | AvgDepsBl_nsubjpass | 0.57 | 0.05 | 1.01 | 0.32 | 1.18 | 0.88 | 0 | 0 | | AvgDepsBl_ccomp | 0.55 | 0.26 | 1.13 | 0.39 | 1.76 | 0.54 | 0 | 0 | | AvgConnSen_addition | 0.54 | 0.25 | 0.59 | 0.25 | 0 | 0.95 | 0 | 0.44 | | AvgConnBl_order | 0.52 | 0.07 | 0.72 | 0.16 | 2.74 | 0.92 | 0 | 0 | | AvgBlVoiceCoOcc | 0.52 | 0 | 1.09 | 0.36 | 0 | 0.73 | 0 | 0 | | AvgInferenceDistChain | 0.51 | 0.32 | 0.74 | 0.45 | 0.3 | 0.79 | 0 | 0.15 | | AvgAOESen_InvLinRegSlo | 0.5 | 0.56 | 0.65 | 0.35 | 0.86 | 0.28 | 0 | 0.73 | | AvgRhythmUnitStreesSyll | 0.49 | 0.29 | 0.03 | 0.3 | 0 | 1.02 | 0 | 0.88 | | AvgConnBl_semi_coords | 0.48 | 0.01 | 0.98 | 0.3 | 0 | 0.68 | 0 | 0 | | LxcSoph | 0.48 | 0.25 | 0.83 | 0.32 | 0 | 0.8 | 0 | 0 | | AvgConnBl_logical_cons | 0.47 | 0.05 | 0.58 | 0.1 | 1.58 | 0.63 | 0 | 0.44 | | AvgDepsBl_neg | 0.47 | 0.04 | 0.81 | 0.21 | 0 | 0.86 | 0 | 0 | | AvgCommaBl | 0.47 | 0.09 | 0.9 | 0.25 | 0 | 0.75 | 0 | 0 | | AvgNounNmdEntBl | 0.47 | 0.08 | 0.67 | 0.14 | 0 | 0.43 | 0 | 0.73 | | AvgNounSen | 0.46 | 0.23 | 0.12 | 0.07 | 0 | 0.62 | 0 | 1.17 | | AvgAdverbSen | 0.45 | 0.17 | 0.46 | 0.55 | 0 | 0.81 | 0 | 0.29 | | AvgConnBl_oppositions | 0.44 | 0.07 | 0.9 | 0.26 | 0 | 0.62 | 0 | 0 | | AvgNmdEntSen | 0.44 | 0.62 | 0.13 | 0.5 | 0 | 0.93 | 0 | 0.44 | | AvgConnBl_contrasts | 0.44 | 0.25 | 1.02 | 0.32 | 0 | 0.41 | 0 | 0 | | AvgConnSen_semi_coords | 0.43 | 0.05 | 0.43 | 0.06 | 0.08 | 0.93 | 0 | 0.29 | | AvgAOASen_Bristol | 0.43 | 0.48 | 0.32 | 0.48 | 1.36 | 0.7 | 0 | 0.29 | | AvgDepsBl_xcomp | 0.43 | 0.11 | 1.15 | 0.41 | 0 | 0.23 | 0 | 0 | | AvgConnSen_simp_subords | 0.43 | 0.74 | 0.28 | 0.31 | 1.49 | 0.97 | 0 | 0 | | WdPolysemyCnt | 0.41 | 0.4 | 0 | 0.71 | 0.46 | 0.17 | 0 | 1.31 | | AvgPronBl_third_person | 0.41 | 0.39 | 0.88 | 0.24 | 0 | 0.44 | 0 | 0 | | AvgAOASen_Bird | 0.4 | 0.25 | 0.58 | 0.51 | 0 | 0.7 | 0 | 0 | | AvgDepsBl_mwe | 0.4 | 0.03 | 0.49 | 0.08 | 3.31 | 0.71 | 0 | 0 | | AvgPronounSen | 0.38 | 0.27 | 0.06 | 0.18 | 0 | 0.59 | 0 | 0.88 | | AvgConnBl_addition | 0.37 | 0.23 | 0.58 | 0.1 | 0 | 0.71 | 0 | 0 | | AvgAOABl_Kuperman | 0.37 | 0.6 | 0.32 | 0.48 | 0 | 0.43 | 0 | 0.44 | | AvgAOASen_Kuperman | 0.36 | 0.16 | 0.62 | 0.55 | 0 | 0.52 | 0 | 0 | | AvgAOABl_Cortese | 0.36 | 0.33 | 0.3 | 0.59 | 2.07 | 0.66 | 0 | 0 | | AvgDepsSen_advcl | 0.35 | 0.22 | 0.5 | 0.51 | 0 | 0.6 | 0 | 0 | | AvgDepsSen_det | 0.33 | 0.32 | 0.08 | 0.27 | 0 | 0.31 | 0 | 0.88 | | AvgDepsBl_acl | 0.33 | 0.01 | 0.54 | 0.1 | 0 | 0.67 | 0 | 0 | | AvgRhythmUnits | 0.33 | 0.54 | 0.6 | 0.43 | 0 | 0.22 | 0 | 0.15 | | AggPronSen_indefinite | 0.32 | 0.14 | 0.24 | 0.49 | 0.23 | 0.55 | 0 | 0.29 | | AvgConnBl_temp_cons | 0.31 | 0.16 | 0.87 | 0.24 | 0 | 0.1 | 0 | 0 | | AvgDepsSen_ccomp | 0.3 | 0.19 | 0.04 | 0.34 | 0 | 0.79 | 0 | 0.29 | | SenAsson | 0.29 | 0.02 | 0.43 | 0.06 | 1 | 0.57 | 0 | 0 | | AggPronSen_third_person | 0.29 | 0.46 | 0.3 | 0.15 | 0 | 0.52 | 0 | 0.15 | | AvgAOABl_Bird | 0.29 | 0.27 | 0.46 | 0.53 | 0 | 0.3 | 0 | 0.15 | | AvgDepsSen_aux | 0.29 | 0.08 | 0.12 | 0.49 | 0 | 0.9 | 0 | 0 | | AvgDepsSen_dobj | 0.28 | 0.1 | 0.11 | 0.06 | 0 | 0.95 | 0 | 0 | | AvgConnSen_oppositions | 0.26 | 0.03 | 0.28 | 0.03 | 0 | 0.69 | 0 | 0 | | AvgDepsSen_nmod | 0.26 | 0.18 | 0.37 | 0.37 | 0 | 0.47 | 0 | 0 | | AvgDepsSen_amod | 0.25 | 0.26 | 0.17 | 0.1 | 0 | 0.47 | 0 | 0.29 | | AvgAOASen_Cortese | 0.24 | 0.2 | 0.59 | 0.44 | 0 | 0.08 | 0 | 0 | | AvgDepsSen_compound | 0.24 | 0.16 | 0.38 | 0.18 | 0 | 0.43 | 0 | 0 | | AvgDepsSen_mark | 0.23 | 0.3 | 0.23 | 0.27 | 0 | 0.48 | 0 | 0 | | AvgDepsSen_mwe | 0.22 | 0.1 | 0.2 | 0.02 | 0 | 0.6 | 0 | 0 | | AvgAdjectiveSen | 0.22 | 0.16 | 0.04 | 0.08 | 0 | 0.78 | 0 | 0 | | AvgAOEBl_IndPolyFAT.3 | 0.21 | 0.26 | 0.05 | 0.29 | 0.04 | 0.24 | 0 | 0.44 | | AvgAOEBl_InvLinRegSlo | 0.21 | 0.44 | 0.17 | 0.25 | 0 | 0.43 | 0 | 0 | | AvgDepsBl_dep | 0.21 | 0.09 | 0.17 | 0.01 | 0.69 | 0.57 | 0 | 0 | | AvgDepsSen_xcomp | 0.2 | 0.26 | 0 | 0.39 | 0 | 0.4 | 0 | 0.29 | | AvgDepsSen_cop | 0.19 | 0.26 | 0.13 | 0.36 | 0 | 0.31 | 0 | 0.15 | | AvgDepsBl_compound | 0.18 | 0.07 | 0.17 | 0.01 | 0.45 | 0.23 | 0 | 0.29 | | LangRhythmDiameter | 0.16 | 0.18 | 0.29 | 0.03 | 0.86 | 0.14 | 0 | 0 | | AvgConnSen_temp_cons | 0.13 | 0.3 | 0.18 | 0.42 | 0 | 0.09 | 0 | 0 | | AvgAOEBl_IndAbThr.0.3. | 0.12 | 0.31 | 0.01 | 0.33 | 0.04 | 0.27 | 0 | 0 | | AvgDepsSen_acl | 0.12 | 0.04 | 0.13 | 0.01 | 0 | 0.3 | 0 | 0 | | AvgConnSen_order | 0.12 | 0.21 | 0.17 | 0.01 | 0 | 0.23 | 0 | 0 | | LangRhythmCoeff | 0.12 | 0.41 | 0 | 0.23 | 0 | 0.31 | 0 | 0 | | AvgSenBlCoh_LeackChod | 0.12 | 0.06 | 0.08 | 0.42 | 0 | 0.28 | 0 | 0 | | AvgUnqWdBl | 0.11 | 0 | 0 | 1.79 | 0 | 0 | 0 | 0 | | AvgBlLen | 0.11 | 0 | 0 | 1.8 | 0 | 0 | 0 | 0 | | AvgVerbBl | 0.1 | 0 | 0 | 1.68 | 0 | 0 | 0 | 0 | | AvgDepsSen_neg | 0.1 | 0.05 | 0.02 | 0 | 0.08 | 0.37 | 0 | 0 | | Words | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 | | Content.words | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 | | AvgWdBl | 0.1 | 0 | 0 | 1.75 | 0 | 0 | 0 | 0 | | AvgDepsBl_case | 0.08 | 0 | 0 | 1.27 | 0 | 0 | 0 | 0 | | AvgNounBl | 0.07 | 0 | 0 | 1.15 | 0 | 0 | 0 | 0 | | LangRhythmId | 0.07 | 0.02 | 0.23 | 0.02 | 0 | 0 | 0 | 0 | | AvgPrepositionBl | 0.07 | 0 | 0 | 1.21 | 0 | 0 | 0 | 0 | | AvgAdverbBl | 0.05 | 0 | 0 | 0.87 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LDA | 0.04 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 | | AvgAOADoc_Shock | 0.04 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_Path | 0.04 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_word2vec | 0.04 | 0 | 0 | 0.67 | 0 | 0 | 0 | 0 | | SynSoph | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | Sentences | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LSA | 0.04 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_word2vec | 0.04 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LSA | 0.03 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgAOESen_InfPointPoly | 0.03 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgUnqWdSen | 0.03 | 0 | 0 | 0.43 | 0 | 0 | 0 | 0 | | AvgConnSen_reas_purp | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LeackChod | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgPrepositionSen | 0.03 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgVerbSen | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_WuPalmer | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | SenStDevUnqWd | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_WuPalmer | 0.03 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgAOADoc_Kuperman | 0.03 | 0 | 0 | 0.48 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LDA | 0.03 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | | SenScoreStDev | 0.03 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgDepsSen_advmod | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 | | AvgUnqNmdEntBl | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 | | AvgAdjectiveBl | 0.03 | 0 | 0 | 0.52 | 0 | 0 | 0 | 0 | | RdbltyFog | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bird | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgConnBl_sentence_link | 0.03 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LSA | 0.03 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_word2vec | 0.03 | 0 | 0 | 0.55 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_Path | 0.03 | 0 | 0 | 0.55 | 0 | 0 | 0 | 0 | | AvgAOADoc_Cortese | 0.03 | 0 | 0 | 0.59 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvLinRegSlo | 0.02 | 0 | 0 | 0.25 | 0 | 0 | 0 | 0 | | AvgConnSen_sentence_link | 0.02 | 0 | 0 | 0.26 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndPolyFAT.3 | 0.02 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgDepsSen_conj | 0.02 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conjs | 0.02 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvAverage | 0.02 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgAOEBl_InvAverage | 0.02 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgSenSyll | 0.02 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndAbThr.0.3. | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgDepsBl_auxpass | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conns | 0.02 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgSemDep | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgSenStressedSyll | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgSenLen | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InfPointPoly | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgAOEBl_InfPointPoly | 0.02 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgAOESen_IndAbThr.0.3. | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LDA | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | AvgDepsSen_nsubj | 0.02 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | AvgAOESen_IndPolyFAT.3 | 0.02 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | AvgWdSen | 0.02 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | AvgVoice | 0.02 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | WdMaxDpthHypernymTree | 0.02 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LeackChod | 0.02 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_WuPalmer | 0.02 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgAOESen_InvAverage | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bristol | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | RdbltyKincaid | 0.02 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgDepsSen_case | 0.02 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgDepsBl_conj | 0.01 | 0 | 0 | 0.11 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conns | 0.01 | 0 | 0 | 0.13 | 0 | 0 | 0 | 0 | | AvgConnSen_conjunctions | 0.01 | 0 | 0 | 0.15 | 0 | 0 | 0 | 0 | | AvgConnBl_conjunctions | 0.01 | 0 | 0 | 0.15 | 0 | 0 | 0 | 0 | | AvgDepsSen_cc | 0.01 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 | | AvgDepsBl_cc | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgRhythmUnitSyll | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgConnSen_logical_cons | 0.01 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgConnSen_contrasts | 0 | 0 | 0 | 0.05 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conjs | 0 | 0 | 0 | 0.06 | 0 | 0 | 0 | 0 | ## ReaderBench Model 1b This model was trained on winter data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:------| | -2.0039 | 0.3112 | 0.1353 | 0.2667 | -0.0102 | 0.1234 | 0.0268 | 0.222 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:--------------------------|:------|:------|:-----|:-----|:-----|:-----|:------|:------| | WdEnt | 11.34 | 23.56 | 2.43 | 1.65 | 0.74 | 3.11 | 31.83 | 12.9 | | AvgDepsBl_det | 6.98 | 12.06 | 2.13 | 1.17 | 2.37 | 2.1 | 9.48 | 12.41 | | AvgPronounBl | 3.71 | 2.92 | 1.9 | 0.99 | 0 | 1.27 | 7.58 | 10.22 | | AvgUnqVerbBl | 3.43 | 6.53 | 2.29 | 1.36 | 1.89 | 2.17 | 0 | 3.65 | | AvgDepsBl_nsubj | 2.97 | 5.88 | 2.3 | 1.43 | 1.14 | 2.11 | 0 | 2.19 | | AvgUnqPrepositionBl | 2.78 | 4.75 | 2.01 | 1.04 | 1.59 | 1.86 | 0 | 3.65 | | LxcDiv | 2.6 | 4 | 2.33 | 1.44 | 0.22 | 1.92 | 0 | 3.16 | | WdDiffWdStem | 1.99 | 0.84 | 0.97 | 0.5 | 0.82 | 0.63 | 0 | 7.3 | | AvgBlScore | 1.56 | 2.98 | 1.74 | 1.22 | 1.18 | 1.81 | 0 | 0 | | AvgDepsBl_punct | 1.52 | 2.67 | 1.81 | 0.85 | 0.54 | 1.23 | 0 | 0.97 | | AvgUnqNoundBl | 1.48 | 0.02 | 1.81 | 0.84 | 0.31 | 1.09 | 11.62 | 2.68 | | AggPronSen_third_person | 1.37 | 0.36 | 0.55 | 0.21 | 0.28 | 0.67 | 18.65 | 2.19 | | TCorefChainDoc | 1.3 | 0.78 | 1.88 | 0.91 | 0.67 | 1.7 | 0 | 2.19 | | RdbltyFlesch | 1.3 | 2.18 | 0.2 | 0.44 | 1.65 | 0.98 | 0 | 2.19 | | AvgPronBl_first_person | 1.19 | 0 | 1.69 | 0.74 | 1.34 | 0.66 | 0 | 3.65 | | AvgSenBl | 1.16 | 2.23 | 1.8 | 0.84 | 0.27 | 0.97 | 0 | 0 | | AvgDepsSen_advcl | 1.14 | 0.06 | 0.01 | 0.49 | 0.72 | 0.76 | 0 | 4.62 | | AvgSenBlCoh_LeackChod | 1.12 | 1.42 | 1.49 | 0.61 | 0.42 | 1.35 | 0 | 1.22 | | AvgSenBlCoh_LDA | 1.07 | 0.19 | 0.94 | 0.65 | 1.67 | 0.94 | 15.45 | 0.49 | | LexChainMaxSp | 0.99 | 1.34 | 1.69 | 0.74 | 2.27 | 1.78 | 0 | 0 | | AvgDepsBl_mark | 0.94 | 0.2 | 1.21 | 0.38 | 1.95 | 0.86 | 0 | 2.68 | | CharEnt | 0.86 | 0.48 | 1.61 | 0.8 | 0.84 | 0.82 | 0 | 1.22 | | AvgWdLen | 0.85 | 0.9 | 1.12 | 0.76 | 0.69 | 0.73 | 0 | 0.97 | | AvgChainSpan | 0.79 | 0.68 | 1.73 | 1.07 | 1.32 | 1.08 | 0 | 0 | | AvgAOESen_InfPointPoly | 0.73 | 0.22 | 0.35 | 0.23 | 0.12 | 0.39 | 0 | 2.68 | | AvgDepsSen_det | 0.72 | 0.92 | 0.1 | 0.35 | 0.83 | 0.57 | 0 | 1.46 | | AvgDepsSen_dobj | 0.69 | 0.6 | 0.53 | 0.4 | 0.94 | 0.52 | 0 | 1.46 | | AvgUnqPronounBl | 0.66 | 0.14 | 1.72 | 0.76 | 0.12 | 1.13 | 0 | 0.49 | | AvgConnBl_temp_conns | 0.65 | 1.22 | 1.12 | 0.33 | 1.18 | 0.69 | 0 | 0 | | AvgDepsSen_compound | 0.64 | 0.99 | 0.55 | 0.18 | 1.46 | 1.18 | 0 | 0.49 | | AvgSenAdjCoh_Path | 0.64 | 0.78 | 1.29 | 0.71 | 0.77 | 0.79 | 0 | 0 | | WdDiffLemmaStem | 0.63 | 1.33 | 0.19 | 0.58 | 0.14 | 0.74 | 0 | 0 | | RdbltyDaleChall | 0.63 | 1.19 | 1.14 | 0.4 | 0.66 | 0.48 | 0 | 0 | | WdMaxDpthHypernymTree | 0.59 | 1.07 | 0.69 | 0.6 | 0.39 | 0.51 | 0 | 0 | | LexChainAvgSpan | 0.57 | 0.28 | 0.88 | 0.7 | 1.27 | 0.59 | 3.91 | 0 | | SenStdDevWd | 0.54 | 0.55 | 0.97 | 0.56 | 0.03 | 0.68 | 0 | 0.24 | | AvgDepsSen_amod | 0.54 | 0.18 | 0.25 | 0.34 | 0.58 | 0.73 | 0 | 1.46 | | AvgDepsBl_dobj | 0.54 | 0.02 | 1.67 | 0.72 | 2.01 | 0.84 | 0 | 0.24 | | AvgDepsSen_dep | 0.52 | 0.87 | 0.47 | 0.4 | 0.18 | 1.08 | 0 | 0 | | AvgDepsBl_nmod | 0.52 | 0.05 | 1.74 | 0.78 | 0.19 | 0.95 | 0 | 0 | | FrqRhythmId | 0.51 | 0.56 | 1.17 | 0.42 | 0.91 | 0.9 | 0 | 0 | | AvgDepsBl_advcl | 0.5 | 0.1 | 1.08 | 0.3 | 1.09 | 0.56 | 0 | 0.97 | | AvgCorefChain | 0.5 | 0.3 | 1.13 | 0.49 | 1.28 | 0.45 | 0 | 0.49 | | AvgDepsSen_nmod | 0.49 | 0.06 | 0.34 | 0.5 | 0.22 | 0.58 | 0 | 1.22 | | AvgConnSen_addition | 0.49 | 0.1 | 0.62 | 0.49 | 0.44 | 0.68 | 0 | 0.97 | | AvgAdjectiveBl | 0.48 | 0.23 | 1.51 | 0.59 | 0.56 | 0.7 | 0 | 0 | | WdLettStdDev | 0.46 | 0.17 | 1.22 | 0.74 | 0.3 | 0.72 | 0 | 0 | | AvgPronBl_indefinite | 0.44 | 0.23 | 1.25 | 0.4 | 0.87 | 1.02 | 0 | 0 | | AvgConnBl_addition | 0.44 | 0.63 | 0.99 | 0.25 | 0.74 | 0.66 | 0 | 0 | | AvgDepsSen_mark | 0.44 | 0.16 | 0.1 | 0.39 | 1.46 | 0.73 | 0 | 0.97 | | LangRhythmCoeff | 0.44 | 0.71 | 0.54 | 0.34 | 0.04 | 0.82 | 0 | 0 | | AvgVoice | 0.43 | 0.26 | 1.31 | 0.44 | 0.83 | 0.69 | 0 | 0 | | AvgUnqAdverbBl | 0.43 | 0.03 | 1.52 | 0.6 | 0.67 | 0.76 | 0 | 0 | | AvgAOABl_Shock | 0.43 | 0.25 | 1.21 | 0.44 | 0.69 | 0.88 | 0 | 0 | | AvgBlLen | 0.41 | 0 | 0 | 1.69 | 0 | 0 | 0 | 0 | | AvgPronBl_third_person | 0.41 | 0.2 | 1.12 | 0.32 | 0.64 | 0.33 | 0 | 0.49 | | AvgUnqWdBl | 0.41 | 0 | 0 | 1.7 | 0 | 0 | 0 | 0 | | Content.words | 0.4 | 0 | 0 | 1.67 | 0 | 0 | 0 | 0 | | AvgWdBl | 0.4 | 0 | 0 | 1.67 | 0 | 0 | 0 | 0 | | Words | 0.39 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 | | AvgAOEBl_InfPointPoly | 0.39 | 0.14 | 0.34 | 0.24 | 0.81 | 0.48 | 0 | 0.97 | | TCorefChainBigSpan | 0.38 | 0 | 1.06 | 0.29 | 1.22 | 1.08 | 1.47 | 0 | | AvgDepsBl_cop | 0.37 | 0 | 1.4 | 0.51 | 1.32 | 0.58 | 0 | 0 | | AvgConnSen_semi_coords | 0.37 | 0.06 | 0.12 | 0.29 | 0.84 | 0.63 | 0 | 0.97 | | AvgVerbBl | 0.37 | 0 | 0 | 1.55 | 0 | 0 | 0 | 0 | | AvgDepsSen_xcomp | 0.35 | 0.38 | 0.05 | 0.6 | 1.2 | 0.67 | 0 | 0 | | AvgRhythmUnitStreesSyll | 0.34 | 0.4 | 0.2 | 0.33 | 1.14 | 0.93 | 0 | 0 | | AvgAOEBl_InvLinRegSlo | 0.34 | 0.7 | 0.09 | 0.23 | 0.67 | 0.59 | 0 | 0 | | AvgNmdEntSen | 0.34 | 0.3 | 0.21 | 0.4 | 0.48 | 1.1 | 0 | 0 | | AvgConnBl_coord_connects | 0.34 | 0 | 1.25 | 0.4 | 1.85 | 0.63 | 0 | 0 | | AvgAOABl_Bird | 0.34 | 0.25 | 0.44 | 0.41 | 1.53 | 0.92 | 0 | 0 | | AvgDepsBl_xcomp | 0.33 | 0.18 | 1.15 | 0.34 | 0 | 0.45 | 0 | 0 | | AvgAdverbSen | 0.33 | 0.27 | 0.2 | 0.4 | 2.52 | 0.97 | 0 | 0 | | AvgConnBl_reas_purp | 0.33 | 0 | 0.96 | 0.24 | 1.1 | 0.45 | 0 | 0.49 | | AvgAdverbBl | 0.33 | 0.04 | 1.29 | 0.43 | 0.75 | 0.5 | 0 | 0 | | TActCorefChainWd | 0.33 | 0.62 | 0.39 | 0.29 | 0.22 | 0.34 | 0 | 0 | | AvgAOABl_Bristol | 0.32 | 0.11 | 0.75 | 0.33 | 0.23 | 1 | 0 | 0 | | AvgAOEBl_IndexPolyFAT.3 | 0.32 | 0.58 | 0 | 0.44 | 0.44 | 0.43 | 0 | 0 | | AvgDepsBl_aux | 0.32 | 0 | 0.52 | 0.07 | 0.8 | 0.39 | 0 | 0.97 | | AvgConnSen_logical_conns | 0.31 | 0.13 | 0.7 | 0.47 | 0.7 | 0.55 | 0 | 0 | | AvgDepsSen_punct | 0.31 | 0.05 | 0.9 | 0.55 | 0.75 | 0.45 | 0 | 0 | | AvgCommaBl | 0.3 | 0 | 1.16 | 0.35 | 0.23 | 0.69 | 0 | 0 | | AvgDepsSen_aux | 0.29 | 0.02 | 0.45 | 0.36 | 0.43 | 1.18 | 0 | 0 | | AvgNounBl | 0.29 | 0 | 0 | 1.2 | 0 | 0 | 0 | 0 | | WdPolysemyCnt | 0.29 | 0.56 | 0.15 | 0.14 | 0.03 | 0.68 | 0 | 0 | | AvgConnSen_reas_purp | 0.29 | 0.02 | 0.16 | 0.26 | 1.13 | 0.85 | 0 | 0.49 | | AvgDepsBl_compound | 0.29 | 0.02 | 0.12 | 0 | 1.6 | 0.57 | 0 | 0.97 | | AvgDepsBl_amod | 0.28 | 0 | 1.1 | 0.32 | 0.78 | 0.53 | 0 | 0 | | AvgAOASen_Bird | 0.28 | 0.51 | 0.15 | 0.23 | 1.13 | 0.46 | 0 | 0 | | WdPathCntHypernymTree | 0.28 | 0.06 | 0.7 | 0.51 | 0.06 | 0.52 | 0 | 0 | | AvgDepsBl_acl | 0.27 | 0.03 | 1 | 0.26 | 0.7 | 0.59 | 0 | 0 | | AvgDepsBl_ccomp | 0.27 | 0 | 0.9 | 0.21 | 0.92 | 0.86 | 0 | 0 | | AvgAOASen_Shock | 0.27 | 0.02 | 0.53 | 0.5 | 0.19 | 0.66 | 0 | 0 | | AggPronSen_indefinite | 0.27 | 0.07 | 0.04 | 0.61 | 0.75 | 0.79 | 0 | 0 | | AvgDepsSen_cop | 0.27 | 0.16 | 0.03 | 0.55 | 0.5 | 0.75 | 0 | 0 | | AvgConnBl_simp_subords | 0.27 | 0.06 | 1.07 | 0.3 | 0.95 | 0.41 | 0 | 0 | | AvgConnBl_semi_coords | 0.26 | 0 | 0.68 | 0.12 | 0.22 | 0.42 | 0 | 0.49 | | AvgAOESen_IndexAbThr.0.3. | 0.26 | 0.1 | 0.23 | 0.53 | 0.02 | 0.64 | 0 | 0 | | AvgAOEBl_IndexAbThr.0.3. | 0.26 | 0.21 | 0 | 0.61 | 0.28 | 0.43 | 0 | 0 | | WdSylCnt | 0.25 | 0.15 | 0.39 | 0.48 | 0.97 | 0.3 | 0 | 0 | | AvgDepsBl_neg | 0.25 | 0.03 | 0.86 | 0.19 | 0.09 | 0.84 | 0 | 0 | | AvgConnBl_contrasts | 0.23 | 0 | 0.73 | 0.14 | 0.65 | 0.85 | 0 | 0 | | AvgNounSen | 0.23 | 0.05 | 0.42 | 0.24 | 0.23 | 0.89 | 0 | 0 | | AvgDepsSen_ccomp | 0.23 | 0.18 | 0.07 | 0.29 | 0.47 | 0.83 | 0 | 0 | | AvgConnBl_logical_conns | 0.23 | 0.16 | 0.87 | 0.19 | 0.05 | 0.27 | 0 | 0 | | AvgConnSen_simp_subords | 0.23 | 0.1 | 0.05 | 0.54 | 0 | 0.54 | 0 | 0 | | AvgAdjectiveSen | 0.23 | 0.1 | 0.21 | 0.34 | 0.85 | 0.79 | 0 | 0 | | AvgConnBl_oppositions | 0.23 | 0 | 0.79 | 0.16 | 0.75 | 0.8 | 0 | 0 | | AvgInferenceDistChain | 0.23 | 0.1 | 0.42 | 0.26 | 0.53 | 0.76 | 0 | 0 | | AvgConnBl_order | 0.22 | 0 | 0.9 | 0.21 | 2.32 | 0.29 | 0 | 0 | | AvgDepsBl_conj | 0.22 | 0 | 0.79 | 0.16 | 1.65 | 0.57 | 0 | 0 | | AvgPrepositionBl | 0.22 | 0 | 0 | 0.9 | 0 | 0 | 0 | 0 | | AvgSenScore | 0.22 | 0.05 | 0.14 | 0.33 | 0.33 | 0.9 | 0 | 0 | | AvgDepsBl_case | 0.22 | 0 | 0 | 0.9 | 0 | 0 | 0 | 0 | | AvgAOESen_IndexPolyFAT.3 | 0.22 | 0.07 | 0.26 | 0.42 | 0.35 | 0.55 | 0 | 0 | | AvgAOABl_Cortese | 0.22 | 0.05 | 0.35 | 0.32 | 1.05 | 0.67 | 0 | 0 | | AvgConnSen_oppositions | 0.21 | 0.03 | 0.03 | 0.35 | 1.28 | 0.84 | 0 | 0 | | AvgIntraBlCoh_word2vec | 0.21 | 0 | 0 | 0.88 | 0 | 0 | 0 | 0 | | AvgRhythmUnits | 0.21 | 0 | 0.34 | 0.42 | 1.08 | 0.53 | 0 | 0 | | AvgNounNmdEntBl | 0.2 | 0.14 | 0.52 | 0.07 | 2.7 | 0.46 | 0 | 0 | | Sentences | 0.2 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgAOABl_Kuperman | 0.2 | 0.13 | 0.06 | 0.32 | 0.93 | 0.64 | 0 | 0 | | AvgIntraBlCoh_LDA | 0.2 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgNmdEntBl | 0.2 | 0.03 | 0.82 | 0.18 | 1.51 | 0.35 | 0 | 0 | | AvgConnSen_order | 0.19 | 0.11 | 0.13 | 0.37 | 0.55 | 0.41 | 0 | 0 | | AvgConnSen_temp_conns | 0.19 | 0.27 | 0.12 | 0 | 1.47 | 0.77 | 0 | 0 | | LxcSoph | 0.19 | 0.05 | 0.18 | 0.34 | 0.45 | 0.61 | 0 | 0 | | LangRhythmDiameter | 0.19 | 0 | 0.3 | 0.02 | 1.45 | 0.37 | 0 | 0.49 | | AvgIntraBlCoh_LSA | 0.19 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_Path | 0.19 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 | | AvgAOASen_Kuperman | 0.18 | 0.28 | 0.06 | 0.17 | 0.06 | 0.45 | 0 | 0 | | AvgDepsBl_nsubjpass | 0.18 | 0.04 | 0.66 | 0.11 | 0.38 | 0.53 | 0 | 0 | | AvgAOASen_Bristol | 0.18 | 0.13 | 0.18 | 0.25 | 0.36 | 0.55 | 0 | 0 | | AvgSenAdjCoh_LDA | 0.17 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_Path | 0.17 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LeackChod | 0.17 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LeackChod | 0.17 | 0 | 0 | 0.71 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_word2vec | 0.17 | 0 | 0 | 0.71 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_WuPalmer | 0.17 | 0 | 0 | 0.72 | 0 | 0 | 0 | 0 | | SenAsson | 0.16 | 0 | 0.47 | 0.06 | 0.6 | 0.7 | 0 | 0 | | AvgDepsBl_nummod | 0.16 | 0 | 0.47 | 0.06 | 0.07 | 0.75 | 0 | 0 | | AvgSenBlCoh_word2vec | 0.16 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_WuPalmer | 0.16 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | SenStDevUnqWd | 0.15 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexAbThr.0.3. | 0.15 | 0 | 0 | 0.61 | 0 | 0 | 0 | 0 | | AvgUnqWdSen | 0.15 | 0 | 0 | 0.62 | 0 | 0 | 0 | 0 | | SynSoph | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgUnqAdjectiveBl | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LSA | 0.15 | 0 | 0 | 0.63 | 0 | 0 | 0 | 0 | | AvgWdSen | 0.14 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgSenLen | 0.14 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgConnBl_sentence_link | 0.14 | 0 | 0 | 0.57 | 0 | 0 | 0 | 0 | | WdAvgDpthHypernymTree | 0.14 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LSA | 0.14 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_WuPalmer | 0.14 | 0 | 0 | 0.6 | 0 | 0 | 0 | 0 | | AvgDepsSen_mwe | 0.13 | 0 | 0.31 | 0.03 | 0.32 | 0.7 | 0 | 0 | | AvgDepsBl_advmod | 0.13 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgDepsSen_advmod | 0.13 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgConnSen_contrasts | 0.13 | 0 | 0.14 | 0.22 | 0.06 | 0.55 | 0 | 0 | | AvgDepsSen_nsubj | 0.12 | 0 | 0 | 0.48 | 0 | 0 | 0 | 0 | | SenScoreStDev | 0.12 | 0 | 0 | 0.49 | 0 | 0 | 0 | 0 | | AvgVerbSen | 0.12 | 0 | 0 | 0.49 | 0 | 0 | 0 | 0 | | AvgSenStressedSyll | 0.12 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | | AvgBlVoiceCoOcc | 0.12 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgAOADoc_Shock | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexPolyFAT.3 | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 | | AvgSemDep | 0.11 | 0 | 0 | 0.44 | 0 | 0 | 0 | 0 | | AvgDepsSen_case | 0.11 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgRhythmUnitSyll | 0.1 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgSenSyll | 0.1 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bird | 0.1 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgDepsSen_acl | 0.1 | 0.06 | 0.2 | 0.01 | 0.6 | 0.43 | 0 | 0 | | RdbltyKincaid | 0.1 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | AvgAOASen_Cortese | 0.1 | 0.12 | 0.06 | 0.25 | 0.22 | 0 | 0 | 0 | | AvgConnBl_conjunctions | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | AvgPronounSen | 0.09 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | RdbltyFog | 0.09 | 0 | 0 | 0.37 | 0 | 0 | 0 | 0 | | AvgAOADoc_Cortese | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgAOADoc_Kuperman | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgPrepositionSen | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | AvgDepsBl_dep | 0.08 | 0.04 | 0.25 | 0.02 | 0.15 | 0.29 | 0 | 0 | | AvgDepsBl_cc | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bristol | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgConnSen_sentence_link | 0.08 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgDepsSen_neg | 0.08 | 0 | 0.13 | 0 | 0.14 | 0.58 | 0 | 0 | | AvgDepsSen_conj | 0.08 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgDepsBl_mwe | 0.08 | 0 | 0.43 | 0.05 | 0.03 | 0.18 | 0 | 0 | | AvgConnSen_coord_connects | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conjs | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvAverage | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgAOEBl_InvAverage | 0.07 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgAOESen_InvAverage | 0.07 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 | | AvgDepsSen_cc | 0.07 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgConnSen_conjunctions | 0.07 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InfPointPoly | 0.06 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 | | AvgUnqNmdEntBl | 0.06 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 | | LangRhythmId | 0.06 | 0.03 | 0.04 | 0 | 0.33 | 0.37 | 0 | 0 | | AvgAOESen_InvLinRegSlo | 0.05 | 0 | 0 | 0.22 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvLinRegSlo | 0.05 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | AvgDepsBl_auxpass | 0.04 | 0 | 0 | 0.18 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conjs | 0.03 | 0 | 0 | 0.12 | 0 | 0 | 0 | 0 | ## ReaderBench Model 1c This model was trained on spring data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:--------| | -5.6692 | 0.1651 | 0.2625 | 0.1043 | -0.0146 | 0.4555 | 0.1632 | -0.0348 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:--------------------------|:-----|:------|:-----|:-----|:-----|:-----|:------|:------| | WdEnt | 7.76 | 15.63 | 2.27 | 1.53 | 2 | 3.41 | 25.87 | 9.83 | | AvgUnqVerbBl | 5.89 | 24.09 | 2.26 | 1.36 | 2.3 | 3.86 | 0 | 12.86 | | AvgBlScore | 2.6 | 7.12 | 1.53 | 1.13 | 0.58 | 2.67 | 0 | 4.69 | | AvgNounSen | 2.57 | 0.61 | 0.6 | 0.39 | 1.06 | 1.04 | 14.54 | 2.12 | | LxcDiv | 2.18 | 3.58 | 2.18 | 1.33 | 0.72 | 2.54 | 0 | 3.78 | | AvgDepsSen_compound | 2.17 | 1.45 | 1.12 | 0.56 | 0.27 | 1.54 | 7.94 | 2.12 | | WdDiffLemmaStem | 2.07 | 0.57 | 0.93 | 0.49 | 0.22 | 1 | 10.51 | 0 | | AvgDepsBl_dobj | 2.06 | 0.09 | 1.63 | 0.74 | 0.92 | 1.13 | 9.09 | 0 | | AvgUnqPrepositionBl | 1.96 | 1.98 | 2.25 | 1.27 | 1.28 | 2.47 | 0 | 5.6 | | AvgDepsBl_punct | 1.85 | 2.48 | 1.93 | 0.93 | 1.96 | 2.21 | 0 | 6.2 | | AvgDepsBl_nsubj | 1.84 | 3.1 | 1.97 | 1.26 | 0.88 | 2.06 | 0 | 1.36 | | RdbltyFlesch | 1.75 | 0.27 | 0.22 | 0.16 | 1.12 | 0.23 | 12.02 | 0.15 | | AvgWdLen | 1.74 | 3.11 | 1.54 | 0.76 | 0.21 | 2.1 | 0 | 3.03 | | AvgDepsBl_nmod | 1.65 | 2.75 | 1.94 | 0.92 | 0.42 | 1.77 | 0 | 2.42 | | AvgUnqNoundBl | 1.28 | 0.06 | 0.99 | 0.68 | 1.79 | 0.1 | 6.94 | 2.72 | | AvgPronounBl | 1.28 | 0.4 | 1.85 | 1.14 | 0.98 | 1.75 | 0 | 1.21 | | AvgUnqPronounBl | 1.23 | 0.36 | 1.69 | 0.74 | 0.27 | 0.86 | 3 | 0.76 | | WdSylCnt | 1.22 | 0.89 | 1.39 | 0.58 | 1.52 | 1.62 | 0 | 5.45 | | AvgUnqAdjectiveBl | 1.19 | 0.24 | 1.37 | 0.54 | 0.34 | 0.6 | 4.3 | 0.76 | | AvgDepsBl_ccomp | 1.18 | 0.35 | 0.61 | 0.05 | 0.35 | 0.61 | 5.78 | 0.76 | | AvgChainSpan | 1.17 | 0.51 | 1.66 | 0.99 | 0.49 | 1.6 | 0 | 0.91 | | LexChainMaxSp | 1.14 | 0.38 | 1.67 | 0.75 | 1.99 | 1.62 | 0 | 0 | | WdDiffWdStem | 1.11 | 0.76 | 1.38 | 0.78 | 0.06 | 1.6 | 0 | 0 | | AvgDepsBl_det | 1.1 | 0.29 | 1.71 | 0.72 | 0.82 | 1.52 | 0 | 0.76 | | WdLettStdDev | 1.02 | 1.03 | 1.66 | 0.85 | 1.56 | 1.04 | 0 | 0.3 | | FrqRhythmId | 1 | 0.65 | 1.44 | 0.59 | 0.95 | 1.32 | 0 | 0.76 | | AvgAOABl_Shock | 0.97 | 0.91 | 1.13 | 0.67 | 0.02 | 1.34 | 0 | 0 | | AvgSenBlCoh_LDA | 0.97 | 1.01 | 1.13 | 0.82 | 1.53 | 1.24 | 0 | 0 | | AvgDepsBl_mark | 0.96 | 0.04 | 1.52 | 0.65 | 1.46 | 1.4 | 0 | 0 | | AvgSenAdjCoh_word2vec | 0.9 | 0.39 | 1.37 | 0.67 | 0.29 | 1.18 | 0 | 1.06 | | TCorefChainDoc | 0.87 | 0.32 | 1.67 | 0.71 | 2.43 | 0.96 | 0 | 0 | | AvgDepsSen_punct | 0.81 | 0.4 | 1 | 0.68 | 0.26 | 1.2 | 0 | 0 | | LangRhythmCoeff | 0.8 | 0.47 | 0.95 | 0.46 | 0.2 | 1.23 | 0 | 0 | | AvgPronBl_first_person | 0.78 | 0.58 | 1.31 | 0.44 | 0.15 | 0.86 | 0 | 1.66 | | RdbltyDaleChall | 0.78 | 1.44 | 0.96 | 0.38 | 1.08 | 0.76 | 0 | 1.06 | | AvgConnBl_sentence_link | 0.74 | 0.09 | 1.31 | 0.45 | 3.5 | 0.92 | 0 | 0.3 | | AvgDepsBl_compound | 0.74 | 1.08 | 0.77 | 0.12 | 1.07 | 0.86 | 0 | 3.48 | | AvgConnSen_logical_conns | 0.71 | 0.31 | 0.54 | 0.51 | 0.72 | 1.24 | 0 | 0.76 | | LexChainAvgSpan | 0.7 | 0.67 | 1.02 | 0.62 | 0.5 | 0.81 | 0 | 0 | | AvgUnqAdverbBl | 0.69 | 0.01 | 1.46 | 0.57 | 0.78 | 0.75 | 0 | 0.76 | | CharEnt | 0.66 | 0.48 | 1.35 | 0.78 | 1.51 | 0.49 | 0 | 0.76 | | AvgDepsBl_amod | 0.66 | 0.23 | 1.31 | 0.48 | 1.2 | 0.73 | 0 | 0 | | AvgRhythmUnitStreesSyll | 0.65 | 0.39 | 0.43 | 0.21 | 0.86 | 1.22 | 0 | 0 | | AvgAdjectiveSen | 0.64 | 0.7 | 0.68 | 0.45 | 0.03 | 0.78 | 0 | 2.72 | | AvgDepsBl_advcl | 0.62 | 0.07 | 1.21 | 0.42 | 1.83 | 0.74 | 0 | 0 | | AvgConnBl_simp_subords | 0.61 | 0.04 | 1.25 | 0.45 | 0.28 | 0.69 | 0 | 0.76 | | AvgConnBl_reas_purp | 0.61 | 0.57 | 0.98 | 0.27 | 0.17 | 0.74 | 0 | 0 | | AvgCorefChain | 0.6 | 0.54 | 1.17 | 0.48 | 1.94 | 0.52 | 0 | 0 | | AvgPronBl_indefinite | 0.59 | 0.2 | 1.33 | 0.5 | 0.05 | 0.57 | 0 | 0 | | AvgDepsBl_xcomp | 0.58 | 0.36 | 1.14 | 0.37 | 0.09 | 0.63 | 0 | 0 | | AvgBlVoiceCoOcc | 0.57 | 0 | 1.44 | 0.56 | 0.17 | 0.51 | 0 | 0 | | AvgDepsBl_aux | 0.56 | 0.17 | 1.12 | 0.32 | 1.5 | 0.64 | 0 | 0 | | AvgDepsBl_mwe | 0.55 | 0 | 0.93 | 0.23 | 1.62 | 0.79 | 0 | 0 | | AvgPronBl_third_person | 0.55 | 0.13 | 1.23 | 0.39 | 0.24 | 0.53 | 0 | 1.06 | | AggPronSen_third_person | 0.55 | 0.37 | 0.47 | 0.21 | 2.96 | 0.87 | 0 | 0.76 | | AvgAOABl_Cortese | 0.55 | 0.56 | 0.64 | 0.7 | 0.31 | 0.68 | 0 | 0 | | AvgDepsBl_neg | 0.54 | 0.08 | 0.71 | 0.15 | 0.29 | 0.91 | 0 | 0 | | AvgConnSen_order | 0.54 | 0.06 | 0.26 | 0.52 | 0.98 | 1.07 | 0 | 0 | | AvgDepsSen_amod | 0.53 | 0.43 | 0.6 | 0.49 | 0.07 | 0.72 | 0 | 0.61 | | AvgInferenceDistChain | 0.53 | 0.52 | 0.46 | 0.31 | 1.67 | 0.81 | 0 | 0 | | TCorefChainBigSpan | 0.52 | 0.04 | 1.24 | 0.35 | 0.13 | 0.53 | 0 | 0 | | SenAsson | 0.51 | 0.33 | 0.85 | 0.21 | 1.05 | 0.63 | 0 | 0.15 | | AvgConnBl_contrasts | 0.5 | 0.06 | 0.85 | 0.21 | 0 | 0.72 | 0 | 0 | | AggPronSen_indefinite | 0.49 | 0.31 | 0.15 | 0.39 | 0.83 | 0.93 | 0 | 0.76 | | AvgDepsSen_xcomp | 0.48 | 0.76 | 0.22 | 0.37 | 0.33 | 0.72 | 0 | 0 | | AvgConnBl_temp_conns | 0.47 | 0.2 | 1.04 | 0.26 | 0.13 | 0.46 | 0 | 0.61 | | AvgDepsBl_cop | 0.46 | 0.25 | 0.92 | 0.23 | 0.13 | 0.5 | 0 | 0 | | AvgAOASen_Kuperman | 0.46 | 0.19 | 0.45 | 0.37 | 1.7 | 0.71 | 0 | 0.61 | | AvgDepsSen_nmod | 0.46 | 0.2 | 0.11 | 0.46 | 1.37 | 0.74 | 0 | 4.39 | | AvgDepsSen_dobj | 0.45 | 0.17 | 0.44 | 0.42 | 0.55 | 0.74 | 0 | 0 | | AvgDepsBl_nummod | 0.44 | 0.07 | 0.35 | 0.02 | 0.46 | 0.88 | 0 | 0 | | AvgAOEBl_IndexPolyFAT.3 | 0.44 | 0.19 | 0.42 | 0.4 | 1.29 | 0.71 | 0 | 0 | | AvgConnBl_order | 0.42 | 0.05 | 0.62 | 0.11 | 0.53 | 0.67 | 0 | 0 | | LxcSoph | 0.42 | 0.28 | 0.06 | 0.07 | 0.93 | 0.88 | 0 | 0.76 | | AvgDepsSen_neg | 0.42 | 0.06 | 0.25 | 0.02 | 0.84 | 0.91 | 0 | 0 | | AvgConnSen_simp_subords | 0.41 | 0.36 | 0.07 | 0.56 | 0.46 | 0.74 | 0 | 0 | | SenStdDevWd | 0.4 | 0.08 | 0.88 | 0.66 | 0.4 | 0.32 | 0 | 0 | | AvgConnBl_oppositions | 0.4 | 0.02 | 0.95 | 0.24 | 1.8 | 0.37 | 0 | 0 | | AvgDepsSen_ccomp | 0.4 | 0.1 | 0.57 | 0.44 | 0.58 | 0.47 | 0 | 2.12 | | AvgAOABl_Kuperman | 0.39 | 0.38 | 0.37 | 0.53 | 0.22 | 0.51 | 0 | 0 | | AvgRhythmUnits | 0.39 | 0.28 | 0.16 | 0.48 | 0.22 | 0.68 | 0 | 0 | | AvgAOEBl_InvLinRegSlo | 0.37 | 0.13 | 0.67 | 0.33 | 0.23 | 0.41 | 0 | 0.76 | | TActCorefChainWd | 0.37 | 0.29 | 0.51 | 0.26 | 0.8 | 0.49 | 0 | 0 | | AvgPronounSen | 0.36 | 0.32 | 0.6 | 0.37 | 0.49 | 0.33 | 0 | 0.76 | | AvgDepsSen_mwe | 0.36 | 0.01 | 0.33 | 0.03 | 0.68 | 0.73 | 0 | 0 | | AvgAOASen_Bristol | 0.35 | 0.6 | 0.24 | 0.13 | 0.88 | 0.42 | 0 | 1.36 | | LangRhythmDiameter | 0.34 | 0.13 | 0.23 | 0.01 | 0.31 | 0.68 | 0 | 0 | | AvgCommaBl | 0.34 | 0.07 | 0.66 | 0.12 | 0.49 | 0.43 | 0 | 0 | | AvgAOASen_Cortese | 0.34 | 0.33 | 0.69 | 0.45 | 0.27 | 0.24 | 0 | 0 | | AvgDepsSen_mark | 0.34 | 0.11 | 0.22 | 0.54 | 0.34 | 0.58 | 0 | 0 | | AvgDepsSen_acl | 0.33 | 0.04 | 0.71 | 0.13 | 0.93 | 0.38 | 0 | 0 | | AvgConnBl_logical_conns | 0.33 | 0.05 | 0.53 | 0.06 | 0.26 | 0.51 | 0 | 0 | | WdPathCntHypernymTree | 0.32 | 0.52 | 0.45 | 0.13 | 0.56 | 0.32 | 0 | 0 | | AvgConnBl_semi_coords | 0.32 | 0.14 | 0.78 | 0.16 | 0.43 | 0.27 | 0 | 0 | | AvgDepsSen_cop | 0.31 | 0.2 | 0.47 | 0.47 | 0.17 | 0.34 | 0 | 0 | | AvgAOESen_InfPointPoly | 0.3 | 0.34 | 0.31 | 0.16 | 1.12 | 0.4 | 0 | 0 | | AvgConnSen_semi_coords | 0.3 | 0.06 | 0.01 | 0.34 | 0.68 | 0.64 | 0 | 0 | | AvgAOASen_Shock | 0.3 | 0.28 | 0.21 | 0.49 | 0.06 | 0.4 | 0 | 0.61 | | AvgConnSen_oppositions | 0.3 | 0.08 | 0.02 | 0 | 0.39 | 0.73 | 0 | 0 | | AvgDepsSen_advmod | 0.29 | 0.32 | 0.18 | 0.47 | 0.28 | 0.42 | 0 | 0 | | AvgConnBl_addition | 0.28 | 0.18 | 0.64 | 0.1 | 2.79 | 0.2 | 0 | 0 | | WdPolysemyCnt | 0.28 | 0.33 | 0.11 | 0.6 | 0.09 | 0.39 | 0 | 0 | | AvgAOABl_Bristol | 0.28 | 0.19 | 0.22 | 0.38 | 0.49 | 0.43 | 0 | 0 | | AvgNmdEntSen | 0.27 | 0.26 | 0.54 | 0.39 | 0.36 | 0.19 | 0 | 0 | | AvgAOEBl_InfPointPoly | 0.27 | 0.2 | 0.35 | 0.23 | 2.11 | 0.3 | 0 | 0.76 | | AvgConnSen_temp_conns | 0.27 | 0.35 | 0.12 | 0 | 0.93 | 0.48 | 0 | 0 | | WdAvgDpthHypernymTree | 0.27 | 0.43 | 0.4 | 0.2 | 0.15 | 0.25 | 0 | 0 | | AvgDepsSen_dep | 0.27 | 0.06 | 0.38 | 0.28 | 2.35 | 0.34 | 0 | 0.3 | | AvgAOESen_InvLinRegSlo | 0.26 | 0.27 | 0.52 | 0.25 | 0.02 | 0.19 | 0 | 0.3 | | AvgAOASen_Bird | 0.26 | 0.53 | 0.04 | 0.16 | 0.75 | 0.39 | 0 | 0.15 | | AvgNmdEntBl | 0.26 | 0.17 | 0.48 | 0.04 | 1.53 | 0.3 | 0 | 0 | | AvgConnSen_reas_purp | 0.25 | 0.12 | 0.08 | 0.48 | 1.14 | 0.4 | 0 | 0 | | AvgDepsSen_det | 0.25 | 0.11 | 0.1 | 0.25 | 0.26 | 0.44 | 0 | 0.76 | | AvgAOABl_Bird | 0.23 | 0.7 | 0.21 | 0.33 | 0.44 | 0.13 | 0 | 0 | | AvgAOESen_IndexPolyFAT.3 | 0.23 | 0.11 | 0.28 | 0.31 | 0.64 | 0.31 | 0 | 0 | | AvgDepsBl_dep | 0.22 | 0.1 | 0.44 | 0.05 | 1.31 | 0.22 | 0 | 0.61 | | AvgDepsBl_nsubjpass | 0.21 | 0.02 | 0.84 | 0.2 | 0.07 | 0 | 0 | 0 | | AvgAOESen_IndexAbThr.0.3. | 0.19 | 0.27 | 0.12 | 0.39 | 0.05 | 0.22 | 0 | 0 | | AvgDepsSen_aux | 0.18 | 0.39 | 0 | 0.49 | 2.05 | 0.17 | 0 | 0 | | AvgUnqWdBl | 0.15 | 0 | 0 | 1.66 | 0 | 0 | 0 | 0 | | AvgBlLen | 0.15 | 0 | 0 | 1.68 | 0 | 0 | 0 | 0 | | AvgDepsBl_acl | 0.14 | 0.01 | 0.42 | 0.04 | 0.2 | 0.1 | 0 | 0 | | AvgVerbBl | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | Content.words | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | AvgWdBl | 0.14 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | AvgNounNmdEntBl | 0.14 | 0.29 | 0.05 | 0 | 1.08 | 0.21 | 0 | 0 | | Words | 0.13 | 0 | 0 | 1.47 | 0 | 0 | 0 | 0 | | AvgPrepositionBl | 0.11 | 0 | 0 | 1.22 | 0 | 0 | 0 | 0 | | AvgDepsSen_advcl | 0.1 | 0.14 | 0.05 | 0.52 | 0.09 | 0.05 | 0 | 0 | | AvgDepsBl_case | 0.09 | 0 | 0 | 0.96 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LDA | 0.09 | 0 | 0 | 0.99 | 0 | 0 | 0 | 0 | | LangRhythmId | 0.09 | 0.01 | 0.19 | 0 | 0.98 | 0.1 | 0 | 0 | | AvgIntraBlCoh_Path | 0.08 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LDA | 0.08 | 0 | 0 | 0.87 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LSA | 0.08 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LSA | 0.07 | 0 | 0 | 0.74 | 0 | 0 | 0 | 0 | | SenScoreStDev | 0.07 | 0 | 0 | 0.75 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_Path | 0.07 | 0 | 0 | 0.76 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_word2vec | 0.07 | 0 | 0 | 0.79 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_Path | 0.07 | 0 | 0 | 0.81 | 0 | 0 | 0 | 0 | | AvgNounBl | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 | | Sentences | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 | | AvgSenBl | 0.07 | 0 | 0 | 0.83 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LSA | 0.07 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_WuPalmer | 0.06 | 0 | 0 | 0.64 | 0 | 0 | 0 | 0 | | SenStDevUnqWd | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_LeackChod | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgSenAdjCoh_LeackChod | 0.06 | 0 | 0 | 0.66 | 0 | 0 | 0 | 0 | | AvgAOADoc_Shock | 0.06 | 0 | 0 | 0.67 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_WuPalmer | 0.06 | 0 | 0 | 0.68 | 0 | 0 | 0 | 0 | | AvgSenBlCoh_WuPalmer | 0.06 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_LeackChod | 0.06 | 0 | 0 | 0.69 | 0 | 0 | 0 | 0 | | AvgIntraBlCoh_word2vec | 0.06 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgAOADoc_Cortese | 0.06 | 0 | 0 | 0.7 | 0 | 0 | 0 | 0 | | AvgVoice | 0.05 | 0 | 0 | 0.5 | 0 | 0 | 0 | 0 | | AvgConnSen_sentence_link | 0.05 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgVerbSen | 0.05 | 0 | 0 | 0.51 | 0 | 0 | 0 | 0 | | AvgDepsSen_nsubj | 0.05 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgAOADoc_Kuperman | 0.05 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | AvgConnSen_conjunctions | 0.05 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_connects | 0.05 | 0 | 0 | 0.54 | 0 | 0 | 0 | 0 | | AvgAdjectiveBl | 0.05 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgDepsSen_case | 0.05 | 0 | 0 | 0.56 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexPolyFAT.3 | 0.04 | 0 | 0 | 0.4 | 0 | 0 | 0 | 0 | | AvgPrepositionSen | 0.04 | 0 | 0 | 0.41 | 0 | 0 | 0 | 0 | | AvgAdverbSen | 0.04 | 0 | 0 | 0.43 | 0 | 0 | 0 | 0 | | AvgSenSyll | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgAOEDoc_IndexAbThr.0.3. | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgAOEBl_IndexAbThr.0.3. | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgConnSen_addition | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgSemDep | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgDepsSen_cc | 0.04 | 0 | 0 | 0.45 | 0 | 0 | 0 | 0 | | AvgDepsBl_advmod | 0.04 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | AvgWdSen | 0.03 | 0 | 0 | 0.29 | 0 | 0 | 0 | 0 | | AvgSenStressedSyll | 0.03 | 0 | 0 | 0.3 | 0 | 0 | 0 | 0 | | AvgConnSen_contrasts | 0.03 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgSenScore | 0.03 | 0 | 0 | 0.31 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvLinRegSlo | 0.03 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bird | 0.03 | 0 | 0 | 0.33 | 0 | 0 | 0 | 0 | | AvgConnSen_coord_conjs | 0.03 | 0 | 0 | 0.34 | 0 | 0 | 0 | 0 | | AvgDepsSen_conj | 0.03 | 0 | 0 | 0.35 | 0 | 0 | 0 | 0 | | SynSoph | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgAOADoc_Bristol | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgAdverbBl | 0.03 | 0 | 0 | 0.38 | 0 | 0 | 0 | 0 | | AvgAOESen_InvAverage | 0.02 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_connects | 0.02 | 0 | 0 | 0.2 | 0 | 0 | 0 | 0 | | AvgDepsBl_auxpass | 0.02 | 0 | 0 | 0.22 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InfPointPoly | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | WdMaxDpthHypernymTree | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | AvgAOEDoc_InvAverage | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | AvgAOEBl_InvAverage | 0.02 | 0 | 0 | 0.23 | 0 | 0 | 0 | 0 | | RdbltyKincaid | 0.02 | 0 | 0 | 0.24 | 0 | 0 | 0 | 0 | | AvgUnqWdSen | 0.02 | 0 | 0 | 0.25 | 0 | 0 | 0 | 0 | | RdbltyFog | 0.02 | 0 | 0 | 0.27 | 0 | 0 | 0 | 0 | | AvgRhythmUnitSyll | 0.02 | 0 | 0 | 0.27 | 0 | 0 | 0 | 0 | | AvgSenLen | 0.02 | 0 | 0 | 0.28 | 0 | 0 | 0 | 0 | | AvgDepsBl_conj | 0.01 | 0 | 0 | 0.11 | 0 | 0 | 0 | 0 | | AvgConnBl_conjunctions | 0.01 | 0 | 0 | 0.13 | 0 | 0 | 0 | 0 | | AvgDepsBl_cc | 0.01 | 0 | 0 | 0.14 | 0 | 0 | 0 | 0 | | AvgConnBl_coord_conjs | 0.01 | 0 | 0 | 0.16 | 0 | 0 | 0 | 0 | | AvgUnqNmdEntBl | 0 | 0 | 0 | 0.05 | 0 | 0 | 0 | 0 | ## ReaderBench Model 1d This model was trained on principal component scores for fall data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:-------| | -8.4195 | 0.0406 | 0.8127 | 0.0694 | -0.0509 | 0.1058 | 0.0038 | 0.0448 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). PC1 = scores on 1st principal component extracted, ... Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:-----|:------|:------|:----|:------|:-----|:------|:------| | PC2 | 50.2 | 61.92 | 52.85 | 0 | 12.57 | 36.4 | 46.85 | 23.81 | | PC1 | 8.03 | 1.95 | 9.25 | 0 | 1.38 | 5.78 | 0.27 | 8.39 | | PC3 | 7.38 | 1.56 | 8.43 | 0 | 3.57 | 5.47 | 0 | 8.39 | | PC5 | 5.42 | 3.46 | 5.74 | 0 | 5.67 | 3.71 | 0 | 13.83 | | PC4 | 2 | 1.81 | 2.27 | 0 | 1.61 | 1.06 | 0 | 0 | | PC24 | 1.9 | 0.9 | 1.48 | 0 | 6.31 | 1.73 | 16.76 | 1.59 | | PC14 | 1.7 | 1.4 | 1.56 | 0 | 3.22 | 1.85 | 3.63 | 3.85 | | PC8 | 1.47 | 0.85 | 1.45 | 0 | 1.88 | 2.26 | 0 | 1.59 | | PC30 | 1.47 | 1.67 | 1.1 | 0 | 6.24 | 2.51 | 0 | 6.58 | | PC6 | 1.09 | 0.73 | 0.95 | 0 | 0.78 | 2.84 | 0 | 0 | | PC31 | 1.09 | 1.01 | 0.92 | 0 | 5.49 | 0.03 | 0 | 9.98 | | PC7 | 1.03 | 0.17 | 1.17 | 0 | 1.27 | 0.95 | 0 | 0 | | PC17 | 1.02 | 1.53 | 0.62 | 0 | 1.24 | 3.12 | 0 | 4.31 | | PC43 | 0.94 | 1.87 | 0.58 | 0 | 5.88 | 1.5 | 0 | 4.76 | | PC33 | 0.87 | 0.6 | 0.88 | 0 | 5.75 | 0.44 | 0 | 0 | | PC32 | 0.82 | 0.21 | 0.6 | 0 | 3.29 | 2.36 | 0 | 1.59 | | PC39 | 0.81 | 1.14 | 0.54 | 0 | 4.22 | 1.87 | 1.81 | 0 | | PC27 | 0.77 | 0.48 | 0.66 | 0 | 2.74 | 0.62 | 0 | 4.99 | | PC13 | 0.73 | 1.28 | 0.73 | 0 | 1.02 | 0.55 | 0 | 0 | | PC34 | 0.7 | 0.46 | 0.2 | 0 | 0.43 | 2.05 | 13.64 | 0 | | PC38 | 0.67 | 0.36 | 0.55 | 0 | 4.08 | 0.86 | 0 | 2.72 | | PC23 | 0.64 | 0.5 | 0.72 | 0 | 2.45 | 0.1 | 0 | 0 | | PC19 | 0.6 | 0.9 | 0.56 | 0 | 1.28 | 0.84 | 0 | 0 | | PC16 | 0.6 | 0.67 | 0.22 | 0 | 0 | 2.25 | 6.99 | 0 | | PC45 | 0.6 | 1.92 | 0.07 | 0 | 0 | 1.77 | 10.06 | 0.91 | | PC20 | 0.56 | 0 | 0.53 | 0 | 1.3 | 1.26 | 0 | 0 | | PC28 | 0.54 | 1.55 | 0.39 | 0 | 1.29 | 0.94 | 0 | 0.45 | | PC40 | 0.54 | 0.77 | 0.52 | 0 | 4.01 | 0.04 | 0 | 0.45 | | PC18 | 0.53 | 0.36 | 0.47 | 0 | 0.92 | 1.27 | 0 | 0 | | PC11 | 0.51 | 0.87 | 0.55 | 0 | 0.48 | 0.18 | 0 | 0 | | PC22 | 0.49 | 0.35 | 0.42 | 0 | 0.99 | 1.2 | 0 | 0 | | PC36 | 0.46 | 0.95 | 0.09 | 0 | 0 | 3.14 | 0 | 0 | | PC12 | 0.45 | 0.22 | 0.46 | 0 | 0.33 | 0.68 | 0 | 0 | | PC29 | 0.44 | 0.95 | 0.29 | 0 | 0.79 | 1.36 | 0 | 0 | | PC41 | 0.43 | 0.27 | 0.35 | 0 | 2.57 | 0.79 | 0 | 0 | | PC42 | 0.4 | 0.32 | 0.31 | 0 | 2.25 | 1.03 | 0 | 0 | | PC9 | 0.37 | 0.34 | 0.4 | 0 | 0.13 | 0.33 | 0 | 0 | | PC44 | 0.29 | 0.31 | 0.22 | 0 | 1.53 | 0.68 | 0 | 0 | | PC46 | 0.28 | 0.91 | 0.05 | 0 | 0 | 1.65 | 0 | 0.45 | | PC15 | 0.26 | 0.87 | 0.14 | 0 | 0 | 0.59 | 0 | 1.36 | | PC26 | 0.25 | 0.16 | 0.29 | 0 | 0.59 | 0 | 0 | 0 | | PC35 | 0.23 | 0.02 | 0.19 | 0 | 0.43 | 0.7 | 0 | 0 | | PC37 | 0.13 | 0.09 | 0.13 | 0 | 0.03 | 0.22 | 0 | 0 | | PC10 | 0.12 | 0.4 | 0 | 0 | 0 | 0.99 | 0 | 0 | | PC25 | 0.08 | 0.29 | 0.08 | 0 | 0 | 0.03 | 0 | 0 | | PC21 | 0.06 | 0.64 | 0.03 | 0 | 0 | 0.01 | 0 | 0 | ### Proportion of Variance by Varimax Rotated Component (RC) Due to space limitations, loadings for only the first five principal components are displayed. | Variable | RC1 | RC2 | RC3 | RC5 | RC4 | |:----------------------|:------|:------|:------|:------|:-----| | SS loadings | 44.56 | 31.07 | 19.29 | 10.07 | 9.38 | | Proportion Var | 0.22 | 0.15 | 0.10 | 0.05 | 0.05 | | Cumulative Var | 0.22 | 0.38 | 0.47 | 0.52 | 0.57 | | Proportion Explained | 0.39 | 0.27 | 0.17 | 0.09 | 0.08 | | Cumulative Proportion | 0.39 | 0.66 | 0.83 | 0.92 | 1.00 | ### Varimax Rotated Loadings | Metric | RC1 | RC2 | RC3 | RC5 | RC4 | |:---------------------------------------|:------:|:------:|:------:|:------:|:------:| | Sentences | -0.589 | 0.59 | -0.023 | -0.072 | -0.157 | | Words | 0.086 | 0.947 | 0.097 | 0.203 | 0.039 | | Content.words | -0.006 | 0.908 | 0.091 | 0.119 | 0.153 | | RdbltyFlesch | -0.891 | -0.159 | -0.034 | -0.06 | 0.033 | | RdbltyFog | 0.95 | 0.117 | 0.07 | 0.127 | -0.022 | | RdbltyKincaid | 0.948 | 0.116 | 0.044 | 0.128 | -0.02 | | RdbltyDaleChall | 0.4 | -0.276 | -0.098 | 0.136 | -0.503 | | AvgBlLen | -0.041 | 0.903 | 0.095 | 0.055 | 0.156 | | AvgCommaBl | -0.1 | 0.309 | 0.05 | -0.227 | 0.006 | | AvgSenLen | 0.91 | 0.198 | 0.058 | 0.044 | 0.17 | | AvgSenBl | -0.589 | 0.59 | -0.023 | -0.072 | -0.157 | | AvgUnqWdBl | -0.015 | 0.901 | 0.103 | 0.081 | 0.118 | | AvgUnqWdSen | 0.922 | 0.161 | 0.089 | 0.096 | 0.134 | | AvgWdLen | -0.089 | 0.407 | 0.562 | -0.271 | 0.362 | | AvgWdBl | -0.006 | 0.908 | 0.091 | 0.119 | 0.153 | | AvgWdSen | 0.931 | 0.147 | 0.057 | 0.086 | 0.135 | | CharEnt | -0.006 | 0.489 | 0.287 | -0.198 | 0.059 | | SenStDevUnqWd | -0.429 | 0.328 | 0.099 | 0.075 | 0.453 | | SenStdDevWd | -0.354 | 0.326 | 0.076 | 0.112 | 0.472 | | WdEnt | 0.01 | 0.886 | 0.21 | 0.066 | 0.085 | | WdLettStdDev | -0.158 | 0.401 | 0.362 | -0.257 | 0.224 | | LxcDiv | 0.065 | 0.791 | 0.172 | 0.047 | 0.204 | | LxcSoph | 0.334 | 0.207 | 0.6 | -0.068 | 0.407 | | SynSoph | 0.825 | 0.348 | 0.092 | 0.118 | 0.259 | | AvgNounBl | -0.017 | 0.705 | 0.149 | 0.361 | 0.031 | | AvgPronounBl | 0.159 | 0.77 | 0.042 | 0.026 | -0.004 | | AvgVerbBl | 0.102 | 0.911 | 0.04 | 0.051 | 0.058 | | AvgAdverbBl | 0.043 | 0.697 | 0.012 | -0.126 | -0.103 | | AvgAdjectiveBl | -0.011 | 0.517 | 0.115 | 0.028 | 0.031 | | AvgPrepositionBl | 0.078 | 0.786 | 0.06 | 0.154 | -0.004 | | AvgNounSen | 0.834 | -0.018 | 0.109 | 0.357 | -0.019 | | AvgPronounSen | 0.928 | 0.03 | 0.053 | 0.017 | -0.023 | | AvgVerbSen | 0.957 | 0.1 | 0.031 | 0.013 | 0.042 | | AvgAdverbSen | 0.701 | 0.22 | -0.042 | -0.147 | -0.05 | | AvgAdjectiveSen | 0.699 | 0.007 | 0.137 | 0.004 | -0.055 | | AvgPrepositionSen | 0.803 | 0.212 | 0.018 | 0.139 | -0.013 | | AvgUnqNoundBl | -0.014 | 0.632 | 0.13 | 0.431 | -0.023 | | AvgUnqPronounBl | -0.007 | 0.533 | 0.088 | 0.074 | 0.058 | | AvgUnqVerbBl | 0.122 | 0.858 | 0.047 | 0.038 | 0.043 | | AvgUnqAdverbBl | -0.007 | 0.719 | 0.011 | -0.17 | -0.142 | | AvgUnqAdjectiveBl | -0.025 | 0.523 | 0.104 | -0.022 | 0.022 | | AvgUnqPrepositionBl | 0.054 | 0.813 | 0.061 | 0.075 | -0.009 | | AvgPronBl_first_person | 0.168 | 0.637 | 0.017 | 0.058 | -0.022 | | AvgPronBl_indefinite | 0.011 | 0.577 | 0.018 | 0.033 | 0.138 | | AggPronSen_indefinite | 0.677 | 0.159 | -0.005 | 0.004 | 0.112 | | AvgPronBl_third_person | 0.131 | 0.424 | 0.057 | -0.018 | 0.019 | | AggPronSen_third_person | 0.782 | -0.078 | 0.043 | -0.033 | 0.007 | | AvgSemDep | 0.967 | 0.07 | 0.088 | 0.181 | 0.001 | | WdDiffLemmaStem | -0.016 | 0.289 | 0.104 | 0.007 | -0.106 | | WdDiffWdStem | 0.041 | 0.483 | 0.031 | -0.313 | 0.132 | | WdMaxDpthHypernymTree | -0.08 | 0.083 | 0.356 | 0.059 | 0.349 | | WdAvgDpthHypernymTree | -0.064 | 0.073 | 0.369 | 0.063 | 0.349 | | WdPathCntHypernymTree | -0.066 | 0.123 | 0.217 | -0.019 | 0.464 | | WdPolysemyCnt | 0 | -0.045 | -0.037 | 0.192 | 0.244 | | WdSylCnt | -0.126 | 0.215 | 0.592 | -0.117 | 0.108 | | AvgAOADoc_Shock | -0.084 | 0.426 | 0.396 | 0.15 | 0.04 | | AvgAOABl_Shock | -0.084 | 0.426 | 0.396 | 0.15 | 0.04 | | AvgAOASen_Shock | 0.289 | 0.196 | 0.419 | 0.105 | 0.111 | | AvgAOADoc_Cortese | 0.011 | 0.056 | 0.743 | -0.088 | 0.221 | | AvgAOABl_Cortese | 0.011 | 0.056 | 0.743 | -0.088 | 0.221 | | AvgAOASen_Cortese | 0.261 | 0.168 | 0.575 | 0.045 | 0.211 | | AvgAOADoc_Kuperman | -0.016 | 0.083 | 0.785 | 0.109 | -0.07 | | AvgAOABl_Kuperman | -0.016 | 0.083 | 0.785 | 0.109 | -0.07 | | AvgAOASen_Kuperman | 0.1 | 0.18 | 0.742 | 0.068 | -0.023 | | AvgAOADoc_Bird | 0.011 | 0.057 | 0.76 | -0.004 | 0.135 | | AvgAOABl_Bird | 0.011 | 0.057 | 0.76 | -0.004 | 0.135 | | AvgAOASen_Bird | 0.262 | 0.14 | 0.554 | 0.029 | 0.173 | | AvgAOADoc_Bristol | 0.018 | 0.308 | 0.513 | 0.022 | 0.201 | | AvgAOABl_Bristol | 0.018 | 0.308 | 0.513 | 0.022 | 0.201 | | AvgAOASen_Bristol | 0.413 | 0.102 | 0.453 | 0.113 | 0.185 | | AvgAOEDoc_IndexPolyFitAbThr.0.3. | -0.049 | 0.027 | 0.591 | 0.176 | -0.597 | | AvgAOEBl_IndexPolyFitAbThr.0.3. | -0.049 | 0.027 | 0.591 | 0.176 | -0.597 | | AvgAOESen_IndexPolyFitAbThr.0.3. | 0.006 | 0.133 | 0.617 | 0.152 | -0.495 | | AvgAOEDoc_InverseLinearRegressionSlope | -0.082 | 0.015 | 0.857 | 0.089 | -0.285 | | AvgAOEBl_InverseLinearRegressionSlope | -0.082 | 0.015 | 0.857 | 0.089 | -0.285 | | AvgAOESen_InverseLinearRegressionSlope | 0.077 | 0.163 | 0.795 | 0.054 | -0.145 | | AvgAOEDoc_InflectionPointPolynomial | -0.099 | 0.06 | 0.848 | -0.017 | -0.214 | | AvgAOEBl_InflectionPointPolynomial | -0.099 | 0.06 | 0.848 | -0.017 | -0.214 | | AvgAOESen_InflectionPointPolynomial | 0.064 | 0.198 | 0.8 | -0.021 | -0.076 | | AvgAOEDoc_InverseAverage | -0.102 | 0.053 | 0.857 | -0.014 | -0.243 | | AvgAOEBl_InverseAverage | -0.102 | 0.053 | 0.857 | -0.014 | -0.243 | | AvgAOESen_InverseAverage | 0.06 | 0.191 | 0.808 | -0.021 | -0.097 | | AvgAOEDoc_IndexAboveThreshold.0.3. | -0.093 | 0.045 | 0.476 | 0.179 | -0.636 | | AvgAOEBl_IndexAboveThreshold.0.3. | -0.093 | 0.045 | 0.476 | 0.179 | -0.636 | | AvgAOESen_IndexAboveThreshold.0.3. | -0.071 | 0.107 | 0.491 | 0.174 | -0.556 | | AvgNmdEntBl | 0.057 | 0.52 | -0.052 | 0.139 | 0.032 | | AvgNounNmdEntBl | 0.088 | 0.363 | -0.01 | 0.198 | -0.015 | | AvgUnqNmdEntBl | 0.119 | 0.537 | -0.058 | 0.14 | -0.003 | | AvgNmdEntSen | 0.752 | 0.106 | -0.084 | 0.167 | 0.06 | | TCorefChainDoc | -0.03 | 0.621 | 0.189 | 0.003 | -0.062 | | AvgCorefChain | 0.143 | 0.47 | 0.111 | 0.016 | 0.079 | | AvgChainSpan | 0.088 | 0.729 | 0.038 | -0.003 | 0.128 | | AvgInferenceDistChain | 0.245 | 0.306 | 0.046 | 0.001 | -0.012 | | TActCorefChainWd | -0.092 | -0.329 | 0.268 | -0.225 | -0.102 | | TCorefChainBigSpan | 0.108 | 0.426 | 0.243 | -0.098 | 0.002 | | AvgConnBl_addition | 0.067 | 0.309 | 0.032 | 0.777 | 0.055 | | AvgConnSen_addition | 0.658 | -0.166 | 0.072 | 0.622 | -0.077 | | AvgConnBl_conjunctions | 0.168 | 0.362 | 0.061 | 0.775 | -0.017 | | AvgConnSen_conjunctions | 0.72 | -0.147 | 0.097 | 0.579 | -0.152 | | AvgConnBl_contrasts | 0.076 | 0.444 | 0.114 | 0.035 | -0.118 | | AvgConnSen_contrasts | 0.512 | 0.196 | 0.125 | -0.059 | -0.149 | | AvgConnBl_coordinating_conjuncts | 0.381 | 0.45 | -0.054 | 0.141 | 0.092 | | AvgConnSen_coordinating_conjuncts | 0.72 | 0.205 | -0.012 | -0.003 | -0.013 | | AvgConnBl_coordinating_connectives | 0.255 | 0.506 | 0.035 | 0.7 | 0.003 | | AvgConnSen_coordinating_connectives | 0.818 | -0.034 | 0.071 | 0.48 | -0.119 | | AvgConnBl_logical_connectors | 0.186 | 0.317 | 0.046 | 0.76 | -0.008 | | AvgConnSen_logical_connectors | 0.693 | -0.154 | 0.069 | 0.563 | -0.121 | | AvgConnBl_oppositions | 0.096 | 0.391 | 0.08 | 0.048 | -0.132 | | AvgConnSen_oppositions | 0.494 | 0.14 | 0.106 | -0.083 | -0.187 | | AvgConnBl_order | -0.115 | 0.255 | -0.013 | 0.188 | 0.176 | | AvgConnSen_order | 0.315 | 0.084 | -0.013 | 0.115 | 0.215 | | AvgConnBl_reason_and_purpose | 0.204 | 0.536 | -0.017 | 0.194 | 0.163 | | AvgConnSen_reason_and_purpose | 0.71 | 0.226 | 0.002 | 0.034 | 0.064 | | AvgConnBl_semi_coordinators | 0.381 | 0.45 | -0.054 | 0.141 | 0.092 | | AvgConnSen_semi_coordinators | 0.72 | 0.205 | -0.012 | -0.003 | -0.013 | | AvgConnBl_sentence_linking | 0.241 | 0.603 | 0.014 | 0.606 | 0.063 | | AvgConnSen_sentence_linking | 0.862 | 0.027 | 0.048 | 0.411 | -0.059 | | AvgConnBl_simple_subordinators | 0.081 | 0.52 | 0.062 | -0.082 | -0.029 | | AvgConnSen_simple_subordinators | 0.681 | 0.165 | -0.053 | -0.039 | 0.023 | | AvgConnBl_temporal_connectors | 0.121 | 0.382 | -0.081 | -0.07 | 0.114 | | AvgConnSen_temporal_connectors | 0.637 | 0.153 | -0.183 | -0.036 | 0.106 | | LexChainAvgSpan | 0.128 | 0.407 | 0.308 | 0.073 | 0.415 | | LexChainMaxSp | -0.007 | 0.663 | 0.02 | 0.171 | 0.207 | | AvgBlScore | 0.148 | 0.801 | 0.043 | 0.165 | 0.285 | | AvgSenScore | 0.909 | 0.12 | 0.044 | 0.055 | 0.165 | | SenScoreStDev | -0.466 | 0.332 | 0.071 | 0.09 | 0.501 | | AvgIntraBlCoh_LeackockChodorow | -0.741 | 0.302 | 0.152 | 0.008 | 0.471 | | AvgSenAdjCoh_LeackockChodorow | -0.735 | 0.263 | 0.166 | -0.002 | 0.488 | | AvgSenBlCoh_LeackockChodorow | 0.434 | -0.139 | 0.696 | 0.053 | 0.24 | | AvgIntraBlCoh_WuPalmer | -0.748 | 0.296 | 0.152 | 0.001 | 0.465 | | AvgSenAdjCoh_WuPalmer | -0.743 | 0.259 | 0.167 | -0.01 | 0.482 | | AvgSenBlCoh_WuPalmer | 0.443 | -0.145 | 0.694 | 0.049 | 0.226 | | AvgIntraBlCoh_Path | -0.749 | 0.277 | 0.154 | -0.007 | 0.46 | | AvgSenAdjCoh_Path | -0.744 | 0.234 | 0.175 | -0.021 | 0.478 | | AvgSenBlCoh_Path | 0.528 | -0.219 | 0.642 | 0.044 | 0.155 | | AvgIntraBlCoh_LSA | -0.739 | 0.338 | 0.148 | 0.005 | 0.446 | | AvgSenAdjCoh_LSA | -0.729 | 0.295 | 0.166 | -0.011 | 0.473 | | AvgSenBlCoh_LSA | 0.598 | -0.209 | 0.547 | 0.09 | 0.148 | | AvgIntraBlCoh_LDA | -0.738 | 0.345 | 0.151 | -0.004 | 0.449 | | AvgSenAdjCoh_LDA | -0.718 | 0.324 | 0.158 | -0.003 | 0.473 | | AvgSenBlCoh_LDA | 0.513 | -0.066 | 0.598 | 0.08 | 0.202 | | AvgIntraBlCoh_word2vec | -0.743 | 0.31 | 0.158 | -0.026 | 0.457 | | AvgSenAdjCoh_word2vec | -0.734 | 0.273 | 0.179 | -0.046 | 0.48 | | AvgSenBlCoh_word2vec | 0.581 | -0.259 | 0.577 | 0.048 | 0.165 | | AvgBlVoiceCoOcc | 0.083 | 0.49 | -0.117 | 0.351 | 0.258 | | AvgVoice | 0.086 | 0.506 | -0.117 | 0.355 | 0.247 | | AvgSenSyll | 0.973 | 0.081 | 0.076 | 0.138 | -0.008 | | AvgSenStressedSyll | 0.939 | 0.141 | 0.049 | 0.106 | 0.117 | | AvgRhythmUnits | 0.315 | 0.184 | -0.003 | -0.262 | 0.043 | | AvgRhythmUnitSyll | 0.807 | -0.027 | 0.082 | 0.284 | -0.045 | | AvgRhythmUnitStreesSyll | 0.809 | 0.024 | 0.053 | 0.235 | 0.073 | | LangRhythmCoeff | 0.24 | 0.048 | -0.011 | 0.129 | -0.121 | | LangRhythmId | -0.173 | -0.023 | -0.06 | 0.028 | -0.387 | | FrqRhythmId | 0.657 | -0.336 | -0.016 | 0.096 | 0.056 | | LangRhythmDiameter | 0.373 | 0.113 | 0.117 | 0.197 | -0.139 | | SenAsson | -0.051 | 0.221 | 0.079 | 0.066 | 0.065 | | AvgDepsBl_acl | 0.092 | 0.23 | 0.074 | 0.206 | 0.093 | | AvgDepsSen_acl | 0.485 | 0.081 | 0.031 | 0.343 | 0.108 | | AvgDepsBl_advcl | 0.394 | 0.519 | -0.037 | 0.042 | 0.175 | | AvgDepsSen_advcl | 0.768 | 0.219 | -0.041 | 0.009 | 0.065 | | AvgDepsBl_advmod | 0.141 | 0.71 | -0.023 | -0.136 | -0.045 | | AvgDepsSen_advmod | 0.753 | 0.193 | -0.054 | -0.132 | -0.006 | | AvgDepsBl_amod | -0.092 | 0.476 | 0.104 | 0.038 | 0.13 | | AvgDepsSen_amod | 0.563 | 0.072 | 0.117 | -0.001 | 0.112 | | AvgDepsBl_aux | -0.02 | 0.402 | 0.09 | -0.282 | 0.005 | | AvgDepsSen_aux | 0.621 | 0.072 | 0.112 | -0.18 | -0.071 | | AvgDepsBl_auxpass | -0.079 | 0.424 | 0.001 | -0.052 | -0.166 | | AvgDepsBl_case | -0.121 | 0.772 | 0.094 | 0.169 | -0.072 | | AvgDepsSen_case | 0.747 | 0.175 | 0.093 | 0.178 | -0.024 | | AvgDepsBl_cc | 0.203 | 0.37 | 0.063 | 0.746 | -0.039 | | AvgDepsSen_cc | 0.73 | -0.143 | 0.088 | 0.553 | -0.149 | | AvgDepsBl_ccomp | 0.383 | 0.496 | 0.036 | -0.022 | 0.173 | | AvgDepsSen_ccomp | 0.781 | 0.073 | 0.031 | -0.041 | 0.09 | | AvgDepsBl_compound | 0.104 | 0.149 | 0.038 | 0.366 | -0.006 | | AvgDepsSen_compound | 0.587 | -0.062 | 0.073 | 0.412 | -0.064 | | AvgDepsBl_conj | 0.31 | 0.297 | 0.122 | 0.722 | -0.045 | | AvgDepsSen_conj | 0.731 | -0.134 | 0.134 | 0.536 | -0.162 | | AvgDepsBl_cop | -0.001 | 0.419 | 0.055 | -0.12 | -0.002 | | AvgDepsSen_cop | 0.583 | 0.061 | 0.018 | -0.107 | 0.003 | | AvgDepsBl_dep | 0.234 | 0.123 | 0.104 | 0.47 | -0.059 | | AvgDepsSen_dep | 0.527 | -0.187 | 0.15 | 0.494 | -0.174 | | AvgDepsBl_det | -0.182 | 0.549 | 0.13 | 0.222 | 0.071 | | AvgDepsSen_det | 0.576 | -0.06 | 0.152 | 0.415 | -0.023 | | AvgDepsBl_dobj | 0.119 | 0.64 | 0.078 | 0.196 | 0.13 | | AvgDepsSen_dobj | 0.856 | -0.002 | 0.047 | 0.128 | 0.028 | | AvgDepsBl_mark | 0.367 | 0.535 | 0.01 | 0.087 | 0.139 | | AvgDepsSen_mark | 0.795 | 0.152 | -0.063 | 0.085 | 0.073 | | AvgDepsBl_mwe | -0.01 | 0.262 | 0.021 | 0.142 | 0.008 | | AvgDepsSen_mwe | 0.383 | 0.097 | 0.025 | 0.044 | 0.031 | | AvgDepsBl_neg | 0.197 | 0.361 | 0.053 | -0.181 | -0.132 | | AvgDepsSen_neg | 0.549 | 0.073 | -0.024 | -0.17 | -0.086 | | AvgDepsBl_nmod | -0.134 | 0.727 | 0.094 | 0.237 | -0.052 | | AvgDepsSen_nmod | 0.661 | 0.15 | 0.086 | 0.281 | -0.02 | | AvgDepsBl_nsubj | 0.141 | 0.88 | 0.062 | 0.041 | 0.08 | | AvgDepsSen_nsubj | 0.969 | 0.02 | 0.051 | 0.018 | 0.003 | | AvgDepsBl_nsubjpass | -0.015 | 0.417 | -0.005 | -0.065 | -0.158 | | AvgDepsBl_nummod | 0.132 | 0.492 | -0.071 | -0.009 | 0.13 | | AvgDepsBl_punct | -0.483 | 0.644 | -0.017 | -0.108 | -0.086 | | AvgDepsSen_punct | -0.192 | 0.389 | 0.063 | -0.232 | 0.191 | | AvgDepsBl_xcomp | 0.053 | 0.501 | 0.03 | 0.186 | 0.068 | | AvgDepsSen_xcomp | 0.666 | 0.075 | 0.061 | 0.196 | 0.024 | ## ReaderBench Model 1e This model was trained on principal component scores for winter data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:-------|:-------| | -8.0185 | 0.0573 | 0.5839 | 0.5269 | -0.3984 | 0.1184 | 0.1066 | 0.0459 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). PC1 = scores on 1st principal component extracted, ... Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:-----|:-----|:------|:----|:-----|:------|:------|:------| | PC2 | 32.8 | 53 | 47.34 | 0 | 9.95 | 26.16 | 39.14 | 22.22 | | PC1 | 7.63 | 1.98 | 13.73 | 0 | 1.84 | 4.83 | 0 | 11.11 | | PC39 | 4.81 | 1.96 | 1.08 | 0 | 7.68 | 2.76 | 17.37 | 8.89 | | PC5 | 4.35 | 2.14 | 4.78 | 0 | 4.24 | 3.9 | 0 | 13.33 | | PC11 | 4.33 | 1.43 | 3.55 | 0 | 5.85 | 1.52 | 4.87 | 11.11 | | PC37 | 4.21 | 1.94 | 1.15 | 0 | 7.47 | 1.75 | 12.15 | 6.67 | | PC6 | 3.45 | 3.05 | 3.87 | 0 | 3.53 | 2.17 | 0 | 8.89 | | PC26 | 3.28 | 2.41 | 1.25 | 0 | 4.41 | 1.69 | 14.49 | 0 | | PC38 | 3.13 | 0.57 | 1.08 | 0 | 7.55 | 0.99 | 0 | 6.67 | | PC14 | 2.89 | 4.55 | 1.94 | 0 | 3.33 | 6.23 | 0 | 6.67 | | PC24 | 2.2 | 1.36 | 1.07 | 0 | 3.24 | 0.49 | 8.2 | 0 | | PC40 | 2.12 | 0.85 | 0.72 | 0 | 5.11 | 1.28 | 0 | 2.22 | | PC9 | 1.75 | 0.72 | 2.06 | 0 | 2.21 | 0.62 | 0 | 2.22 | | PC33 | 1.63 | 0.91 | 0.67 | 0 | 2.86 | 2.41 | 2.5 | 0 | | PC45 | 1.61 | 0.26 | 0.5 | 0 | 4.33 | 0.63 | 0 | 0 | | PC44 | 1.51 | 0.65 | 0.49 | 0 | 3.6 | 1.94 | 0 | 0 | | PC32 | 1.3 | 0.87 | 0.67 | 0 | 2.61 | 1.92 | 0 | 0 | | PC20 | 1.29 | 1.94 | 1 | 0 | 2.28 | 0.76 | 0 | 0 | | PC4 | 1.16 | 1.23 | 1.61 | 0 | 0.82 | 1.5 | 0 | 0 | | PC19 | 1.07 | 1.79 | 0.78 | 0 | 1.41 | 2.19 | 0 | 0 | | PC34 | 0.98 | 1.11 | 0.5 | 0 | 1.9 | 1.2 | 0.26 | 0 | | PC43 | 0.97 | 0.27 | 0.42 | 0 | 2.53 | 0 | 0 | 0 | | PC8 | 0.95 | 0.61 | 1.17 | 0 | 0.8 | 1.73 | 0 | 0 | | PC15 | 0.94 | 1.05 | 0.88 | 0 | 1.21 | 1.47 | 0 | 0 | | PC23 | 0.9 | 0.73 | 0.64 | 0 | 1.33 | 1.89 | 0 | 0 | | PC17 | 0.9 | 0.77 | 0.85 | 0 | 1.41 | 0.6 | 0 | 0 | | PC35 | 0.86 | 1.17 | 0.46 | 0 | 1.61 | 1.24 | 0 | 0 | | PC3 | 0.81 | 0.64 | 1.28 | 0 | 0.16 | 1.75 | 0 | 0 | | PC12 | 0.76 | 0.42 | 0.78 | 0 | 0.71 | 1.86 | 0 | 0 | | PC28 | 0.73 | 0.92 | 0.48 | 0 | 1.01 | 1.9 | 0 | 0 | | PC16 | 0.69 | 0.41 | 0.71 | 0 | 0.86 | 0.92 | 0 | 0 | | PC25 | 0.59 | 0.21 | 0.47 | 0 | 0.73 | 1.59 | 0 | 0 | | PC30 | 0.42 | 1.15 | 0.31 | 0 | 0.25 | 1.74 | 0 | 0 | | PC42 | 0.41 | 0.61 | 0.22 | 0 | 0.34 | 1.92 | 0 | 0 | | PC36 | 0.36 | 0.14 | 0.29 | 0 | 0.51 | 0.88 | 0 | 0 | | PC27 | 0.35 | 0.38 | 0.35 | 0 | 0.34 | 0.77 | 0 | 0 | | PC7 | 0.34 | 1.67 | 0.06 | 0 | 0 | 1.66 | 1.01 | 0 | | PC13 | 0.28 | 0.89 | 0.01 | 0 | 0 | 2.56 | 0 | 0 | | PC29 | 0.28 | 0.97 | 0.05 | 0 | 0 | 2.36 | 0 | 0 | | PC10 | 0.27 | 0.54 | 0.27 | 0 | 0 | 1.27 | 0 | 0 | | PC18 | 0.24 | 0.52 | 0.14 | 0 | 0 | 1.68 | 0 | 0 | | PC31 | 0.18 | 0.35 | 0.08 | 0 | 0 | 1.51 | 0 | 0 | | PC46 | 0.11 | 0.43 | 0.15 | 0 | 0 | 0.24 | 0 | 0 | | PC21 | 0.07 | 0.18 | 0 | 0 | 0 | 0.64 | 0 | 0 | | PC22 | 0.07 | 0.25 | 0.09 | 0 | 0 | 0.27 | 0 | 0 | | PC41 | 0.06 | 0 | 0 | 0 | 0 | 0.6 | 0 | 0 | ### Proportion of Variance by Varimax Rotated Component (RC) Due to space limitations, loadings for only the first five principal components are displayed. | Variable | RC1 | RC2 | RC3 | RC4 | RC5 | |:----------------------|:------|:------|:------|:-----|:-----| | SS loadings | 46.99 | 34.24 | 14.94 | 7.95 | 6.95 | | Proportion Var | 0.23 | 0.17 | 0.07 | 0.04 | 0.03 | | Cumulative Var | 0.23 | 0.40 | 0.48 | 0.52 | 0.55 | | Proportion Explained | 0.42 | 0.31 | 0.13 | 0.07 | 0.06 | | Cumulative Proportion | 0.42 | 0.73 | 0.87 | 0.94 | 1.00 | ### Varimax Rotated Loadings | Metric | RC1 | RC2 | RC3 | RC4 | RC5 | |:--------------------------|:------|:------|:------|:------|:------| | Sentences | -0.66 | 0.48 | -0.05 | -0.04 | -0.26 | | Words | 0.07 | 0.97 | 0 | -0.08 | -0.05 | | Content.words | -0.05 | 0.94 | 0.07 | 0.07 | -0.11 | | RdbltyFlesch | -0.82 | -0.19 | -0.02 | -0.12 | 0.09 | | RdbltyFog | 0.91 | 0.17 | 0.01 | 0.04 | 0.01 | | RdbltyKincaid | 0.91 | 0.18 | 0.01 | 0.05 | 0 | | RdbltyDaleChall | 0.5 | -0.34 | 0.11 | -0.06 | -0.22 | | AvgBlLen | -0.11 | 0.91 | 0.08 | 0.13 | -0.16 | | AvgCommaBl | -0.23 | 0.38 | 0.06 | -0.1 | -0.1 | | AvgSenLen | 0.9 | 0.26 | 0.08 | 0.15 | 0.03 | | AvgSenBl | -0.66 | 0.48 | -0.05 | -0.04 | -0.26 | | AvgUnqWdBl | -0.08 | 0.91 | 0.05 | 0.12 | -0.15 | | AvgUnqWdSen | 0.93 | 0.22 | 0.08 | 0.12 | 0.05 | | AvgWdLen | -0.21 | 0.26 | -0.17 | 0.25 | -0.46 | | AvgWdBl | -0.05 | 0.94 | 0.07 | 0.07 | -0.11 | | AvgWdSen | 0.93 | 0.23 | 0.07 | 0.09 | 0.06 | | CharEnt | -0.14 | 0.54 | 0.11 | -0.02 | -0.03 | | SenStDevUnqWd | -0.23 | 0.5 | 0.04 | 0.09 | 0.41 | | SenStdDevWd | -0.16 | 0.48 | 0.04 | 0.07 | 0.45 | | WdEnt | 0.03 | 0.86 | 0.04 | 0.1 | -0.13 | | WdLettStdDev | -0.09 | 0.4 | 0.37 | 0.24 | 0.04 | | LxcDiv | 0 | 0.81 | 0.09 | 0.2 | -0.07 | | LxcSoph | 0.36 | 0.09 | -0.28 | 0.11 | -0.35 | | SynSoph | 0.82 | 0.44 | 0.12 | 0.14 | 0.12 | | AvgNounBl | 0.01 | 0.76 | 0.11 | 0.03 | -0.35 | | AvgPronounBl | 0.1 | 0.79 | -0.1 | -0.13 | 0.18 | | AvgVerbBl | 0.03 | 0.91 | -0.05 | -0.02 | 0.05 | | AvgAdverbBl | 0.09 | 0.63 | -0.01 | -0.22 | 0.11 | | AvgAdjectiveBl | -0.06 | 0.55 | 0.12 | -0.07 | -0.09 | | AvgPrepositionBl | 0.11 | 0.75 | -0.04 | 0.27 | -0.1 | | AvgNounSen | 0.9 | 0.06 | 0.06 | 0.06 | -0.14 | | AvgPronounSen | 0.91 | 0.1 | -0.08 | -0.02 | 0.15 | | AvgVerbSen | 0.94 | 0.16 | -0.01 | 0.04 | 0.09 | | AvgAdverbSen | 0.77 | 0.15 | -0.02 | -0.13 | 0.18 | | AvgAdjectiveSen | 0.71 | 0.11 | 0.13 | -0.13 | 0.1 | | AvgPrepositionSen | 0.85 | 0.2 | -0.07 | 0.26 | -0.04 | | AvgUnqNoundBl | 0.02 | 0.71 | 0.05 | 0.02 | -0.36 | | AvgUnqPronounBl | 0.09 | 0.66 | -0.01 | -0.09 | 0.02 | | AvgUnqVerbBl | 0.03 | 0.85 | -0.05 | 0.01 | 0.04 | | AvgUnqAdverbBl | -0.01 | 0.61 | -0.02 | -0.13 | 0.05 | | AvgUnqAdjectiveBl | -0.06 | 0.57 | 0.12 | -0.09 | -0.13 | | AvgUnqPrepositionBl | 0.09 | 0.71 | 0 | 0.3 | -0.14 | | AvgPronBl_first_person | 0.09 | 0.69 | -0.05 | -0.14 | 0.12 | | AvgPronBl_indefinite | -0.05 | 0.45 | 0.01 | 0.24 | -0.04 | | AggPronSen_indefinite | 0.62 | 0.17 | 0.05 | 0.23 | 0.02 | | AvgPronBl_third_person | 0.13 | 0.48 | -0.08 | -0.12 | 0.14 | | AggPronSen_third_person | 0.84 | -0.02 | -0.09 | -0.09 | 0.12 | | AvgSemDep | 0.97 | 0.12 | -0.01 | -0.06 | 0.06 | | WdDiffLemmaStem | -0.11 | 0.06 | -0.23 | -0.01 | -0.24 | | WdDiffWdStem | -0.28 | 0.19 | 0.03 | 0.16 | -0.28 | | WdMaxDpthHypernymTree | -0.02 | -0.26 | -0.09 | 0.14 | -0.23 | | WdAvgDpthHypernymTree | 0 | -0.27 | -0.11 | 0.13 | -0.24 | | WdPathCntHypernymTree | -0.1 | -0.26 | -0.17 | 0.16 | -0.09 | | WdPolysemyCnt | 0.06 | 0.11 | -0.09 | -0.09 | 0.35 | | WdSylCnt | -0.1 | 0.05 | -0.26 | 0.11 | -0.46 | | AvgAOADoc_Shock | 0.03 | 0.43 | 0.12 | 0.27 | -0.28 | | AvgAOABl_Shock | 0.03 | 0.43 | 0.12 | 0.27 | -0.28 | | AvgAOASen_Shock | 0.46 | 0.21 | 0.05 | 0.31 | -0.27 | | AvgAOADoc_Cortese | -0.17 | 0.07 | 0.69 | 0.22 | 0.19 | | AvgAOABl_Cortese | -0.17 | 0.07 | 0.69 | 0.22 | 0.19 | | AvgAOASen_Cortese | 0.14 | 0.01 | 0.55 | 0.3 | 0.17 | | AvgAOADoc_Kuperman | -0.22 | -0.03 | 0.43 | 0.31 | -0.38 | | AvgAOABl_Kuperman | -0.22 | -0.03 | 0.43 | 0.31 | -0.38 | | AvgAOASen_Kuperman | -0.03 | -0.04 | 0.42 | 0.41 | -0.3 | | AvgAOADoc_Bird | -0.12 | 0.17 | 0.55 | 0.32 | 0.21 | | AvgAOABl_Bird | -0.12 | 0.17 | 0.55 | 0.32 | 0.21 | | AvgAOASen_Bird | 0.23 | 0.12 | 0.43 | 0.36 | 0.21 | | AvgAOADoc_Bristol | -0.06 | 0.24 | 0.54 | 0.25 | -0.04 | | AvgAOABl_Bristol | -0.06 | 0.24 | 0.54 | 0.25 | -0.04 | | AvgAOASen_Bristol | 0.37 | 0.08 | 0.34 | 0.26 | 0 | | AvgAOEDoc_IndexPolyFAT.3 | -0.02 | -0.07 | 0.77 | -0.22 | -0.14 | | AvgAOEBl_IndexPolyFAT.3 | -0.02 | -0.07 | 0.77 | -0.22 | -0.14 | | AvgAOESen_IndexPolyFAT.3 | 0.02 | -0.02 | 0.74 | -0.12 | -0.17 | | AvgAOEDoc_InvLinRegSlo | 0 | -0.1 | 0.82 | -0.22 | -0.09 | | AvgAOEBl_InvLinRegSlo | 0 | -0.1 | 0.82 | -0.22 | -0.09 | | AvgAOESen_InvLinRegSlo | 0.17 | -0.02 | 0.67 | -0.02 | -0.11 | | AvgAOEDoc_InfPointPoly | -0.1 | 0.01 | 0.86 | -0.15 | 0.1 | | AvgAOEBl_InfPointPoly | -0.1 | 0.01 | 0.86 | -0.15 | 0.1 | | AvgAOESen_InfPointPoly | 0.07 | 0.03 | 0.73 | 0.02 | 0.05 | | AvgAOEDoc_InvAverage | -0.11 | 0 | 0.88 | -0.14 | 0.06 | | AvgAOEBl_InvAverage | -0.11 | 0 | 0.88 | -0.14 | 0.06 | | AvgAOESen_InvAverage | 0.06 | 0.02 | 0.74 | 0.03 | 0.02 | | AvgAOEDoc_IndexAbThr.0.3. | 0.03 | -0.04 | 0.77 | -0.12 | -0.21 | | AvgAOEBl_IndexAbThr.0.3. | 0.03 | -0.04 | 0.77 | -0.12 | -0.21 | | AvgAOESen_IndexAbThr.0.3. | 0.05 | 0 | 0.75 | -0.04 | -0.27 | | AvgNmdEntBl | -0.07 | 0.32 | 0.15 | -0.11 | -0.44 | | AvgNounNmdEntBl | -0.02 | 0.24 | 0.24 | -0.13 | -0.43 | | AvgUnqNmdEntBl | -0.07 | 0.34 | 0.12 | -0.09 | -0.48 | | AvgNmdEntSen | 0.59 | 0.01 | 0.19 | -0.09 | -0.26 | | TCorefChainDoc | 0.04 | 0.69 | 0.06 | -0.2 | -0.01 | | AvgCorefChain | 0.02 | 0.46 | -0.18 | 0 | 0.11 | | AvgChainSpan | 0.06 | 0.73 | -0.03 | 0.07 | -0.1 | | AvgInferenceDistChain | 0.45 | 0.31 | -0.02 | 0.31 | 0.07 | | TActCorefChainWd | -0.02 | -0.25 | -0.04 | -0.19 | 0.04 | | TCorefChainBigSpan | 0.19 | 0.52 | -0.02 | -0.09 | -0.01 | | AvgConnBl_addition | 0.1 | 0.53 | 0.1 | -0.58 | -0.08 | | AvgConnSen_addition | 0.74 | 0.04 | 0.05 | -0.4 | -0.02 | | AvgConnBl_conjunctions | 0.15 | 0.59 | 0.09 | -0.57 | 0.06 | | AvgConnSen_conjunctions | 0.83 | 0.04 | 0.06 | -0.36 | 0.06 | | AvgConnBl_contrasts | 0.19 | 0.37 | 0 | -0.11 | 0.28 | | AvgConnSen_contrasts | 0.63 | 0.11 | 0.04 | -0.11 | 0.22 | | AvgConnBl_coord_conjs | 0.17 | 0.47 | -0.17 | 0.14 | 0.23 | | AvgConnSen_coord_conjs | 0.63 | 0.25 | -0.14 | 0.21 | 0.25 | | AvgConnBl_coord_connects | 0.21 | 0.7 | -0.02 | -0.4 | 0.18 | | AvgConnSen_coord_connects | 0.88 | 0.1 | -0.02 | -0.23 | 0.17 | | AvgConnBl_logical_conn | 0.09 | 0.5 | 0.07 | -0.64 | 0.05 | | AvgConnSen_logical_conn | 0.67 | -0.02 | 0.03 | -0.5 | 0.1 | | AvgConnBl_oppositions | 0.19 | 0.37 | 0.05 | 0.02 | 0.22 | | AvgConnSen_oppositions | 0.6 | 0.16 | 0.07 | 0.12 | 0.11 | | AvgConnBl_order | 0.12 | 0.38 | 0.05 | -0.29 | -0.16 | | AvgConnSen_order | 0.57 | 0.14 | 0.02 | -0.07 | -0.1 | | AvgConnBl_reas_purp | 0.18 | 0.58 | -0.07 | -0.03 | 0.09 | | AvgConnSen_reas_purp | 0.73 | 0.25 | -0.06 | 0.08 | 0.09 | | AvgConnBl_semi_coords | 0.17 | 0.47 | -0.17 | 0.14 | 0.23 | | AvgConnSen_semi_coords | 0.63 | 0.25 | -0.14 | 0.21 | 0.25 | | AvgConnBl_sentence_link | 0.2 | 0.77 | -0.01 | -0.33 | 0.13 | | AvgConnSen_sentence_link | 0.92 | 0.14 | -0.03 | -0.13 | 0.12 | | AvgConnBl_simp_subords | -0.02 | 0.41 | -0.05 | 0.27 | 0.11 | | AvgConnSen_simp_subords | 0.52 | 0.18 | -0.09 | 0.34 | 0.16 | | AvgConnBl_temp_conn | -0.14 | 0.36 | 0.05 | -0.23 | 0.08 | | AvgConnSen_temp_conn | 0.36 | 0.06 | 0.04 | -0.25 | 0.27 | | LexChainAvgSpan | 0.07 | 0.49 | 0.03 | -0.01 | 0.16 | | LexChainMaxSp | 0 | 0.72 | 0 | -0.02 | 0.08 | | AvgBlScore | 0.14 | 0.82 | 0.01 | 0 | 0.13 | | AvgSenScore | 0.9 | 0.22 | 0.02 | 0.04 | 0.14 | | SenScoreStDev | -0.27 | 0.5 | 0.04 | 0.05 | 0.45 | | AvgIntraBlCoh_LeackChod | -0.73 | 0.44 | 0.16 | 0.2 | 0.17 | | AvgSenAdjCoh_LeackChod | -0.7 | 0.43 | 0.19 | 0.22 | 0.17 | | AvgSenBlCoh_LeackChod | 0.77 | -0.38 | -0.13 | -0.08 | 0.09 | | AvgIntraBlCoh_WuPalmer | -0.74 | 0.44 | 0.16 | 0.2 | 0.18 | | AvgSenAdjCoh_WuPalmer | -0.7 | 0.43 | 0.19 | 0.22 | 0.17 | | AvgSenBlCoh_WuPalmer | 0.79 | -0.39 | -0.13 | -0.09 | 0.08 | | AvgIntraBlCoh_Path | -0.73 | 0.43 | 0.16 | 0.2 | 0.19 | | AvgSenAdjCoh_Path | -0.7 | 0.42 | 0.19 | 0.22 | 0.2 | | AvgSenBlCoh_Path | 0.82 | -0.42 | -0.12 | -0.13 | 0.08 | | AvgIntraBlCoh_LSA | -0.73 | 0.46 | 0.16 | 0.18 | 0.18 | | AvgSenAdjCoh_LSA | -0.7 | 0.44 | 0.2 | 0.22 | 0.18 | | AvgSenBlCoh_LSA | 0.8 | -0.35 | -0.11 | -0.12 | 0.11 | | AvgIntraBlCoh_LDA | -0.73 | 0.48 | 0.16 | 0.18 | 0.15 | | AvgSenAdjCoh_LDA | -0.7 | 0.47 | 0.2 | 0.2 | 0.15 | | AvgSenBlCoh_LDA | 0.75 | -0.21 | -0.13 | -0.11 | 0.08 | | AvgIntraBlCoh_word2vec | -0.73 | 0.45 | 0.16 | 0.19 | 0.18 | | AvgSenAdjCoh_word2vec | -0.7 | 0.44 | 0.2 | 0.22 | 0.18 | | AvgSenBlCoh_word2vec | 0.8 | -0.39 | -0.1 | -0.11 | 0.11 | | AvgBlVoiceCoOcc | 0.03 | 0.63 | -0.05 | -0.04 | -0.01 | | AvgVoice | -0.01 | 0.6 | -0.08 | -0.05 | 0 | | AvgSenSyll | 0.98 | 0.12 | 0.01 | 0.02 | 0.02 | | AvgSenStressedSyll | 0.94 | 0.22 | 0.07 | 0.08 | 0.05 | | AvgRhythmUnits | 0.23 | 0.26 | 0.15 | 0.01 | 0.28 | | AvgRhythmUnitSyll | 0.84 | 0.01 | -0.09 | 0.02 | -0.09 | | AvgRhythmUnitStreesSyll | 0.81 | 0.11 | -0.01 | 0.08 | -0.05 | | LangRhythmCoeff | 0.39 | -0.1 | -0.21 | -0.17 | -0.06 | | LangRhythmId | -0.11 | -0.02 | -0.4 | -0.27 | -0.14 | | FrqRhythmId | 0.72 | -0.3 | 0.01 | -0.12 | 0.04 | | LangRhythmDiameter | 0.27 | -0.01 | -0.34 | -0.25 | -0.19 | | SenAsson | 0.13 | 0.25 | 0.12 | -0.11 | -0.17 | | AvgDepsBl_acl | -0.03 | 0.24 | 0.15 | 0.19 | -0.16 | | AvgDepsSen_acl | 0.44 | 0.03 | 0.26 | 0.09 | -0.13 | | AvgDepsBl_advcl | 0.23 | 0.52 | -0.05 | 0.1 | 0.16 | | AvgDepsSen_advcl | 0.67 | 0.21 | -0.08 | 0.2 | 0.09 | | AvgDepsBl_advmod | 0.07 | 0.67 | 0.03 | -0.25 | 0.13 | | AvgDepsSen_advmod | 0.78 | 0.15 | 0.02 | -0.17 | 0.2 | | AvgDepsBl_amod | -0.07 | 0.39 | 0.16 | 0.06 | -0.2 | | AvgDepsSen_amod | 0.62 | 0.1 | 0.17 | 0 | -0.01 | | AvgDepsBl_aux | 0.13 | 0.37 | -0.13 | 0.13 | 0.24 | | AvgDepsSen_aux | 0.56 | 0.02 | -0.14 | -0.01 | 0.27 | | AvgDepsBl_auxpass | 0.06 | 0.26 | 0.08 | 0.05 | 0.03 | | AvgDepsBl_case | 0.06 | 0.69 | 0.01 | 0.15 | -0.26 | | AvgDepsSen_case | 0.84 | 0.13 | -0.02 | 0.15 | -0.17 | | AvgDepsBl_cc | 0.17 | 0.59 | 0.08 | -0.61 | 0.09 | | AvgDepsSen_cc | 0.8 | 0.02 | 0.05 | -0.42 | 0.11 | | AvgDepsBl_ccomp | 0.37 | 0.51 | -0.03 | -0.01 | 0.32 | | AvgDepsSen_ccomp | 0.78 | 0.16 | -0.01 | -0.01 | 0.19 | | AvgDepsBl_compound | 0.17 | 0.16 | 0.19 | -0.05 | -0.36 | | AvgDepsSen_compound | 0.61 | -0.12 | 0.18 | -0.07 | -0.27 | | AvgDepsBl_conj | 0.23 | 0.52 | 0.06 | -0.58 | 0.14 | | AvgDepsSen_conj | 0.77 | 0.02 | 0.04 | -0.4 | 0.13 | | AvgDepsBl_cop | 0.04 | 0.49 | 0.09 | 0.03 | -0.1 | | AvgDepsSen_cop | 0.71 | 0.16 | 0.14 | 0.04 | -0.03 | | AvgDepsBl_dep | 0.2 | 0.22 | 0.08 | -0.34 | 0.07 | | AvgDepsSen_dep | 0.56 | -0.05 | 0.1 | -0.31 | 0.11 | | AvgDepsBl_det | -0.09 | 0.58 | -0.03 | 0.05 | -0.28 | | AvgDepsSen_det | 0.86 | 0.1 | -0.02 | 0.04 | -0.16 | | AvgDepsBl_dobj | 0 | 0.73 | 0.03 | -0.2 | 0.01 | | AvgDepsSen_dobj | 0.91 | 0.07 | -0.01 | -0.04 | 0.1 | | AvgDepsBl_mark | 0.22 | 0.63 | -0.11 | 0.28 | 0.09 | | AvgDepsSen_mark | 0.78 | 0.25 | -0.1 | 0.28 | 0.05 | | AvgDepsBl_mwe | -0.07 | 0.19 | 0.06 | 0 | -0.02 | | AvgDepsSen_mwe | 0.3 | 0 | 0.03 | -0.09 | 0.05 | | AvgDepsBl_neg | 0.03 | 0.37 | -0.08 | -0.03 | 0.25 | | AvgDepsSen_neg | 0.38 | 0.12 | -0.04 | 0.05 | 0.28 | | AvgDepsBl_nmod | 0.05 | 0.64 | -0.01 | 0.17 | -0.27 | | AvgDepsSen_nmod | 0.83 | 0.07 | -0.06 | 0.14 | -0.18 | | AvgDepsBl_nsubj | 0.03 | 0.9 | -0.1 | -0.1 | 0.08 | | AvgDepsSen_nsubj | 0.94 | 0.14 | -0.07 | 0.01 | 0.14 | | AvgDepsBl_nsubjpass | 0.05 | 0.2 | 0.03 | 0.08 | 0.16 | | AvgDepsBl_nummod | -0.09 | 0.17 | -0.05 | 0.04 | -0.2 | | AvgDepsBl_punct | -0.53 | 0.52 | -0.01 | -0.06 | -0.17 | | AvgDepsSen_punct | -0.13 | 0.35 | 0.14 | 0.12 | 0.26 | | AvgDepsBl_xcomp | 0.08 | 0.49 | -0.03 | 0.01 | -0.11 | | AvgDepsSen_xcomp | 0.63 | 0.15 | 0.02 | -0.06 | -0.1 | ## ReaderBench Model 1f This model was trained on principal component scores for spring data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:--------|:-------|:-------| | -9.2262 | 0.1219 | 0.7713 | 0.1603 | -0.3706 | -0.0217 | 0.3129 | 0.0233 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). PC1 = scores on 1st principal component extracted, ... Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:------|:------|:------|:------| | PC2 | 31.31 | 56.99 | 35.09 | 0 | 11.67 | 23.86 | 35.8 | 21.83 | | PC1 | 11.79 | 6.72 | 17.18 | 0 | 3.69 | 5.61 | 8.89 | 21.83 | | PC4 | 10.38 | 7.69 | 9.11 | 0 | 8.29 | 5.24 | 16.85 | 12.66 | | PC44 | 4.45 | 0.7 | 0.97 | 0 | 7.79 | 1.41 | 10.78 | 3.71 | | PC9 | 4.36 | 2.62 | 4.85 | 0 | 7.31 | 1.81 | 0 | 10.92 | | PC27 | 3.4 | 0.68 | 0.79 | 0 | 1.88 | 1.75 | 12.84 | 0 | | PC14 | 3.37 | 1.1 | 3.45 | 0 | 6.67 | 1.95 | 0 | 7.21 | | PC43 | 2.78 | 1.23 | 1.19 | 0 | 9.29 | 1.77 | 0 | 0 | | PC6 | 2.51 | 0.56 | 3.24 | 0 | 3.33 | 1.2 | 0.08 | 9.17 | | PC16 | 2.38 | 0.82 | 1.17 | 0 | 1.74 | 1.2 | 6.85 | 0 | | PC15 | 2.31 | 1.41 | 2.25 | 0 | 4.31 | 1.89 | 0 | 10.92 | | PC5 | 2.22 | 0.82 | 2.97 | 0 | 2.79 | 2.34 | 0.2 | 1.75 | | PC28 | 1.47 | 0.78 | 1.17 | 0 | 3.69 | 1.94 | 0 | 0 | | PC10 | 1.46 | 1.85 | 1.73 | 0 | 2.08 | 1.98 | 0 | 0 | | PC11 | 1.33 | 1.77 | 1.55 | 0 | 1.91 | 2.26 | 0 | 0 | | PC37 | 1.29 | 0.57 | 0.8 | 0 | 3.79 | 1.57 | 0 | 0 | | PC12 | 1.19 | 0.4 | 1.5 | 0 | 1.87 | 2.16 | 0 | 0 | | PC34 | 1.08 | 0.19 | 0.77 | 0 | 3.06 | 0.68 | 0 | 0 | | PC31 | 1.05 | 0.52 | 0.79 | 0 | 2.74 | 2.09 | 0 | 0 | | PC30 | 0.9 | 0.05 | 0.21 | 0 | 0 | 0.29 | 4.02 | 0 | | PC19 | 0.84 | 1.4 | 0.92 | 0 | 1.29 | 0.05 | 0 | 0 | | PC23 | 0.8 | 0.38 | 0.83 | 0 | 1.59 | 1.37 | 0 | 0 | | PC3 | 0.77 | 0.92 | 0.36 | 0 | 0 | 2.84 | 2.68 | 0 | | PC40 | 0.75 | 0.22 | 0.5 | 0 | 2.14 | 1.16 | 0 | 0 | | PC21 | 0.74 | 0.02 | 0.7 | 0 | 0.87 | 1.03 | 1 | 0 | | PC45 | 0.73 | 0.12 | 0.43 | 0 | 2.23 | 1.46 | 0 | 0 | | PC25 | 0.65 | 0.57 | 0.67 | 0 | 1.22 | 1.65 | 0 | 0 | | PC18 | 0.43 | 0.14 | 0.62 | 0 | 0.45 | 2.08 | 0 | 0 | | PC20 | 0.38 | 0.49 | 0.55 | 0 | 0.3 | 1.36 | 0 | 0 | | PC38 | 0.37 | 0.35 | 0.35 | 0 | 0.72 | 1.69 | 0 | 0 | | PC35 | 0.37 | 0.16 | 0.4 | 0 | 0.7 | 0 | 0 | 0 | | PC32 | 0.27 | 0.8 | 0.34 | 0 | 0.17 | 1.33 | 0 | 0 | | PC17 | 0.24 | 0.08 | 0.47 | 0 | 0 | 1.41 | 0 | 0 | | PC39 | 0.23 | 0.46 | 0.29 | 0 | 0.25 | 1.31 | 0 | 0 | | PC13 | 0.23 | 1.06 | 0.32 | 0 | 0 | 2.26 | 0 | 0 | | PC22 | 0.21 | 0.42 | 0.36 | 0 | 0 | 1.86 | 0 | 0 | | PC41 | 0.2 | 0.42 | 0.27 | 0 | 0.16 | 1.77 | 0 | 0 | | PC7 | 0.18 | 0.41 | 0.3 | 0 | 0 | 1.37 | 0 | 0 | | PC26 | 0.17 | 1.46 | 0.12 | 0 | 0 | 2.05 | 0 | 0 | | PC24 | 0.12 | 1.74 | 0 | 0 | 0 | 1.36 | 0 | 0 | | PC42 | 0.1 | 0.32 | 0.15 | 0 | 0 | 1.77 | 0 | 0 | | PC8 | 0.07 | 0.36 | 0.06 | 0 | 0 | 1.67 | 0 | 0 | | PC33 | 0.06 | 0.12 | 0.08 | 0 | 0 | 1.34 | 0 | 0 | | PC36 | 0.06 | 0 | 0.11 | 0 | 0 | 0.72 | 0 | 0 | | PC29 | 0.02 | 0.13 | 0.02 | 0 | 0 | 2.08 | 0 | 0 | ### Proportion of Variance by Varimax Rotated Component (RC) Due to space limitations, loadings for only the first five principal components are displayed. | RC1 | RC2 | RC3 | RC4 | RC5 | | |:----------------------|:------|:------|:------|:------|:-----| | SS loadings | 49.09 | 28.90 | 15.04 | 12.78 | 8.00 | | Proportion Var | 0.24 | 0.14 | 0.07 | 0.06 | 0.04 | | Cumulative Var | 0.24 | 0.39 | 0.46 | 0.53 | 0.57 | | Proportion Explained | 0.43 | 0.25 | 0.13 | 0.11 | 0.07 | | Cumulative Proportion | 0.43 | 0.69 | 0.82 | 0.93 | 1.00 | ### Varimax Rotated Loadings | Metric | RC1 | RC2 | RC3 | RC4 | RC5 | |:--------------------------|:------|:------|:------|:------|:------| | Sentences | -0.62 | 0.59 | -0.01 | -0.09 | 0.19 | | Words | 0.13 | 0.87 | -0.07 | 0.37 | 0.19 | | Content.words | 0.02 | 0.85 | 0.01 | 0.36 | 0.07 | | RdbltyFlesch | -0.83 | -0.11 | -0.08 | 0.04 | 0.04 | | RdbltyFog | 0.9 | 0.08 | 0.01 | -0.01 | 0 | | RdbltyKincaid | 0.89 | 0.07 | 0.02 | 0 | -0.01 | | RdbltyDaleChall | 0.47 | -0.12 | 0.41 | -0.23 | 0.28 | | AvgBlLen | -0.01 | 0.88 | 0.03 | 0.29 | -0.02 | | AvgCommaBl | -0.18 | 0.28 | -0.06 | -0.18 | 0.05 | | AvgSenLen | 0.94 | 0.15 | 0.04 | 0 | -0.04 | | AvgSenBl | -0.62 | 0.59 | -0.01 | -0.09 | 0.19 | | AvgUnqWdBl | 0.02 | 0.89 | 0.06 | 0.22 | 0.02 | | AvgUnqWdSen | 0.95 | 0.12 | 0.03 | 0.01 | -0.04 | | AvgWdLen | -0.12 | 0.48 | 0.04 | -0.15 | -0.47 | | AvgWdBl | 0.02 | 0.85 | 0.01 | 0.36 | 0.07 | | AvgWdSen | 0.95 | 0.1 | 0.02 | 0.01 | 0.03 | | CharEnt | -0.18 | 0.59 | -0.07 | -0.04 | 0.21 | | SenStDevUnqWd | -0.35 | 0.23 | 0.05 | 0.67 | 0.07 | | SenStdDevWd | -0.29 | 0.21 | 0.03 | 0.67 | 0.08 | | WdEnt | -0.01 | 0.88 | 0 | 0.12 | 0.16 | | WdLettStdDev | -0.04 | 0.56 | 0.08 | 0.02 | -0.16 | | LxcDiv | 0.08 | 0.81 | 0.05 | 0.16 | -0.06 | | LxcSoph | 0.56 | 0.08 | 0.09 | -0.08 | -0.48 | | SynSoph | 0.89 | 0.25 | 0.04 | 0.2 | 0.03 | | AvgNounBl | 0.13 | 0.55 | 0.22 | 0.28 | 0.52 | | AvgPronounBl | 0.04 | 0.71 | -0.19 | 0.37 | 0.11 | | AvgVerbBl | 0.04 | 0.84 | -0.17 | 0.36 | 0.02 | | AvgAdverbBl | 0.2 | 0.61 | -0.01 | 0.25 | -0.12 | | AvgAdjectiveBl | -0.08 | 0.65 | 0.05 | 0.06 | 0.04 | | AvgPrepositionBl | 0.11 | 0.79 | -0.1 | 0.18 | 0.08 | | AvgNounSen | 0.89 | -0.04 | 0.07 | -0.01 | 0.26 | | AvgPronounSen | 0.9 | -0.01 | -0.07 | 0.03 | 0.02 | | AvgVerbSen | 0.97 | 0.08 | -0.05 | 0 | -0.04 | | AvgAdverbSen | 0.8 | 0.17 | 0.04 | 0.09 | -0.15 | | AvgAdjectiveSen | 0.81 | 0 | 0 | -0.16 | -0.05 | | AvgPrepositionSen | 0.9 | 0.17 | 0 | 0 | 0.02 | | AvgUnqNoundBl | 0.13 | 0.52 | 0.2 | 0.16 | 0.48 | | AvgUnqPronounBl | -0.01 | 0.54 | -0.15 | 0.18 | 0.09 | | AvgUnqVerbBl | 0.01 | 0.8 | -0.09 | 0.29 | 0.01 | | AvgUnqAdverbBl | 0.12 | 0.69 | -0.01 | 0.1 | -0.11 | | AvgUnqAdjectiveBl | -0.07 | 0.65 | 0.04 | -0.02 | 0.04 | | AvgUnqPrepositionBl | 0.04 | 0.77 | -0.11 | 0.17 | 0.05 | | AvgPronBl_first_person | -0.02 | 0.47 | -0.22 | 0.38 | 0.16 | | AvgPronBl_indefinite | -0.03 | 0.57 | -0.13 | -0.04 | 0.14 | | AggPronSen_indefinite | 0.74 | 0.09 | -0.09 | -0.12 | 0.15 | | AvgPronBl_third_person | 0.08 | 0.54 | -0.03 | 0.14 | -0.03 | | AggPronSen_third_person | 0.78 | 0.06 | -0.02 | -0.05 | -0.13 | | AvgSemDep | 0.98 | 0.06 | -0.01 | 0.01 | 0.08 | | WdDiffLemmaStem | -0.11 | 0.34 | 0.02 | -0.16 | -0.19 | | WdDiffWdStem | -0.11 | 0.39 | -0.03 | -0.1 | -0.42 | | WdMaxDpthHypernymTree | 0.02 | -0.21 | 0.13 | 0.1 | 0.23 | | WdAvgDpthHypernymTree | 0.02 | -0.22 | 0.09 | 0.1 | 0.19 | | WdPathCntHypernymTree | 0.01 | -0.23 | 0.09 | 0.04 | 0.27 | | WdPolysemyCnt | -0.04 | -0.04 | -0.4 | 0.25 | 0.07 | | WdSylCnt | -0.11 | 0.5 | 0.32 | -0.08 | -0.3 | | AvgAOADoc_Shock | -0.06 | 0.48 | -0.01 | 0.02 | -0.15 | | AvgAOABl_Shock | -0.06 | 0.48 | -0.01 | 0.02 | -0.15 | | AvgAOASen_Shock | 0.49 | 0.17 | -0.02 | 0.08 | -0.39 | | AvgAOADoc_Cortese | -0.09 | -0.14 | 0.49 | -0.16 | -0.23 | | AvgAOABl_Cortese | -0.09 | -0.14 | 0.49 | -0.16 | -0.23 | | AvgAOASen_Cortese | 0.25 | -0.19 | 0.35 | 0.03 | -0.39 | | AvgAOADoc_Kuperman | -0.1 | -0.04 | 0.64 | -0.1 | -0.12 | | AvgAOABl_Kuperman | -0.1 | -0.04 | 0.64 | -0.1 | -0.12 | | AvgAOASen_Kuperman | 0.11 | -0.05 | 0.62 | 0 | -0.32 | | AvgAOADoc_Bird | -0.01 | 0.07 | 0.58 | 0.1 | -0.3 | | AvgAOABl_Bird | -0.01 | 0.07 | 0.58 | 0.1 | -0.3 | | AvgAOASen_Bird | 0.35 | -0.02 | 0.26 | 0.18 | -0.53 | | AvgAOADoc_Bristol | -0.07 | 0.14 | 0.5 | 0.02 | -0.31 | | AvgAOABl_Bristol | -0.07 | 0.14 | 0.5 | 0.02 | -0.31 | | AvgAOASen_Bristol | 0.41 | -0.05 | 0.1 | 0.01 | -0.51 | | AvgAOEDoc_IndexPolyFAT.3 | -0.04 | -0.1 | 0.89 | 0.02 | 0.22 | | AvgAOEBl_IndexPolyFAT.3 | -0.04 | -0.1 | 0.89 | 0.02 | 0.22 | | AvgAOESen_IndexPolyFAT.3 | 0.06 | -0.03 | 0.83 | 0.04 | 0.08 | | AvgAOEDoc_InvLinRegSlo | -0.05 | -0.2 | 0.84 | -0.02 | 0.3 | | AvgAOEBl_InvLinRegSlo | -0.05 | -0.2 | 0.84 | -0.02 | 0.3 | | AvgAOESen_InvLinRegSlo | 0.17 | -0.15 | 0.69 | 0.07 | -0.03 | | AvgAOEDoc_InfPointPoly | -0.03 | -0.05 | 0.85 | -0.11 | 0.22 | | AvgAOEBl_InfPointPoly | -0.03 | -0.05 | 0.85 | -0.11 | 0.22 | | AvgAOESen_InfPointPoly | 0.21 | -0.05 | 0.7 | 0.01 | -0.14 | | AvgAOEDoc_InvAverage | -0.04 | -0.08 | 0.88 | -0.13 | 0.22 | | AvgAOEBl_InvAverage | -0.04 | -0.08 | 0.88 | -0.13 | 0.22 | | AvgAOESen_InvAverage | 0.2 | -0.06 | 0.73 | -0.01 | -0.13 | | AvgAOEDoc_IndexAbThr.0.3. | -0.07 | -0.07 | 0.81 | 0.06 | 0.23 | | AvgAOEBl_IndexAbThr.0.3. | -0.07 | -0.07 | 0.81 | 0.06 | 0.23 | | AvgAOESen_IndexAbThr.0.3. | -0.03 | 0.02 | 0.78 | 0.04 | 0.18 | | AvgNmdEntBl | 0.06 | 0.31 | 0.06 | 0.04 | 0.62 | | AvgNounNmdEntBl | 0.04 | 0.16 | 0.2 | 0 | 0.67 | | AvgUnqNmdEntBl | 0.03 | 0.34 | 0.06 | 0.03 | 0.63 | | AvgNmdEntSen | 0.57 | -0.06 | 0.08 | 0 | 0.43 | | TCorefChainDoc | 0.02 | 0.51 | -0.04 | 0.33 | 0.25 | | AvgCorefChain | 0 | 0.44 | -0.14 | 0.18 | 0.03 | | AvgChainSpan | 0.17 | 0.66 | -0.09 | 0.15 | -0.05 | | AvgInferenceDistChain | 0.26 | 0.29 | -0.06 | 0.06 | 0.15 | | TActCorefChainWd | -0.09 | -0.34 | -0.03 | -0.01 | 0.03 | | TCorefChainBigSpan | 0.21 | 0.34 | -0.02 | 0.16 | 0.02 | | AvgConnBl_addition | 0.45 | 0.25 | -0.03 | 0.6 | 0.11 | | AvgConnSen_addition | 0.85 | -0.02 | 0.02 | 0.17 | 0.09 | | AvgConnBl_conjunctions | 0.48 | 0.32 | -0.03 | 0.43 | 0.19 | | AvgConnSen_conjunctions | 0.91 | -0.02 | 0.02 | 0.05 | 0.11 | | AvgConnBl_contrasts | 0.16 | 0.51 | 0.12 | -0.15 | 0 | | AvgConnSen_contrasts | 0.67 | 0.22 | 0.05 | -0.24 | -0.1 | | AvgConnBl_coord_conjs | 0.11 | 0.28 | -0.06 | 0.38 | -0.23 | | AvgConnSen_coord_conjs | 0.5 | 0.11 | 0.01 | 0.15 | -0.25 | | AvgConnBl_coord_connects | 0.47 | 0.41 | -0.03 | 0.52 | 0.09 | | AvgConnSen_coord_connects | 0.94 | 0.01 | 0.02 | 0.08 | 0.05 | | AvgConnBl_logical_conns | 0.44 | 0.24 | -0.04 | 0.5 | 0.17 | | AvgConnSen_logical_conns | 0.86 | -0.04 | 0.03 | 0.12 | 0.1 | | AvgConnBl_oppositions | 0.17 | 0.49 | 0.07 | -0.22 | 0.04 | | AvgConnSen_oppositions | 0.66 | 0.19 | 0.03 | -0.29 | -0.05 | | AvgConnBl_order | 0.04 | 0.16 | -0.02 | 0.45 | -0.06 | | AvgConnSen_order | 0.4 | -0.09 | 0.01 | 0.28 | 0.01 | | AvgConnBl_reas_purp | 0.07 | 0.32 | -0.03 | 0.49 | -0.16 | | AvgConnSen_reas_purp | 0.58 | 0.04 | 0.03 | 0.25 | -0.17 | | AvgConnBl_semi_coords | 0.11 | 0.28 | -0.06 | 0.38 | -0.23 | | AvgConnSen_semi_coords | 0.5 | 0.11 | 0.01 | 0.15 | -0.25 | | AvgConnBl_sentence_link | 0.39 | 0.51 | -0.1 | 0.58 | 0.07 | | AvgConnSen_sentence_link | 0.95 | 0 | 0 | 0.11 | 0.06 | | AvgConnBl_simp_subords | -0.13 | 0.52 | -0.14 | -0.04 | 0.01 | | AvgConnSen_simp_subords | 0.52 | 0.16 | -0.03 | -0.1 | -0.02 | | AvgConnBl_temp_conns | -0.1 | 0.34 | -0.21 | 0.22 | 0.04 | | AvgConnSen_temp_conns | 0.33 | 0 | -0.12 | 0.07 | 0.06 | | LexChainAvgSpan | -0.04 | 0.29 | -0.21 | 0.47 | 0.17 | | LexChainMaxSp | 0.04 | 0.61 | -0.1 | 0.45 | 0.02 | | AvgBlScore | 0.22 | 0.56 | -0.18 | 0.53 | 0.08 | | AvgSenScore | 0.93 | 0.05 | -0.06 | 0.06 | 0.07 | | SenScoreStDev | -0.39 | 0.27 | -0.02 | 0.67 | 0.07 | | AvgIntraBlCoh_LeackChod | -0.71 | 0.35 | 0.01 | 0.54 | 0.08 | | AvgSenAdjCoh_LeackChod | -0.69 | 0.33 | 0.02 | 0.56 | 0.07 | | AvgSenBlCoh_LeackChod | 0.76 | -0.5 | -0.07 | -0.05 | -0.17 | | AvgIntraBlCoh_WuPalmer | -0.71 | 0.35 | 0.01 | 0.53 | 0.08 | | AvgSenAdjCoh_WuPalmer | -0.7 | 0.33 | 0.01 | 0.55 | 0.08 | | AvgSenBlCoh_WuPalmer | 0.76 | -0.51 | -0.08 | -0.07 | -0.17 | | AvgIntraBlCoh_Path | -0.72 | 0.34 | 0 | 0.53 | 0.09 | | AvgSenAdjCoh_Path | -0.7 | 0.32 | 0 | 0.55 | 0.08 | | AvgSenBlCoh_Path | 0.78 | -0.52 | -0.07 | -0.17 | -0.14 | | AvgIntraBlCoh_LSA | -0.7 | 0.35 | -0.01 | 0.55 | 0.09 | | AvgSenAdjCoh_LSA | -0.68 | 0.34 | -0.01 | 0.57 | 0.09 | | AvgSenBlCoh_LSA | 0.76 | -0.49 | -0.13 | -0.04 | -0.15 | | AvgIntraBlCoh_LDA | -0.7 | 0.37 | -0.02 | 0.53 | 0.08 | | AvgSenAdjCoh_LDA | -0.68 | 0.35 | -0.02 | 0.55 | 0.07 | | AvgSenBlCoh_LDA | 0.71 | -0.39 | -0.2 | 0.02 | -0.19 | | AvgIntraBlCoh_word2vec | -0.69 | 0.35 | 0.01 | 0.55 | 0.08 | | AvgSenAdjCoh_word2vec | -0.69 | 0.33 | 0 | 0.56 | 0.09 | | AvgSenBlCoh_word2vec | 0.76 | -0.53 | -0.09 | -0.07 | -0.13 | | AvgBlVoiceCoOcc | -0.05 | 0.47 | -0.14 | 0.51 | 0.08 | | AvgVoice | -0.05 | 0.46 | -0.15 | 0.54 | 0.08 | | AvgSenSyll | 0.98 | 0.06 | 0.01 | -0.01 | 0.06 | | AvgSenStressedSyll | 0.96 | 0.08 | 0.03 | 0.01 | 0.06 | | AvgRhythmUnits | 0.18 | 0 | -0.08 | -0.18 | 0.01 | | AvgRhythmUnitSyll | 0.81 | -0.04 | 0.02 | 0.09 | 0.09 | | AvgRhythmUnitStreesSyll | 0.8 | -0.03 | 0.04 | 0.11 | 0.1 | | LangRhythmCoeff | 0.27 | -0.25 | -0.03 | 0.24 | 0.22 | | LangRhythmId | -0.19 | 0.11 | 0.02 | -0.04 | 0.21 | | FrqRhythmId | 0.68 | -0.44 | -0.04 | -0.07 | -0.11 | | LangRhythmDiameter | 0.34 | 0.07 | 0 | 0.11 | 0.35 | | SenAsson | -0.05 | 0.28 | -0.05 | 0.17 | -0.09 | | AvgDepsBl_acl | -0.14 | 0.18 | -0.12 | 0.02 | -0.13 | | AvgDepsSen_acl | 0.12 | -0.23 | -0.17 | -0.14 | -0.17 | | AvgDepsBl_advcl | 0.14 | 0.49 | -0.08 | 0.25 | -0.2 | | AvgDepsSen_advcl | 0.66 | 0.18 | 0.02 | 0.03 | -0.23 | | AvgDepsBl_advmod | 0.14 | 0.61 | -0.03 | 0.33 | -0.12 | | AvgDepsSen_advmod | 0.79 | 0.15 | 0.01 | 0.15 | -0.12 | | AvgDepsBl_amod | -0.1 | 0.58 | 0.09 | 0.11 | 0.15 | | AvgDepsSen_amod | 0.7 | 0 | 0.02 | -0.07 | 0.09 | | AvgDepsBl_aux | 0.16 | 0.54 | -0.15 | 0.06 | -0.03 | | AvgDepsSen_aux | 0.76 | 0.21 | -0.09 | -0.06 | -0.08 | | AvgDepsBl_auxpass | 0.04 | 0.4 | -0.05 | 0.2 | -0.08 | | AvgDepsBl_case | 0.08 | 0.67 | -0.08 | 0.16 | 0.28 | | AvgDepsSen_case | 0.84 | 0.1 | -0.03 | -0.01 | 0.17 | | AvgDepsBl_cc | 0.49 | 0.35 | 0 | 0.43 | 0.19 | | AvgDepsSen_cc | 0.92 | 0 | 0.03 | 0.05 | 0.09 | | AvgDepsBl_ccomp | 0.2 | 0.28 | -0.25 | 0.31 | 0.11 | | AvgDepsSen_ccomp | 0.76 | -0.05 | -0.15 | -0.02 | 0.11 | | AvgDepsBl_compound | -0.02 | -0.06 | 0.5 | 0.1 | 0.4 | | AvgDepsSen_compound | 0.37 | -0.22 | 0.24 | 0.02 | 0.28 | | AvgDepsBl_conj | 0.6 | 0.31 | 0.02 | 0.37 | 0.15 | | AvgDepsSen_conj | 0.88 | 0.02 | 0.02 | 0.03 | 0.08 | | AvgDepsBl_cop | -0.05 | 0.49 | -0.09 | -0.11 | -0.03 | | AvgDepsSen_cop | 0.64 | 0.02 | -0.13 | -0.16 | -0.15 | | AvgDepsBl_dep | 0.38 | 0.28 | 0.03 | 0.25 | 0.25 | | AvgDepsSen_dep | 0.73 | 0.01 | 0.05 | 0.02 | 0.18 | | AvgDepsBl_det | 0.14 | 0.58 | 0.03 | 0.31 | 0.22 | | AvgDepsSen_det | 0.76 | 0.11 | -0.01 | 0.03 | 0.19 | | AvgDepsBl_dobj | 0.14 | 0.56 | -0.02 | 0.47 | 0.08 | | AvgDepsSen_dobj | 0.9 | -0.03 | 0 | 0.06 | 0.06 | | AvgDepsBl_mark | 0.09 | 0.6 | -0.1 | 0.25 | -0.23 | | AvgDepsSen_mark | 0.75 | 0.1 | -0.03 | 0 | -0.24 | | AvgDepsBl_mwe | 0.14 | 0.34 | 0.04 | 0.06 | 0.15 | | AvgDepsSen_mwe | 0.45 | 0.15 | 0.08 | 0.04 | 0.14 | | AvgDepsBl_neg | 0 | 0.38 | 0.01 | -0.11 | 0.02 | | AvgDepsSen_neg | 0.39 | 0.05 | 0.05 | -0.21 | -0.07 | | AvgDepsBl_nmod | 0.09 | 0.62 | -0.1 | 0.16 | 0.3 | | AvgDepsSen_nmod | 0.83 | 0.08 | -0.03 | -0.03 | 0.2 | | AvgDepsBl_nsubj | 0.07 | 0.79 | -0.16 | 0.34 | 0.15 | | AvgDepsSen_nsubj | 0.97 | 0.03 | -0.04 | 0 | 0.04 | | AvgDepsBl_nsubjpass | 0.05 | 0.34 | -0.04 | 0.2 | -0.04 | | AvgDepsBl_nummod | 0.02 | 0.2 | -0.15 | 0.06 | 0.24 | | AvgDepsBl_punct | -0.53 | 0.62 | -0.01 | -0.15 | 0.13 | | AvgDepsSen_punct | -0.11 | 0.31 | 0 | -0.06 | -0.15 | | AvgDepsBl_xcomp | 0.03 | 0.41 | 0.03 | 0.34 | -0.22 | | AvgDepsSen_xcomp | 0.71 | 0.07 | 0.07 | 0.06 | -0.24 | --- # ReaderBench Model 2 {#readerbench-model-2} ## General Description ReaderBench Model 2 is a simplified version of [Model 1](#readerbench-model-1) that better handles multi-paragraph compositions, and Model 2 is recommended over Model 1. Model 2 is an ensemble (formed by averaging predicted quality scores) of three sub-models that are described below. All of these models used ReaderBench scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019]. Highly correlated ReaderBench metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). ## ReaderBench Model 2a This model was trained with fall data from [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * pls = partial least squares regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * svm = support vector machines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | rf | mars | svm | cube | |:----------|:-------|:-------|:-------|:-------|:-------| | -4.338 | 0.2371 | 0.1755 | 0.1780 | 0.2234 | 0.2532 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | overall | pls | rf | mars | svm | cube | |:------------------------------------------------|:--------|:-----|:------|:------|:-----|:------| | WdEnt | 20.53 | 4.67 | 10.12 | 73.84 | 5.16 | 18.67 | | AvgDepsSen_dep | 4.65 | 1.23 | 0.88 | 16.82 | 0.88 | 5.25 | | Content.words | 4.59 | 4.32 | 4.77 | 0 | 4.68 | 7.87 | | Words | 3.72 | 4.44 | 4.67 | 0 | 4.67 | 4.17 | | LxcDiv | 3.1 | 4.08 | 3.29 | 0 | 4.06 | 3.4 | | AvgAOASen_Shock | 2.77 | 1.45 | 1.16 | 9.34 | 1.39 | 1.7 | | TCorefChainDoc | 2.62 | 2.98 | 0.81 | 0 | 2.06 | 5.86 | | AvgChainSpan | 2.59 | 3.27 | 3.83 | 0 | 3.1 | 2.47 | | WdDiffWdStem | 2.46 | 2.73 | 3.07 | 0 | 2.24 | 3.7 | | SynSoph | 2.12 | 1.71 | 0.92 | 0 | 1.82 | 5.09 | | AvgDepsSen_punct | 2.03 | 2.48 | 1.68 | 0 | 1.74 | 3.55 | | TActCorefChainWd | 1.93 | 1.6 | 1.91 | 0 | 1.47 | 4.01 | | WdDiffLemmaStem | 1.66 | 1.52 | 0.72 | 0 | 2.44 | 2.93 | | RdbltyFlesch | 1.55 | 0.77 | 1.22 | 0 | 1.09 | 4.01 | | WdLettStdDev | 1.44 | 2.35 | 1.51 | 0 | 2.13 | 0.93 | | AvgAOESen_InverseAverage | 1.37 | 1.44 | 1.22 | 0 | 1.09 | 2.62 | | Sentences | 1.3 | 2.84 | 1.77 | 0 | 1.82 | 0 | | AvgWdLen | 1.27 | 2.65 | 1.57 | 0 | 2.02 | 0 | | LexChainMaxSp | 1.26 | 2.89 | 1.19 | 0 | 2.02 | 0 | | AvgAOADoc_Shock | 1.26 | 2.36 | 1.89 | 0 | 1.68 | 0.31 | | AvgAOADoc_Kuperman | 1.25 | 0.72 | 1.01 | 0 | 1.3 | 2.78 | | WdSylCnt | 1.15 | 1.57 | 1.83 | 0 | 1.51 | 0.77 | | CharEnt | 1.14 | 2.65 | 0.96 | 0 | 1.85 | 0 | | LexChainAvgSpan | 1.12 | 2.18 | 1.5 | 0 | 1.86 | 0 | | AvgDepsSen_advcl | 1.07 | 0.93 | 0.85 | 0 | 1.35 | 1.85 | | AvgAOASen_Kuperman | 1.04 | 1.23 | 1.48 | 0 | 1.46 | 0.93 | | AvgCorefChain | 1 | 1.86 | 0.85 | 0 | 0.9 | 1.08 | | WdAvgDpthHypernymTree | 1 | 1.14 | 0.87 | 0 | 0.97 | 1.7 | | SenStdDevWd | 0.98 | 1.96 | 1.43 | 0 | 1.49 | 0 | | TCorefChainBigSpan | 0.95 | 2.16 | 1.44 | 0 | 1.13 | 0 | | AvgAOADoc_Bristol | 0.94 | 1.75 | 1.03 | 0 | 1.1 | 0.62 | | LxcSoph | 0.92 | 1.64 | 1.2 | 0 | 0.85 | 0.77 | | AvgAdverbSen | 0.88 | 0.89 | 1.38 | 0 | 1.46 | 0.62 | | RdbltyDaleChall | 0.87 | 1.75 | 1.63 | 0 | 1 | 0 | | AvgSenAdjCoh_LDA | 0.82 | 1.97 | 0.64 | 0 | 1.33 | 0 | | AvgRhythmUnits | 0.82 | 1.12 | 1.13 | 0 | 1.15 | 0.62 | | FrqRhythmId | 0.8 | 1.69 | 1.07 | 0 | 1.18 | 0 | | AvgAOADoc_Bird | 0.78 | 0.95 | 0.3 | 0 | 1.43 | 0.93 | | AvgVoice | 0.78 | 2.01 | 0.76 | 0 | 0.99 | 0 | | AvgAOADoc_Cortese | 0.77 | 0.69 | 1.3 | 0 | 1.57 | 0.31 | | WdPathCntHypernymTree | 0.71 | 1.45 | 0.84 | 0 | 1.17 | 0 | | AvgConnSen_simple_subordinators | 0.7 | 0.51 | 2.49 | 0 | 0.82 | 0 | | AvgAOASen_Bristol | 0.68 | 0.66 | 0.71 | 0 | 1.29 | 0.62 | | AvgRhythmUnitStreesSyll | 0.63 | 0.08 | 0.91 | 0 | 0.81 | 1.23 | | AvgInferenceDistChain | 0.62 | 1.39 | 0.34 | 0 | 1.2 | 0 | | AggPronSen_indefinite | 0.62 | 0.45 | 0.63 | 0 | 1.31 | 0.62 | | AvgAOASen_Bird | 0.6 | 1.13 | 0.37 | 0 | 1.37 | 0 | | AvgDepsSen_compound | 0.6 | 0.72 | 0.5 | 0 | 0.48 | 1.08 | | WdPolysemyCnt | 0.58 | 0 | 1.09 | 0 | 1.9 | 0 | | AvgDepsSen_ccomp | 0.57 | 0.09 | 1.32 | 0 | 0.9 | 0.62 | | AvgAOASen_Cortese | 0.55 | 1.15 | 0.3 | 0 | 1.17 | 0 | | AvgDepsSen_cop | 0.54 | 0.24 | 0.58 | 0 | 0.97 | 0.77 | | AvgPronounSen | 0.54 | 0.12 | 0.93 | 0 | 0.48 | 1.08 | | AvgNmdEntSen | 0.52 | 0.24 | 1.12 | 0 | 1.33 | 0 | | AvgNounSen | 0.52 | 0.24 | 0.15 | 0 | 0.18 | 1.7 | | AvgDepsSen_nmod | 0.48 | 0.7 | 0.69 | 0 | 1 | 0 | | AvgDepsSen_aux | 0.48 | 0.24 | 0.92 | 0 | 1.31 | 0 | | AvgConnSen_addition | 0.48 | 1.1 | 0.6 | 0 | 0.66 | 0 | | AvgDepsSen_dobj | 0.48 | 0.23 | 1.51 | 0 | 0.16 | 0.62 | | AvgAOEDoc_InverseLinearRegressionSlope | 0.44 | 0.4 | 0.8 | 0 | 0.68 | 0.31 | | AvgDepsSen_mark | 0.41 | 0.43 | 0.95 | 0 | 0.73 | 0 | | AvgConnSen_temporal_connectors | 0.41 | 0.32 | 0.64 | 0 | 1.11 | 0 | | AvgDepsSen_det | 0.4 | 0.18 | 0.4 | 0 | 0.72 | 0.62 | | AvgConnSen_semi_coordinators | 0.38 | 0.8 | 0.15 | 0 | 0.16 | 0.62 | | AvgConnSen_order | 0.36 | 0.31 | 1.74 | 0 | 0.03 | 0 | | AggPronSen_third_person | 0.36 | 0.57 | 0.91 | 0 | 0.41 | 0 | | LangRhythmDiameter | 0.35 | 0.57 | 0.79 | 0 | 0.08 | 0.31 | | SenAsson | 0.35 | 0.8 | 0.83 | 0 | 0.16 | 0 | | AvgAOEDoc_IndexAboveThreshold.0.3. | 0.33 | 0.03 | 0.43 | 0 | 0.87 | 0.31 | | AvgDepsSen_amod | 0.29 | 0.33 | 0.98 | 0 | 0.27 | 0 | | AvgAdjectiveSen | 0.28 | 0.1 | 1.28 | 0 | 0.21 | 0 | | AvgConnSen_oppositions | 0.27 | 0.54 | 0.82 | 0 | 0.07 | 0 | | AvgDepsSen_xcomp | 0.24 | 0.01 | 0.13 | 0 | 1.04 | 0 | | AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.21 | 0.12 | 0.1 | 0 | 0.78 | 0 | | LangRhythmId | 0.19 | 0.47 | 0.45 | 0 | 0.05 | 0 | | AvgDepsSen_neg | 0.18 | 0.03 | 1.05 | 0 | 0 | 0 | | AvgDepsSen_mwe | 0.17 | 0.38 | 0.47 | 0 | 0.04 | 0 | | LangRhythmCoeff | 0.16 | 0 | 0.22 | 0 | 0.61 | 0 | | AvgDepsSen_acl | 0.06 | 0.25 | 0 | 0 | 0.02 | 0 | ## ReaderBench Model 2b This model was trained with winter data from [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * pls = partial least squares regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * svm = support vector machines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | rf | mars | svm | cube | |:----------|:-------|:-------|:-------|:-------|:-------| | -5.4658 | 0.2205 | 0.5768 | 0.2047 | 0.0528 | 0.0400 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | overall | pls | rf | mars | svm | cube | |:------------------------------------------------|:--------|:-----|:-----|:------|:-----|:------| | Content.words | 11.94 | 5.23 | 4.51 | 41.33 | 4.49 | 15.7 | | WdEnt | 8.27 | 5.15 | 5.03 | 19.27 | 4.44 | 21.01 | | SynSoph | 4.17 | 1.03 | 2.06 | 14.97 | 1.69 | 0 | | LxcDiv | 3.24 | 4.93 | 3.5 | 0 | 3.86 | 5.8 | | AvgDepsSen_det | 3.18 | 0.25 | 1.12 | 12.65 | 0.94 | 3.38 | | TCorefChainDoc | 2.63 | 4 | 2.76 | 0 | 2.45 | 6.76 | | AvgChainSpan | 2.28 | 3.64 | 2.57 | 0 | 2.88 | 1.45 | | LexChainMaxSp | 2.25 | 3.56 | 2.72 | 0 | 1.98 | 0 | | TActCorefChainWd | 2.22 | 0.83 | 0.71 | 7.46 | 0.78 | 6.76 | | Sentences | 2.18 | 3.8 | 2.48 | 0 | 2.25 | 0 | | AvgNounSen | 2.07 | 0.89 | 1.54 | 4.33 | 0.64 | 6.52 | | CharEnt | 1.7 | 3.41 | 1.66 | 0 | 2.16 | 0.97 | | RdbltyFlesch | 1.36 | 0.46 | 1.91 | 0 | 1.18 | 5.56 | | WdLettStdDev | 1.31 | 2.58 | 1.31 | 0 | 2 | 0 | | AvgSenAdjCoh_LeackockChodorow | 1.3 | 2.75 | 1.24 | 0 | 1.87 | 0 | | FrqRhythmId | 1.28 | 2.48 | 1.39 | 0 | 1.13 | 0 | | AvgDepsSen_aux | 1.25 | 0.94 | 1.69 | 0 | 0.97 | 3.38 | | AvgWdLen | 1.24 | 2.36 | 1.26 | 0 | 2.04 | 0 | | AvgAOADoc_Bristol | 1.21 | 1.59 | 1.61 | 0 | 0.88 | 0 | | AvgDepsSen_compound | 1.2 | 1.17 | 1.78 | 0 | 0.48 | 0 | | AvgVoice | 1.2 | 2.75 | 1.13 | 0 | 1.19 | 0 | | AvgAOADoc_Shock | 1.16 | 2.54 | 1.13 | 0 | 1.17 | 0 | | TCorefChainBigSpan | 1.13 | 2.24 | 1.22 | 0 | 0.78 | 0 | | AvgConnSen_addition | 1.07 | 1.31 | 1.29 | 0 | 1.31 | 1.69 | | WdDiffWdStem | 1.04 | 2.05 | 1.06 | 0 | 1.35 | 0 | | AvgConnSen_logical_connectors | 1.03 | 1.49 | 1.11 | 0 | 1.27 | 2.17 | | AvgCorefChain | 1.01 | 2.38 | 0.9 | 0 | 1.32 | 0 | | AggPronSen_third_person | 0.98 | 1.18 | 1.23 | 0 | 0.56 | 1.93 | | AvgDepsSen_punct | 0.98 | 1.89 | 1.01 | 0 | 1.48 | 0 | | AvgDepsSen_dep | 0.95 | 1.01 | 1.31 | 0 | 1.07 | 0 | | AvgRhythmUnitStreesSyll | 0.95 | 0.44 | 1.46 | 0 | 0.9 | 1.21 | | AvgDepsSen_dobj | 0.95 | 1.12 | 1.04 | 0 | 1.07 | 3.38 | | AvgAdjectiveSen | 0.91 | 0.44 | 1.47 | 0 | 0.9 | 0 | | SenStdDevWd | 0.9 | 2.04 | 0.69 | 0 | 1.52 | 1.21 | | LexChainAvgSpan | 0.87 | 1.85 | 0.77 | 0 | 1.88 | 0 | | WdPathCntHypernymTree | 0.86 | 1.46 | 0.94 | 0 | 1.38 | 0 | | AvgAOESen_InverseAverage | 0.85 | 0.71 | 1.27 | 0 | 0.81 | 0 | | AvgDepsSen_mark | 0.83 | 0.22 | 1.4 | 0 | 1.06 | 0 | | WdPolysemyCnt | 0.83 | 0.32 | 1.43 | 0 | 0.39 | 0 | | AvgConnSen_reason_and_purpose | 0.82 | 0.35 | 1.3 | 0 | 0.7 | 0.97 | | LangRhythmCoeff | 0.8 | 1.15 | 1 | 0 | 0.92 | 0 | | AvgConnSen_simple_subordinators | 0.78 | 0.11 | 1.3 | 0 | 1.47 | 0 | | AvgDepsSen_xcomp | 0.76 | 0.11 | 1.25 | 0 | 1.6 | 0 | | AvgAOASen_Bird | 0.76 | 0.33 | 1.19 | 0 | 0.62 | 0.97 | | AvgDepsSen_ccomp | 0.75 | 0.16 | 1.29 | 0 | 0.79 | 0 | | RdbltyDaleChall | 0.75 | 2.41 | 0.41 | 0 | 1.07 | 0 | | AvgAOEDoc_InflectionPointPolynomial | 0.73 | 0.7 | 1.06 | 0 | 0.65 | 0 | | AvgAOESen_IndexAboveThreshold.0.3. | 0.7 | 0.47 | 1.01 | 0 | 1.43 | 0 | | AvgAOESen_IndexPolynomialFitAboveThreshold.0.3. | 0.7 | 0.55 | 1.01 | 0 | 1.13 | 0 | | AggPronSen_indefinite | 0.7 | 0.09 | 1.07 | 0 | 1.64 | 1.21 | | AvgDepsSen_cop | 0.7 | 0.09 | 1.16 | 0 | 1.49 | 0 | | AvgNmdEntSen | 0.68 | 0.45 | 1.02 | 0 | 1.07 | 0 | | AvgConnSen_contrasts | 0.68 | 0.32 | 1.03 | 0 | 0.59 | 1.21 | | AvgConnSen_oppositions | 0.68 | 0.07 | 1.18 | 0 | 0.94 | 0 | | AvgDepsSen_advcl | 0.67 | 0.03 | 1.13 | 0 | 1.31 | 0 | | AvgAdverbSen | 0.67 | 0.43 | 1.01 | 0 | 1.08 | 0 | | AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.66 | 0 | 1.14 | 0 | 1.18 | 0 | | AvgDepsSen_nmod | 0.66 | 0.74 | 0.76 | 0 | 1.35 | 1.21 | | AvgAOADoc_Bird | 0.65 | 0.95 | 0.77 | 0 | 1.11 | 0 | | AvgDepsSen_amod | 0.65 | 0.53 | 0.69 | 0 | 0.9 | 3.86 | | AvgConnSen_semi_coordinators | 0.64 | 0.24 | 1.04 | 0 | 0.78 | 0 | | WdMaxDpthHypernymTree | 0.62 | 1.46 | 0.46 | 0 | 1.61 | 0 | | AvgAOASen_Shock | 0.62 | 1.13 | 0.62 | 0 | 1.34 | 0 | | AvgAOASen_Kuperman | 0.6 | 0.15 | 1.01 | 0 | 0.45 | 0.48 | | AvgConnSen_temporal_connectors | 0.58 | 0.27 | 0.99 | 0 | 0.01 | 0 | | AvgAOASen_Bristol | 0.57 | 0.38 | 0.87 | 0 | 0.67 | 0 | | LangRhythmDiameter | 0.56 | 0.65 | 0.81 | 0 | 0.06 | 0 | | AvgConnSen_order | 0.52 | 0.29 | 0.7 | 0 | 1 | 1.21 | | AvgAOEDoc_IndexAboveThreshold.0.3. | 0.5 | 0.01 | 0.79 | 0 | 1.65 | 0 | | AvgRhythmUnits | 0.5 | 0.73 | 0.57 | 0 | 1.13 | 0 | | AvgAOADoc_Kuperman | 0.5 | 0.14 | 0.82 | 0 | 0.86 | 0 | | AvgAOASen_Cortese | 0.49 | 0.13 | 0.83 | 0 | 0.66 | 0 | | AvgInferenceDistChain | 0.48 | 0.87 | 0.51 | 0 | 0.71 | 0 | | WdDiffLemmaStem | 0.48 | 0.4 | 0.62 | 0 | 1.55 | 0 | | SenAsson | 0.42 | 0.99 | 0.4 | 0 | 0.15 | 0 | | AvgDepsSen_mwe | 0.41 | 0.66 | 0.52 | 0 | 0.07 | 0 | | AvgDepsSen_neg | 0.39 | 0.28 | 0.64 | 0 | 0 | 0 | | AvgDepsSen_acl | 0.33 | 0.45 | 0.46 | 0 | 0.03 | 0 | | LxcSoph | 0.31 | 0.39 | 0.35 | 0 | 0.92 | 0 | | AvgAOEDoc_InverseLinearRegressionSlope | 0.27 | 0.19 | 0.39 | 0 | 0.61 | 0 | | AvgAOADoc_Cortese | 0.24 | 0.76 | 0.09 | 0 | 0.85 | 0 | | WdSylCnt | 0.23 | 0.83 | 0 | 0 | 1.29 | 0 | | LangRhythmId | 0.03 | 0.09 | 0.03 | 0 | 0 | 0 | ## ReaderBench Model 2c This model was trained on spring data from [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * pls = partial least squares regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * gbm = stochastic gradient boosted trees * svm = support vector machines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | rf | mars | gbm | svm | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------| | -7.3027 | 0.2354 | 0.1868 | 0.1595 | 0.1816 | 0.2191 | 0.0704 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | overall | pls | rf | mars | gbm | svm | cube | |:------------------------------------------------|:--------|:-----|:-----|:------|:------|:-----|:------| | Content.words | 11.99 | 4.55 | 5.81 | 30.16 | 21.71 | 4.24 | 11.11 | | WdEnt | 7.28 | 4.3 | 5.74 | 0 | 21.09 | 4.12 | 12.09 | | AvgDepsSen_compound | 3.97 | 2.07 | 1.98 | 13.22 | 2.22 | 1.52 | 6.82 | | AvgWdLen | 3.87 | 2.64 | 2.65 | 7.11 | 4.85 | 2.04 | 7.02 | | LxcDiv | 3.77 | 4.06 | 4.13 | 0 | 7.72 | 3.59 | 0.78 | | AvgChainSpan | 3.36 | 3.09 | 2.64 | 5.1 | 4.15 | 2.66 | 2.34 | | TCorefChainBigSpan | 2.64 | 2.23 | 1.48 | 10.59 | 0.43 | 0.93 | 0 | | Sentences | 2.37 | 3.33 | 2.15 | 0 | 2.63 | 2.24 | 4.87 | | AvgDepsSen_mark | 2.21 | 0.38 | 1.17 | 10.59 | 0.08 | 1.45 | 0 | | AvgDepsSen_dobj | 2 | 0.81 | 0.96 | 8.72 | 0.14 | 1.13 | 0.97 | | AvgSenAdjCoh_LSA | 1.95 | 2.68 | 1.87 | 0 | 3.17 | 2.26 | 0 | | AvgCorefChain | 1.94 | 2.2 | 1 | 5.1 | 0.28 | 1.28 | 2.73 | | WdDiffWdStem | 1.92 | 2.4 | 1.86 | 0 | 2.95 | 2.09 | 1.56 | | LexChainMaxSp | 1.82 | 3.13 | 2.35 | 0 | 1.28 | 2.01 | 0.97 | | WdLettStdDev | 1.79 | 3 | 1.66 | 0 | 1.64 | 2.28 | 0.97 | | TCorefChainDoc | 1.62 | 3.23 | 1.85 | 0 | 0.17 | 1.92 | 2.14 | | CharEnt | 1.59 | 2.56 | 0.9 | 0 | 0.29 | 2.1 | 5.46 | | WdSylCnt | 1.53 | 2.45 | 1.7 | 0 | 1.52 | 1.55 | 1.36 | | FrqRhythmId | 1.47 | 2.67 | 1.7 | 0 | 1.03 | 1.59 | 0.97 | | AvgDepsSen_punct | 1.36 | 1.82 | 1.57 | 0 | 0.72 | 1.83 | 2.53 | | AvgAOEDoc_InverseLinearRegressionSlope | 1.32 | 1.31 | 0.73 | 4.26 | 0.24 | 0.89 | 0.39 | | RdbltyDaleChall | 1.25 | 1.81 | 1.27 | 0 | 1.04 | 1.02 | 3.51 | | AvgAOADoc_Shock | 1.2 | 2.2 | 1.24 | 0 | 0.69 | 1.8 | 0 | | LangRhythmCoeff | 1.06 | 1.61 | 1.41 | 0 | 1.03 | 1.24 | 0.19 | | LexChainAvgSpan | 1.05 | 1.94 | 1.36 | 0 | 0.16 | 1.66 | 0 | | SenAsson | 1.05 | 1.63 | 0.58 | 3.07 | 0.02 | 0.56 | 0 | | AvgVoice | 1 | 2.62 | 0.58 | 0 | 0 | 1.36 | 0.39 | | AvgNounSen | 0.97 | 1.09 | 1.59 | 0 | 0.47 | 1.06 | 2.14 | | WdDiffLemmaStem | 0.94 | 1.65 | 1.01 | 0 | 0.36 | 1.32 | 0.78 | | TActCorefChainWd | 0.94 | 0.93 | 0.74 | 0 | 1.05 | 0.71 | 4.09 | | AvgAOADoc_Cortese | 0.93 | 1.16 | 0.9 | 0 | 0.56 | 1.89 | 0.39 | | AvgAOASen_Bristol | 0.92 | 0.35 | 1.28 | 2.08 | 1.15 | 0.34 | 0.39 | | SenStdDevWd | 0.92 | 1.6 | 0.97 | 0 | 0.09 | 1.78 | 0 | | AvgDepsSen_xcomp | 0.83 | 0.41 | 1.61 | 0 | 1.42 | 0.99 | 0 | | AvgAdjectiveSen | 0.83 | 1.24 | 1 | 0 | 0.18 | 1.21 | 1.36 | | AvgDepsSen_nmod | 0.81 | 0.16 | 1.23 | 0 | 0.38 | 1.25 | 3.51 | | AvgAOADoc_Kuperman | 0.8 | 0.7 | 1.18 | 0 | 0.65 | 1.44 | 0.39 | | AvgDepsSen_amod | 0.79 | 1.11 | 0.91 | 0 | 0.1 | 1.33 | 1.36 | | AvgDepsSen_ccomp | 0.78 | 1.06 | 1.41 | 0 | 0.27 | 1.18 | 0 | | AvgAOASen_Kuperman | 0.78 | 0.78 | 0.83 | 0 | 0.41 | 0.99 | 2.73 | | AvgNmdEntSen | 0.78 | 0.93 | 1 | 0 | 1.05 | 1.05 | 0 | | AvgAOESen_IndexPolynomialFitAboveThreshold.0.3. | 0.76 | 0.58 | 1.12 | 0 | 0.4 | 0.84 | 2.73 | | AvgConnSen_simple_subordinators | 0.74 | 0.25 | 0.86 | 0 | 1.12 | 1.52 | 0.39 | | AvgPronounSen | 0.72 | 1.09 | 1.13 | 0 | 0.02 | 0.99 | 0.97 | | AvgAOASen_Shock | 0.69 | 0.48 | 1.51 | 0 | 0.21 | 1.32 | 0 | | AvgConnSen_reason_and_purpose | 0.68 | 0.16 | 1.31 | 0 | 0.82 | 1.29 | 0 | | AvgAOASen_Cortese | 0.66 | 1.25 | 0.45 | 0 | 0.24 | 1.22 | 0 | | AvgAOESen_InverseLinearRegressionSlope | 0.66 | 0.99 | 1.02 | 0 | 0.31 | 0.68 | 0.97 | | AvgAOEDoc_InflectionPointPolynomial | 0.65 | 0.64 | 0.36 | 0 | 0.37 | 0.61 | 3.7 | | AvgConnSen_addition | 0.65 | 0.88 | 0.88 | 0 | 0.12 | 1.21 | 0.39 | | AvgConnSen_order | 0.64 | 0.44 | 0.48 | 0 | 0.92 | 1.41 | 0 | | AvgInferenceDistChain | 0.64 | 0.8 | 0.91 | 0 | 0.7 | 0.83 | 0 | | WdPolysemyCnt | 0.62 | 0.27 | 0.93 | 0 | 0.37 | 1.61 | 0 | | AvgAOEDoc_IndexPolynomialFitAboveThreshold.0.3. | 0.61 | 0.83 | 0.71 | 0 | 0.07 | 1.07 | 0.97 | | AvgRhythmUnits | 0.61 | 0.3 | 1.24 | 0 | 0.32 | 1.3 | 0 | | AvgDepsSen_aux | 0.57 | 0.03 | 1.26 | 0 | 0.38 | 1.32 | 0 | | SynSoph | 0.57 | 0.59 | 0.85 | 0 | 0.08 | 1.02 | 0.97 | | AvgDepsSen_cop | 0.55 | 0.87 | 0.48 | 0 | 0.05 | 1.25 | 0 | | AvgRhythmUnitStreesSyll | 0.52 | 0.76 | 1.17 | 0 | 0.14 | 0.56 | 0 | | AvgDepsSen_advmod | 0.48 | 0.33 | 0.6 | 0 | 0.2 | 1.26 | 0 | | AvgDepsSen_det | 0.48 | 0.22 | 1.04 | 0 | 0.45 | 0.68 | 0.39 | | AggPronSen_third_person | 0.47 | 0.86 | 0.8 | 0 | 0.08 | 0.58 | 0 | | AvgAOADoc_Bristol | 0.45 | 0.36 | 0.71 | 0 | 0.12 | 1.02 | 0.19 | | AvgDepsSen_acl | 0.44 | 1.28 | 0.29 | 0 | 0.16 | 0.36 | 0 | | AvgAOADoc_Bird | 0.44 | 0.38 | 0.84 | 0 | 0.13 | 0.89 | 0 | | WdAvgDpthHypernymTree | 0.43 | 0.79 | 0.71 | 0 | 0.06 | 0.54 | 0 | | RdbltyFlesch | 0.43 | 0.42 | 1.35 | 0 | 0.03 | 0.44 | 0 | | AvgDepsSen_dep | 0.42 | 0.68 | 0.6 | 0 | 0.02 | 0.75 | 0 | | AggPronSen_indefinite | 0.41 | 0.34 | 0.51 | 0 | 0.14 | 1.05 | 0 | | AvgConnSen_semi_coordinators | 0.39 | 0 | 1.01 | 0 | 0.13 | 0.92 | 0 | | AvgDepsSen_mwe | 0.38 | 0.6 | 1.17 | 0 | 0.1 | 0.07 | 0 | | AvgDepsSen_advcl | 0.38 | 0.06 | 0.44 | 0 | 0.01 | 1.4 | 0 | | AvgDepsSen_neg | 0.37 | 0.51 | 0.97 | 0 | 0.4 | 0.05 | 0 | | WdPathCntHypernymTree | 0.36 | 0.89 | 0.33 | 0 | 0.19 | 0.35 | 0 | | AvgAOESen_IndexAboveThreshold.0.3. | 0.35 | 0.27 | 0 | 0 | 0.41 | 1.05 | 0 | | AvgAOASen_Bird | 0.33 | 0.04 | 0.75 | 0 | 0.59 | 0.42 | 0 | | LxcSoph | 0.31 | 0.02 | 0.61 | 0 | 0.16 | 0.18 | 1.95 | | AvgConnSen_oppositions | 0.26 | 0.11 | 0.98 | 0 | 0.36 | 0 | 0 | | LangRhythmDiameter | 0.24 | 0.3 | 0.84 | 0 | 0.12 | 0.03 | 0 | | AvgConnSen_temporal_connectors | 0.17 | 0.23 | 0.58 | 0 | 0.09 | 0.01 | 0 | | LangRhythmId | 0.09 | 0.22 | 0.23 | 0 | 0.02 | 0.01 | 0 | --- # ReaderBench Model 3 {#readerbench-model-3} ## General Description ReaderBench Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below. The models were trained on ReaderBench scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons). Highly correlated ReaderBench metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). More details on the sample will be provided once peer review is complete on the main study using this model. ## ReaderBench Model 3narr This model was trained on 15-minute narrative writing samples. ### Algorithm Weightings in Ensemble Abbreviations: * overall = ensemble model * pls = partial least squares regression * gbm = stochastic gradient boosted trees * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | rf | mars | gbm | svm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------|:-------| | 0.0000 | 0.1419 | 0.0945 | 0.3143 | 0.0729 | 0.0816 | 0.1792 | 0.1538 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). |Metric |overall|pls |rf |mars |gbm |svm |enet|cube| |-------------------------------------------------|-------|----|----|-----|-----|----|----|----| |Content.words |13.76 |2 |2.22|32.41|16.24|2.38|4.95|8.73| |RB.AvgWdLen |7.5 |0.81|1.06|21.05|1.14 |1.01|1.38|3.55| |RB.AvgDepsBl_compound |4.66 |0.89|1.01|11.15|2.17 |0.45|2.45|3.09| |RB.WdEnt |4.59 |1.7 |2.06|7.03 |4.52 |1.75|4.27|5.73| |RB.LangRhythmId |3.17 |0.69|0.35|8.26 |0.06 |0.21|2.85|0.18| |RB.RdbltyDaleChall |3.15 |1.27|1.24|4.63 |2.86 |0.95|4.05|3.27| |RB.AvgUnqWdBl |2.75 |1.63|1.92|4.02 |0.46 |1.82|0 |6.45| |RB.LxcDiv |2.67 |1.87|1.89|0 |11.82|2.04|3.58|4.27| |Sentences |2.62 |1.71|2.13|0 |5.6 |1.52|4.67|5.91| |RB.TCorefChainDoc |2.35 |1.99|1.91|0 |5.33 |1.97|5.29|3.09| |RB.AvgAOADoc_Cortese |1.8 |0.21|0.37|3.53 |0.76 |0.8 |2.06|1.36| |RB.CAF |1.78 |1.83|1.52|0 |1.84 |1.87|3.65|3.27| |RB.AvgNounNmdEntBl |1.6 |0.53|0.45|4.63 |0.21 |0.19|0 |0.36| |RB.AvgDepsBl_nsubjpass |1.42 |0.81|0.27|3.29 |0.2 |0.41|1.23|0.18| |RB.AvgDepsBl_aux |1.21 |1.48|1.75|0 |2.12 |1.53|1.3 |2.36| |RB.TActCorefChainWd |1.06 |0.33|0.95|0 |1.19 |1.17|2.65|2 | |RB.AvgDepsBl_nsubj |1.05 |1.67|1.78|0 |2.94 |1.85|0 |2.09| |RB.AvgPronounBl |0.99 |1.72|1.84|0 |3.59 |2.06|0 |1.18| |RB.AvgUnqNoundBl |0.93 |0.71|0.59|0 |0.37 |0.65|2.75|1.55| |RB.TCorefChainBigSpan |0.89 |1.61|1.19|0 |0.16 |1.33|2.58|0 | |RB.AvgAOESen_InflectionPointPolynomial |0.87 |0.97|1.18|0 |0.57 |1.09|2.33|0.73| |RB.AvgBlScore |0.83 |1.46|1.28|0 |0.92 |1.62|0 |2.18| |RB.AvgConnBl_addition |0.81 |0.99|0.83|0 |0.77 |0.66|1.36|1.73| |RB.AvgChainSpan |0.81 |1.52|1.28|0 |2.39 |1.7 |0 |1.27| |RB.AvgPrepositionBl |0.79 |1.51|1.03|0 |0.76 |1.6 |0.95|1 | |RB.AvgUnqPrepositionBl |0.76 |1.48|1.09|0 |0.48 |1.55|0.8 |1.09| |RB.SenStdDevWd |0.74 |0.86|1.25|0 |1.41 |1.28|1.45|0.36| |RB.AvgAOADoc_Shock |0.72 |0.86|0.94|0 |1.36 |1.15|0.8 |1.27| |RB.AvgDepsBl_punct |0.68 |1.26|1.34|0 |1 |1 |0.35|1.18| |RB.AvgCorefChain |0.68 |1.19|1.03|0 |0.34 |1.28|1.1 |0.73| |RB.AvgNmdEntSen |0.67 |0.27|0.42|0 |0.17 |0.72|2.16|1 | |RB.AvgPronBl_indefinite |0.65 |1.4 |1.66|0 |1.98 |1.31|0.13|0.27| |RB.AvgDepsBl_det |0.65 |1.18|0.96|0 |0.43 |0.97|0.92|0.91| |RB.AvgDepsBl_dobj |0.65 |1.4 |0.87|0 |0.37 |1.26|0 |1.73| |RB.SynDiv |0.6 |0.64|0.7 |0 |0.42 |0.71|1.55|0.64| |RB.AvgAOEBl_InflectionPointPolynomial |0.6 |0.88|0.95|0 |2.01 |1.2 |0 |1.09| |RB.FrqRhythmId |0.59 |1.12|1.27|0 |0.11 |0.77|1.31|0.18| |RB.LangRhythmDiameter |0.58 |0.18|0.35|0 |0.17 |0.01|2.28|0.82| |RB.AvgDepsBl_expl |0.58 |0.64|0.28|0 |0.35 |0.2 |1.79|0.82| |RB.CharEnt |0.57 |1.37|1.01|0 |0.53 |1.35|0.86|0 | |RB.AvgNounSen |0.55 |0.74|0.91|0 |0.31 |0.46|1.46|0.36| |RB.AvgDepsBl_amod |0.55 |0.92|0.33|0 |0.1 |0.55|1.08|1.09| |RB.AvgUnqVerbBl |0.54 |1.56|1.01|0 |0.48 |1.48|0.16|0.36| |RB.AvgUnqPronounBl |0.54 |1.6 |0.94|0 |1.5 |1.7 |0 |0 | |RB.AvgPronBl_first_person |0.53 |1.35|0.7 |0 |0.34 |1.23|0.61|0.36| |RB.AvgConnBl_sentence_linking |0.53 |1.45|1.07|0 |0.41 |1.41|0 |0.64| |RB.LxcSoph |0.52 |0.32|0.77|0 |0.46 |0.6 |0.1 |2.09| |RB.AvgAOEBl_IndexPolynomialFitAboveThreshold.0.3.|0.5 |0.94|1.01|0 |0.41 |1.09|0.73|0.27| |RB.AvgRhythmUnitStreesSyll |0.49 |0.65|0.67|0 |0.45 |0.43|0.75|1 | |RB.AvgDepsBl_mark |0.47 |1.36|0.95|0 |0.08 |1.19|0 |0.64| |RB.AvgDepsBl_nmod |0.47 |1.27|0.79|0 |0.43 |1.1 |0 |0.73| |RB.WdDiffLemmaStem |0.45 |0.64|0.88|0 |0.65 |1.02|0.74|0.18| |RB.AvgDepsBl_conj |0.45 |0.95|0.52|0 |0.28 |0.64|0.41|0.91| |RB.AvgAOABl_Bird |0.44 |0.56|0.39|0 |0.58 |0.57|0.96|0.55| |RB.AvgPronBl_third_person |0.43 |1.36|1.14|0 |0.33 |1.26|0 |0.09| |RB.AvgDepsBl_ccomp |0.43 |0.93|0.86|0 |0.04 |0.47|1.06|0 | |RB.AggPronSen_third_person |0.43 |0.52|0.51|0 |0.11 |1.01|1.15|0.18| |RB.AvgDepsSen_punct |0.43 |0.38|0.62|0 |0.19 |0.94|0.81|0.64| |RB.AvgConnBl_simple_subordinators |0.42 |1.31|1.02|0 |0.73 |1.08|0 |0.09| |RB.AvgConnSen_simple_subordinators |0.41 |0.15|0.52|0 |0.09 |0.52|1.54|0.18| |RB.AvgSenBlCoh_LDA |0.4 |0.74|0.95|0 |0.2 |1.21|0 |0.73| |RB.AvgDepsBl_xcomp |0.4 |1.15|0.82|0 |0.37 |0.88|0.45|0 | |RB.AvgCommaBl |0.4 |0.72|0.45|0 |0.05 |0.39|0.96|0.45| |RB.AvgAOASen_Shock |0.39 |0.4 |0.74|0 |0.45 |0.9 |0.79|0.18| |RB.AvgSenBlCoh_word2vec |0.36 |1.11|0.78|0 |0.18 |1.03|0 |0.27| |RB.WdLettStdDev |0.34 |0.72|0.63|0 |0.39 |0.7 |0.46|0.18| |RB.AvgConnBl_temporal_connectors |0.34 |1.03|0.92|0 |0.02 |0.77|0.1 |0.27| |RB.AvgDepsBl_acl |0.34 |0.58|0.44|0 |0.07 |0.2 |1.19|0 | |RB.LangRhythmCoeff |0.33 |0.7 |0.59|0 |0.25 |0.66|0.61|0 | |RB.WdSylCnt |0.33 |0.38|0.9 |0 |0.27 |0.79|0.26|0.45| |RB.AvgDepsBl_auxpass |0.33 |0.86|0.55|0 |0.01 |0.5 |0.7 |0 | |RB.AvgConnBl_oppositions |0.33 |0.85|0.49|0 |0 |0.42|0.63|0.18| |RB.AvgAdverbBl |0.32 |1.1 |0.72|0 |0.11 |0.83|0 |0.18| |RB.AvgConnBl_order |0.32 |0.73|0.27|0 |0.01 |0.29|1 |0 | |RB.AvgAOABl_Bristol |0.31 |0.75|0.29|0 |0.7 |0.77|0.39|0 | |RB.AvgDepsSen_nmod |0.31 |0.06|0.43|0 |0.11 |0.34|0.45|1 | |RB.AvgPronounSen |0.31 |0.36|0.7 |0 |0.05 |0.55|0 |1 | |RB.AvgIntraBlCoh_Path |0.3 |1.14|0.3 |0 |0.16 |0.97|0 |0.18| |RB.AvgAOABl_Kuperman |0.3 |0.51|0.51|0 |0.71 |0.59|0.1 |0.45| |RB.AvgDepsSen_nsubj |0.3 |0.05|0.83|0 |0.04 |0.49|0 |1.18| |RB.AvgDepsSen_aux |0.3 |0.17|0.62|0 |0.21 |0.49|0.68|0.36| |RB.AvgInferenceDistChain |0.29 |0.8 |0.71|0 |0.19 |0.74|0.25|0 | |RB.AvgConnBl_conditions |0.29 |0.9 |0.45|0 |0.15 |0.49|0.44|0 | |RB.AvgDepsBl_cop |0.28 |1.07|0.53|0 |0.08 |0.7 |0 |0.18| |RB.RdbltyFlesch |0.28 |0.49|0.88|0 |0.38 |0.54|0 |0.45| |RB.AvgConnSen_temporal_connectors |0.28 |0.28|0.82|0 |0.17 |0.05|0.75|0.18| |RB.AvgUnqAdjectiveBl |0.27 |1.2 |0.24|0 |0.01 |0.97|0.03|0 | |RB.AvgDepsBl_advcl |0.27 |1.18|0.37|0 |0.08 |0.9 |0.01|0 | |RB.AvgDepsSen_advcl |0.27 |0.16|0.53|0 |0.12 |0.69|0.66|0.18| |RB.AggPronSen_indefinite |0.25 |0.42|0.84|0 |0.26 |1.17|0.03|0 | |RB.WdDiffWdStem |0.25 |0.65|0.56|0 |0.42 |0.81|0.13|0 | |RB.AvgDepsBl_neg |0.24 |0.45|0.08|0 |0.02 |0.12|0.9 |0 | |RB.AvgDepsBl_nummod |0.23 |0.45|0.11|0 |0 |0.12|0.89|0 | |RB.AvgDepsBl_mwe |0.22 |0.29|0.46|0 |0 |0.06|0.76|0 | |RB.AvgDepsSen_amod |0.22 |0.3 |0.64|0 |0.32 |0.72|0.22|0 | |RB.AvgAOASen_Bird |0.21 |0.32|0.74|0 |0.27 |0.42|0.25|0 | |RB.AvgPrepositionSen |0.21 |0.07|0.66|0 |0.05 |0.35|0 |0.73| |RB.AvgConnBl_contrasts |0.21 |1.04|0.19|0 |0.01 |0.64|0 |0 | |RB.AvgAOASen_Kuperman |0.21 |0.5 |0.2 |0 |0.88 |0.48|0 |0.18| |RB.AvgDepsSen_xcomp |0.21 |0.19|0.71|0 |0.05 |1.01|0.23|0 | |RB.AvgDepsBl_root |0.2 |0.04|0.29|0 |0.02 |0 |0.99|0 | |RB.AvgDepsSen_cop |0.2 |0.06|0.49|0 |0.28 |0.36|0.6 |0 | |RB.AvgConnSen_reason_and_purpose |0.19 |0.14|0.21|0 |0.11 |0.61|0.53|0 | |RB.AvgDepsSen_conj |0.19 |0.14|0.56|0 |0.02 |0.42|0 |0.55| |RB.AvgDepsSen_dobj |0.19 |0.06|0.6 |0 |0.23 |0.38|0 |0.55| |RB.AvgDepsSen_dep |0.19 |0.49|0.57|0 |0.29 |0.65|0 |0 | |RB.AvgAdverbSen |0.19 |0 |0.72|0 |0.05 |0.87|0 |0.36| |RB.AvgSenLen |0.18 |0.06|0.76|0 |0.12 |0.29|0 |0.45| |RB.AvgPronBl_second_person |0.18 |0.7 |0.62|0 |0.01 |0.3 |0 |0 | |RB.AvgConnBl_disjunctions |0.18 |0.73|0.25|0 |0.01 |0.35|0.16|0 | |RB.AvgConnBl_reason_and_purpose |0.18 |0.64|0.26|0 |0.03 |0.26|0.28|0 | |RB.AggPronSen_second_person |0.18 |0.32|0.47|0 |0.01 |0.08|0.53|0 | |RB.AvgConnBl_semi_coordinators |0.16 |0.35|0.31|0 |0.01 |0.09|0.43|0 | |RB.AvgPronBl_interrogative |0.16 |0.7 |0.35|0 |0.01 |0.28|0.07|0 | |RB.AvgAOASen_Bristol |0.15 |0.41|0.39|0 |0.14 |0.58|0 |0 | |RB.AvgDepsSen_ccomp |0.15 |0.18|0.59|0 |0.11 |0.47|0 |0.18| |RB.AvgDepsBl_iobj |0.15 |0.62|0.33|0 |0.01 |0.29|0 |0.09| |RB.AvgDepsSen_det |0.15 |0.21|0.32|0 |0.24 |0.07|0.12|0.36| |RB.AvgConnSen_addition |0.13 |0.11|0.27|0 |0.75 |0.46|0 |0 | |RB.AvgDepsSen_acl |0.13 |0.38|0.58|0 |0.11 |0.08|0 |0.09| |RB.AvgDepsSen_mark |0.12 |0.09|0.35|0 |0.04 |0.36|0 |0.27| |RB.AvgConnSen_oppositions |0.12 |0.25|0.65|0 |0.08 |0.05|0.13|0 | |RB.AvgDepsBl_dep |0.11 |0.58|0.04|0 |0.06 |0.19|0.04|0 | |RB.AvgConnSen_semi_coordinators |0.11 |0.22|0.65|0 |0.23 |0.04|0 |0 | |RB.AvgConnBl_complex_subordinators |0.11 |0.39|0.19|0 |0 |0.12|0.19|0 | |RB.AvgAOASen_Cortese |0.11 |0.12|0.25|0 |0.35 |0.64|0 |0 | |RB.AvgAdjectiveSen |0.1 |0.09|0.4 |0 |0.05 |0.56|0 |0 | |RB.AvgDepsSen_iobj |0.07 |0.16|0.45|0 |0.02 |0.02|0 |0 | |RB.AggPronSen_interrogative |0.07 |0.11|0.4 |0 |0.21 |0.01|0 |0 | |RB.AvgConnSen_order |0.07 |0.02|0.48|0 |0.13 |0 |0 |0.09| |RB.SenAsson |0.07 |0.24|0.41|0 |0 |0.02|0 |0 | |RB.AvgConnSen_conditions |0.06 |0.07|0.49|0 |0.08 |0 |0 |0 | |RB.AvgDepsBl_csubj |0.06 |0.02|0.31|0 |0.06 |0 |0.17|0 | |RB.AvgDepsSen_neg |0.06 |0.17|0.4 |0 |0.03 |0.03|0 |0 | |RB.AvgDepsBl_parataxis |0.05 |0.24|0.12|0 |0 |0.04|0 |0 | |RB.AvgDepsBl_appos |0.04 |0.2 |0.08|0 |0 |0.04|0 |0 | |RB.AvgDepsSen_nummod |0.04 |0.04|0.35|0 |0.02 |0 |0 |0 | |RB.AggPronSen_first_person |0.04 |0.02|0.2 |0 |0.14 |0.1 |0 |0 | |RB.AvgConnSen_disjunctions |0.03 |0.11|0.13|0 |0.04 |0.01|0 |0 | |RB.SenAllit |0.03 |0.03|0.3 |0 |0 |0 |0 |0 | |RB.AvgDepsSen_mwe |0.01 |0.07|0 |0 |0 |0.01|0 |0 | ## ReaderBench Model 3exp This model was trained on 15 min expository writing samples. ### Algorithm Weightings in Ensemble Abbreviations: * overall = ensemble model * pls = partial least squares regression * gbm = stochastic gradient boosted trees * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | rf | mars | gbm | svm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------| | -0.0156 | 0.0826 | 0.3112 | 0.0319 | 0.1360 | 0.3306 | 0.1259 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). |Metric |overall|rf |mars |gbm |svm |enet |cube | |--------------------------------------|-------|----|-----|-----|----|-----|-----| |Content.words |20.83 |5.13|35.84|48.17|3.43|17.49|14.66| |RB.AvgWdLen |4.5 |1.29|10.64|1.06 |0.42|1.6 |4.32 | |RB.AvgDepsBl_compound |4.11 |0.71|8.06 |0.36 |0.06|3.36 |3.92 | |RB.AvgConnBl_order |3.63 |0.6 |6.17 |0.15 |0.21|4.43 |1.81 | |RB.SenStdDevWd |3.56 |1.02|5.2 |0.29 |1.18|4.38 |2.41 | |RB.LangRhythmId |3.46 |0.96|10.64|0.29 |0.26|0 |0.7 | |RB.TCorefChainDoc |3.34 |1.93|0 |4.43 |2.48|7.01 |3.51 | |RB.WdEnt |3.21 |2.23|0 |1.79 |2.35|6.08 |5.52 | |RB.AggPronSen_first_person |2.93 |0.99|8.76 |0.77 |0.12|0 |1.1 | |Sentences |2.9 |1.37|0 |2.38 |1.47|6.99 |2.01 | |RB.AvgSenAdjCoh_Path |2.68 |1.05|0 |1.28 |1.09|5.93 |3.92 | |RB.CAF |2.51 |1.33|0 |1.24 |1.75|5.87 |1.81 | |RB.AvgPronBl_third_person |2.49 |0.76|7.52 |0.04 |0.79|0 |0.2 | |RB.AvgBlScore |2.27 |2.1 |4.33 |1.67 |2.38|0 |3.31 | |RB.AvgPronBl_second_person |2.03 |1.17|0 |0.64 |0.95|4.73 |2.01 | |RB.LangRhythmDiameter |1.92 |0.73|2.84 |0.28 |0.15|2.24 |1.91 | |RB.TActCorefChainWd |1.4 |0.76|0 |0.47 |1.04|2.97 |1.81 | |RB.TCorefChainBigSpan |1.31 |0.44|0 |1.17 |1.63|2.75 |1 | |RB.AvgUnqAdjectiveBl |1.06 |1.01|0 |0.23 |1.7 |1.95 |0.9 | |RB.WdDiffWdStem |0.99 |1.06|0 |1.8 |0.52|2.17 |0.6 | |RB.AvgDepsSen_nmod |0.94 |0.95|0 |0.52 |1.14|1.67 |1.2 | |RB.AvgDepsBl_expl |0.89 |0.79|0 |0.42 |0.42|1.86 |1.2 | |RB.RdbltyDaleChall |0.86 |1.25|0 |0.91 |0.78|1.01 |2.41 | |RB.AvgAOEBl_InflectionPointPolynomial |0.77 |0.72|0 |0.27 |0.7 |1.85 |0.1 | |RB.AvgConnBl_temporal_connectors |0.76 |0.91|0 |0.04 |0.5 |1.37 |1.41 | |RB.AvgPronBl_indefinite |0.75 |2.03|0 |5.56 |1.56|0 |1.61 | |RB.SynDiv |0.71 |0.61|0 |0.28 |1.04|1.39 |0.5 | |RB.LxcDiv |0.69 |1.51|0 |1.57 |2.14|0 |1.91 | |RB.AvgAOASen_Bristol |0.66 |0.56|0 |0.14 |0.31|1.56 |0.5 | |RB.AvgDepsBl_root |0.65 |0.09|0 |0.04 |0.04|1.95 |0 | |RB.AvgDepsBl_nsubj |0.62 |1.9 |0 |0.88 |2.19|0 |1.2 | |RB.AvgPronounBl |0.59 |1.54|0 |0.25 |1.94|0 |1.61 | |RB.AvgPrepositionBl |0.59 |1.37|0 |1.41 |2.07|0 |1.31 | |RB.AvgUnqNoundBl |0.49 |0.83|0 |0.41 |1.02|0 |2.21 | |RB.AvgDepsBl_parataxis |0.47 |0.53|0 |0.01 |0.15|1.26 |0 | |RB.LangRhythmCoeff |0.44 |0.58|0 |0.33 |0.4 |0.93 |0.2 | |RB.AvgUnqPrepositionBl |0.43 |0.94|0 |0.2 |2.05|0 |0.6 | |RB.AvgAOASen_Bird |0.43 |0.63|0 |0.4 |0.79|0.63 |0.5 | |RB.WdSylCnt |0.42 |0.96|0 |0.54 |0.18|0.5 |1.1 | |RB.AvgDepsBl_nmod |0.42 |1.01|0 |0.52 |1.59|0 |0.9 | |RB.AvgChainSpan |0.41 |1.04|0 |0.4 |1.58|0 |0.8 | |RB.AvgDepsBl_nummod |0.41 |0.7 |0 |0.01 |0.21|1 |0 | |RB.AvgDepsSen_expl |0.4 |0.41|0 |0.23 |0.06|1.1 |0 | |RB.AvgPronBl_first_person |0.39 |0.71|0 |0.51 |0.5 |0.25 |1.41 | |RB.AvgUnqVerbBl |0.38 |0.91|0 |0.06 |1.71|0 |0.6 | |RB.AvgDepsBl_aux |0.37 |0.59|0 |0.19 |0.93|0.4 |0.5 | |RB.AvgAdverbBl |0.33 |0.6 |0 |0.11 |1.31|0 |0.8 | |RB.AvgDepsBl_punct |0.33 |1.26|0 |0.3 |1.17|0 |0.5 | |RB.AvgNounSen |0.33 |0.99|0 |0.05 |0.22|0 |1.81 | |RB.LxcSoph |0.32 |0.79|0 |0.3 |0.75|0 |1.2 | |RB.CharEnt |0.31 |0.49|0 |1.05 |1.09|0.13 |0.4 | |RB.AvgDepsSen_cop |0.31 |0.86|0 |0.55 |0.55|0 |1.2 | |RB.AvgDepsBl_mark |0.31 |1.04|0 |0.56 |1.59|0 |0 | |RB.AvgSenBlCoh_LDA |0.3 |0.82|0 |0.16 |1.15|0 |0.6 | |RB.RdbltyFlesch |0.29 |0.47|0 |0.19 |0.17|0 |1.81 | |RB.AvgCorefChain |0.28 |0.76|0 |0.2 |1.05|0 |0.6 | |RB.AvgDepsBl_dobj |0.28 |0.92|0 |0.09 |1.36|0 |0.2 | |RB.AvgDepsBl_cop |0.27 |0.59|0 |0.07 |0.97|0 |0.7 | |RB.AvgDepsBl_det |0.27 |0.92|0 |0.09 |1.36|0 |0.1 | |RB.AvgDepsSen_mark |0.27 |0.68|0 |0.19 |1.12|0 |0.5 | |RB.AvgDepsBl_amod |0.26 |0.58|0 |0.27 |1.23|0 |0.3 | |RB.AvgDepsBl_mwe |0.25 |0.8 |0 |0.09 |0.61|0.3 |0 | |RB.AvgUnqAdverbBl |0.25 |0.6 |0 |0.03 |1.39|0 |0.1 | |RB.AvgPrepositionSen |0.24 |0.44|0 |0.16 |0.91|0 |0.6 | |RB.AvgConnBl_simple_subordinators |0.23 |0.76|0 |0.05 |1.22|0 |0 | |RB.AvgAOASen_Kuperman |0.23 |0.53|0 |0.51 |0.39|0.2 |0.4 | |RB.AvgDepsSen_compound |0.23 |1.22|0 |0.33 |0.33|0 |0.6 | |RB.AvgDepsBl_ccomp |0.22 |0.51|0 |0.05 |0.54|0.2 |0.3 | |RB.AvgUnqPronounBl |0.22 |0.46|0 |0 |1.33|0 |0 | |RB.FrqRhythmId |0.22 |0.94|0 |0.3 |0.68|0.06 |0.2 | |RB.AggPronSen_indefinite |0.22 |0.76|0 |0.37 |0.93|0 |0.2 | |RB.AvgDepsSen_dobj |0.21 |0.98|0 |0.1 |0.49|0 |0.5 | |RB.AggPronSen_second_person |0.2 |0.81|0 |0.23 |0.64|0 |0.3 | |RB.AvgAOADoc_Shock |0.2 |0.98|0 |0.42 |0.82|0 |0 | |RB.AvgConnSen_semi_coordinators |0.19 |0.59|0 |0.29 |0 |0.38 |0.1 | |RB.AvgConnBl_addition |0.18 |0.7 |0 |0.23 |0.65|0 |0.2 | |RB.AvgRhythmUnitStreesSyll |0.18 |0.89|0 |0.17 |0.47|0 |0.3 | |RB.AvgDepsSen_ccomp |0.18 |0.31|0 |0.22 |0.94|0 |0.2 | |RB.AvgAdverbSen |0.17 |0.38|0 |0.06 |0.99|0 |0 | |RB.AvgCommaSen |0.17 |0.62|0 |0.25 |0.8 |0 |0 | |RB.AvgAOEDoc_IndexAboveThreshold.0.3. |0.17 |0.72|0 |0.12 |0.36|0 |0.5 | |RB.AvgConnBl_contrasts |0.17 |0.46|0 |0.08 |0.82|0 |0.2 | |RB.AvgConnSen_simple_subordinators |0.16 |0.44|0 |0.13 |0.88|0 |0 | |RB.AvgConnBl_reason_and_purpose |0.16 |0.73|0 |0.14 |0.62|0 |0.1 | |RB.AvgAOADoc_Bird |0.16 |0.79|0 |0.14 |0.68|0 |0 | |RB.AvgDepsSen_amod |0.16 |0.29|0 |0.25 |0.5 |0 |0.5 | |RB.AvgConnBl_oppositions |0.16 |0.65|0 |0.05 |0.6 |0.02 |0.2 | |RB.AvgAOABl_Kuperman |0.15 |0.11|0 |0.18 |0.45|0 |0.6 | |RB.AvgDepsSen_xcomp |0.15 |0.63|0 |0.06 |0.73|0 |0 | |RB.AvgPronounSen |0.14 |0.62|0 |0.03 |0.26|0 |0.4 | |RB.AvgDepsBl_advcl |0.14 |0.21|0 |0.02 |0.89|0 |0 | |RB.AvgInferenceDistChain |0.14 |0.56|0 |0.2 |0.45|0 |0.2 | |RB.AvgNounNmdEntBl |0.14 |0.49|0 |0.87 |0.55|0 |0 | |RB.AggPronSen_third_person |0.14 |0.65|0 |0.14 |0.64|0 |0 | |RB.WdLettStdDev |0.14 |0.65|0 |0.18 |0.63|0 |0 | |RB.AvgConnSen_addition |0.13 |0.47|0 |0.23 |0.63|0 |0 | |RB.AvgNmdEntSen |0.13 |0.18|0 |0.36 |0.81|0 |0 | |RB.WdDiffLemmaStem |0.12 |0.71|0 |0.26 |0.29|0 |0.1 | |RB.AvgDepsSen_aux |0.12 |0.4 |0 |0.03 |0.64|0 |0 | |RB.AvgCommaBl |0.12 |0.66|0 |0.04 |0.4 |0 |0.1 | |RB.AvgAOASen_Shock |0.12 |0.28|0 |0.05 |0.73|0 |0 | |RB.AvgDepsBl_acl |0.12 |0.47|0 |0.13 |0.6 |0 |0 | |RB.AvgAOABl_Cortese |0.12 |0.28|0 |0.1 |0.64|0 |0.1 | |RB.AvgDepsSen_advcl |0.12 |0.46|0 |0.25 |0.59|0 |0 | |RB.AvgDepsBl_xcomp |0.12 |0.23|0 |0.09 |0.78|0 |0 | |RB.AvgConnSen_temporal_connectors |0.11 |0.73|0 |0.06 |0.09|0.06 |0.1 | |RB.AvgAOESen_InflectionPointPolynomial|0.11 |0.28|0 |0.11 |0.52|0 |0.1 | |RB.AvgDepsSen_dep |0.11 |0.49|0 |0.17 |0.38|0 |0.1 | |RB.AvgAOASen_Cortese |0.11 |0.22|0 |0.17 |0.66|0 |0 | |RB.AvgDepsSen_det |0.11 |0.14|0 |0.12 |0.54|0 |0.2 | |RB.AvgConnSen_reason_and_purpose |0.11 |0.39|0 |0.12 |0.58|0 |0 | |RB.AvgAOABl_Bristol |0.1 |0.45|0 |0.15 |0.37|0 |0.1 | |RB.AvgDepsBl_iobj |0.09 |0.74|0 |0.21 |0.17|0 |0 | |RB.AvgDepsSen_mwe |0.09 |0.64|0 |0.44 |0.21|0 |0 | |RB.AvgConnSen_order |0.08 |0.69|0 |0.67 |0.01|0 |0 | |RB.AvgConnSen_oppositions |0.08 |0.63|0 |0.11 |0.09|0 |0.1 | |RB.AvgConnBl_disjunctions |0.08 |0.5 |0 |0 |0.32|0 |0 | |RB.AvgConnSen_contrasts |0.07 |0.6 |0 |0.11 |0.11|0 |0 | |RB.AvgDepsBl_auxpass |0.07 |0.56|0 |0.01 |0.17|0 |0 | |RB.AvgDepsSen_neg |0.07 |0.47|0 |0.31 |0 |0 |0.2 | |RB.AvgConnBl_conditions |0.07 |0.49|0 |0.11 |0.23|0 |0 | |RB.AvgDepsBl_neg |0.06 |0.19|0 |0.03 |0.23|0 |0.1 | |RB.AvgPronBl_interrogative |0.06 |0.54|0 |0.04 |0.14|0 |0 | |RB.SenAsson |0.05 |0.25|0 |0 |0.23|0 |0 | |RB.AvgConnSen_disjunctions |0.05 |0.55|0 |0.03 |0.05|0 |0 | |RB.AvgConnBl_semi_coordinators |0.04 |0.16|0 |0.1 |0.16|0 |0 | |RB.AvgDepsSen_nummod |0.04 |0.46|0 |0.03 |0 |0 |0 | |RB.AvgDepsSen_acl |0.04 |0.46|0 |0.01 |0.01|0 |0 | |RB.AvgDepsBl_csubj |0.04 |0.4 |0 |0.01 |0.05|0 |0 | |RB.AvgDepsBl_nsubjpass |0.04 |0.25|0 |0 |0.16|0 |0 | |RB.AvgDepsBl_appos |0.04 |0.51|0 |0 |0.02|0 |0 | |RB.AvgDepsBl_dep |0.02 |0 |0 |0.08 |0.15|0 |0 | |RB.SenAllit |0.02 |0.3 |0 |0 |0 |0 |0 | ## ReaderBench Model 3per This modelwas trained on 15 min persuasive writing samples. ### Algorithm Weightings in Ensemble Abbreviations: * overall = ensemble model * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | mars | gbm | svm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------|:-------| | -0.0141 | 0.0326 | 0.2043 | 0.2331 | 0.1507 | 0.3202 | 0.0801 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). |Metric |overall|pls |mars |gbm |svm |enet |cube| |--------------------------------------------------|-------|----|-----|-----|----|-----|----| |RB.WdEnt |9.44 |1.97|0 |16.45|2.58|14.62|8.38| |RB.AvgPrepositionBl |8.44 |1.96|20.57|11.55|2.48|2.8 |4.83| |Sentences |6.71 |1.67|19.13|4.09 |1.41|4.14 |5.01| |RB.AvgBlScore |5.39 |2 |0 |15.75|2.73|2.76 |5.92| |RB.CAF |4.59 |1.59|19.13|1.27 |1.43|0 |2.73| |RB.AvgSenScore |3.72 |0.51|8.2 |0.15 |0.47|5.74 |2 | |RB.TCorefChainDoc |3.36 |1.97|0 |5.74 |2.16|4.69 |2.46| |RB.AvgWdLen |2.49 |1.32|0 |4.98 |1.13|2.84 |3.28| |RB.AvgAOADoc_Shock |2.39 |1.4 |8.2 |1.65 |1.21|0.2 |1.18| |RB.AvgPronBl_indefinite |2.34 |1.76|0 |3.91 |2.01|2.57 |3.73| |RB.RdbltyDaleChall |2.32 |0.79|0 |1.32 |0.68|5.11 |3.73| |RB.AvgDepsBl_compound |2.28 |0.23|7.3 |0.29 |0.02|1.88 |2 | |RB.AvgUnqNoundBl |2.11 |0.75|1.47 |0.3 |1.41|4.17 |2.64| |RB.AvgConnBl_simple_subordinators |1.75 |1.76|0 |3.08 |1.98|2.11 |0.46| |RB.AvgAOESen_InflectionPointPolynomial |1.61 |0.53|5.45 |0.22 |0.77|0.99 |0.36| |RB.AvgPronBl_interrogative |1.23 |0.61|0 |0.53 |0.13|2.92 |2 | |RB.AvgDepsBl_nsubj |1.12 |1.87|0 |1.87 |2.47|0 |3.37| |RB.AvgDepsBl_mark |1.1 |1.63|0 |0.89 |1.7 |1.41 |1.91| |RB.AvgNmdEntSen |1.07 |0.07|0 |0.25 |0.41|2.79 |0.91| |RB.AvgDepsBl_amod |1.06 |0.82|0 |0.33 |0.41|2.72 |0.64| |RB.AvgCorefChain |1.04 |0.95|0 |0.08 |1.1 |2.38 |1 | |RB.AvgDepsSen_advmod |1.01 |0.09|0 |0.22 |0.22|2.68 |1 | |RB.AvgPronBl_first_person |0.96 |0.7 |2.97 |0.06 |0.27|0.75 |0.64| |RB.LangRhythmCoeff |0.95 |0.77|0 |1.43 |0.81|1.35 |0.73| |RB.AvgAOABl_Bird |0.93 |0.47|3.59 |0.46 |0.63|0 |0 | |RB.AvgDepsSen_aux |0.92 |0 |4 |0.21 |0.18|0 |0.55| |RB.AvgSenAdjCoh_Path |0.87 |1.19|0 |2.13 |1.26|0.33 |0.73| |RB.AvgDepsBl_det |0.83 |1.51|0 |0.54 |1.48|1.21 |0.73| |RB.AvgConnSen_oppositions |0.8 |0.24|0 |0.41 |0.01|2.1 |0.46| |RB.AvgAOASen_Shock |0.8 |0.62|0 |0.13 |1.02|1.67 |0.91| |RB.LxcDiv |0.8 |1.45|0 |1.85 |1.4 |0 |1.55| |RB.AvgUnqPronounBl |0.77 |1.68|0 |0.64 |1.73|0.63 |1.46| |RB.AvgAOADoc_Cortese |0.69 |0.02|0 |0.3 |0.59|1.5 |0.73| |RB.AvgUnqAdjectiveBl |0.69 |1.13|0 |0.03 |0.81|1.58 |0.36| |RB.AvgDepsBl_nsubjpass |0.69 |0.48|0 |0.02 |0.16|2.05 |0.09| |RB.AvgAOASen_Bird |0.64 |0.52|0 |0.31 |0.39|1.46 |0.46| |RB.AvgDepsBl_cop |0.63 |1.09|0 |0.1 |0.8 |1.3 |0.55| |RB.TCorefChainBigSpan |0.6 |1.53|0 |0.34 |1.38|0.62 |1 | |RB.AvgChainSpan |0.59 |1.42|0 |1 |1.73|0 |0.73| |RB.AvgDepsBl_aux |0.59 |1.42|0 |0.59 |1.36|0.5 |0.64| |RB.AvgUnqPrepositionBl |0.58 |1.83|0 |0.66 |2.15|0 |0.64| |RB.AggPronSen_second_person |0.54 |0.32|0 |0.04 |0.55|1.28 |0.46| |RB.SynDiv |0.51 |1.15|0 |0.48 |1.14|0.52 |0.46| |RB.CharEnt |0.49 |1.19|0 |1.21 |1.21|0 |0 | |RB.AvgAOASen_Bristol |0.48 |0.35|0 |0.13 |0.36|1.1 |0.46| |RB.AvgDepsBl_punct |0.47 |1.51|0 |0.72 |1.39|0 |0.64| |RB.AvgDepsBl_nmod |0.46 |1.6 |0 |0.48 |1.72|0 |0.55| |RB.AvgUnqVerbBl |0.43 |1.51|0 |0.41 |1.47|0 |0.91| |RB.WdDiffLemmaStem |0.42 |0.86|0 |0.43 |0.9 |0.48 |0.18| |RB.AvgDepsSen_mark |0.42 |0.28|0 |0.1 |0.29|0.68 |1.73| |RB.WdDiffWdStem |0.42 |0.67|0 |0.31 |0.66|0.71 |0.18| |RB.AvgPronounBl |0.41 |1.67|0 |0.06 |1.67|0 |1.18| |RB.AvgAOASen_Cortese |0.41 |0.08|0 |0.15 |0.3 |0.88 |0.73| |RB.AvgConnBl_temporal_connectors |0.41 |0.71|0 |0.02 |0.38|1.05 |0 | |RB.AvgRhythmUnitStreesSyll |0.38 |0.09|0 |0.12 |0.17|0.87 |0.73| |RB.LxcSoph |0.37 |0.75|0 |0.68 |0.65|0 |1.18| |RB.AvgDepsBl_ccomp |0.34 |1.38|0 |0.08 |1.26|0.12 |0.64| |RB.AvgDepsSen_neg |0.34 |0.23|0 |0.09 |0.52|0.74 |0 | |RB.AvgPronBl_third_person |0.34 |1.34|0 |0.39 |1.16|0 |0.46| |RB.AvgDepsBl_root |0.33 |0.09|0 |0.06 |0 |1 |0 | |RB.TActCorefChainWd |0.33 |0.36|0 |0.36 |0.81|0.14 |0.91| |RB.WdSylCnt |0.3 |0.76|0 |0.54 |0.7 |0 |0.64| |RB.AvgUnqAdverbBl |0.29 |1.34|0 |0.09 |1.2 |0 |0.64| |RB.AvgDepsSen_punct |0.27 |0.44|0 |0.09 |0.16|0.68 |0 | |RB.AvgNmdEntBl |0.26 |1.25|0 |0.11 |1.05|0 |0.55| |RB.AvgConnBl_addition |0.25 |1.13|0 |0.16 |0.9 |0 |0.55| |RB.AvgDepsSen_compound |0.25 |0.5 |0 |0.19 |0.56|0 |1.37| |RB.AggPronSen_indefinite |0.25 |0.42|0 |0.24 |0.9 |0 |0.64| |RB.AvgDepsBl_dobj |0.25 |1.37|0 |0.02 |1.15|0 |0.46| |RB.AvgConnBl_order |0.24 |0.56|0 |0.03 |0.21|0.59 |0 | |RB.AvgAOADoc_Bristol |0.24 |0.8 |0 |0.28 |0.89|0 |0.27| |RB.SenStdDevWd |0.24 |0.98|0 |0.18 |1.06|0 |0.18| |RB.FrqRhythmId |0.23 |1.1 |0 |0.02 |0.72|0.21 |0.18| |RB.AvgDepsBl_advmod |0.23 |1.21|0 |0.12 |0.96|0 |0.27| |RB.AvgDepsBl_advcl |0.23 |1.36|0 |0.01 |1.26|0 |0 | |RB.AvgAdverbBl |0.23 |1.25|0 |0.1 |1.01|0 |0.27| |RB.AvgConnBl_logical_connectors |0.22 |1.14|0 |0.2 |0.89|0 |0.09| |RB.AvgConnBl_semi_coordinators |0.21 |0.2 |0 |0.02 |0.03|0.56 |0.18| |RB.AvgPronounSen |0.21 |0.33|0 |0.15 |0.72|0 |0.73| |RB.AvgUnqNmdEntBl |0.21 |1 |0 |0.18 |0.65|0 |0.55| |RB.AvgConnSen_simple_subordinators |0.2 |0.46|0 |0.17 |0.74|0 |0.46| |RB.AvgSenBlCoh_LDA |0.2 |0.59|0 |0.06 |0.91|0.01 |0.36| |RB.AvgConnBl_reason_and_purpose |0.2 |1.2 |0 |0.09 |0.96|0 |0 | |RB.AvgDepsSen_amod |0.2 |0.18|0 |0.18 |0.62|0 |0.82| |RB.AvgInferenceDistChain |0.19 |0.33|0 |0.23 |0.81|0 |0.09| |RB.AvgAOESen_IndexPolynomialFitAboveThreshold.0.3.|0.19 |0.68|0 |0.32 |0.65|0 |0 | |RB.AvgSenBlCoh_LSA |0.19 |0.97|0 |0.09 |0.86|0 |0.18| |RB.AvgAOEDoc_InverseAverage |0.18 |0.62|0 |0.17 |0.82|0 |0 | |RB.AvgAOEBl_IndexAboveThreshold.0.3. |0.18 |0.59|0 |0.17 |0.71|0.06 |0 | |RB.SenAllit |0.18 |0.53|0 |0.01 |0.19|0.43 |0 | |RB.AvgDepsSen_dep |0.18 |0.19|0 |0.2 |0.42|0 |0.91| |RB.AvgDepsBl_nummod |0.17 |0.66|0 |0.11 |0.32|0.24 |0 | |RB.AvgDepsSen_det |0.16 |0.21|0 |0.08 |0.89|0 |0 | |RB.AvgDepsBl_conj |0.16 |1.02|0 |0.11 |0.63|0 |0.09| |RB.AvgDepsSen_ccomp |0.16 |0.3 |0 |0.15 |0.55|0 |0.46| |RB.AvgConnSen_addition |0.15 |0.01|0 |0.11 |0.52|0 |0.55| |RB.AvgDepsSen_acl |0.15 |0.15|0 |0.13 |0.01|0.36 |0 | |RB.AvgDepsBl_xcomp |0.14 |0.89|0 |0.04 |0.56|0 |0.18| |RB.AvgPronBl_second_person |0.14 |0.91|0 |0.05 |0.55|0 |0.18| |RB.AvgAOABl_Kuperman |0.14 |0.08|0 |0.23 |0.45|0 |0.18| |RB.AvgNounSen |0.14 |0.2 |0 |0.03 |0.22|0 |1.18| |RB.AvgConnBl_contrasts |0.14 |1.03|0 |0.05 |0.67|0 |0 | |RB.WdLettStdDev |0.13 |0.6 |0 |0.19 |0.39|0 |0.09| |RB.AvgDepsBl_neg |0.13 |0.3 |0 |0.03 |0.05|0.33 |0 | |RB.AvgDepsSen_xcomp |0.13 |0.06|0 |0.05 |0.58|0 |0.36| |RB.AvgDepsSen_advcl |0.13 |0.14|0 |0.13 |0.66|0 |0 | |RB.AvgConnBl_oppositions |0.13 |0.98|0 |0.03 |0.54|0 |0.18| |RB.AggPronSen_first_person |0.13 |0.06|0 |0.22 |0.56|0 |0 | |RB.AggPronSen_third_person |0.12 |0.38|0 |0.02 |0.69|0 |0 | |RB.AvgDepsSen_dobj |0.1 |0.07|0 |0.07 |0.31|0 |0.46| |RB.AvgAdjectiveSen |0.1 |0.04|0 |0.09 |0.4 |0 |0.27| |RB.AvgDepsSen_cop |0.1 |0.14|0 |0.04 |0.59|0 |0 | |RB.AvgConnSen_reason_and_purpose |0.09 |0.05|0 |0.06 |0.25|0 |0.46| |RB.AvgConnBl_conditions |0.09 |0.72|0 |0.04 |0.39|0 |0 | |RB.LangRhythmDiameter |0.09 |0.29|0 |0.13 |0.06|0.15 |0 | |RB.AvgDepsBl_acl |0.09 |0.74|0 |0.03 |0.39|0 |0.09| |RB.AvgAOASen_Kuperman |0.08 |0.09|0 |0.11 |0.26|0 |0.18| |RB.AvgConnBl_disjunctions |0.08 |0.52|0 |0.03 |0.21|0.08 |0 | |RB.AvgDepsSen_nmod |0.08 |0.02|0 |0.15 |0.17|0 |0.27| |RB.AvgCommaBl |0.08 |0.78|0 |0.02 |0.36|0 |0 | |RB.AvgDepsBl_mwe |0.07 |0.67|0 |0.01 |0.33|0 |0 | |RB.AvgDepsBl_dep |0.07 |0.64|0 |0.07 |0.22|0 |0.09| |RB.AvgConnSen_semi_coordinators |0.06 |0.13|0 |0.01 |0.01|0.11 |0.27| |RB.AvgConnSen_conditions |0.05 |0.01|0 |0.2 |0 |0 |0 | |RB.AvgConnBl_conjuncts |0.04 |0.42|0 |0.01 |0.14|0 |0 | |RB.LangRhythmId |0.04 |0.39|0 |0.04 |0.1 |0 |0 | |RB.AvgDepsBl_csubj |0.03 |0.39|0 |0 |0.09|0 |0 | |RB.AvgDepsBl_iobj |0.03 |0.29|0 |0.03 |0.09|0 |0 | |RB.AvgDepsSen_nummod |0.03 |0.13|0 |0.09 |0.01|0 |0 | |RB.AvgDepsBl_auxpass |0.03 |0.36|0 |0 |0.11|0 |0 | |RB.AvgDepsBl_expl |0.03 |0.29|0 |0 |0.09|0.03 |0 | |RB.SenAsson |0.03 |0.37|0 |0.02 |0.1 |0 |0 | |RB.AvgCommaSen |0.03 |0.14|0 |0.05 |0.02|0 |0.18| |RB.AvgDepsSen_csubj |0.01 |0.04|0 |0.02 |0 |0 |0 | |RB.AvgConnSen_disjunctions |0.01 |0.07|0 |0.03 |0.01|0 |0 | |RB.AvgDepsBl_parataxis |0.01 |0.2 |0 |0 |0.03|0 |0 | |RB.AvgConnBl_complex_subordinators |0 |0.06|0 |0 |0.01|0 |0 | |RB.AvgConnSen_temporal_connectors |0 |0.01|0 |0.01 |0 |0 |0 | --- # Coh-Metrix Model 1 {#cohmetrix-model-1} ## General Description Model 1 has been replaced by the greatly simplified [Model 2](#cohmetrix-model-2). Model 2 is recommended for current use. Coh-Metrix Model 1 is an ensemble (formed by averaging predicted quality scores) of six sub-models that are detailed below. All of these models used Coh-Metrix scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019]. This scoring model was evaluated in the following publications: [@Matta2022; @Keller-Margulis2021] ## Coh-Metrix Model 1a This model was trained on fall Coh-Metrix scores from data described in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:------|:-------|:------| | -10.8465 | 0.0266 | 0.1506 | 0.2663 | -0.0302 | 0.296 | 0.2609 | 0.136 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | DESWC | 19.42 | 42.06 | 5.86 | 4.76 | 26.22 | 7.84 | 45.19 | 26.59 | | DESWLlt | 6.35 | 2.43 | 2.98 | 1.91 | 5.72 | 2.26 | 13.18 | 13.19 | | LDMTLD | 4.88 | 8.17 | 4.01 | 3.11 | 6.4 | 4.13 | 4.19 | 10.99 | | PCCONNp | 4.75 | 0.03 | 0 | 0.53 | 1.42 | 0.59 | 18.28 | 0 | | PCNARp | 3.13 | 0 | 1.76 | 0.8 | 0 | 1.17 | 10.1 | 0 | | WRDHYPn | 2.82 | 3.15 | 2.9 | 1.83 | 7.62 | 2.16 | 0 | 10.99 | | PCVERBp | 2.35 | 0 | 0.99 | 0.6 | 0 | 1.23 | 7.35 | 0 | | DESPL | 1.57 | 0.67 | 3.22 | 1.72 | 2.63 | 1.65 | 1.53 | 0 | | SYNSTRUTa | 1.39 | 1.21 | 0.85 | 1.12 | 3.49 | 2.74 | 0 | 2.42 | | PCDCp | 1.28 | 1.12 | 2 | 1.08 | 0 | 2.72 | 0 | 0.88 | | DESWLsy | 1.26 | 0.48 | 2.09 | 1.23 | 1.3 | 1.34 | 0 | 3.08 | | CNCTempx | 1.25 | 1.24 | 0.89 | 1.99 | 1.97 | 1.73 | 0 | 1.54 | | LDTTRa | 1.25 | 0.48 | 2.01 | 0.89 | 2.69 | 1.65 | 0 | 3.08 | | WRDFRQa | 1.23 | 1.38 | 1.86 | 1.58 | 1.86 | 0.82 | 0 | 3.08 | | WRDVERB | 1.16 | 1.48 | 1.04 | 0.69 | 3.13 | 1.79 | 0 | 3.08 | | LSASSpd | 1.11 | 0.1 | 1.95 | 1.43 | 3.46 | 1.39 | 0 | 1.54 | | CNCTemp | 1.07 | 1.06 | 0.93 | 1.51 | 1.89 | 1.48 | 0 | 1.54 | | CNCADC | 1 | 0.88 | 1.12 | 1.89 | 0 | 1.61 | 0 | 0 | | SMINTEp | 0.99 | 1.86 | 0.75 | 1.33 | 3.22 | 1.3 | 0 | 1.54 | | SMCAUSwn | 0.99 | 0.82 | 1.4 | 1.72 | 0 | 1.64 | 0 | 0 | | DESWLsyd | 0.98 | 2.02 | 1.97 | 1.03 | 2.25 | 1.54 | 0 | 0.66 | | CRFCWO1d | 0.97 | 1.03 | 1.67 | 1.5 | 0 | 1.64 | 0 | 0 | | PCNARz | 0.95 | 0.74 | 1.29 | 0.88 | 3.68 | 1.5 | 0 | 1.54 | | WRDHYPnv | 0.94 | 0.1 | 1.9 | 0.97 | 0 | 1.23 | 0 | 1.54 | | WRDPRO | 0.94 | 1.26 | 1.77 | 1.28 | 0 | 1.68 | 0 | 0 | | DESSLd | 0.93 | 0.15 | 1.67 | 1.09 | 0 | 1.79 | 0.18 | 0 | | DESWLltd | 0.93 | 1.34 | 2.3 | 1.28 | 1.04 | 1.35 | 0 | 0 | | DRPP | 0.93 | 0.99 | 2.18 | 1.12 | 0.47 | 1.62 | 0 | 0 | | CNCLogic | 0.91 | 1.16 | 1.42 | 1.36 | 1.86 | 0.99 | 0 | 1.1 | | LSAGN | 0.86 | 0.52 | 2.55 | 1.44 | 0 | 0.94 | 0 | 0 | | PCCONNz | 0.83 | 1.35 | 1.09 | 1.09 | 0 | 1.21 | 0 | 0.88 | | CRFCWOad | 0.82 | 0.32 | 1.51 | 1.44 | 0 | 1.24 | 0 | 0 | | WRDADV | 0.81 | 0.07 | 1.3 | 1.24 | 0 | 1.51 | 0 | 0 | | RDFRE | 0.81 | 0.86 | 1.29 | 0.98 | 2.81 | 1.65 | 0 | 0 | | LSASS1d | 0.8 | 0.4 | 1.55 | 1.38 | 0.22 | 1.17 | 0 | 0 | | WRDFRQmc | 0.8 | 1.22 | 1.35 | 0.32 | 0 | 1.92 | 0 | 0.66 | | LDTTRc | 0.8 | 0.36 | 1.82 | 0.98 | 0.02 | 1.48 | 0 | 0 | | PCVERBz | 0.8 | 0.11 | 1.55 | 0.87 | 0 | 0.93 | 0 | 1.54 | | DRNP | 0.77 | 0.25 | 1.73 | 0.66 | 0 | 1.71 | 0 | 0 | | WRDCNCc | 0.72 | 1.35 | 1.07 | 0.64 | 3.32 | 0 | 0 | 3.08 | | CRFNOa | 0.72 | 0.4 | 0.74 | 1.33 | 0.63 | 1.25 | 0 | 0 | | CNCPos | 0.71 | 0.33 | 0.88 | 1.02 | 0.09 | 0.64 | 0 | 1.54 | | SYNMEDpos | 0.71 | 1.89 | 1.75 | 1.08 | 0 | 0.86 | 0 | 0 | | LSAGNd | 0.68 | 0.78 | 1.5 | 1.1 | 0.83 | 0.94 | 0 | 0 | | CNCCaus | 0.63 | 0.09 | 1.11 | 1.17 | 0 | 0.91 | 0 | 0 | | DRVP | 0.63 | 1.02 | 0.82 | 0.6 | 0.03 | 0.7 | 0 | 1.54 | | DRNEG | 0.63 | 0.4 | 0.76 | 1.13 | 0.03 | 1.09 | 0 | 0 | | CRFCWOa | 0.61 | 0.32 | 0.46 | 1.25 | 0 | 0.98 | 0 | 0 | | RDL2 | 0.61 | 0.3 | 0.31 | 0.87 | 0 | 1.45 | 0 | 0 | | WRDPOLc | 0.61 | 0.56 | 0.85 | 1.57 | 0 | 0.5 | 0 | 0 | | WRDIMGc | 0.61 | 0.19 | 0.14 | 0.74 | 0 | 0.87 | 0 | 1.54 | | SMCAUSr | 0.6 | 0.32 | 1.58 | 0.23 | 0.02 | 1.5 | 0 | 0 | | WRDFAMc | 0.6 | 1.32 | 1.09 | 0.86 | 1.25 | 0.96 | 0 | 0 | | LSASSp | 0.59 | 0.14 | 0.79 | 1.05 | 0 | 0.99 | 0 | 0 | | SMINTEr | 0.59 | 0.58 | 1.83 | 0.36 | 0 | 1.22 | 0 | 0 | | SMCAUSlsa | 0.59 | 0.48 | 0.24 | 1.01 | 0.69 | 1.25 | 0 | 0 | | WRDAOAc | 0.59 | 0.24 | 1.66 | 0.94 | 0.16 | 0.74 | 0 | 0 | | SMCAUSvp | 0.54 | 0.02 | 0.93 | 1.33 | 0 | 0.46 | 0 | 0 | | DRAP | 0.54 | 0.23 | 0.46 | 0.94 | 0 | 1.05 | 0 | 0 | | WRDHYPv | 0.54 | 1.15 | 0.28 | 1.17 | 1.97 | 0.75 | 0 | 0 | | PCTEMPp | 0.53 | 0.38 | 1.05 | 0.38 | 0 | 0.82 | 0 | 0.88 | | SYNLE | 0.49 | 0.34 | 1.13 | 0.62 | 1.27 | 0.83 | 0 | 0 | | PCDCz | 0.48 | 0 | 0 | 1.93 | 0 | 0 | 0 | 0 | | CRFANPa | 0.46 | 0.18 | 0.76 | 0.77 | 0 | 0.78 | 0 | 0 | | WRDNOUN | 0.44 | 0.54 | 0.56 | 0.53 | 0 | 0.97 | 0 | 0 | | DESSC | 0.43 | 0 | 0 | 1.72 | 0 | 0 | 0 | 0 | | CNCNeg | 0.43 | 0 | 0 | 1.74 | 0 | 0 | 0 | 0 | | WRDPRP3s | 0.41 | 0.28 | 0.76 | 0.45 | 1.91 | 0.85 | 0 | 0 | | PCREFp | 0.4 | 0 | 0.06 | 0.46 | 0 | 1.13 | 0 | 0 | | PCREFz | 0.39 | 0.33 | 0.47 | 0.43 | 0 | 0.94 | 0 | 0 | | WRDADJ | 0.38 | 0.08 | 0.99 | 0.59 | 0 | 0.51 | 0 | 0 | | PCCNCz | 0.38 | 0.26 | 1.18 | 0.32 | 0 | 0.73 | 0 | 0 | | CRFCWO1 | 0.37 | 0 | 0 | 1.5 | 0 | 0 | 0 | 0 | | SMCAUSv | 0.35 | 0.38 | 0.39 | 0.53 | 0 | 0.68 | 0 | 0 | | WRDMEAc | 0.34 | 1.04 | 0.21 | 0.57 | 2.34 | 0.56 | 0 | 0 | | CNCAdd | 0.34 | 0 | 0 | 1.38 | 0 | 0 | 0 | 0 | | WRDFRQc | 0.34 | 0.53 | 0.54 | 0.83 | 0 | 0.28 | 0 | 0 | | PCCNCp | 0.32 | 0 | 0.57 | 0.13 | 0 | 0.93 | 0 | 0 | | SYNSTRUTt | 0.32 | 0 | 0 | 1.28 | 0 | 0 | 0 | 0 | | SYNNP | 0.32 | 0.14 | 0.03 | 0.26 | 0 | 1.02 | 0 | 0 | | LSASS1 | 0.32 | 0 | 0 | 1.3 | 0 | 0 | 0 | 0 | | CRFSOa | 0.31 | 0 | 0 | 1.25 | 0 | 0 | 0 | 0 | | PCSYNp | 0.29 | 0.55 | 0.68 | 0 | 0.13 | 0.85 | 0 | 0 | | SYNMEDlem | 0.28 | 0 | 0 | 1.12 | 0 | 0 | 0 | 0 | | SYNMEDwrd | 0.26 | 0 | 0 | 1.03 | 0 | 0 | 0 | 0 | | RDFKGL | 0.26 | 0 | 0 | 1.03 | 0 | 0 | 0 | 0 | | WRDPRP3p | 0.25 | 0.01 | 0.84 | 0.02 | 0 | 0.66 | 0 | 0 | | CNCAll | 0.22 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 | | CRFAOa | 0.21 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | PCTEMPz | 0.21 | 0 | 0 | 0.85 | 0 | 0 | 0 | 0 | | DESSL | 0.15 | 0 | 0 | 0.59 | 0 | 0 | 0 | 0 | | SMTEMP | 0.13 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | CRFAO1 | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | CRFANP1 | 0.08 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | PCSYNz | 0.04 | 0 | 0 | 0.17 | 0 | 0 | 0 | 0 | | CRFSO1 | 0.03 | 0 | 0 | 0.14 | 0 | 0 | 0 | 0 | | CRFNO1 | 0.02 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 | ## Coh-Metrix Model 1b This model used Coh-Metrix scores from 7 min narrative writing samples in winter [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:--------|:-------|:-------|:-------|:-------|:-------| | -9.468 | 0.2532 | -0.0876 | 0.2097 | 0.0554 | 0.2458 | 0.2979 | 0.0974 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | DESWC | 20.79 | 30 | 7.75 | 3.44 | 27.38 | 7.68 | 37.87 | 27.6 | | LDMTLD | 6.27 | 12.21 | 3.55 | 1.94 | 3.85 | 4.41 | 8.17 | 3.39 | | DESWLlt | 6.11 | 3.24 | 1.99 | 1.08 | 6.03 | 2.23 | 14.98 | 13.8 | | CRFCWO1d | 5.2 | 0.54 | 1.57 | 1.71 | 0.35 | 1.98 | 19.51 | 0 | | LSAGN | 4.28 | 7.08 | 2.73 | 2.24 | 1.41 | 3.97 | 0 | 17.19 | | RDL2 | 3.21 | 0.71 | 1.62 | 1.3 | 0 | 1.12 | 11.39 | 0 | | DESPL | 2.56 | 0.19 | 3.82 | 1.84 | 6.29 | 2.53 | 5.18 | 0 | | SYNLE | 1.48 | 3.13 | 0.87 | 1.41 | 1.45 | 2.12 | 0 | 0.48 | | LSAGNd | 1.29 | 0.53 | 1.67 | 1.39 | 0 | 1.34 | 0 | 7.02 | | WRDVERB | 1.28 | 1.66 | 1.91 | 1.32 | 3.02 | 2.19 | 0 | 0 | | WRDPRP3s | 1.2 | 1.06 | 2.64 | 1.2 | 3.48 | 2.15 | 0 | 0 | | RDFRE | 1.17 | 1.26 | 2.36 | 0.2 | 0 | 2.28 | 0 | 3.39 | | WRDIMGc | 1.1 | 1.23 | 0.57 | 0.96 | 3.24 | 1.06 | 0 | 3.39 | | DESSLd | 1.07 | 0.9 | 0.91 | 1.18 | 0 | 0.93 | 1.95 | 0 | | LDTTRa | 1.06 | 0.96 | 3.04 | 0.83 | 0 | 1.12 | 0 | 3.39 | | SYNMEDpos | 1.05 | 0.29 | 2.03 | 1.57 | 0 | 1.54 | 0 | 3.39 | | WRDNOUN | 1.02 | 0.83 | 1.48 | 1.39 | 5.64 | 1.14 | 0 | 0 | | PCCNCz | 1.01 | 0.78 | 1.56 | 1.26 | 0 | 1.22 | 0 | 3.39 | | WRDHYPnv | 1.01 | 1.06 | 0.45 | 1.07 | 1.76 | 1.1 | 0 | 3.39 | | DESWLltd | 0.95 | 1.21 | 0.88 | 1.2 | 2.76 | 1.52 | 0 | 0 | | LSASSp | 0.91 | 0.91 | 1.57 | 1.34 | 0 | 1.92 | 0 | 0 | | WRDHYPv | 0.88 | 0.72 | 1.87 | 1.14 | 2.79 | 1.32 | 0 | 0 | | PCCNCp | 0.88 | 0 | 1.25 | 1.28 | 1.4 | 1.19 | 0 | 3.39 | | SMCAUSwn | 0.87 | 1.08 | 1.35 | 1.6 | 2.18 | 0.78 | 0 | 0 | | LSASS1d | 0.86 | 0.25 | 1.96 | 1.47 | 3.26 | 1.3 | 0 | 0 | | CNCAdd | 0.84 | 0.74 | 1.13 | 0.8 | 1.85 | 0.53 | 0 | 3.39 | | SYNNP | 0.82 | 1.25 | 1.32 | 0.72 | 4.14 | 0.68 | 0 | 0 | | DESWLsy | 0.82 | 0.76 | 0.59 | 1.01 | 0 | 1.33 | 0.84 | 0 | | DESWLsyd | 0.81 | 1.52 | 0.69 | 1.28 | 0 | 0.96 | 0.13 | 0 | | WRDFRQmc | 0.75 | 0.84 | 0.92 | 0.99 | 2.76 | 1.03 | 0 | 0 | | PCVERBz | 0.74 | 0.37 | 1.53 | 1.4 | 0 | 1.59 | 0 | 0 | | PCREFp | 0.73 | 0 | 0.41 | 0.55 | 5.81 | 0.26 | 0 | 3.39 | | CRFAOa | 0.72 | 0.22 | 1.88 | 1.32 | 0 | 1.6 | 0 | 0 | | DRVP | 0.71 | 1.3 | 0.17 | 0.73 | 2.1 | 1.01 | 0 | 0 | | CRFCWOad | 0.71 | 0.1 | 1.93 | 1.51 | 0 | 1.49 | 0 | 0 | | SMCAUSv | 0.68 | 1.02 | 1.48 | 0.65 | 0 | 1.26 | 0 | 0 | | PCSYNz | 0.68 | 0.41 | 1.89 | 0.69 | 0 | 1.78 | 0 | 0 | | PCNARz | 0.67 | 0.59 | 1.31 | 1.29 | 0 | 1.15 | 0 | 0 | | CRFCWO1 | 0.67 | 0.42 | 1.72 | 1.38 | 0 | 1.12 | 0 | 0 | | CRFANPa | 0.66 | 0.22 | 1.25 | 1.25 | 0 | 1.57 | 0 | 0 | | CRFNOa | 0.66 | 0.63 | 0.71 | 1.5 | 0 | 1.08 | 0 | 0 | | DRPP | 0.66 | 0.69 | 1.38 | 0.81 | 2.9 | 0.69 | 0 | 0 | | CNCTemp | 0.65 | 0.55 | 0.28 | 1.47 | 0.68 | 1.12 | 0 | 0 | | SMINTEr | 0.63 | 0.72 | 1.23 | 0.55 | 0 | 1.56 | 0 | 0 | | CNCNeg | 0.62 | 1.15 | 0.39 | 0.92 | 0 | 0.95 | 0 | 0 | | LDTTRc | 0.62 | 0.65 | 1.34 | 1.03 | 0.22 | 0.99 | 0 | 0 | | WRDADV | 0.62 | 0.62 | 0.79 | 0.93 | 0 | 1.4 | 0 | 0 | | SMINTEp | 0.61 | 0.13 | 1.26 | 1.31 | 0 | 1.38 | 0 | 0 | | WRDFRQa | 0.6 | 1.08 | 1.08 | 0.8 | 0 | 0.8 | 0 | 0 | | SMCAUSlsa | 0.58 | 0.83 | 0.64 | 0.6 | 0 | 1.33 | 0 | 0 | | SMCAUSvp | 0.58 | 0.33 | 1.47 | 1.12 | 0 | 1.08 | 0 | 0 | | CNCAll | 0.58 | 0.92 | 1.12 | 0.75 | 1.02 | 0.64 | 0 | 0 | | SYNSTRUTa | 0.56 | 0.4 | 1.51 | 0.94 | 0 | 1.03 | 0 | 0 | | WRDADJ | 0.55 | 1.43 | 0.3 | 0.4 | 0.51 | 0.72 | 0 | 0 | | WRDMEAc | 0.54 | 0.65 | 0.09 | 1.1 | 0 | 1.05 | 0 | 0 | | DRNP | 0.53 | 0.53 | 1.42 | 0.45 | 0 | 1.25 | 0 | 0 | | PCTEMPp | 0.52 | 0.51 | 1.21 | 0.8 | 0 | 0.95 | 0 | 0 | | PCVERBp | 0.52 | 0 | 0.91 | 1.17 | 0 | 1.3 | 0 | 0 | | SMCAUSr | 0.52 | 0.25 | 1.49 | 0.08 | 0 | 1.87 | 0 | 0 | | PCSYNp | 0.51 | 0.01 | 1.27 | 0.44 | 0 | 1.82 | 0 | 0 | | CNCCaus | 0.5 | 0.62 | 0.42 | 0.72 | 0 | 1.12 | 0 | 0 | | WRDPRO | 0.49 | 0.65 | 0.75 | 0.93 | 0.14 | 0.62 | 0 | 0 | | WRDAOAc | 0.48 | 0.84 | 0.97 | 0.55 | 0 | 0.7 | 0 | 0 | | PCNARp | 0.46 | 0 | 1.14 | 1.07 | 0 | 0.96 | 0 | 0 | | CNCTempx | 0.45 | 0.46 | 0.67 | 0.68 | 0 | 0.99 | 0 | 0 | | WRDFRQc | 0.42 | 0.49 | 0.25 | 0.78 | 0 | 0.85 | 0 | 0 | | PCCONNp | 0.39 | 0.8 | 0.87 | 0.29 | 0 | 0.57 | 0 | 0 | | DRNEG | 0.37 | 0.17 | 0.47 | 0.85 | 0.01 | 0.75 | 0 | 0 | | WRDPOLc | 0.36 | 0.19 | 0.72 | 0.73 | 0 | 0.72 | 0 | 0 | | CNCLogic | 0.35 | 0.26 | 0.34 | 0.7 | 1.58 | 0.35 | 0 | 0 | | DESSC | 0.34 | 0 | 0 | 1.84 | 0 | 0 | 0 | 0 | | PCREFz | 0.33 | 0.51 | 0.84 | 0.55 | 0 | 0.3 | 0 | 0 | | SYNMEDwrd | 0.32 | 0 | 0 | 1.72 | 0 | 0 | 0 | 0 | | WRDFAMc | 0.31 | 0.46 | 0 | 0.71 | 0 | 0.42 | 0 | 0 | | LSASSpd | 0.3 | 0 | 0 | 1.61 | 0 | 0 | 0 | 0 | | PCDCp | 0.29 | 0.25 | 0.78 | 0.71 | 0 | 0.25 | 0 | 0 | | SYNMEDlem | 0.29 | 0 | 0 | 1.57 | 0 | 0 | 0 | 0 | | WRDHYPn | 0.29 | 0.36 | 1.3 | 0.66 | 0 | 0 | 0 | 0 | | CRFSOa | 0.28 | 0 | 0 | 1.53 | 0 | 0 | 0 | 0 | | DRAP | 0.26 | 0.3 | 0.38 | 0.69 | 0 | 0.21 | 0 | 0 | | LSASS1 | 0.26 | 0 | 0 | 1.44 | 0 | 0 | 0 | 0 | | CRFCWOa | 0.25 | 0 | 0 | 1.37 | 0 | 0 | 0 | 0 | | WRDCNCc | 0.21 | 0 | 0 | 1.12 | 0 | 0 | 0 | 0 | | SYNSTRUTt | 0.21 | 0 | 0 | 1.13 | 0 | 0 | 0 | 0 | | PCTEMPz | 0.21 | 0 | 0 | 1.14 | 0 | 0 | 0 | 0 | | SMTEMP | 0.19 | 0 | 0 | 1.02 | 0 | 0 | 0 | 0 | | CRFANP1 | 0.19 | 0 | 0 | 1.02 | 0 | 0 | 0 | 0 | | PCDCz | 0.18 | 0 | 0 | 0.99 | 0 | 0 | 0 | 0 | | WRDPRP3p | 0.17 | 0 | 0.55 | 0 | 0 | 0.71 | 0 | 0 | | CNCADC | 0.15 | 0 | 0 | 0.84 | 0 | 0 | 0 | 0 | | CNCPos | 0.14 | 0 | 0 | 0.75 | 0 | 0 | 0 | 0 | | PCCONNz | 0.12 | 0 | 0 | 0.65 | 0 | 0 | 0 | 0 | | CRFAO1 | 0.1 | 0 | 0 | 0.53 | 0 | 0 | 0 | 0 | | DESSL | 0.08 | 0 | 0 | 0.42 | 0 | 0 | 0 | 0 | | RDFKGL | 0.06 | 0 | 0 | 0.32 | 0 | 0 | 0 | 0 | | CRFSO1 | 0.04 | 0 | 0 | 0.2 | 0 | 0 | 0 | 0 | | CRFNO1 | 0.02 | 0 | 0 | 0.08 | 0 | 0 | 0 | 0 | ## Coh-Metrix Model 1c This model used Coh-Metrix scores from 7 min narrative writing samples in spring [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:--------|:-------|:--------| | -4.8423 | 0.5169 | 0.1348 | 0.6009 | -0.2375 | -0.4134 | 0.4001 | -0.0098 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-----|:-----|:------|:-----|:------|:------| | DESWC | 20.66 | 36.45 | 5.76 | 2.78 | 21.32 | 6.39 | 47.19 | 16.14 | | WRDVERB | 5.51 | 3.32 | 1.88 | 1.07 | 0.58 | 2.06 | 22.99 | 2.79 | | WRDHYPn | 4.27 | 2.98 | 2.38 | 1.13 | 3.13 | 1.75 | 14.74 | 1.28 | | DESSLd | 3.23 | 1.17 | 0.87 | 1.74 | 3.05 | 2.13 | 10.33 | 0 | | PCNARp | 2.65 | 2.58 | 2.72 | 1.77 | 10.29 | 2.17 | 0 | 4.99 | | DESPL | 2.51 | 2.16 | 3.29 | 1.6 | 2.76 | 2.2 | 4.27 | 2.56 | | DESWLltd | 1.8 | 4.36 | 1.75 | 1.09 | 1.39 | 1.51 | 0 | 6.97 | | WRDNOUN | 1.59 | 1.69 | 2.13 | 0.8 | 5.92 | 1.42 | 0 | 5.81 | | WRDFRQmc | 1.59 | 2.27 | 2.04 | 1.46 | 2.86 | 1.51 | 0 | 5.34 | | DESWLlt | 1.55 | 0.7 | 2.05 | 0.77 | 6.89 | 1.97 | 0 | 3.72 | | CRFANPa | 1.45 | 1.93 | 1.23 | 1.64 | 0.05 | 2.84 | 0 | 0 | | LSASS1d | 1.43 | 2.3 | 1.5 | 1.82 | 0.08 | 1.92 | 0 | 0 | | LDMTLD | 1.43 | 2.6 | 1.79 | 1.39 | 0 | 2.14 | 0 | 0 | | CRFCWOa | 1.4 | 2.06 | 1.9 | 1.74 | 0 | 2.09 | 0 | 0 | | WRDHYPv | 1.31 | 1.51 | 2.37 | 0.97 | 2.75 | 1.62 | 0 | 1.63 | | PCDCz | 1.31 | 2.21 | 1.05 | 1.47 | 0 | 2 | 0 | 1.97 | | WRDPRP3s | 1.28 | 1.96 | 1.36 | 0.66 | 3.27 | 1.43 | 0 | 0.81 | | SMCAUSwn | 1.26 | 1.16 | 2.26 | 1.16 | 3.45 | 1.13 | 0 | 2.9 | | SMCAUSvp | 1.25 | 2.1 | 0.97 | 1.55 | 0 | 1.8 | 0 | 0 | | SYNSTRUTa | 1.17 | 1.47 | 1.74 | 1.81 | 0 | 1.38 | 0 | 3.72 | | PCDCp | 1.16 | 0 | 1.81 | 1.28 | 3.29 | 2.05 | 0 | 4.53 | | LSAGN | 1.12 | 0.71 | 1.72 | 1.81 | 0 | 2.08 | 0 | 2.09 | | RDL2 | 1.04 | 0.86 | 2.17 | 0.93 | 2.07 | 1.45 | 0 | 1.28 | | SMCAUSlsa | 1.01 | 0.63 | 1.59 | 1.07 | 3.13 | 0.94 | 0 | 3.72 | | DRPP | 0.98 | 1.61 | 1.78 | 0.67 | 0.69 | 1.47 | 0 | 1.28 | | LSAGNd | 0.97 | 0.08 | 2.29 | 2 | 0 | 1.63 | 0 | 0 | | SYNMEDpos | 0.94 | 0.3 | 1.78 | 1.48 | 0 | 2.09 | 0 | 0.93 | | CNCTemp | 0.9 | 1.06 | 0.85 | 0.91 | 1.61 | 1.18 | 0 | 0 | | WRDADV | 0.89 | 1.56 | 1.9 | 0.67 | 0 | 1.4 | 0 | 0 | | CNCPos | 0.87 | 0.61 | 0.82 | 0.74 | 1.97 | 1.58 | 0 | 2.44 | | SMCAUSv | 0.86 | 1.16 | 0.91 | 1.27 | 0 | 1.22 | 0 | 0 | | PCTEMPp | 0.85 | 0.19 | 1.68 | 1.09 | 2.36 | 0.96 | 0 | 2.09 | | PCVERBz | 0.85 | 0.24 | 1.73 | 1.52 | 0 | 1.6 | 0 | 3.37 | | LDTTRc | 0.84 | 1.39 | 1.26 | 0.82 | 0 | 1.37 | 0 | 0 | | PCREFz | 0.78 | 0.25 | 1.2 | 0.56 | 2.5 | 1.41 | 0 | 0 | | RDFKGL | 0.77 | 0.36 | 2.06 | 0.91 | 0 | 1.86 | 0 | 0 | | LSASSp | 0.76 | 0.04 | 1.97 | 1.62 | 0 | 1.16 | 0 | 0.93 | | PCVERBp | 0.75 | 0.3 | 1.09 | 1.28 | 0 | 1.55 | 0 | 0 | | CRFCWO1d | 0.73 | 0.13 | 1.54 | 1.52 | 0 | 1.18 | 0 | 0 | | DRNP | 0.7 | 0.54 | 1.62 | 0.83 | 0 | 1.51 | 0 | 0 | | WRDPRO | 0.69 | 0.48 | 0.73 | 0.6 | 1.85 | 0.96 | 0 | 4.18 | | SMCAUSr | 0.69 | 0.9 | 0.05 | 0.72 | 0.63 | 1.33 | 0 | 0 | | WRDAOAc | 0.69 | 0.92 | 0.69 | 0.84 | 0.51 | 0.98 | 0 | 0 | | LDTTRa | 0.68 | 0.05 | 2.31 | 0.95 | 0 | 1.57 | 0 | 0 | | WRDCNCc | 0.67 | 0.53 | 1.3 | 0.38 | 3.63 | 0 | 0 | 1.28 | | PCSYNz | 0.62 | 0.24 | 1.76 | 0.77 | 0 | 1.45 | 0 | 1.28 | | CNCCaus | 0.61 | 0.72 | 0.73 | 0.73 | 0.12 | 1.1 | 0 | 0 | | WRDMEAc | 0.57 | 0.51 | 0.68 | 0.45 | 2.11 | 0.46 | 0 | 0 | | CNCTempx | 0.57 | 0.45 | 0.26 | 1.36 | 0.06 | 0.53 | 0 | 0 | | DESWLsy | 0.56 | 0.34 | 0.85 | 0.55 | 0.35 | 0.89 | 0.49 | 1.28 | | WRDPOLc | 0.56 | 0.48 | 0.89 | 0.63 | 0 | 1.31 | 0 | 0 | | SYNLE | 0.55 | 0.35 | 0.09 | 0.62 | 0 | 1.67 | 0 | 0 | | PCCNCz | 0.54 | 0.14 | 1.87 | 0.96 | 0 | 0.72 | 0 | 3.25 | | DRNEG | 0.54 | 0.05 | 0.73 | 0.81 | 1.44 | 0.74 | 0 | 0 | | WRDFRQa | 0.52 | 0.4 | 0.36 | 0.71 | 0 | 1.2 | 0 | 1.28 | | WRDFRQc | 0.52 | 0.97 | 0.3 | 0.33 | 0 | 1.09 | 0 | 1.28 | | DRVP | 0.5 | 0.11 | 0.79 | 0.77 | 0.61 | 0.88 | 0 | 0.81 | | SYNNP | 0.5 | 0.21 | 0.65 | 0.88 | 0 | 1.04 | 0 | 0 | | LSASSpd | 0.5 | 0 | 0 | 1.94 | 0 | 0 | 0 | 0 | | CNCLogic | 0.49 | 0.55 | 0.41 | 0.68 | 0 | 0.9 | 0 | 0 | | CNCADC | 0.49 | 0.38 | 1.61 | 0.46 | 0.24 | 0.93 | 0 | 0 | | SYNSTRUTt | 0.48 | 0 | 0 | 1.83 | 0 | 0 | 0 | 0 | | PCNARz | 0.46 | 0 | 0 | 1.77 | 0 | 0 | 0 | 0 | | PCCNCp | 0.46 | 0 | 1.15 | 0.87 | 0 | 0.93 | 0 | 0 | | WRDADJ | 0.45 | 0.29 | 1.12 | 0.72 | 0 | 0.76 | 0 | 0 | | CRFCWOad | 0.44 | 0 | 0 | 1.69 | 0 | 0 | 0 | 0 | | CRFAOa | 0.43 | 0 | 0 | 1.64 | 0 | 0 | 0 | 0 | | PCSYNp | 0.43 | 0 | 1.45 | 0.62 | 0 | 1.01 | 0 | 0 | | DESSC | 0.42 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 | | CRFCWO1 | 0.42 | 0 | 0 | 1.6 | 0 | 0 | 0 | 0 | | SYNMEDwrd | 0.42 | 0 | 0 | 1.61 | 0 | 0 | 0 | 0 | | CRFAO1 | 0.41 | 0 | 0 | 1.55 | 0 | 0 | 0 | 0 | | LSASS1 | 0.41 | 0 | 0 | 1.59 | 0 | 0 | 0 | 0 | | SYNMEDlem | 0.41 | 0 | 0 | 1.59 | 0 | 0 | 0 | 0 | | DRAP | 0.4 | 0.23 | 1.23 | 0.64 | 0 | 0.58 | 0 | 0 | | SMINTEp | 0.38 | 0.36 | 1.06 | 0.68 | 0 | 0.29 | 0 | 0.81 | | PCTEMPz | 0.38 | 0 | 0 | 1.45 | 0 | 0 | 0 | 0 | | WRDPRP3p | 0.38 | 0.08 | 0.44 | 0.01 | 1.83 | 0.83 | 0 | 0 | | SMTEMP | 0.37 | 0 | 0 | 1.44 | 0 | 0 | 0 | 0 | | WRDHYPnv | 0.36 | 0.3 | 0.02 | 0.78 | 0 | 0.5 | 0 | 0 | | CRFANP1 | 0.35 | 0 | 0 | 1.35 | 0 | 0 | 0 | 0 | | CRFNOa | 0.34 | 0 | 0 | 1.32 | 0 | 0 | 0 | 0 | | CRFSOa | 0.3 | 0 | 0 | 1.17 | 0 | 0 | 0 | 0 | | PCREFp | 0.28 | 0 | 0.32 | 0.32 | 0 | 0.97 | 0 | 0 | | DESWLsyd | 0.27 | 0.32 | 0.32 | 0.45 | 0 | 0.34 | 0 | 1.28 | | PCCONNp | 0.26 | 0.07 | 1.18 | 0.05 | 0 | 0.87 | 0 | 0 | | PCCONNz | 0.26 | 0.18 | 0.53 | 0.23 | 0 | 0.69 | 0 | 0 | | DESSL | 0.24 | 0 | 0 | 0.91 | 0 | 0 | 0 | 0 | | CRFNO1 | 0.22 | 0 | 0.98 | 0.06 | 0 | 0.81 | 0 | 0 | | RDFRE | 0.21 | 0 | 0 | 0.79 | 0 | 0 | 0 | 0 | | WRDFAMc | 0.21 | 0.29 | 0 | 0.07 | 1.19 | 0.02 | 0 | 0 | | SMINTEr | 0.21 | 0.07 | 0.37 | 0.32 | 0 | 0.5 | 0 | 0 | | CNCNeg | 0.15 | 0 | 0 | 0.58 | 0 | 0 | 0 | 0 | | CNCAll | 0.12 | 0 | 0 | 0.46 | 0 | 0 | 0 | 0 | | CNCAdd | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | WRDIMGc | 0.09 | 0 | 0 | 0.36 | 0 | 0 | 0 | 0 | | CRFSO1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | ## Coh-Metrix Model 1d This model used principal components scores from 7 min narrative writing samples in fall [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:-------|:------|:-------| | -20.0773 | 0.0971 | 0.7558 | 0.5784 | -0.4401 | -4e-04 | 0.002 | 0.0227 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). PC1 = scores on 1st principal component extracted, ... Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:-----|:------|:------|:------| | PC3 | 16.71 | 31.88 | 19.04 | 0 | 8.86 | 20.39 | 26.66 | 17.12 | | PC5 | 12.28 | 18.78 | 13.53 | 0 | 8.6 | 14.58 | 19.92 | 8.98 | | PC1 | 11.48 | 6.86 | 17.08 | 0 | 3.49 | 5.37 | 14.52 | 8.98 | | PC8 | 7.3 | 4.81 | 7.77 | 0 | 7.07 | 4.77 | 9.88 | 8.47 | | PC9 | 5 | 7.06 | 4.69 | 0 | 4.66 | 7.51 | 11.76 | 8.47 | | PC4 | 4.73 | 2.76 | 5.98 | 0 | 3.09 | 2.6 | 0 | 7.97 | | PC11 | 4.57 | 2.64 | 4.52 | 0 | 5.05 | 1.91 | 0 | 8.47 | | PC7 | 2.91 | 4.92 | 2.9 | 0 | 2.16 | 5.93 | 0 | 8.47 | | PC34 | 2.6 | 0.6 | 0.87 | 0 | 5.78 | 0.91 | 0 | 7.97 | | PC14 | 2.37 | 0.66 | 2.18 | 0 | 3.18 | 0.56 | 0 | 2.2 | | PC16 | 2.3 | 0.66 | 1.87 | 0 | 3.52 | 1.18 | 0 | 1.69 | | PC21 | 2.27 | 0.65 | 1.5 | 0 | 3.75 | 1.41 | 0 | 6.78 | | PC10 | 2.26 | 1.45 | 2.34 | 0 | 2.39 | 1.39 | 0 | 1.69 | | PC30 | 2.07 | 0.38 | 0.88 | 0 | 4.61 | 1.45 | 0 | 0.51 | | PC15 | 1.81 | 0 | 1.61 | 0 | 2.7 | 1.24 | 0 | 0.51 | | PC31 | 1.8 | 1.2 | 0.69 | 0 | 3.86 | 1.42 | 0 | 1.19 | | PC6 | 1.7 | 0.76 | 2.11 | 0 | 1.37 | 1.29 | 0 | 0 | | PC35 | 1.7 | 0.15 | 0.51 | 0 | 4.07 | 1.11 | 5.51 | 0 | | PC17 | 1.5 | 0.35 | 1.24 | 0 | 2.35 | 1.14 | 0 | 0 | | PC12 | 1.4 | 0.88 | 1.45 | 0 | 1.59 | 0.23 | 0 | 0 | | PC24 | 1.39 | 0.83 | 0.85 | 0 | 2.51 | 1.02 | 0 | 0 | | PC13 | 1.3 | 0.62 | 1.24 | 0 | 1.66 | 0.98 | 0 | 0 | | PC22 | 1.27 | 1.34 | 0.81 | 0 | 2.08 | 2.74 | 0 | 0 | | PC19 | 1.11 | 2.25 | 0.72 | 0 | 1.51 | 4.51 | 0 | 0.51 | | PC32 | 1.09 | 1.42 | 0.37 | 0 | 2.27 | 0.42 | 0 | 0 | | PC2 | 0.89 | 1.16 | 1.19 | 0 | 0.33 | 2.11 | 0.87 | 0 | | PC26 | 0.67 | 0.85 | 0.34 | 0 | 1.2 | 2.9 | 0 | 0 | | PC33 | 0.64 | 0.61 | 0.17 | 0 | 1.32 | 0.89 | 7.71 | 0 | | PC18 | 0.63 | 0.17 | 0.5 | 0 | 0.93 | 0.58 | 3.18 | 0 | | PC29 | 0.6 | 0.71 | 0.24 | 0 | 1.18 | 2.26 | 0 | 0 | | PC28 | 0.51 | 0.26 | 0.23 | 0 | 1.08 | 0.92 | 0 | 0 | | PC20 | 0.43 | 0.87 | 0.27 | 0 | 0.63 | 0 | 0 | 0 | | PC25 | 0.34 | 0.22 | 0.19 | 0 | 0.64 | 0.93 | 0 | 0 | | PC27 | 0.33 | 0.86 | 0.12 | 0 | 0.52 | 2.07 | 0 | 0 | | PC23 | 0.04 | 0.37 | 0 | 0 | 0 | 1.27 | 0 | 0 | ### Proportion of Variance by Varimax Rotated Component (RC) Due to space limitations, loadings for only the first ten principal components are displayed. | Variable | RC2 | RC1 | RC4 | RC3 | RC8 | RC5 | RC6 | RC7 | RC10 | RC9 | |:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----| | SS loadings | 14.67 | 14.32 | 5.99 | 5.95 | 5.14 | 4.89 | 4.76 | 4.01 | 4.00 | 3.01 | | Proportion Var | 0.15 | 0.15 | 0.06 | 0.06 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.03 | | Cumulative Var | 0.15 | 0.30 | 0.36 | 0.42 | 0.47 | 0.53 | 0.57 | 0.62 | 0.66 | 0.69 | | Proportion Explained | 0.22 | 0.21 | 0.09 | 0.09 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.05 | | Cumulative Proportion | 0.22 | 0.43 | 0.52 | 0.61 | 0.69 | 0.76 | 0.83 | 0.90 | 0.95 | 1.00 | ### Varimax Rotated Loadings | Metric | RC2 | RC1 | RC4 | RC3 | RC8 | RC5 | RC6 | RC7 | RC10 | RC9 | |:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------| | DESSC | -0.01 | 0.8 | 0.31 | -0.06 | -0.06 | 0.02 | -0.04 | 0.05 | 0.29 | 0.01 | | DESWC | 0.06 | 0.11 | 0.34 | 0.23 | -0.09 | -0.06 | -0.07 | 0.1 | 0.78 | -0.03 | | DESPL | -0.01 | 0.8 | 0.31 | -0.06 | -0.06 | 0.02 | -0.04 | 0.05 | 0.29 | 0.01 | | DESSL | -0.37 | -0.76 | -0.03 | 0.23 | 0.06 | 0.01 | -0.06 | 0.01 | 0.3 | 0.14 | | DESSLd | 0.52 | -0.1 | 0.04 | 0.32 | 0.1 | -0.04 | -0.04 | -0.01 | 0.25 | -0.45 | | DESWLsy | 0.08 | 0.09 | 0.71 | -0.01 | -0.06 | 0.24 | 0.25 | -0.07 | 0.02 | -0.02 | | DESWLsyd | 0.08 | 0.07 | 0.66 | 0.01 | 0.01 | 0.22 | 0.18 | -0.13 | 0 | 0.13 | | DESWLlt | 0.07 | 0.29 | 0.71 | 0.12 | 0.11 | 0.05 | 0.19 | 0.11 | 0.01 | -0.08 | | DESWLltd | 0.22 | 0.24 | 0.69 | 0.08 | 0.06 | 0.21 | -0.09 | -0.06 | 0.02 | 0.2 | | PCNARz | 0.74 | 0.23 | -0.02 | 0.1 | -0.22 | 0.02 | -0.51 | -0.1 | 0.01 | 0.07 | | PCNARp | 0.61 | 0.35 | 0.13 | 0.06 | -0.2 | 0.07 | -0.48 | -0.03 | 0.06 | -0.01 | | PCSYNz | -0.09 | 0.88 | 0.13 | -0.06 | -0.04 | -0.1 | 0.01 | 0.14 | -0.33 | 0.08 | | PCSYNp | -0.19 | 0.84 | 0.16 | 0.01 | -0.01 | -0.04 | -0.03 | 0.15 | -0.22 | 0.12 | | PCCNCz | -0.35 | -0.48 | -0.08 | 0.04 | 0.61 | -0.35 | 0.08 | 0.08 | -0.02 | 0.27 | | PCCNCp | -0.12 | -0.35 | -0.02 | 0.03 | 0.57 | -0.38 | 0.1 | 0.14 | -0.08 | 0.25 | | PCREFz | 0.71 | -0.34 | -0.25 | -0.08 | 0.02 | 0.04 | -0.03 | 0.14 | 0.07 | 0.48 | | PCREFp | 0.45 | -0.36 | -0.26 | -0.03 | -0.03 | 0.04 | -0.1 | 0.13 | 0.16 | 0.52 | | PCDCz | 0.12 | 0.17 | 0.12 | 0.9 | -0.09 | -0.15 | -0.02 | 0.22 | 0.04 | 0.04 | | PCDCp | 0.12 | 0.13 | 0.14 | 0.85 | -0.14 | -0.14 | 0.02 | 0.16 | -0.01 | 0.08 | | PCVERBz | -0.63 | -0.44 | -0.46 | 0.05 | -0.06 | 0.12 | 0.2 | 0.21 | 0.12 | 0.05 | | PCVERBp | -0.47 | -0.21 | -0.5 | 0.06 | -0.01 | 0.13 | 0.21 | 0.3 | 0.11 | -0.08 | | PCCONNz | 0.04 | 0.09 | 0.24 | 0.09 | -0.02 | 0.87 | -0.06 | -0.06 | 0.05 | 0.05 | | PCCONNp | -0.08 | 0.01 | 0.02 | 0.01 | 0.01 | 0.81 | 0 | -0.1 | -0.12 | 0.01 | | PCTEMPz | 0.67 | 0.64 | 0.17 | 0.06 | 0.06 | 0.01 | 0.01 | 0 | 0.01 | -0.28 | | PCTEMPp | 0.4 | 0.37 | 0.04 | 0.04 | 0.17 | -0.16 | 0.01 | 0 | 0.14 | -0.35 | | CRFNO1 | 0.61 | -0.16 | 0.11 | -0.12 | 0.06 | 0.06 | 0.49 | 0.2 | 0.17 | 0.09 | | CRFAO1 | 0.88 | 0.21 | 0.13 | 0.02 | -0.01 | 0.08 | -0.06 | 0.09 | 0 | 0.08 | | CRFSO1 | 0.64 | -0.13 | 0.12 | -0.06 | 0.04 | 0.01 | 0.51 | 0.18 | 0.19 | 0.01 | | CRFNOa | 0.64 | -0.21 | 0.11 | -0.12 | 0.1 | 0.06 | 0.48 | 0.16 | 0.13 | 0.08 | | CRFAOa | 0.91 | 0.21 | 0.07 | 0.06 | 0.06 | 0.06 | -0.09 | 0.01 | -0.07 | 0.06 | | CRFSOa | 0.65 | -0.16 | 0.12 | -0.07 | 0.09 | 0.01 | 0.53 | 0.15 | 0.13 | 0.02 | | CRFCWO1 | 0.88 | 0.18 | -0.08 | -0.03 | 0 | 0.07 | -0.06 | 0.14 | -0.01 | 0.2 | | CRFCWO1d | 0.2 | 0.62 | 0.03 | 0.16 | 0.01 | 0.06 | -0.13 | -0.03 | 0.29 | 0.02 | | CRFCWOa | 0.9 | 0.14 | -0.11 | -0.03 | 0.04 | 0.04 | -0.08 | 0.07 | -0.1 | 0.14 | | CRFCWOad | 0.2 | 0.76 | -0.03 | 0.09 | 0.03 | 0.04 | 0.04 | -0.02 | 0.24 | -0.06 | | CRFANP1 | 0.82 | 0.29 | 0.09 | 0.06 | 0 | 0.02 | -0.22 | -0.02 | -0.02 | 0 | | CRFANPa | 0.85 | 0.09 | 0.01 | 0.12 | 0.05 | -0.03 | -0.25 | -0.06 | -0.13 | 0.04 | | LSASS1 | 0.83 | 0.06 | 0.02 | -0.02 | 0.05 | -0.01 | 0.11 | -0.01 | 0.14 | -0.01 | | LSASS1d | 0.16 | 0.7 | 0.09 | 0.05 | -0.01 | 0.05 | 0.08 | -0.05 | 0.26 | 0.06 | | LSASSp | 0.85 | 0.01 | 0.03 | 0 | 0.08 | -0.01 | 0.11 | -0.04 | 0.08 | -0.04 | | LSASSpd | 0.24 | 0.76 | 0.12 | 0.05 | 0 | 0.08 | 0.13 | 0.02 | 0.28 | 0.05 | | LSAGN | 0.56 | 0.66 | 0.2 | -0.01 | -0.03 | 0 | 0.08 | 0.02 | 0.3 | -0.02 | | LSAGNd | 0.78 | 0.41 | 0.12 | 0 | 0.08 | 0.03 | 0.07 | -0.02 | 0.08 | -0.15 | | LDTTRc | -0.06 | -0.01 | 0.07 | -0.11 | 0.02 | 0.03 | -0.35 | -0.41 | -0.59 | 0.03 | | LDTTRa | -0.16 | 0.12 | 0.12 | -0.07 | -0.11 | 0.11 | -0.04 | -0.17 | -0.76 | -0.16 | | LDMTLD | -0.08 | 0.2 | 0.52 | 0.21 | -0.08 | 0.15 | -0.17 | 0.08 | 0.09 | -0.19 | | CNCAll | -0.12 | -0.12 | -0.21 | 0.51 | 0.01 | -0.77 | 0.05 | -0.03 | -0.03 | 0.07 | | CNCCaus | 0 | 0.01 | 0.08 | 0.86 | -0.02 | 0.08 | 0 | 0.06 | -0.09 | 0.01 | | CNCLogic | 0.02 | 0 | 0.05 | 0.64 | -0.3 | -0.25 | -0.04 | 0.3 | 0.01 | 0.16 | | CNCADC | 0.07 | 0.02 | 0.16 | 0.01 | -0.37 | -0.25 | -0.16 | 0.43 | 0.06 | 0.04 | | CNCTemp | 0.01 | 0.11 | 0.09 | 0.31 | -0.1 | -0.28 | -0.01 | 0.11 | 0.03 | 0.22 | | CNCTempx | 0.11 | 0.11 | -0.06 | 0.29 | 0.05 | 0.1 | 0.06 | 0 | 0.06 | 0.28 | | CNCAdd | -0.11 | -0.19 | -0.33 | -0.04 | 0.09 | -0.83 | 0.05 | -0.08 | -0.06 | -0.04 | | CNCPos | -0.13 | -0.1 | -0.22 | 0.53 | 0.12 | -0.67 | 0.09 | -0.14 | -0.06 | 0.09 | | CNCNeg | 0.03 | 0.01 | 0.13 | 0.02 | -0.35 | -0.26 | -0.19 | 0.44 | 0.03 | 0.01 | | SMCAUSv | -0.02 | 0.62 | 0.09 | 0.06 | 0.06 | -0.02 | -0.2 | 0.16 | -0.33 | 0.12 | | SMCAUSvp | -0.02 | 0.44 | 0.11 | 0.52 | 0.02 | -0.01 | -0.17 | 0.2 | -0.32 | 0.05 | | SMINTEp | 0.06 | 0.59 | 0.19 | 0.14 | 0.1 | -0.06 | -0.1 | 0.05 | -0.2 | 0.35 | | SMCAUSr | -0.02 | -0.34 | 0 | 0.72 | -0.07 | 0.07 | 0.02 | 0.05 | 0.27 | -0.14 | | SMINTEr | -0.07 | -0.31 | 0.01 | 0.72 | -0.05 | 0.1 | 0.06 | 0.01 | 0.26 | -0.13 | | SMCAUSlsa | 0.03 | 0.03 | -0.29 | -0.15 | -0.03 | 0.34 | 0.35 | 0.48 | 0.1 | -0.02 | | SMCAUSwn | 0.04 | 0.1 | -0.14 | 0.32 | 0.08 | 0.08 | 0.02 | 0.73 | 0.11 | 0.02 | | SMTEMP | 0.67 | 0.64 | 0.17 | 0.04 | 0.07 | 0.02 | 0.01 | 0.01 | 0 | -0.26 | | SYNLE | 0.01 | -0.19 | -0.02 | 0.34 | 0.03 | 0.02 | -0.06 | 0.07 | 0.34 | -0.19 | | SYNNP | -0.05 | -0.04 | 0.03 | 0 | 0.14 | -0.09 | 0.61 | 0 | 0.1 | -0.32 | | SYNMEDpos | 0.63 | 0.56 | 0.2 | 0.04 | 0.02 | 0.05 | 0.01 | -0.05 | -0.05 | -0.38 | | SYNMEDwrd | 0.61 | 0.64 | 0.24 | 0.05 | 0.04 | 0.05 | -0.01 | -0.01 | -0.03 | -0.33 | | SYNMEDlem | 0.61 | 0.63 | 0.24 | 0.05 | 0.04 | 0.03 | -0.02 | -0.01 | -0.02 | -0.34 | | SYNSTRUTa | 0.06 | 0.8 | -0.01 | -0.05 | 0.08 | 0.19 | 0.1 | -0.06 | -0.16 | 0.1 | | SYNSTRUTt | 0.08 | 0.83 | 0.03 | -0.03 | 0.05 | 0.2 | 0.07 | -0.08 | -0.2 | 0.08 | | DRNP | 0.02 | -0.01 | -0.27 | -0.05 | 0.27 | 0.43 | 0.02 | 0.07 | -0.19 | 0.07 | | DRVP | 0.15 | -0.03 | 0.14 | -0.08 | -0.09 | 0.18 | -0.63 | 0.14 | 0.02 | -0.02 | | DRAP | -0.11 | 0.06 | 0.26 | 0.12 | -0.57 | -0.19 | 0.03 | -0.01 | -0.16 | 0.23 | | DRPP | 0.13 | 0.23 | 0.27 | 0.09 | 0.2 | 0.09 | 0.04 | -0.16 | 0.27 | 0.25 | | DRNEG | -0.13 | -0.23 | 0.1 | -0.1 | -0.33 | -0.01 | -0.18 | -0.01 | 0.28 | 0.05 | | WRDNOUN | -0.15 | 0.11 | 0.07 | -0.07 | 0.29 | 0.09 | 0.7 | -0.09 | -0.09 | -0.03 | | WRDVERB | -0.05 | -0.05 | 0.03 | -0.05 | 0.13 | -0.1 | -0.54 | 0.18 | 0.12 | -0.12 | | WRDADJ | -0.01 | -0.04 | -0.13 | 0.01 | -0.11 | 0.1 | 0.33 | 0.01 | -0.16 | -0.5 | | WRDADV | -0.11 | 0.01 | 0.29 | 0.2 | -0.7 | -0.18 | -0.04 | 0.02 | -0.01 | 0.17 | | WRDPRO | 0.13 | -0.08 | -0.25 | -0.09 | -0.1 | 0.16 | -0.6 | -0.03 | -0.21 | 0.13 | | WRDPRP3s | -0.14 | -0.02 | -0.1 | 0.13 | -0.01 | 0.02 | -0.17 | 0.12 | 0.05 | -0.17 | | WRDPRP3p | -0.06 | 0.04 | 0.12 | 0.25 | 0.04 | -0.08 | 0 | -0.01 | 0.13 | -0.1 | | WRDFRQc | -0.1 | -0.09 | -0.49 | 0.07 | -0.54 | 0.25 | -0.02 | -0.07 | 0.09 | 0.11 | | WRDFRQa | -0.11 | -0.21 | -0.59 | -0.13 | -0.21 | -0.01 | 0.1 | -0.44 | 0.07 | 0.04 | | WRDFRQmc | 0.09 | 0.64 | 0.01 | 0.13 | -0.03 | 0.07 | -0.16 | 0.25 | 0 | -0.08 | | WRDAOAc | 0.1 | -0.02 | 0.39 | 0.12 | 0.03 | 0 | -0.19 | 0.27 | 0.11 | -0.04 | | WRDFAMc | 0 | 0.02 | -0.28 | 0.15 | -0.19 | 0.19 | 0.02 | -0.15 | 0.31 | 0.18 | | WRDCNCc | 0.09 | 0.1 | 0.24 | -0.08 | 0.77 | -0.05 | 0.11 | 0.15 | 0.14 | 0.15 | | WRDIMGc | 0.09 | 0.04 | 0.22 | -0.14 | 0.86 | -0.03 | 0.12 | 0.06 | 0.03 | 0.02 | | WRDMEAc | 0.06 | 0.04 | 0.22 | 0.03 | 0.73 | -0.04 | 0.03 | -0.06 | -0.09 | 0.06 | | WRDPOLc | -0.01 | 0.08 | -0.23 | 0.32 | -0.03 | -0.03 | -0.13 | 0.66 | -0.04 | -0.05 | | WRDHYPn | 0.12 | 0.15 | 0.38 | 0.13 | 0.13 | 0 | -0.1 | 0.66 | 0.13 | -0.02 | | WRDHYPv | 0.01 | -0.01 | 0.02 | 0.14 | 0.27 | -0.19 | -0.22 | 0.39 | -0.03 | 0.26 | | WRDHYPnv | 0.01 | 0.08 | 0.37 | 0.05 | 0.38 | 0.07 | 0.22 | 0.6 | 0.09 | 0.06 | | RDFRE | 0.35 | 0.74 | -0.17 | -0.23 | -0.05 | -0.08 | -0.02 | 0.02 | -0.29 | -0.13 | | RDFKGL | -0.36 | -0.76 | 0.04 | 0.23 | 0.06 | 0.03 | -0.03 | 0.01 | 0.3 | 0.14 | | RDL2 | 0.5 | 0.5 | -0.34 | -0.01 | -0.27 | 0.29 | 0.01 | 0.01 | -0.04 | 0.24 | ## Coh-Metrix Model 1e This model used principal components scores from 7 min narrative writing samples in winter [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:------|:------|:-------|:--------|:-------|:-------|:--------| | -10.9566 | 0.338 | 0.385 | 0.5608 | -0.2444 | 0.0443 | 0.0047 | -0.0256 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). PC1 = scores on 1st principal component extracted, ... Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:-----|:------|:------|:------| | PC1 | 15.14 | 27.79 | 9.8 | 0 | 4.09 | 18.64 | 24.42 | 21.1 | | PC4 | 9.3 | 10.16 | 11.41 | 0 | 4.87 | 7.73 | 13.26 | 10.85 | | PC7 | 8.4 | 8.15 | 9.97 | 0 | 6.48 | 6.21 | 19.43 | 10.26 | | PC11 | 6.67 | 7.23 | 6.54 | 0 | 5.69 | 5.83 | 11.19 | 10.26 | | PC9 | 6.08 | 3.67 | 9.2 | 0 | 6.76 | 2 | 1.16 | 1.97 | | PC10 | 5.49 | 3.61 | 7.03 | 0 | 5.42 | 3.37 | 8.76 | 9.47 | | PC28 | 4.63 | 2.51 | 3.16 | 0 | 9.22 | 3.54 | 0 | 10.26 | | PC16 | 3.93 | 2.03 | 4.15 | 0 | 5.33 | 2.65 | 0 | 10.26 | | PC12 | 3.92 | 2.11 | 4.88 | 0 | 4.59 | 1.8 | 0 | 8.88 | | PC33 | 3.09 | 2.27 | 1.71 | 0 | 7.03 | 2.57 | 0 | 1.97 | | PC17 | 2.46 | 1.81 | 2.53 | 0 | 3.42 | 2.91 | 0 | 0.59 | | PC31 | 2.32 | 2.4 | 1.32 | 0 | 4.7 | 1.59 | 0 | 0 | | PC20 | 2.3 | 4.01 | 1.31 | 0 | 1.99 | 2.63 | 0 | 0 | | PC18 | 2.29 | 1.13 | 2.55 | 0 | 3.68 | 2.3 | 16.31 | 0.59 | | PC14 | 2.21 | 2.99 | 1.75 | 0 | 1.83 | 3.19 | 0 | 0.79 | | PC22 | 2.1 | 0.93 | 2.13 | 0 | 3.83 | 2.78 | 0 | 0 | | PC25 | 2.05 | 0.96 | 2.05 | 0 | 4.55 | 0 | 0 | 1.38 | | PC19 | 2.02 | 1.4 | 2.05 | 0 | 2.95 | 2.31 | 0 | 0.99 | | PC6 | 1.88 | 2.76 | 1.76 | 0 | 0.89 | 2.87 | 0 | 0 | | PC3 | 1.79 | 0.55 | 3.72 | 0 | 0.94 | 1 | 0 | 0 | | PC2 | 1.49 | 0.77 | 2.52 | 0 | 0.49 | 2.87 | 0.24 | 0 | | PC15 | 1.38 | 0 | 2.17 | 0 | 2.43 | 0.98 | 0 | 0 | | PC8 | 1.38 | 1.43 | 1.51 | 0 | 0.9 | 2.69 | 0 | 0 | | PC13 | 1.31 | 1.74 | 1.23 | 0 | 1.17 | 1.25 | 0 | 0 | | PC26 | 1.13 | 1.03 | 0.71 | 0 | 1.66 | 2.56 | 0 | 0 | | PC32 | 1.03 | 2.12 | 0.2 | 0 | 0.73 | 2.06 | 0 | 0 | | PC29 | 0.93 | 0.43 | 0.75 | 0 | 2.25 | 0.69 | 0 | 0 | | PC34 | 0.68 | 1.1 | 0.12 | 0 | 0.53 | 2.38 | 0 | 0 | | PC21 | 0.58 | 0.44 | 0.44 | 0 | 0.59 | 1.47 | 5.24 | 0.39 | | PC5 | 0.5 | 0.16 | 0.73 | 0 | 0.16 | 1.73 | 0 | 0 | | PC27 | 0.44 | 0.88 | 0.15 | 0 | 0.22 | 0.95 | 0 | 0 | | PC23 | 0.37 | 0.46 | 0.17 | 0 | 0.16 | 1.67 | 0 | 0 | | PC24 | 0.36 | 0.15 | 0.28 | 0 | 0.43 | 1.4 | 0 | 0 | | PC30 | 0.36 | 0.82 | 0 | 0 | 0 | 1.41 | 0 | 0 | ### Proportion of Variance by Varimax Rotated Component (RC) Due to space limitations, loadings for only the first ten principal components are displayed. | Variable | RC1 | RC3 | RC2 | RC4 | RC5 | RC6 | RC7 | RC10 | RC9 | RC8 | |:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----| | SS loadings | 17.16 | 14.47 | 6.31 | 5.42 | 5.10 | 5.09 | 4.07 | 3.84 | 3.40 | 3.20 | | Proportion Var | 0.18 | 0.15 | 0.07 | 0.06 | 0.05 | 0.05 | 0.04 | 0.04 | 0.04 | 0.03 | | Cumulative Var | 0.18 | 0.33 | 0.39 | 0.45 | 0.50 | 0.55 | 0.59 | 0.63 | 0.67 | 0.70 | | Proportion Explained | 0.25 | 0.21 | 0.09 | 0.08 | 0.07 | 0.07 | 0.06 | 0.06 | 0.05 | 0.05 | | Cumulative Proportion | 0.25 | 0.46 | 0.56 | 0.64 | 0.71 | 0.79 | 0.85 | 0.90 | 0.95 | 1.00 | ### Varimax Rotated Loadings | Metric | RC1 | RC3 | RC2 | RC4 | RC5 | RC6 | RC7 | RC10 | RC9 | RC8 | |:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------| | DESSC | 0.86 | -0.02 | 0.02 | 0.05 | -0.06 | 0.01 | 0.08 | 0.07 | 0.18 | 0.11 | | DESWC | 0.18 | 0.16 | -0.09 | 0.21 | 0.05 | -0.09 | 0.19 | 0.21 | 0.63 | 0.38 | | DESPL | 0.86 | -0.02 | 0.02 | 0.05 | -0.06 | 0.01 | 0.08 | 0.07 | 0.18 | 0.11 | | DESSL | -0.81 | -0.25 | -0.04 | 0.15 | 0 | -0.07 | 0.1 | 0.12 | 0.34 | 0.01 | | DESSLd | -0.04 | 0.46 | -0.08 | 0.14 | 0.1 | -0.15 | 0.07 | 0.17 | 0.01 | 0.51 | | DESWLsy | 0.14 | 0.21 | 0.02 | -0.1 | -0.16 | 0.84 | -0.05 | -0.06 | -0.06 | -0.05 | | DESWLsyd | 0.13 | 0.17 | -0.07 | -0.04 | -0.14 | 0.79 | -0.08 | 0.03 | -0.14 | -0.04 | | DESWLlt | 0.22 | 0.12 | 0.21 | -0.04 | -0.01 | 0.83 | -0.1 | 0.02 | 0.02 | -0.05 | | DESWLltd | 0.18 | 0.13 | -0.05 | -0.07 | -0.17 | 0.67 | -0.11 | 0.11 | -0.13 | -0.09 | | PCNARz | 0.33 | 0.64 | -0.37 | 0.18 | 0 | -0.25 | 0.34 | 0.28 | -0.03 | 0.05 | | PCNARp | 0.42 | 0.53 | -0.34 | 0.22 | -0.04 | -0.17 | 0.23 | 0.28 | 0 | 0.04 | | PCSYNz | 0.9 | -0.07 | 0.06 | -0.01 | 0.11 | 0.14 | -0.05 | -0.09 | -0.14 | -0.18 | | PCSYNp | 0.85 | -0.16 | 0.07 | 0.14 | 0.08 | 0.17 | -0.04 | -0.07 | 0.01 | -0.15 | | PCCNCz | -0.56 | -0.3 | 0.65 | -0.14 | 0.17 | -0.11 | -0.1 | 0.03 | -0.04 | -0.19 | | PCCNCp | -0.42 | -0.11 | 0.69 | 0.05 | 0.09 | -0.03 | -0.06 | 0.13 | 0.09 | -0.16 | | PCREFz | -0.36 | 0.73 | -0.06 | -0.07 | -0.02 | -0.19 | -0.04 | 0.08 | 0.16 | -0.46 | | PCREFp | -0.42 | 0.45 | -0.14 | -0.17 | -0.08 | -0.26 | 0 | 0.07 | 0.18 | -0.46 | | PCDCz | 0.14 | 0.24 | -0.15 | 0.86 | 0.18 | 0.01 | 0.13 | 0.04 | 0.15 | 0.06 | | PCDCp | 0.19 | 0.2 | -0.14 | 0.82 | 0.21 | 0.02 | 0.09 | -0.01 | 0.09 | 0.03 | | PCVERBz | -0.56 | -0.65 | -0.12 | 0.01 | -0.04 | -0.12 | -0.39 | 0.14 | 0.05 | 0 | | PCVERBp | -0.33 | -0.56 | -0.15 | 0.09 | 0.05 | -0.14 | -0.51 | 0.17 | -0.07 | 0.13 | | PCCONNz | 0.05 | -0.04 | -0.1 | 0.13 | -0.93 | 0.06 | -0.11 | -0.02 | 0.07 | -0.03 | | PCCONNp | 0.04 | -0.03 | -0.02 | -0.08 | -0.75 | 0 | -0.13 | -0.09 | -0.13 | -0.04 | | PCTEMPz | 0.68 | 0.62 | -0.03 | 0.12 | 0.05 | 0.15 | 0 | 0.11 | -0.07 | 0.25 | | PCTEMPp | 0.42 | 0.46 | 0.13 | 0.08 | 0.04 | 0.03 | 0.02 | 0.21 | -0.11 | 0.17 | | CRFNO1 | -0.11 | 0.73 | 0.26 | 0.11 | -0.01 | 0.12 | -0.16 | -0.08 | 0.1 | 0.08 | | CRFAO1 | 0.33 | 0.86 | -0.07 | 0.05 | 0.01 | 0.1 | 0.01 | 0.09 | -0.03 | -0.04 | | CRFSO1 | -0.1 | 0.77 | 0.27 | 0.09 | -0.03 | 0.16 | -0.1 | -0.12 | 0.12 | 0.14 | | CRFNOa | -0.16 | 0.77 | 0.25 | 0.07 | 0.05 | 0.07 | -0.12 | -0.07 | 0.13 | 0.12 | | CRFAOa | 0.31 | 0.88 | -0.08 | 0.03 | 0.01 | 0.1 | 0.03 | 0.1 | -0.06 | -0.01 | | CRFSOa | -0.14 | 0.78 | 0.25 | 0.08 | 0.02 | 0.13 | -0.1 | -0.11 | 0.11 | 0.14 | | CRFCWO1 | 0.24 | 0.86 | -0.11 | 0.03 | 0.05 | 0.05 | -0.05 | 0.08 | 0 | -0.19 | | CRFCWO1d | 0.8 | 0.14 | -0.04 | 0.01 | -0.06 | 0.1 | 0.01 | 0.07 | 0.14 | 0.03 | | CRFCWOa | 0.21 | 0.89 | -0.14 | 0.01 | 0.06 | 0.01 | -0.02 | 0.09 | -0.01 | -0.16 | | CRFCWOad | 0.82 | 0.2 | -0.03 | -0.02 | -0.07 | 0.02 | 0.05 | 0.09 | 0.17 | 0.13 | | CRFANP1 | 0.47 | 0.66 | -0.19 | 0.05 | 0.05 | 0.06 | 0.02 | 0.18 | -0.04 | -0.07 | | CRFANPa | 0.28 | 0.71 | -0.21 | 0.03 | 0.04 | 0.11 | 0.03 | 0.18 | -0.16 | -0.07 | | LSASS1 | 0.22 | 0.83 | 0.09 | 0 | -0.01 | 0.04 | 0.03 | 0.04 | 0.17 | 0.03 | | LSASS1d | 0.71 | 0.21 | 0.01 | -0.04 | -0.09 | 0.03 | 0.04 | 0.08 | 0.26 | 0.14 | | LSASSp | 0.17 | 0.87 | 0.06 | -0.01 | 0.04 | 0 | 0.06 | 0 | 0.17 | 0.02 | | LSASSpd | 0.72 | 0.28 | 0.04 | -0.08 | -0.09 | 0.07 | 0.06 | 0.05 | 0.32 | 0.1 | | LSAGN | 0.75 | 0.52 | 0.02 | -0.01 | -0.02 | 0.04 | 0.1 | 0.06 | 0.28 | 0.06 | | LSAGNd | 0.48 | 0.78 | 0.02 | 0.03 | 0.03 | 0.05 | 0.06 | 0.05 | 0.16 | 0.14 | | LDTTRc | -0.07 | -0.35 | 0.08 | -0.11 | -0.11 | 0.09 | 0.1 | 0.03 | -0.74 | 0.19 | | LDTTRa | 0.05 | -0.3 | 0.3 | -0.02 | -0.14 | 0.4 | 0.06 | -0.1 | -0.66 | 0.04 | | LDMTLD | 0.24 | -0.14 | 0.17 | 0.28 | -0.11 | 0.51 | 0.27 | 0.03 | 0.02 | 0.41 | | CNCAll | -0.12 | 0.07 | -0.06 | 0.32 | 0.86 | -0.16 | 0.08 | -0.15 | 0.02 | -0.07 | | CNCCaus | 0.03 | 0.08 | 0.03 | 0.84 | -0.1 | -0.02 | -0.14 | 0.04 | -0.06 | 0.06 | | CNCLogic | -0.07 | -0.09 | -0.31 | 0.62 | 0.35 | -0.01 | 0.33 | -0.07 | 0.19 | -0.13 | | CNCADC | -0.04 | -0.12 | -0.05 | 0.1 | 0.36 | 0.13 | 0.52 | 0.33 | -0.02 | 0.06 | | CNCTemp | 0.14 | 0.09 | -0.31 | 0.08 | 0.34 | -0.01 | 0.32 | -0.21 | 0.2 | -0.23 | | CNCTempx | 0.15 | -0.11 | 0.03 | 0.39 | 0.07 | 0.02 | 0.05 | -0.02 | -0.22 | -0.1 | | CNCAdd | -0.14 | 0.01 | 0.04 | -0.19 | 0.89 | -0.14 | -0.04 | -0.08 | -0.04 | -0.01 | | CNCPos | -0.08 | 0.11 | -0.05 | 0.33 | 0.73 | -0.2 | -0.16 | -0.26 | 0 | -0.1 | | CNCNeg | -0.07 | -0.13 | -0.11 | 0.03 | 0.37 | 0.11 | 0.55 | 0.31 | 0.02 | 0.01 | | SMCAUSv | 0.64 | 0.04 | 0.07 | 0.05 | -0.09 | 0.14 | -0.05 | 0.06 | -0.15 | -0.33 | | SMCAUSvp | 0.49 | 0.12 | 0.11 | 0.55 | -0.14 | 0.06 | -0.03 | 0.09 | -0.06 | -0.17 | | SMINTEp | 0.66 | 0.08 | 0.13 | 0.05 | -0.18 | 0.11 | 0.09 | 0.03 | 0.12 | -0.27 | | SMCAUSr | -0.36 | 0.16 | -0.05 | 0.58 | -0.13 | -0.13 | 0.05 | 0.08 | 0.18 | 0.3 | | SMINTEr | -0.43 | 0.12 | -0.08 | 0.68 | -0.06 | -0.07 | -0.06 | 0.04 | 0.12 | 0.25 | | SMCAUSlsa | -0.01 | 0.06 | -0.17 | 0.13 | 0.12 | 0.04 | -0.54 | 0.07 | 0.29 | -0.01 | | SMCAUSwn | 0.28 | 0.09 | -0.01 | 0.23 | 0.02 | 0.22 | -0.35 | 0.54 | 0.22 | 0.22 | | SMTEMP | 0.69 | 0.62 | -0.03 | 0.11 | 0.04 | 0.15 | -0.01 | 0.11 | -0.07 | 0.24 | | SYNLE | -0.13 | -0.11 | -0.1 | -0.14 | 0.05 | 0.15 | 0.11 | 0.03 | 0.18 | -0.06 | | SYNNP | 0 | -0.05 | 0.29 | -0.17 | 0 | 0.38 | -0.21 | -0.5 | 0.09 | 0.35 | | SYNMEDpos | 0.66 | 0.51 | -0.06 | 0.11 | 0.04 | 0.13 | 0.05 | 0.1 | -0.11 | 0.4 | | SYNMEDwrd | 0.71 | 0.53 | -0.04 | 0.12 | 0.03 | 0.17 | 0.01 | 0.09 | -0.07 | 0.35 | | SYNMEDlem | 0.7 | 0.54 | -0.04 | 0.12 | 0.03 | 0.16 | 0.02 | 0.09 | -0.08 | 0.35 | | SYNSTRUTa | 0.81 | 0.14 | -0.08 | 0.08 | -0.12 | 0.17 | -0.06 | 0.05 | -0.05 | -0.13 | | SYNSTRUTt | 0.85 | 0.15 | -0.05 | 0.06 | -0.12 | 0.16 | -0.05 | 0.02 | -0.04 | -0.15 | | DRNP | -0.04 | -0.1 | 0.15 | 0.02 | -0.43 | -0.03 | -0.26 | -0.02 | -0.12 | -0.28 | | DRVP | -0.05 | 0.07 | -0.08 | -0.14 | -0.05 | -0.14 | 0 | 0.65 | 0 | -0.02 | | DRAP | 0.09 | 0 | -0.21 | 0.22 | 0.26 | -0.09 | 0.42 | -0.01 | 0.06 | -0.36 | | DRPP | 0.11 | -0.02 | 0.05 | 0.09 | -0.01 | 0.06 | -0.45 | -0.03 | 0 | 0.05 | | DRNEG | 0.03 | 0.03 | -0.14 | 0.08 | 0.1 | -0.15 | 0.53 | -0.11 | 0.03 | 0.09 | | WRDNOUN | 0.04 | -0.06 | 0.49 | -0.12 | -0.16 | 0.28 | -0.41 | -0.4 | 0.02 | 0.06 | | WRDVERB | 0.17 | 0.17 | 0.05 | -0.13 | -0.06 | 0 | 0.17 | 0.65 | -0.12 | 0.03 | | WRDADJ | 0.05 | 0.01 | -0.09 | 0.04 | -0.06 | 0.37 | 0.01 | -0.26 | -0.03 | 0.29 | | WRDADV | 0.04 | 0.04 | -0.24 | 0.38 | 0.29 | 0 | 0.47 | -0.08 | 0.22 | -0.19 | | WRDPRO | -0.11 | 0.01 | -0.27 | -0.03 | -0.23 | -0.54 | 0.37 | 0.2 | -0.13 | -0.19 | | WRDPRP3s | 0.05 | 0.01 | -0.09 | 0.14 | -0.03 | -0.03 | 0.45 | -0.03 | 0.1 | 0.22 | | WRDPRP3p | -0.23 | -0.17 | -0.12 | 0.23 | 0.04 | 0.13 | -0.02 | 0.04 | 0.19 | 0.04 | | WRDFRQc | -0.12 | -0.25 | -0.58 | 0.26 | -0.15 | -0.24 | -0.05 | 0.31 | 0.05 | 0.06 | | WRDFRQa | -0.23 | -0.19 | -0.49 | -0.11 | 0.06 | -0.47 | -0.23 | -0.06 | -0.26 | 0.07 | | WRDFRQmc | 0.66 | -0.02 | -0.21 | 0.05 | 0.11 | 0.1 | -0.08 | 0.15 | -0.07 | 0.18 | | WRDAOAc | 0.12 | 0.14 | 0.17 | 0.16 | 0.04 | 0.32 | 0.03 | -0.09 | 0.07 | 0.13 | | WRDFAMc | -0.09 | -0.12 | -0.47 | 0.22 | -0.18 | -0.23 | -0.08 | 0.22 | 0.12 | -0.06 | | WRDCNCc | 0.03 | 0.02 | 0.83 | -0.12 | 0.04 | -0.03 | -0.06 | 0.02 | -0.09 | 0.05 | | WRDIMGc | 0.08 | 0.07 | 0.88 | -0.06 | -0.05 | 0 | -0.11 | -0.02 | -0.15 | 0.03 | | WRDMEAc | 0.06 | 0.04 | 0.71 | 0.06 | -0.19 | 0.03 | -0.13 | 0.17 | -0.2 | 0.03 | | WRDPOLc | 0.05 | 0.15 | 0.08 | 0.3 | -0.06 | 0.01 | -0.27 | 0.67 | 0.18 | 0.11 | | WRDHYPn | 0.24 | 0.16 | 0.25 | 0.27 | -0.11 | -0.07 | 0.07 | 0.3 | 0.17 | -0.18 | | WRDHYPv | 0.21 | 0 | 0.23 | 0.1 | -0.12 | 0.2 | -0.02 | 0.54 | 0.31 | -0.01 | | WRDHYPnv | 0.13 | 0.08 | 0.61 | 0.11 | -0.23 | 0.22 | -0.34 | 0.12 | 0.24 | -0.05 | | RDFRE | 0.82 | 0.19 | 0.04 | -0.11 | 0.05 | -0.19 | -0.09 | -0.1 | -0.32 | 0 | | RDFKGL | -0.81 | -0.23 | -0.04 | 0.15 | -0.02 | 0.02 | 0.1 | 0.11 | 0.34 | 0.01 | | RDL2 | 0.44 | 0.48 | -0.44 | 0.2 | -0.1 | -0.03 | -0.09 | 0.25 | 0.01 | -0.15 | ## Coh-Metrix Model 1f This model used principal component scores from 7 min narrative writing samples in spring [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * gbm = stochastic gradient boosted trees * pls = partial least squares regression * svm = support vector machines * enet = elastic net regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | gbm | pls | svm | enet | rf | mars | cube | |:----------|:-------|:-------|:-------|:--------|:--------|:------|:--------| | -16.5845 | 0.1071 | 0.5091 | 0.6984 | -0.2708 | -0.0323 | 0.036 | -0.0221 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). PC1 = scores on 1st principal component extracted, ... Note: Importance is unavailable for support vector machines when PCA-based pre-processing is used (so all values for svm are 0). | Metric | all | gbm | pls | svm | enet | rf | mars | cube | |:-------|:------|:------|:------|:----|:------|:------|:------|:-----| | PC7 | 14.02 | 14.78 | 14.62 | 0 | 12.36 | 11.93 | 19.81 | 10 | | PC1 | 13.63 | 30.49 | 12.95 | 0 | 6.54 | 26.18 | 26.11 | 20 | | PC8 | 12.82 | 12.75 | 13.42 | 0 | 11.91 | 8.44 | 15.73 | 10 | | PC6 | 10.25 | 12.69 | 10.89 | 0 | 7.93 | 8.88 | 10.09 | 16.6 | | PC3 | 5.8 | 2.48 | 8 | 0 | 3.23 | 3.41 | 0.8 | 10 | | PC9 | 4.73 | 3.43 | 5.34 | 0 | 4.67 | 2.71 | 0 | 6.6 | | PC4 | 4.32 | 2.1 | 5.78 | 0 | 2.94 | 0.55 | 0.09 | 6.6 | | PC16 | 3.92 | 1.27 | 3.57 | 0 | 6.22 | 0.83 | 0 | 3.4 | | PC20 | 3.79 | 3.76 | 2.76 | 0 | 6.1 | 2.84 | 0 | 6.6 | | PC24 | 3.35 | 2.04 | 2.26 | 0 | 6.11 | 5.13 | 0 | 3.4 | | PC21 | 2.5 | 0 | 1.85 | 0 | 3.81 | 0.86 | 7.63 | 3.4 | | PC32 | 2.2 | 0.74 | 1.2 | 0 | 5.05 | 1.77 | 0 | 0 | | PC18 | 2.06 | 0.28 | 1.83 | 0 | 3.34 | 0.48 | 0 | 3.4 | | PC15 | 2.06 | 0.86 | 2.12 | 0 | 2.84 | 1.45 | 0 | 0 | | PC14 | 1.87 | 0.22 | 2.13 | 0 | 2.42 | 0.49 | 0 | 0 | | PC19 | 1.76 | 0.83 | 1.58 | 0 | 2.82 | 1.1 | 0 | 0 | | PC10 | 1.73 | 0.88 | 2.1 | 0 | 1.55 | 3.01 | 0 | 0 | | PC31 | 1.69 | 0.48 | 1.03 | 0 | 3.81 | 0.38 | 0 | 0 | | PC12 | 1.2 | 0.41 | 1.48 | 0 | 1.15 | 1.21 | 0 | 0 | | PC30 | 1.06 | 0.58 | 0.76 | 0 | 1.97 | 1.51 | 0 | 0 | | PC22 | 1.02 | 0.47 | 0.99 | 0 | 1.56 | 0.27 | 0 | 0 | | PC17 | 0.83 | 1.24 | 0.22 | 0 | 0 | 3.14 | 13.07 | 0 | | PC26 | 0.72 | 0.25 | 0.7 | 0 | 1.14 | 0.26 | 0 | 0 | | PC5 | 0.58 | 1.09 | 0.47 | 0 | 0 | 2.15 | 4.42 | 0 | | PC25 | 0.49 | 0.83 | 0.54 | 0 | 0.47 | 0.05 | 0 | 0 | | PC13 | 0.46 | 0.3 | 0.67 | 0 | 0.07 | 2.34 | 0 | 0 | | PC2 | 0.35 | 0.99 | 0.31 | 0 | 0 | 0 | 2.25 | 0 | | PC11 | 0.23 | 1.78 | 0 | 0 | 0 | 3.42 | 0 | 0 | | PC28 | 0.21 | 0.07 | 0.31 | 0 | 0 | 1.49 | 0 | 0 | | PC29 | 0.11 | 0.55 | 0.08 | 0 | 0 | 0.75 | 0 | 0 | | PC23 | 0.09 | 0.33 | 0.01 | 0 | 0 | 2.16 | 0 | 0 | | PC33 | 0.07 | 0.57 | 0.03 | 0 | 0 | 0.32 | 0 | 0 | | PC27 | 0.05 | 0.43 | 0 | 0 | 0 | 0.5 | 0 | 0 | ### Proportion of Variance by Varimax Rotated Component (RC) Due to space limitations, loadings for only the first ten principal components are displayed. | Variable | RC1 | RC2 | RC4 | RC10 | RC5 | RC3 | RC8 | RC6 | RC7 | RC9 | |:----------------------|:------|:------|:-----|:-----|:-----|:-----|:-----|:-----|:-----|:-----| | SS loadings | 18.09 | 16.04 | 6.14 | 4.86 | 4.81 | 4.44 | 4.37 | 4.20 | 3.86 | 2.98 | | Proportion Var | 0.19 | 0.17 | 0.06 | 0.05 | 0.05 | 0.05 | 0.05 | 0.04 | 0.04 | 0.03 | | Cumulative Var | 0.19 | 0.35 | 0.42 | 0.47 | 0.51 | 0.56 | 0.61 | 0.65 | 0.69 | 0.72 | | Proportion Explained | 0.26 | 0.23 | 0.09 | 0.07 | 0.07 | 0.06 | 0.06 | 0.06 | 0.06 | 0.04 | | Cumulative Proportion | 0.26 | 0.49 | 0.58 | 0.65 | 0.72 | 0.78 | 0.84 | 0.90 | 0.96 | 1.00 | ### Varimax Rotated Loadings | Metric | RC1 | RC2 | RC4 | RC10 | RC5 | RC3 | RC8 | RC6 | RC7 | RC9 | |:----------|:------|:------|:------|:------|:------|:------|:------|:------|:------|:------| | DESSC | 0.85 | 0.02 | -0.07 | 0.2 | 0.07 | 0.05 | 0.04 | -0.04 | 0.13 | 0.11 | | DESWC | 0.17 | 0.32 | 0.15 | 0.44 | -0.02 | 0.18 | 0.04 | -0.17 | 0.38 | 0.43 | | DESPL | 0.85 | 0.02 | -0.07 | 0.2 | 0.07 | 0.05 | 0.04 | -0.04 | 0.13 | 0.11 | | DESSL | -0.75 | -0.24 | 0.1 | 0.12 | -0.09 | 0.12 | -0.11 | -0.15 | 0.25 | 0.25 | | DESSLd | 0.02 | 0.66 | 0.16 | 0.1 | -0.03 | 0.1 | 0.07 | 0.06 | -0.06 | 0.2 | | DESWLsy | 0.09 | -0.08 | 0 | -0.05 | -0.14 | -0.16 | 0.86 | 0.02 | 0.06 | -0.13 | | DESWLsyd | 0.03 | -0.07 | 0 | -0.04 | -0.07 | 0.06 | 0.82 | -0.1 | -0.04 | -0.09 | | DESWLlt | 0.14 | -0.17 | -0.05 | -0.01 | -0.12 | -0.32 | 0.72 | 0.1 | 0.17 | -0.07 | | DESWLltd | 0.12 | 0.06 | 0.09 | -0.05 | 0.15 | -0.05 | 0.74 | 0.06 | -0.13 | 0.18 | | PCNARz | 0.35 | 0.7 | 0.16 | 0.21 | 0.45 | 0.21 | -0.14 | 0.03 | -0.03 | 0.17 | | PCNARp | 0.47 | 0.57 | 0.1 | 0.27 | 0.41 | 0.2 | -0.13 | -0.02 | -0.02 | 0.16 | | PCSYNz | 0.89 | -0.07 | -0.03 | -0.17 | 0.08 | -0.17 | 0.09 | 0.06 | -0.07 | -0.23 | | PCSYNp | 0.88 | -0.13 | 0.05 | -0.15 | 0.01 | -0.14 | 0.02 | 0.04 | 0.08 | -0.11 | | PCCNCz | -0.64 | -0.38 | -0.18 | -0.4 | -0.06 | -0.31 | -0.04 | -0.22 | 0.07 | 0.23 | | PCCNCp | -0.48 | -0.13 | -0.1 | -0.45 | -0.04 | -0.36 | 0.03 | -0.14 | 0.05 | 0.22 | | PCREFz | -0.4 | 0.72 | 0.01 | -0.14 | 0.29 | 0.09 | -0.24 | -0.13 | 0.1 | -0.1 | | PCREFp | -0.49 | 0.4 | -0.04 | -0.04 | 0.4 | 0.15 | -0.24 | -0.1 | 0.05 | -0.05 | | PCDCz | 0.17 | 0.3 | 0.9 | 0.06 | 0.06 | 0.07 | 0 | 0.06 | 0.13 | 0.02 | | PCDCp | 0.18 | 0.32 | 0.79 | 0.15 | 0.09 | 0.08 | 0.05 | 0.04 | 0.09 | 0.07 | | PCVERBz | -0.62 | -0.62 | -0.01 | -0.1 | -0.18 | 0.3 | -0.09 | -0.05 | 0.2 | -0.09 | | PCVERBp | -0.41 | -0.49 | 0.06 | -0.17 | -0.22 | 0.43 | -0.1 | -0.03 | 0.16 | -0.11 | | PCCONNz | 0.24 | 0.02 | 0.09 | -0.06 | -0.06 | 0.08 | -0.01 | 0.9 | 0.11 | 0.03 | | PCCONNp | 0.13 | -0.07 | 0.03 | -0.24 | 0.12 | -0.05 | -0.12 | 0.75 | 0.11 | -0.14 | | PCTEMPz | 0.72 | 0.62 | 0.09 | 0.09 | 0.01 | 0.01 | 0.13 | 0.05 | 0.05 | 0.12 | | PCTEMPp | 0.47 | 0.54 | 0.14 | 0.17 | -0.11 | 0.02 | 0.17 | 0.07 | 0 | 0.17 | | CRFNO1 | 0.07 | 0.7 | 0.15 | 0.07 | -0.37 | -0.13 | -0.22 | -0.03 | 0.11 | -0.07 | | CRFAO1 | 0.36 | 0.85 | 0.06 | 0.07 | 0.17 | 0.02 | 0.11 | 0 | 0.06 | 0.06 | | CRFSO1 | 0.09 | 0.77 | 0.1 | -0.01 | -0.32 | -0.16 | -0.22 | -0.07 | 0.06 | -0.11 | | CRFNOa | -0.03 | 0.77 | 0.08 | 0.01 | -0.32 | -0.08 | -0.21 | -0.05 | 0.08 | -0.1 | | CRFAOa | 0.3 | 0.86 | 0.07 | 0.08 | 0.25 | 0.07 | 0.12 | 0.02 | 0.05 | 0.06 | | CRFSOa | -0.01 | 0.81 | 0.07 | -0.04 | -0.28 | -0.12 | -0.2 | -0.09 | 0.05 | -0.1 | | CRFCWO1 | 0.27 | 0.89 | 0.08 | -0.08 | 0.17 | 0.05 | -0.01 | -0.04 | 0.08 | -0.03 | | CRFCWO1d | 0.81 | 0.27 | -0.05 | 0.07 | -0.03 | 0.02 | 0.02 | -0.04 | 0.03 | 0.06 | | CRFCWOa | 0.18 | 0.9 | 0.08 | -0.07 | 0.19 | 0.07 | 0 | -0.04 | 0.07 | -0.04 | | CRFCWOad | 0.83 | 0.33 | -0.02 | 0.12 | -0.01 | 0.02 | 0.02 | 0.02 | 0.03 | 0.1 | | CRFANP1 | 0.38 | 0.8 | 0.09 | 0.08 | 0.29 | 0.05 | 0.12 | 0.03 | 0.03 | 0.08 | | CRFANPa | 0.18 | 0.84 | 0.11 | 0.04 | 0.29 | 0.11 | 0.15 | 0.05 | 0.02 | 0.04 | | LSASS1 | 0.29 | 0.81 | 0.07 | -0.08 | 0.11 | -0.13 | -0.14 | -0.03 | 0.03 | 0.01 | | LSASS1d | 0.67 | 0.34 | 0.01 | 0.13 | -0.09 | -0.13 | -0.16 | 0.02 | 0.03 | 0.1 | | LSASSp | 0.19 | 0.85 | 0.06 | -0.1 | 0.11 | -0.07 | -0.09 | -0.03 | 0.03 | -0.02 | | LSASSpd | 0.76 | 0.38 | -0.02 | 0.09 | -0.12 | -0.1 | -0.1 | 0.06 | 0.05 | 0.09 | | LSAGN | 0.73 | 0.58 | -0.01 | 0.13 | 0.09 | -0.05 | -0.01 | -0.04 | 0.12 | 0.04 | | LSAGNd | 0.59 | 0.72 | 0.03 | 0.02 | -0.05 | -0.08 | -0.07 | 0 | 0.04 | 0.02 | | LDTTRc | -0.11 | -0.48 | 0.03 | -0.19 | 0.15 | -0.09 | 0.22 | 0.27 | -0.56 | 0.16 | | LDTTRa | -0.03 | -0.46 | 0.01 | -0.15 | -0.02 | -0.31 | 0.18 | 0.5 | -0.42 | -0.11 | | LDMTLD | 0.19 | -0.13 | 0.15 | 0.25 | -0.03 | -0.16 | 0.28 | 0.37 | -0.19 | 0.32 | | CNCAll | -0.3 | 0.07 | 0.62 | -0.12 | 0.17 | -0.01 | -0.06 | -0.62 | -0.02 | -0.09 | | CNCCaus | 0.04 | 0.02 | 0.86 | -0.19 | 0 | 0.12 | 0.13 | 0.03 | -0.01 | -0.08 | | CNCLogic | -0.09 | 0.11 | 0.78 | 0.31 | 0.08 | 0.06 | -0.09 | -0.01 | -0.05 | -0.13 | | CNCADC | 0.05 | -0.11 | -0.1 | 0.71 | -0.04 | 0.06 | 0.12 | -0.24 | -0.16 | 0.05 | | CNCTemp | 0.05 | 0.24 | 0.35 | 0.16 | 0.09 | -0.15 | -0.26 | 0.14 | 0.14 | 0.03 | | CNCTempx | -0.02 | 0.11 | 0.14 | 0.09 | -0.16 | 0.33 | 0.08 | -0.01 | -0.19 | 0.25 | | CNCAdd | -0.36 | -0.05 | 0 | -0.12 | 0.11 | -0.07 | -0.09 | -0.84 | -0.11 | -0.05 | | CNCPos | -0.27 | 0.11 | 0.61 | -0.31 | 0.19 | -0.05 | -0.1 | -0.53 | 0.03 | -0.1 | | CNCNeg | 0 | -0.14 | -0.08 | 0.63 | -0.06 | 0.08 | 0.07 | -0.22 | -0.17 | 0.02 | | SMCAUSv | 0.76 | -0.13 | 0.04 | -0.2 | 0.05 | -0.1 | 0.23 | 0.11 | 0.13 | 0.06 | | SMCAUSvp | 0.58 | -0.1 | 0.55 | -0.22 | 0.01 | -0.03 | 0.22 | 0.15 | 0.13 | -0.03 | | SMINTEp | 0.62 | 0.03 | -0.13 | -0.29 | 0.33 | -0.08 | -0.06 | 0.16 | 0.08 | -0.01 | | SMCAUSr | -0.25 | 0.24 | 0.61 | 0.15 | -0.05 | 0.2 | -0.11 | 0.04 | 0.1 | 0.13 | | SMINTEr | -0.23 | 0.03 | 0.66 | 0.03 | -0.13 | 0.26 | 0.11 | -0.11 | 0.13 | 0.18 | | SMCAUSlsa | -0.16 | 0.03 | -0.08 | -0.11 | -0.21 | 0.43 | 0.17 | -0.07 | 0.38 | -0.38 | | SMCAUSwn | 0.11 | 0.06 | 0.17 | -0.08 | 0.17 | 0.12 | 0.02 | 0.09 | 0.71 | 0.06 | | SMTEMP | 0.73 | 0.62 | 0.07 | 0.08 | 0 | 0.01 | 0.13 | 0.05 | 0.05 | 0.12 | | SYNLE | -0.17 | -0.03 | -0.01 | 0.03 | 0.02 | -0.18 | 0.08 | 0.23 | 0.35 | 0.02 | | SYNNP | -0.03 | -0.11 | -0.04 | -0.14 | -0.65 | -0.04 | 0.11 | 0.03 | 0 | 0.01 | | SYNMEDpos | 0.74 | 0.57 | 0.07 | 0.11 | 0.01 | 0.02 | 0.12 | 0.04 | 0.04 | 0.12 | | SYNMEDwrd | 0.76 | 0.56 | 0.07 | 0.11 | 0 | 0 | 0.14 | 0.07 | 0.05 | 0.13 | | SYNMEDlem | 0.76 | 0.55 | 0.07 | 0.11 | -0.01 | 0 | 0.14 | 0.08 | 0.05 | 0.14 | | SYNSTRUTa | 0.77 | 0.18 | -0.04 | -0.11 | 0.04 | 0.05 | 0.05 | 0.15 | 0.07 | -0.01 | | SYNSTRUTt | 0.82 | 0.16 | -0.08 | -0.11 | 0.04 | 0.04 | 0.01 | 0.18 | 0.08 | 0.01 | | DRNP | -0.17 | 0.01 | -0.27 | -0.2 | -0.15 | 0.04 | -0.25 | 0.33 | -0.36 | 0.1 | | DRVP | 0.16 | 0.04 | 0.09 | -0.05 | 0.64 | -0.04 | 0.03 | -0.04 | 0.22 | -0.22 | | DRAP | -0.1 | 0.09 | 0.25 | 0.55 | 0.08 | -0.01 | -0.11 | -0.01 | 0.25 | -0.07 | | DRPP | 0.04 | -0.06 | -0.09 | -0.13 | -0.13 | 0 | 0.02 | 0.18 | 0 | 0.66 | | DRNEG | 0.03 | 0.03 | -0.11 | 0.5 | 0.05 | 0.06 | -0.08 | 0.13 | -0.12 | -0.06 | | WRDNOUN | 0.05 | -0.06 | -0.13 | -0.26 | -0.71 | -0.31 | 0.03 | 0.24 | 0.05 | 0.02 | | WRDVERB | 0.23 | 0.1 | 0.08 | -0.06 | 0.61 | -0.04 | 0.07 | 0.07 | 0.21 | 0.07 | | WRDADJ | 0.09 | -0.11 | -0.09 | 0.09 | -0.24 | 0.1 | 0.31 | 0.1 | -0.1 | -0.56 | | WRDADV | -0.09 | 0.14 | 0.33 | 0.71 | 0.06 | 0.06 | -0.1 | -0.05 | 0.19 | -0.08 | | WRDPRO | -0.04 | 0.14 | -0.08 | 0 | 0.61 | 0.27 | -0.34 | 0.06 | -0.27 | 0.02 | | WRDPRP3s | 0.03 | 0.12 | 0 | 0.21 | 0.06 | -0.03 | -0.04 | 0.08 | 0.01 | 0.09 | | WRDPRP3p | -0.06 | -0.18 | -0.05 | -0.03 | 0 | -0.17 | 0.07 | 0.18 | 0.04 | -0.3 | | WRDFRQc | -0.08 | -0.06 | 0.23 | 0.22 | 0.16 | 0.75 | -0.34 | 0.07 | 0.04 | -0.14 | | WRDFRQa | -0.26 | -0.09 | 0.17 | 0.07 | 0.13 | 0.58 | -0.43 | -0.17 | -0.25 | 0.22 | | WRDFRQmc | 0.79 | 0.09 | 0.01 | 0.04 | 0.12 | 0.07 | 0.05 | 0 | 0.09 | 0.08 | | WRDAOAc | 0.22 | -0.06 | 0.11 | 0.24 | -0.11 | -0.2 | 0.07 | 0.1 | 0.07 | -0.14 | | WRDFAMc | -0.02 | -0.03 | 0.12 | 0.02 | 0.12 | 0.67 | -0.08 | 0.09 | 0.19 | 0.09 | | WRDCNCc | 0.04 | 0.03 | -0.31 | -0.41 | -0.07 | -0.5 | 0.03 | 0.02 | 0.1 | 0.51 | | WRDIMGc | 0.01 | -0.02 | -0.38 | -0.48 | -0.16 | -0.48 | 0.12 | -0.01 | 0.05 | 0.38 | | WRDMEAc | 0.11 | -0.11 | -0.36 | -0.47 | -0.13 | -0.18 | 0.25 | -0.1 | 0.09 | 0.37 | | WRDPOLc | 0.15 | 0.08 | 0.25 | -0.21 | 0.13 | 0.2 | -0.04 | 0.15 | 0.67 | -0.04 | | WRDHYPn | 0.11 | 0.25 | 0.03 | 0.04 | 0.22 | -0.32 | -0.03 | 0.07 | 0.48 | 0.07 | | WRDHYPv | 0.14 | 0.04 | -0.01 | -0.07 | 0.57 | -0.12 | 0.01 | -0.03 | 0.46 | 0.06 | | WRDHYPnv | 0.04 | 0.16 | -0.1 | -0.23 | -0.23 | -0.52 | -0.02 | 0.14 | 0.49 | 0.03 | | RDFRE | 0.79 | 0.26 | -0.08 | -0.09 | 0.14 | -0.09 | -0.13 | 0.11 | -0.23 | -0.18 | | RDFKGL | -0.75 | -0.25 | 0.1 | 0.11 | -0.11 | 0.11 | -0.03 | -0.15 | 0.26 | 0.24 | | RDL2 | 0.48 | 0.59 | 0.15 | 0.01 | 0.2 | 0.44 | -0.16 | 0.08 | 0.1 | -0.1 | --- # Coh-Metrix Model 2 {#cohmetrix-model-2} ## General Description Coh-Metrix Model 2 is a simplified version of [Model 1](#coh-metrix-model-1). Model 2 is recommended for use over Model 1. Coh-Metrix Model 2 is an ensemble (formed by averaging predicted quality scores) of the three sub-models described below. Highly correlated Coh-Metrix metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). All of these models used Coh-Metrix scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019] to predict holistic writing quality on the samples (elo ratings calculated from paired comparisons). More details on the sample are available in [@Mercer2019]. ## Coh-Metrix Model 2a This model was trained on fall data in [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * pls = partial least squares regression * rf = random forest regression * mars = bagged multivariate adaptive regression splines * gbm = stochastic gradient boosted trees * svm = support vector machines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | rf | mars | gbm | svm | cube | |:----------|:-------|:-------|:-------|:-------|:------|:-------| | -11.0081 | 0.1741 | 0.0413 | 0.1875 | 0.2353 | 0.206 | 0.2108 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | overall | pls | rf | mars | gbm | svm | cube | |:----------|:--------|:-----|:------|:------|:------|:-----|:------| | DESWC | 29.91 | 5.86 | 14.51 | 55.78 | 44.87 | 5.96 | 36.49 | | DESWLlt | 8.67 | 2.98 | 2.69 | 18.79 | 3.07 | 2.39 | 17.89 | | LDMTLD | 7.35 | 4.01 | 5.57 | 0 | 9.55 | 3.9 | 17.89 | | WRDHYPn | 7.16 | 2.9 | 2.68 | 9.55 | 3.85 | 2.29 | 17.89 | | LDTTRa | 2.84 | 2.01 | 0.97 | 11.14 | 0.26 | 1.12 | 1.05 | | CNCPos | 1.36 | 0.88 | 1.11 | 4.75 | 0.05 | 1.28 | 0.35 | | DESWLsy | 1.3 | 2.09 | 1.9 | 0 | 1.66 | 1.54 | 1.05 | | CNCTempx | 1.11 | 0.89 | 1.03 | 0 | 1.63 | 2.48 | 0.35 | | CNCLogic | 1.09 | 1.42 | 1.39 | 0 | 1.81 | 1.7 | 0.35 | | PCDCp | 1.08 | 2 | 2.56 | 0 | 1.71 | 1.36 | 0 | | DESPL | 1.04 | 3.22 | 2.18 | 0 | 0.04 | 2.15 | 0 | | DESWLltd | 1.02 | 2.3 | 1.13 | 0 | 1.26 | 1.6 | 0 | | WRDFRQa | 0.96 | 1.86 | 0.58 | 0 | 0.17 | 1.97 | 1.05 | | DESWLsyd | 0.92 | 1.97 | 1.87 | 0 | 1.21 | 1.29 | 0 | | DESSLd | 0.9 | 1.67 | 2 | 0 | 1.27 | 1.36 | 0 | | CNCTemp | 0.89 | 0.93 | 1.39 | 0 | 1.08 | 1.89 | 0.35 | | LSAGN | 0.87 | 2.55 | 1.18 | 0 | 0.22 | 1.8 | 0 | | LSASSpd | 0.86 | 1.95 | 0.86 | 0 | 0.36 | 1.79 | 0.35 | | CNCADC | 0.85 | 1.12 | 1.41 | 0 | 0.67 | 2.37 | 0 | | DRPP | 0.84 | 2.18 | 2.11 | 0 | 0.57 | 1.4 | 0 | | PCCONNz | 0.82 | 1.09 | 0.4 | 0 | 1.61 | 1.36 | 0 | | WRDPRO | 0.82 | 1.77 | 1.44 | 0 | 0.73 | 1.61 | 0 | | SYNSTRUTa | 0.8 | 0.85 | 3.07 | 0 | 0.58 | 1.4 | 0.7 | | CRFCWO1d | 0.79 | 1.67 | 1.39 | 0 | 0.43 | 1.88 | 0 | | LSASS1d | 0.78 | 1.55 | 0.48 | 0 | 0.78 | 1.72 | 0 | | SMCAUSwn | 0.76 | 1.4 | 1.04 | 0 | 0.31 | 2.16 | 0 | | SYNMEDpos | 0.75 | 1.75 | 0.6 | 0 | 0.77 | 1.36 | 0 | | SMINTEp | 0.73 | 0.75 | 0.85 | 0 | 0.79 | 1.66 | 0.35 | | LDTTRc | 0.73 | 1.82 | 0.7 | 0 | 0.72 | 1.23 | 0 | | CRFCWOad | 0.73 | 1.51 | 1.12 | 0 | 0.38 | 1.81 | 0 | | WRDVERB | 0.71 | 1.04 | 0.93 | 0 | 0.58 | 0.86 | 1.05 | | WRDFAMc | 0.7 | 1.09 | 0.5 | 0 | 1.32 | 1.08 | 0 | | WRDHYPnv | 0.69 | 1.9 | 0.91 | 0 | 0.13 | 1.22 | 0.35 | | WRDFRQmc | 0.69 | 1.35 | 2.5 | 0 | 1.3 | 0.4 | 0 | | WRDCNCc | 0.69 | 1.07 | 0.44 | 0 | 0.58 | 0.8 | 1.05 | | PCNARz | 0.68 | 1.29 | 1.29 | 0 | 0.58 | 1.1 | 0.35 | | WRDPOLc | 0.67 | 0.85 | 0.77 | 0 | 0.51 | 1.97 | 0 | | RDFRE | 0.66 | 1.29 | 0.91 | 0 | 0.76 | 1.23 | 0 | | CRFNOa | 0.63 | 0.74 | 1.34 | 0 | 0.56 | 1.67 | 0 | | PCVERBz | 0.63 | 1.55 | 0.67 | 0 | 0.28 | 1.09 | 0.35 | | LSAGNd | 0.61 | 1.5 | 0.95 | 0 | 0.24 | 1.38 | 0 | | SMCAUSvp | 0.59 | 0.93 | 0.73 | 0 | 0.36 | 1.67 | 0 | | WRDADV | 0.58 | 1.3 | 0.95 | 0 | 0.13 | 1.55 | 0 | | WRDAOAc | 0.58 | 1.66 | 0.46 | 0 | 0.27 | 1.18 | 0 | | DRNP | 0.57 | 1.73 | 1.5 | 0 | 0.27 | 0.82 | 0 | | WRDHYPv | 0.56 | 0.28 | 1.01 | 0 | 0.83 | 1.47 | 0 | | DRVP | 0.56 | 0.82 | 0.62 | 0 | 0.82 | 0.75 | 0.35 | | PCNARp | 0.54 | 1.76 | 1.29 | 0 | 0 | 1 | 0 | | CNCCaus | 0.51 | 1.11 | 0.58 | 0 | 0.11 | 1.46 | 0 | | CRFCWOa | 0.49 | 0.46 | 0.56 | 0 | 0.37 | 1.57 | 0 | | SMCAUSr | 0.47 | 1.58 | 1.07 | 0 | 0.47 | 0.29 | 0 | | SMCAUSlsa | 0.46 | 0.24 | 0.86 | 0 | 0.63 | 1.26 | 0 | | LSASSp | 0.46 | 0.79 | 1.01 | 0 | 0.15 | 1.31 | 0 | | SMINTEr | 0.45 | 1.83 | 0.9 | 0 | 0.13 | 0.45 | 0 | | DRAP | 0.44 | 0.46 | 1.01 | 0 | 0.41 | 1.17 | 0 | | DRNEG | 0.44 | 0.76 | 0.43 | 0 | 0.09 | 1.41 | 0 | | WRDNOUN | 0.43 | 0.56 | 0.96 | 0 | 0.75 | 0.66 | 0 | | WRDFRQc | 0.43 | 0.54 | 0.67 | 0 | 0.48 | 1.04 | 0 | | WRDPRP3s | 0.43 | 0.76 | 0.84 | 0 | 0.74 | 0.56 | 0 | | CRFANPa | 0.41 | 0.76 | 1.36 | 0 | 0.18 | 0.96 | 0 | | SYNLE | 0.4 | 1.13 | 0.95 | 0 | 0.1 | 0.77 | 0 | | PCTEMPp | 0.4 | 1.05 | 0.22 | 0 | 0.55 | 0.47 | 0 | | WRDADJ | 0.39 | 0.99 | 0.94 | 0 | 0.2 | 0.74 | 0 | | RDL2 | 0.39 | 0.31 | 1.06 | 0 | 0.39 | 1.09 | 0 | | WRDIMGc | 0.37 | 0.14 | 0.74 | 0 | 0.3 | 0.93 | 0.35 | | PCVERBp | 0.36 | 0.99 | 1.39 | 0 | 0 | 0.75 | 0 | | PCCNCz | 0.35 | 1.18 | 0.1 | 0 | 0.32 | 0.39 | 0 | | WRDMEAc | 0.29 | 0.21 | 0.45 | 0 | 0.44 | 0.71 | 0 | | PCREFz | 0.29 | 0.47 | 1.07 | 0 | 0.3 | 0.53 | 0 | | SMCAUSv | 0.25 | 0.39 | 0.77 | 0 | 0.13 | 0.66 | 0 | | SYNNP | 0.22 | 0.03 | 0.64 | 0 | 0.58 | 0.33 | 0 | | PCSYNp | 0.21 | 0.68 | 1.55 | 0 | 0.15 | 0 | 0 | | PCCONNp | 0.17 | 0 | 0.33 | 0 | 0.11 | 0.66 | 0 | | PCCNCp | 0.16 | 0.57 | 0.78 | 0 | 0 | 0.16 | 0 | | PCREFp | 0.15 | 0.06 | 0.74 | 0 | 0 | 0.58 | 0 | | WRDPRP3p | 0.14 | 0.84 | 0 | 0 | 0 | 0.03 | 0 | ## Coh-Metrix Model 2b This model was trained on winter data [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * mars = bagged multivariate adaptive regression splines * gbm = stochastic gradient boosted trees * svm = support vector machines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | mars | gbm | svm | cube | |:----------|:-------|:-------|:-------|:-------| | -7.2585 | 0.2289 | 0.5300 | 0.1527 | 0.1150 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | overall | mars | gbm | svm | cube | |:----------|:--------|:------|:-----|:-----|:------| | DESWC | 30.39 | 45.46 | 34.5 | 4.37 | 16.04 | | LSAGN | 7.18 | 0 | 9.31 | 2.85 | 17.43 | | DESWLlt | 6.73 | 19 | 2.42 | 1.37 | 9.31 | | LDMTLD | 5.59 | 0 | 8.91 | 2.47 | 5.54 | | SYNLE | 5.43 | 13.58 | 3.65 | 1.79 | 2.18 | | WRDIMGc | 3.6 | 9.36 | 1.84 | 1.22 | 3.37 | | WRDNOUN | 3.19 | 6.64 | 1.38 | 1.77 | 6.53 | | CNCAdd | 1.87 | 5.96 | 0.45 | 1.02 | 1.39 | | WRDVERB | 1.42 | 0 | 2.02 | 1.67 | 1.19 | | SMCAUSwn | 1.25 | 0 | 1.83 | 2.04 | 0 | | DESWLltd | 1.16 | 0 | 1.21 | 1.53 | 2.77 | | CRFCWO1d | 1.09 | 0 | 1.18 | 2.17 | 1.39 | | WRDHYPnv | 1.02 | 0 | 0.98 | 1.36 | 2.77 | | CRFNOa | 0.99 | 0 | 1.37 | 1.9 | 0 | | RDFRE | 0.99 | 0 | 1.54 | 0.25 | 1.39 | | LSAGNd | 0.85 | 0 | 0.53 | 1.76 | 2.77 | | DESWLsy | 0.77 | 0 | 1.11 | 1.28 | 0 | | SYNMEDpos | 0.76 | 0 | 0.29 | 2 | 2.77 | | WRDPRP3s | 0.74 | 0 | 0.91 | 1.53 | 0.4 | | DESPL | 0.73 | 0 | 0.44 | 2.34 | 1.39 | | CNCAll | 0.72 | 0 | 0.44 | 0.96 | 3.17 | | RDL2 | 0.71 | 0 | 0.6 | 1.65 | 1.39 | | PCCNCz | 0.71 | 0 | 0.62 | 1.6 | 1.39 | | PCVERBz | 0.69 | 0 | 0.82 | 1.78 | 0 | | WRDPRO | 0.69 | 0 | 0.69 | 1.18 | 1.39 | | CNCTemp | 0.65 | 0 | 0.71 | 1.87 | 0 | | WRDFRQc | 0.65 | 0 | 0.98 | 0.99 | 0 | | WRDFRQmc | 0.63 | 0 | 0.56 | 1.25 | 1.39 | | DRVP | 0.62 | 0 | 0.93 | 0.92 | 0 | | LSASS1d | 0.61 | 0 | 0.65 | 1.87 | 0 | | SMCAUSlsa | 0.6 | 0 | 0.94 | 0.77 | 0 | | CRFCWOad | 0.6 | 0 | 0.61 | 1.92 | 0 | | WRDMEAc | 0.59 | 0 | 0.53 | 1.4 | 0.99 | | PCTEMPp | 0.56 | 0 | 0.78 | 1.02 | 0 | | SMCAUSv | 0.56 | 0 | 0.85 | 0.83 | 0 | | PCCNCp | 0.56 | 0 | 0.02 | 1.62 | 2.77 | | WRDFRQa | 0.53 | 0 | 0.73 | 1.01 | 0 | | LSASSp | 0.53 | 0 | 0.54 | 1.71 | 0 | | LDTTRc | 0.52 | 0 | 0.64 | 1.3 | 0 | | DRPP | 0.51 | 0 | 0.7 | 1.02 | 0 | | PCREFp | 0.5 | 0 | 0 | 0.7 | 3.56 | | CRFCWO1 | 0.47 | 0 | 0.4 | 1.75 | 0 | | SMCAUSvp | 0.46 | 0 | 0.47 | 1.42 | 0 | | PCNARz | 0.45 | 0 | 0.26 | 1.64 | 0.59 | | SYNNP | 0.45 | 0 | 0.3 | 0.91 | 1.39 | | PCSYNz | 0.45 | 0 | 0.32 | 0.87 | 1.39 | | SMINTEp | 0.45 | 0 | 0.4 | 1.66 | 0 | | LDTTRa | 0.43 | 0 | 0.23 | 1.06 | 1.39 | | DESWLsyd | 0.42 | 0 | 0.35 | 1.62 | 0 | | CRFANPa | 0.41 | 0 | 0.34 | 1.59 | 0 | | SMINTEr | 0.41 | 0 | 0.6 | 0.69 | 0 | | CNCLogic | 0.4 | 0 | 0.51 | 0.89 | 0 | | WRDAOAc | 0.4 | 0 | 0.58 | 0.69 | 0 | | WRDHYPv | 0.4 | 0 | 0.36 | 1.45 | 0 | | CNCNeg | 0.4 | 0 | 0.32 | 1.17 | 0.59 | | CNCCaus | 0.38 | 0 | 0.47 | 0.92 | 0 | | WRDFAMc | 0.37 | 0 | 0.46 | 0.91 | 0 | | SYNSTRUTa | 0.36 | 0 | 0.36 | 1.2 | 0 | | CRFAOa | 0.35 | 0 | 0.19 | 1.68 | 0 | | WRDADV | 0.34 | 0 | 0.32 | 1.19 | 0 | | SMCAUSr | 0.33 | 0 | 0.61 | 0.1 | 0 | | DESSLd | 0.32 | 0 | 0.18 | 1.5 | 0 | | PCCONNp | 0.29 | 0 | 0.46 | 0.37 | 0 | | WRDPOLc | 0.28 | 0 | 0.28 | 0.93 | 0 | | WRDADJ | 0.28 | 0 | 0.4 | 0.51 | 0 | | DRAP | 0.26 | 0 | 0.24 | 0.87 | 0 | | DRNP | 0.26 | 0 | 0.33 | 0.57 | 0 | | WRDHYPn | 0.26 | 0 | 0.27 | 0.84 | 0 | | CNCTempx | 0.25 | 0 | 0.23 | 0.86 | 0 | | DRNEG | 0.23 | 0 | 0.13 | 1.08 | 0 | | PCREFz | 0.23 | 0 | 0.24 | 0.7 | 0 | | PCVERBp | 0.22 | 0 | 0 | 1.48 | 0 | | PCNARp | 0.2 | 0 | 0.01 | 1.35 | 0 | | PCDCp | 0.19 | 0 | 0.11 | 0.9 | 0 | | PCSYNp | 0.09 | 0 | 0.02 | 0.56 | 0 | | WRDPRP3p | 0 | 0 | 0 | 0 | 0 | ## Coh-Metrix Model 2c This model was trained on spring data [@Mercer2019]. ### Algorithm Weightings in Ensemble Abbreviations: * all = ensemble model * pls = partial least squares regression * mars = bagged multivariate adaptive regression splines * gbm = stochastic gradient boosted trees * svm = support vector machines The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | mars | gbm | svm | |:----------|:-------|:-------|:------|:-------| | -10.8192 | 0.0374 | 0.2735 | 0.243 | 0.5377 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). | Metric | overall | pls | mars | gbm | svm | |:----------|:--------|:-----|:------|:------|:-----| | DESWC | 22.58 | 5.76 | 48.45 | 37.37 | 3.91 | | WRDVERB | 7.27 | 1.88 | 23.22 | 2.92 | 1.5 | | DESWLltd | 5.72 | 1.75 | 16.17 | 3.85 | 1.53 | | CRFCWOa | 4.57 | 1.9 | 12.16 | 1.14 | 2.44 | | PCNARp | 2.04 | 2.72 | 0 | 3.23 | 2.49 | | PCDCz | 1.78 | 1.05 | 0 | 3.24 | 2.07 | | CRFANPa | 1.67 | 1.23 | 0 | 2.21 | 2.3 | | LSASS1d | 1.67 | 1.5 | 0 | 1.62 | 2.56 | | WRDHYPn | 1.59 | 2.38 | 0 | 3.29 | 1.58 | | SYNSTRUTa | 1.59 | 1.74 | 0 | 1.28 | 2.54 | | LDMTLD | 1.57 | 1.79 | 0 | 2.47 | 1.95 | | LSAGN | 1.57 | 1.72 | 0 | 1.17 | 2.55 | | SMCAUSvp | 1.53 | 0.97 | 0 | 1.91 | 2.17 | | DESSLd | 1.52 | 0.87 | 0 | 1.28 | 2.45 | | LSAGNd | 1.49 | 2.29 | 0 | 0.11 | 2.81 | | WRDFRQmc | 1.41 | 2.04 | 0 | 1.45 | 2.06 | | DESPL | 1.41 | 3.29 | 0 | 0.85 | 2.25 | | PCVERBz | 1.25 | 1.73 | 0 | 0.62 | 2.13 | | SYNMEDpos | 1.24 | 1.78 | 0 | 0.72 | 2.08 | | LSASSp | 1.22 | 1.97 | 0 | 0.13 | 2.28 | | SMCAUSv | 1.16 | 0.91 | 0 | 1.16 | 1.78 | | CRFCWO1d | 1.15 | 1.54 | 0 | 0.21 | 2.14 | | SMCAUSwn | 1.09 | 2.26 | 0 | 0.94 | 1.63 | | CNCTempx | 1.08 | 0.26 | 0 | 0.57 | 1.92 | | WRDHYPv | 1.06 | 2.37 | 0 | 1.41 | 1.36 | | SMCAUSlsa | 1.03 | 1.59 | 0 | 1.07 | 1.5 | | WRDNOUN | 1 | 2.13 | 0 | 1.68 | 1.11 | | PCDCp | 1 | 1.81 | 0 | 0.23 | 1.8 | | PCVERBp | 0.93 | 1.09 | 0 | 0.05 | 1.79 | | PCTEMPp | 0.92 | 1.68 | 0 | 0.51 | 1.52 | | LDTTRc | 0.9 | 1.26 | 0 | 1.34 | 1.14 | | WRDPRP3s | 0.87 | 1.36 | 0 | 1.67 | 0.92 | | DESWLlt | 0.85 | 2.05 | 0 | 1.12 | 1.07 | | CNCTemp | 0.85 | 0.85 | 0 | 0.9 | 1.27 | | RDL2 | 0.84 | 2.17 | 0 | 0.54 | 1.31 | | DRPP | 0.83 | 1.78 | 0 | 1.37 | 0.94 | | PCCNCz | 0.8 | 1.87 | 0 | 0.31 | 1.35 | | DRNP | 0.8 | 1.62 | 0 | 0.79 | 1.16 | | LDTTRa | 0.79 | 2.31 | 0 | 0.24 | 1.34 | | WRDAOAc | 0.78 | 0.69 | 0 | 0.78 | 1.17 | | RDFKGL | 0.73 | 2.06 | 0 | 0.13 | 1.27 | | SYNNP | 0.68 | 0.65 | 0 | 0.24 | 1.23 | | CNCPos | 0.68 | 0.82 | 0 | 0.67 | 1.03 | | CNCCaus | 0.67 | 0.73 | 0 | 0.63 | 1.02 | | WRDADV | 0.67 | 1.9 | 0 | 0.66 | 0.93 | | PCSYNz | 0.66 | 1.76 | 0 | 0.33 | 1.07 | | WRDPRO | 0.64 | 0.73 | 0 | 0.9 | 0.84 | | CNCLogic | 0.64 | 0.41 | 0 | 0.71 | 0.95 | | PCCNCp | 0.64 | 1.15 | 0 | 0 | 1.22 | | DRVP | 0.63 | 0.79 | 0 | 0.32 | 1.08 | | WRDADJ | 0.62 | 1.12 | 0 | 0.38 | 1 | | SMINTEp | 0.62 | 1.06 | 0 | 0.53 | 0.95 | | DRNEG | 0.6 | 0.73 | 0 | 0.06 | 1.13 | | WRDPOLc | 0.6 | 0.89 | 0 | 0.61 | 0.88 | | WRDHYPnv | 0.58 | 0.02 | 0 | 0.17 | 1.09 | | WRDFRQa | 0.58 | 0.36 | 0 | 0.35 | 0.99 | | SYNLE | 0.57 | 0.09 | 0 | 0.62 | 0.87 | | SMCAUSr | 0.56 | 0.05 | 0 | 0.3 | 1 | | DRAP | 0.55 | 1.23 | 0 | 0.32 | 0.89 | | PCREFz | 0.53 | 1.2 | 0 | 0.46 | 0.78 | | DESWLsy | 0.5 | 0.85 | 0 | 0.43 | 0.76 | | WRDMEAc | 0.49 | 0.68 | 0 | 0.7 | 0.63 | | PCSYNp | 0.48 | 1.45 | 0 | 0.02 | 0.86 | | CNCADC | 0.46 | 1.61 | 0 | 0.41 | 0.63 | | WRDFRQc | 0.42 | 0.3 | 0 | 0.83 | 0.45 | | WRDCNCc | 0.39 | 1.3 | 0 | 0.39 | 0.53 | | DESWLsyd | 0.39 | 0.32 | 0 | 0.33 | 0.63 | | SMINTEr | 0.26 | 0.37 | 0 | 0.11 | 0.44 | | PCCONNz | 0.23 | 0.53 | 0 | 0.23 | 0.32 | | PCREFp | 0.23 | 0.32 | 0 | 0.01 | 0.44 | | WRDFAMc | 0.1 | 0 | 0 | 0.24 | 0.09 | | CRFNO1 | 0.09 | 0.98 | 0 | 0.1 | 0.08 | | PCCONNp | 0.08 | 1.18 | 0 | 0.04 | 0.06 | | WRDPRP3p | 0.02 | 0.44 | 0 | 0.03 | 0 | --- # Coh-Metrix Model 3 {#cohmetrix-model-3} ## General Description Coh-Metrix Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below. The models were trained on Coh-Metrix scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons). Highly correlated CohMetrix metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). More details on the sample will be provided once peer review is complete on the main study using this model. ## CohMetrix Model 3narr This model was trained on CohMetrix scores from 15 min narrative writing samples. ### Algorithm Weightings in Ensemble Abbreviations: * overall = ensemble model * pls = partial least squares regression * mars = bagged multivariate adaptive regression splines * enet = elastic net regression * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | mars | gbm | enet | cube | |:----------|:-------|:-------|:-------|:-------|:-------| | 0.0000 | 0.1419 | 0.3143 | 0.0729 | 0.0816 | 0.1792 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). |Metric |overall|pls |gbm |mars |enet |cube | |---------|-------|----|-----|-----|-----|-----| |DESWC |24.87 |4.56|36.54|23.26|28.45|14.11| |WRDHYPn |8.87 |1.84|2.7 |14.41|7.18 |4.52 | |WRDNOUN |7.12 |2.27|2.89 |10.61|7.1 |4.37 | |DESSL |7 |1.2 |0.27 |14.41|0 |0.77 | |SYNNP |5.71 |1.81|3.49 |7.61 |8.68 |2.38 | |DESWLlt |5.25 |0.86|1.94 |7.61 |5.68 |4.14 | |LDVOCD |4.17 |2.88|3.99 |6.34 |0 |0.69 | |LDTTRa |2.6 |3.58|4.54 |0 |7.74 |3.99 | |SMCAUSwn |2.18 |2 |5.72 |0 |3.96 |2.15 | |SYNLE |2.05 |0.5 |1.59 |3.39 |0 |0.31 | |WRDPRP1s |1.71 |0.5 |0.83 |2.89 |0 |0.84 | |WRDHYPnv |1.55 |0.44|0.28 |2.61 |0 |1.61 | |PCDCp |1.44 |1.37|0.04 |2.61 |0.04 |1 | |PCREFp |1.41 |0.55|0 |2.65 |0 |1 | |PCNARz |1.32 |2.01|3.19 |0 |0 |3.14 | |CRFANPa |1.3 |1.59|3.69 |0 |0 |2.3 | |LSAGN |1.29 |2.26|2.08 |0 |2.54 |2.99 | |WRDFRQmc |0.94 |1.95|0.78 |0 |2.77 |2.68 | |CNCLogic |0.91 |0.73|0.37 |1.61 |0 |0.23 | |PCDCz |0.91 |1.12|3.1 |0 |0 |0.77 | |SYNMEDpos|0.9 |1.78|0.13 |0 |4.2 |2.61 | |SMCAUSlsa|0.84 |0.78|0.76 |0 |1.34 |3.37 | |PCCONNz |0.75 |1.46|1.11 |0 |2.09 |1.46 | |DRPP |0.65 |1.2 |0.36 |0 |2.51 |1.76 | |WRDAOAc |0.65 |1.77|1.13 |0 |1.24 |1.23 | |WRDPRO |0.63 |1.35|1.42 |0 |0 |1.53 | |DESPL |0.61 |2.71|0.51 |0 |1.6 |1.46 | |PCREFz |0.59 |1.06|0.56 |0 |2.75 |0.92 | |WRDMEAc |0.53 |0.99|1.17 |0 |0.82 |0.84 | |LDTTRc |0.52 |2.28|0.15 |0 |0 |2.68 | |DRNP |0.49 |0.62|0.24 |0 |1.25 |1.92 | |DESWLsy |0.49 |1.25|0.33 |0 |0.72 |1.99 | |WRDPOLc |0.44 |1.45|0.91 |0 |0 |1.07 | |SMINTEr |0.4 |0.35|0.26 |0 |2.03 |0.84 | |PCSYNz |0.39 |0.91|0.59 |0 |0 |1.46 | |PCCONNp |0.37 |2.07|0.2 |0 |1.9 |0.31 | |SMINTEp |0.35 |0.56|1.24 |0 |0 |0.15 | |DRPVAL |0.34 |1.5 |0.33 |0 |1.65 |0.23 | |LSASS1d |0.33 |1.49|0.06 |0 |0 |1.76 | |PCCNCz |0.32 |1.44|0.1 |0 |0 |1.61 | |CRFCWOa |0.3 |1.73|0.04 |0 |0 |1.46 | |RDL2 |0.29 |1.91|0.31 |0 |0 |0.92 | |CNCPos |0.28 |0.22|0.87 |0 |0 |0.38 | |PCVERBz |0.28 |1.5 |0.14 |0 |0 |1.23 | |LSAGNd |0.28 |1.86|0.19 |0 |0 |1.07 | |CRFCWO1d |0.25 |1.57|0.74 |0 |0 |0 | |WRDFRQa |0.25 |0.44|0.67 |0 |0 |0.46 | |CNCCaus |0.25 |0.36|0.29 |0 |0 |1.15 | |CRFAOa |0.24 |1.74|0.04 |0 |0 |1.07 | |DESWLltd |0.23 |0.2 |0.49 |0 |0 |0.69 | |WRDADJ |0.22 |0.25|0.43 |0 |0 |0.69 | |WRDPRP3p |0.21 |1.25|0.51 |0 |0 |0.23 | |LDMTLD |0.21 |0.26|0.42 |0 |0 |0.69 | |CRFCWOad |0.21 |1.68|0.35 |0 |0 |0.38 | |CNCTemp |0.2 |0.55|0.06 |0 |1.45 |0.15 | |WRDIMGc |0.19 |0.4 |0.22 |0 |0 |0.84 | |DESWLsyd |0.19 |0.69|0.26 |0 |0 |0.69 | |WRDVERB |0.18 |0.55|0.44 |0 |0 |0.31 | |CRFNOa |0.15 |1.06|0.07 |0 |0 |0.61 | |CNCADC |0.14 |1.46|0.12 |0 |0 |0.31 | |WRDHYPv |0.14 |0.35|0.32 |0 |0 |0.31 | |WRDFRQc |0.14 |0.58|0.38 |0 |0 |0.15 | |WRDCNCc |0.14 |0.37|0.51 |0 |0 |0 | |LSASSp |0.14 |1.54|0.08 |0 |0 |0.38 | |LSASSpd |0.14 |1.55|0.05 |0 |0 |0.46 | |PCTEMPp |0.13 |1.36|0.06 |0 |0 |0.38 | |DRVP |0.12 |0.92|0.15 |0 |0 |0.31 | |WRDPRP2 |0.12 |1.54|0.13 |0 |0.23 |0 | |DRGERUND |0.12 |0.49|0.15 |0 |0 |0.46 | |SMCAUSvp |0.11 |0.54|0.28 |0 |0 |0.15 | |PCSYNp |0.1 |0.68|0.15 |0 |0 |0.23 | |DRINF |0.1 |0.68|0.11 |0 |0 |0.31 | |DRAP |0.09 |0.43|0.29 |0 |0 |0 | |WRDADV |0.09 |0.6 |0.28 |0 |0 |0 | |DESSLd |0.09 |1 |0.12 |0 |0.1 |0.08 | |SYNSTRUTa|0.08 |1.31|0.09 |0 |0 |0 | |PCCNCp |0.07 |1.21|0 |0 |0 |0.15 | |WRDFAMc |0.07 |0.63|0.2 |0 |0 |0 | |SMCAUSv |0.06 |0.53|0.14 |0 |0 |0 | |CNCTempx |0.05 |0 |0.16 |0 |0 |0.08 | |SMCAUSr |0.04 |0.77|0.02 |0 |0 |0 | |PCVERBp |0.04 |0.9 |0 |0 |0 |0 | |WRDPRP1p |0.03 |0.68|0.02 |0 |0 |0 | |WRDPRP3s |0.02 |0.4 |0 |0 |0 |0 | |DRNEG |0.01 |0.24|0.02 |0 |0 |0 | ## Coh-Metrix Model 3exp This model was trained on Coh-Metrix scores from 15 min expository writing samples. ### Algorithm Weightings in Ensemble Abbreviations: * overall = ensemble model * pls = partial least squares regression * gbm = stochastic gradient boosted trees * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | mars | gbm | cube | |:----------|:-------|:-------|:-------|:-------| | -0.0577 | 0.1306 | 0.3136 | 0.3991 | 0.1752 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). |Metric |overall|mars |pls |gbm |cube | |---------|-------|-----|----|-----|-----| |DESWC |26.13 |25.33|5.56|47.08|15.82| |LSAGN |3.77 |0 |2.75|5.56 |4.35 | |LDTTRa |3.68 |0 |4.12|4.55 |3.68 | |DESSLd |3.13 |9.32 |1.26|2.42 |3.51 | |DESWLlt |2.98 |10.88|0.99|1.35 |4.35 | |WRDPRP2 |2.25 |0 |2.09|2.72 |3.18 | |LDVOCD |2.16 |2.46 |3.82|0.71 |2.26 | |DRPP |2.11 |0 |2.07|3.28 |1.09 | |WRDPOLc |2.1 |12.49|0.93|0.31 |0.5 | |LDTTRc |1.96 |0 |3.31|1.89 |1.17 | |SMCAUSwn |1.62 |0 |0.98|2.12 |2.85 | |WRDNOUN |1.61 |3.23 |1.38|0.53 |3.26 | |WRDPRP1s |1.55 |5.37 |1.05|0.83 |1.26 | |WRDPRP1p |1.5 |5.2 |0.08|0.9 |2.68 | |CNCTemp |1.39 |7.18 |1.01|0.27 |0.33 | |PCNARz |1.37 |0 |2.21|0.63 |2.59 | |LSASS1d |1.29 |4.14 |1.67|0.51 |0.25 | |PCREFz |1.27 |5.37 |1.14|0.18 |0.92 | |PCCONNz |1.25 |0 |1.16|2.11 |0.42 | |DRNP |1.22 |3.67 |0.17|1.18 |1.34 | |WRDMEAc |1.2 |5.37 |0.64|0.47 |0.75 | |WRDFRQa |1.19 |0 |1.04|1.52 |1.59 | |PCCONNp |1.19 |0 |2.2 |0.62 |1.59 | |SYNMEDpos|1.19 |0 |1.84|0.24 |3.1 | |WRDHYPn |1.18 |0 |0.89|1.18 |2.59 | |DESPL |1.03 |0 |2.71|0.38 |0.25 | |WRDHYPnv |1.02 |0 |0.83|0.73 |2.76 | |PCCNCz |0.94 |0 |1.35|0.26 |2.43 | |RDL2 |0.93 |0 |1.89|0.83 |0.17 | |LSASSp |0.9 |0 |1.68|0.69 |0.67 | |PCVERBz |0.9 |0 |1.38|0.38 |1.92 | |WRDHYPv |0.87 |0 |0.92|0.94 |1.26 | |LSASSpd |0.84 |0 |1.66|0.21 |1.42 | |WRDADJ |0.82 |0 |1.39|0.89 |0.25 | |CRFCWOa |0.81 |0 |1.79|0.37 |0.67 | |CRFANPa |0.78 |0 |1.28|0.52 |1.09 | |LSAGNd |0.77 |0 |2.06|0.08 |0.59 | |PCREFp |0.77 |0 |0.97|0 |2.76 | |CRFAOa |0.74 |0 |1.87|0.02 |0.92 | |DESSL |0.74 |0 |0.94|0.46 |1.59 | |CRFCWO1d |0.69 |0 |1.77|0.29 |0.17 | |SYNNP |0.69 |0 |1.25|0.31 |1.09 | |CRFNOa |0.64 |0 |1.2 |0.47 |0.5 | |PCTEMPp |0.62 |0 |1.26|0.49 |0.25 | |WRDAOAc |0.61 |0 |1.21|0.47 |0.33 | |CNCNeg |0.6 |0 |1.7 |0.2 |0 | |DRAP |0.58 |0 |1.08|0.21 |0.92 | |DRGERUND |0.57 |0 |0.97|0.69 |0 | |PCDCz |0.55 |0 |1.33|0.16 |0.42 | |PCDCp |0.53 |0 |1.35|0.09 |0.42 | |DESWLsy |0.53 |0 |0.58|0.34 |1.26 | |PCSYNz |0.51 |0 |0.75|0.12 |1.34 | |DESWLltd |0.5 |0 |0.25|0.53 |1.26 | |SMCAUSr |0.47 |0 |1.38|0.11 |0 | |WRDFRQmc |0.47 |0 |1.21|0.11 |0.33 | |WRDIMGc |0.45 |0 |0.32|0.27 |1.42 | |LDMTLD |0.45 |0 |0.51|0.64 |0.25 | |SMCAUSlsa|0.44 |0 |0.33|0.24 |1.42 | |SYNSTRUTa|0.43 |0 |1.13|0.22 |0 | |WRDADV |0.42 |0 |0.56|0.12 |1.17 | |CNCTempx |0.37 |0 |1.07|0.09 |0 | |WRDFRQc |0.37 |0 |0.68|0.16 |0.59 | |CNCLogic |0.36 |0 |0.95|0.17 |0 | |SMINTEr |0.36 |0 |1.12|0.03 |0 | |DRINF |0.36 |0 |0.89|0.21 |0 | |DESWLsyd |0.36 |0 |0.28|0.55 |0.33 | |SMINTEp |0.35 |0 |0.99|0.07 |0.08 | |SMCAUSvp |0.33 |0 |0.99|0.06 |0 | |CNCPos |0.33 |0 |0.17|0.61 |0.25 | |PCVERBp |0.31 |0 |0.49|0.01 |0.92 | |WRDPRP3s |0.29 |0 |0.42|0.26 |0.33 | |PCCNCp |0.29 |0 |0.91|0.03 |0 | |WRDPRO |0.29 |0 |0.64|0.14 |0.25 | |WRDFAMc |0.26 |0 |0.55|0.24 |0 | |DRVP |0.25 |0 |0.57|0.18 |0 | |SMCAUSv |0.23 |0 |0.69|0.04 |0 | |SYNLE |0.2 |0 |0.03|0.33 |0.33 | |PCSYNp |0.2 |0 |0.53|0.02 |0.17 | |WRDPRP3p |0.19 |0 |0.25|0.29 |0 | |CNCCaus |0.18 |0 |0.47|0.09 |0 | |WRDVERB |0.17 |0 |0.1 |0.34 |0 | |DRNEG |0.03 |0 |0 |0.08 |0 | ## Coh-Metrix Model 3per This model was trained on Coh-Metrix scores from 15 min persuasive writing samples. ### Algorithm Weightings in Ensemble Abbreviations: * overall = ensemble model * pls = partial least squares regression * gbm = stochastic gradient boosted trees * mars = bagged multivariate adaptive regression splines * cube = cubist regression The table below presents the linear weightings of each algorithm for the ensemble model. | Intercept | pls | mars | gbm | cube | |:----------|:-------|:-------|:-------|:-------| | -0.0381 | 0.0558 | 0.4924 | 0.4425 | 0.0259 | ### Metric Importance in Each Algorithm and Ensemble Each column sums to 100 (so values can be interpreted as % contribution to the model). |Metric |overall|pls |mars |gbm |cube | |---------|-------|----|-----|-----|-----| |DESWC |32.09 |4.68|34.34|33.8 |19.05| |WRDHYPn |10.41 |2.03|17.13|4.3 |5.17 | |LDVOCD |9.59 |3.43|0 |21.44|2.53 | |DESWLlt |8.27 |1.29|13.16|3.77 |7.4 | |LSAGN |6.13 |2.92|8.63 |3.88 |4.05 | |WRDNOUN |4.46 |1.39|7.36 |1.66 |3.95 | |WRDADV |3.05 |1.18|5.53 |0.52 |3.24 | |WRDFRQa |2.13 |0.25|3.86 |0.55 |0.2 | |SMCAUSwn |2.11 |1.08|3.7 |0.54 |1.01 | |CNCAdd |1.76 |0.66|3.37 |0.19 |0.2 | |WRDADJ |1.67 |0.59|2.93 |0.26 |4.15 | |LDTTRa |1.47 |4.05|0 |2.58 |4.76 | |DESSC |1.38 |3.27|0 |2.6 |2.74 | |LDTTRc |1.17 |3.17|0 |2.21 |1.32 | |DESWLltd |0.63 |1.31|0 |1.2 |1.42 | |WRDPRO |0.57 |1.57|0 |1.1 |0.3 | |PCDCz |0.5 |1.55|0 |0.94 |0.3 | |DESSLd |0.47 |1.61|0 |0.84 |0.61 | |CRFCWO1d |0.47 |2.36|0 |0.74 |0.81 | |DRNEG |0.46 |1.75|0 |0.72 |1.82 | |WRDPOLc |0.46 |0.78|0 |0.89 |1.22 | |SYNNP |0.45 |0.82|0 |0.92 |0 | |CNCCaus |0.44 |0.24|0 |0.98 |0 | |WRDPRP3p |0.4 |0.59|0 |0.82 |0.3 | |WRDHYPv |0.39 |0.94|0 |0.76 |0.2 | |LDMTLD |0.35 |0.38|0 |0.67 |1.72 | |WRDAOAc |0.34 |1.9 |0 |0.52 |0.3 | |CNCPos |0.32 |0.48|0 |0.66 |0 | |WRDMEAc |0.32 |1.11|0 |0.56 |0.61 | |CRFANPa |0.31 |1.37|0 |0.52 |0.2 | |DRVP |0.31 |0.47|0 |0.58 |1.22 | |WRDFRQc |0.31 |0.22|0 |0.65 |0.71 | |SMCAUSlsa|0.26 |0.65|0 |0.5 |0 | |LSASS1d |0.26 |2.04|0 |0.34 |0 | |CRFAO1 |0.25 |2.03|0 |0.3 |0.2 | |WRDHYPnv |0.25 |0.33|0 |0.48 |0.71 | |CNCTempx |0.23 |0.5 |0 |0.32 |2.23 | |SYNSTRUTa|0.22 |0.75|0 |0.39 |0.41 | |SYNMEDpos|0.21 |2.04|0 |0.14 |1.42 | |WRDPRP1s |0.2 |0.81|0 |0.26 |1.62 | |PCVERBz |0.2 |1.75|0 |0.15 |1.62 | |CRFCWO1 |0.19 |1.81|0 |0.18 |0.41 | |RDL2 |0.19 |1.54|0 |0.22 |0.51 | |PCNARz |0.18 |2.01|0 |0.05 |1.72 | |LSAGNd |0.18 |2.08|0 |0.1 |0.91 | |RDFKGL |0.17 |0.52|0 |0.19 |2.13 | |PCSYNz |0.17 |0.67|0 |0.19 |2.13 | |LSASSpd |0.15 |1.95|0 |0.08 |0.2 | |SMCAUSv |0.15 |0.37|0 |0.26 |0.61 | |SYNLE |0.14 |0.44|0 |0.26 |0 | |RDFRE |0.14 |0.45|0 |0.14 |2.13 | |DRNP |0.14 |0.41|0 |0.28 |0 | |PCDCp |0.14 |1.89|0 |0 |1.62 | |SMCAUSr |0.13 |1.29|0 |0.13 |0 | |LSASS1 |0.13 |1.84|0 |0.04 |0.41 | |CRFCWOad |0.13 |1.93|0 |0.04 |0.3 | |WRDPRP2 |0.12 |1.32|0 |0.1 |0 | |WRDFAMc |0.12 |0.32|0 |0.23 |0.1 | |PCREFz |0.12 |1.13|0 |0.06 |1.42 | |DESWLsy |0.11 |0.61|0 |0.13 |0.51 | |CNCLogic |0.11 |0.55|0 |0.17 |0.1 | |CRFNO1 |0.11 |1.61|0 |0.04 |0.2 | |DRGERUND |0.1 |0.38|0 |0.16 |0.41 | |DRPP |0.1 |0.34|0 |0.19 |0 | |PCTEMPp |0.1 |1.16|0 |0.07 |0.2 | |SMINTEr |0.1 |1.5 |0 |0.03 |0.2 | |PCCNCz |0.1 |1.31|0 |0.05 |0.41 | |DESWLsyd |0.1 |0.76|0 |0.13 |0.3 | |CNCNeg |0.09 |0.55|0 |0.13 |0.3 | |WRDVERB |0.08 |0.23|0 |0.15 |0 | |SMCAUSvp |0.08 |0.24|0 |0.14 |0.2 | |PCCONNz |0.08 |0.49|0 |0.11 |0.3 | |PCCONNp |0.07 |1.11|0 |0.01 |0 | |DRAP |0.07 |0.12|0 |0.14 |0 | |WRDCNCc |0.07 |0.03|0 |0.14 |0.2 | |WRDFRQmc |0.07 |1.06|0 |0.03 |0 | |PCVERBp |0.07 |1.19|0 |0 |0.3 | |PCREFp |0.06 |0.92|0 |0 |0.3 | |PCCNCp |0.05 |0.83|0 |0 |0 | |WRDPRP1p |0.05 |0.16|0 |0.09 |0 | |WRDIMGc |0.05 |0 |0 |0.09 |0.51 | |SMINTEp |0.05 |0.71|0 |0.03 |0 | |DRINF |0.05 |0.3 |0 |0.06 |0.3 | |CNCADC |0.05 |0.44|0 |0.05 |0.2 | |CNCTemp |0.04 |0.58|0 |0.02 |0 | |PCSYNp |0.04 |0.42|0 |0.01 |0.71 | |DRPVAL |0.01 |0.07|0 |0.01 |0 | --- # Automated Written Expression CBM (aWE-CBM) Model 1 {#awecbm-model-1} ## General Description Total Words Written(TWW) scores are generated directly from the GAMET word count score. Words Spelled Correctly (WSC) scores are generated by subtracting the GAMET misspelling score from the GAMET word count score. Correct Word Sequences (CWS) and Correct Minus Incorrect Word Sequences (CIWS) scores are based on emsemble models originally trained to predict CBM scores on 7 min narrative writing samples ("I once had a magic pencil and ...") from students in the fall, winter, and spring of Grades 2-5 [@Mercer2019]. More details on the sample are available in [@Mercer2019]. The CWS and CIWS models are detailed below (from Mercer et al., 2021). ## Correct Word Sequences Model | Metric | Overall | GBM | SVM | ENET | MARS | |:------------|:--------|:------|:------|:------|:------| | Word Count | 75.48 | 86.79 | 67.10 | 77.17 | 77.84 | | Spelling | 14.26 | 0.62 | 0.00 | 21.41 | 22.05 | | %Spelling | 8.78 | 12.28 | 27.95 | 0.40 | 0.11 | | Grammar | 0.85 | 0.05 | 2.77 | 0.11 | 0.00 | | %Grammar | 0.01 | 0.06 | 0.01 | 0.00 | 0.00 | | Duplication | 0.04 | 0.12 | 0.12 | 0.00 | 0.00 | | Typography | 0.38 | 0.08 | 1.33 | 0.00 | 0.00 | | White Space | 0.20 | 0.00 | 0.71 | 0.92 | 0.00 | _Note._ The weightings sum to 100; thus, they can be viewed as the percentage contribution of each metric to the predicted scores. Overall = the ensemble model of all algorithms, GBM = stochastic gradient boosted regression trees, SVM = support vector machines (radial kernel), ENET = elastic net regression, MARS = bagged multivariate adaptive regression splines. The following regression equation was used to weight the algorithms in the CWS ensemble model: .162 + .074 * GBM + .281 * SVM + .001 * ENET + .642 * MARS. ## Correct Minus Incorrect Word Sequences Model | Metric | Overall | GBM | SVM | ENET | MARS | |:------------|:--------|:------|:------|:------|:------| | Word Count | 55.60 | 55.76 | 47.57 | 61.43 | 61.35 | | Spelling | 19.25 | 1.48 | 6.57 | 35.80 | 35.04 | | %Spelling | 22.31 | 41.99 | 42.74 | 0.00 | 0.00 | | Grammar | 0.82 | 0.00 | 1.69 | 0.00 | 0.62 | | %Grammar | 0.04 | 0.23 | 0.00 | 0.00 | 0.00 | | Duplication | 0.28 | 0.10 | 0.76 | 0.00 | 0.00 | | Typography | 1.37 | 0.41 | 0.07 | 1.55 | 2.97 | | White Space | 0.34 | 0.04 | 0.60 | 1.22 | 0.00 | _Note._ The weightings sum to 100; thus, they can be viewed as the percentage contribution of each metric to the predicted scores. Overall = the ensemble model of all algorithms, GBM = stochastic gradient boosted regression trees, SVM = support vector machines (radial kernel), ENET = elastic net regression, MARS = bagged multivariate adaptive regression splines. The following equation was used for the CIWS model: -.170 + .180 * GBM + .346 * SVM + .100 * ENET + .375 * MARS. --- # References