Comment on “Being literate, behaving literate? A mixed-methods approach to adolescents’ algorithm literacy and behavioral strategies on social media”

Selected via the production queue; critique generated by the AGISS engine.

This is a careful, well-disclosed mixed-methods study (focus groups + diary + a quota-matched survey of 610 German adolescents) whose headline finding is a set of small structural-equation associations: more algorithm awareness goes with more 'indifferent' scrolling, and more algorithm knowledge goes with LESS interactive and critical behavior. The full text resolves most abstract-era worries — it openly concedes cross-sectional data, qualitative-sample education skew, and inability to reproduce the full literacy model. The one hard-to-refute problem is statistical bookkeeping: the significance legend under Table 3 (the table carrying every literacy→behavior result) says a single asterisk means p<.01, but the four key asterisked paths are reported in the text at p=.028, .022, .047 and (for awareness) .028 — all between .01 and .05. Taken literally, the legend would render the paper's entire central finding non-significant; the legend is almost certainly a typo for p<.05, but as printed it is an internal contradiction at the exact point the conclusions rest. A secondary, milder issue: the paper sets its own SRMR cutoff (<.05) in note 5, then calls the central SEM (SRMR=.055) and a CFA (SRMR=.050) 'acceptable.' Severity is low-to-moderate: the substantive associations are small and the flaws are presentational/threshold-consistency rather than evidence that the analysis is wrong.

Table 3 — which carries every algorithm-literacy→behavior result and thus the paper's central claims — has a significance legend stating a single asterisk denotes p<.01, yet the four asterisked paths it summarizes are reported in the running text at p=.028 (awareness→indifferent), p=.022 (knowledge→interactive), and p=.047 (knowledge→critical), all of which lie between .01 and .05 and therefore cannot satisfy a p<.01 threshold. The legend is almost certainly a typo for *<.05, but as printed the table contradicts the text: read literally, NONE of the four headline literacy-to-behavior paths clears the stated single-asterisk bar, which would nullify every confirmatory finding the abstract advertises ('higher algorithm awareness is associated with increased indifferent behavior, whereas greater algorithm knowledge correlates with reduced interaction'). This is an internal inconsistency the reader cannot defeat — both numbers are the paper's own.

The authors define their OWN fit criterion in note 5 — 'a standardized root mean square residual (SRMR) with values lower than .05 indicated an acceptable model fit' — then label as 'acceptable' two models that violate it: the central SEM carrying all hypothesis tests reports SRMR = .055 ('the model showed acceptable fit values') and the three-factor behavioral CFA reports SRMR = .050 (not lower than .05). Judged against the paper's self-imposed threshold, the key model does not meet the SRMR cutoff, yet is described as acceptable without comment.

The survey is cross-sectional (Study 3) and the limitations section explicitly concedes the data 'did not allow for any indications of whether more algorithm literacy would be related to different behavioral strategies of adolescents over time,' yet the results narrate associations with directional/causal verbs — attitudes 'driving engagement in these behaviors' — implying a causal push from attitudes to behavior that the design cannot license. This is softened by the disclosed limitation and by SEM-path convention, so it is a secondary rather than headline flaw.

The paper is methodologically conscientious and the full text dissolves most abstract-era suspicions. It is genuinely mixed-methods with a sensible sequential design (focus groups → diary → survey each informing the next); the survey is quota-matched to German population margins on gender, age, education, location, and migration background, with lower-education adolescents recruited offline precisely to correct the qualitative samples' skew. Measures are borrowed from validated instruments (Zarouali et al. AMCA scale; Dogruel et al. knowledge test), full CFA fit statistics, alphas, and item-deletion decisions are reported, analysis code/package versions are given (lavaan 0.6-19, ML estimation), and materials plus Study-3 data are posted on OSF. Crucially, the limitations section pre-empts the obvious objections: it concedes the cross-sectional design forecloses over-time inference, that the qualitative samples under-represent lower education, and that the full Dogruel literacy model could not be reproduced. The effects are honestly reported as small and the discussion is appropriately hedged ('complex,' 'first holistic insight,' 'initial quantitative validation'). The asterisk legend is best read as a printing erratum rather than a substantive analytic error, and the SRMR overshoot (.055) is trivial by conventional (<.08) standards.