NEPSroutinesThe NEPS data is scaled using the NEPSroutines package.
For this purpose, the latest stable version of NEPSroutines should be
installed (see README file for an how-to-guide).
Once installed, NEPSroutines must be loaded at the beginning of each R
session:
Working with the NEPSroutines package requires a specific data
structure. This will be explained in the following section. How to
transform the raw data into the required data structure and which
functions of the NEPSroutines package may be helpful in this process
is described in vignette prepare_data.
The package is mainly based on the two datasets resp and vars. The
bold marked variable names must be retained as specified. In all
other cases, variable names can be freely chosen.
Sometimes parts of the scaling procedure are conducted separately for different groups, e.g. different test versions, settings, measurement occasions, or cohorts. The terms group and grouping refer to such cases. However, the functions of the package can only handle one grouping at a time. If multiple groupings are present (e.g. different test booklets and different cohorts), there are two possible approaches:
On the one hand, analyses can be conducted separately. Since the
functions of the package can handle at least one grouping, it is
sufficient to run the analyses separately for the groups of the second
grouping (e.g. once for cohort A and once for cohort B). The first
grouping (in our case test booklet version) can then be handled
normally within the functions. In this case, remember to assign a
group name using the function argument name_group, so that results
are saved under different names and can be clearly assigned to the
respective groups (cohort A, cohort B).
On the other hand, a grouping variable can be created that contains all possible combinations of groupings. This can then be used as a standard grouping variable in the functions. For two groupings with two levels each (e.g. cohorts A and B, test booklets A and B), a new grouping variable with four categories would need to be created:
Group 1 for cohort A & test booklet A
Group 2 for cohort A & test booklet B
Group 3 for cohort B & test booklet A
Group 4 for cohort B & test booklet B
respThis data.frame contains all competence items — both scored and
unscored — as well as all information about the test participants, e.g.
sociodemographic variables. Information about each participant is stored
in a single row. How the test items are scored and the sociodemographic
variables prepared is described in vignette prepare_data. The
following variables are particularly required:
| Variable | Explanation |
|---|---|
ID_t |
A variable that contains the IDs of the persons. |
| All sociodemographic and methodological variables available per test person |
|
| All unscored, scored, and aggregated competence items |
|
| Additional variables that identify specific persons in the dataset |
|
For an example of resp see the example data dat1 to dat4.
Information on how to access the example data and its documentation is
available in the vignette example_data_and_scripts.
varsThis data.frame contains all information about the competence items,
e.g. item positions, assignment to a test booklet, and item type. Each
version of a competence item (e.g. scored and unscored) is stored in a
separate row (for an overview of the different item types, see Section
Competence items in vignette scaling_NEPS_data). How the variables
are constructed is described in vignette prepare_data. The following
variables are particularly required:
| Variable | Explanation |
|---|---|
item |
A variable containing the item name |
correct_response |
A variable containing the correct answer of the respective item e.g. 3 if the third response option is correct. Can be a character or numeric vector. |
type |
Variable defining the item type MC, CMC, etc. |
| Position variables | Indicate the position of the item in the test booklet
|
| Logical variables identifying specific items |
|
| Depending on scaling additional variables |
|
For an example of vars see the example data dat1 to dat4.
Information on how to access the example data and its documentation is
available in the vignette example_data_and_scripts.
The package is strongly oriented toward the scaling of NEPS competence tests and includes functions for the following sections of the scaling process:
Data preparation
Descriptive analyses
Analysis of missing values (per person & per item)
IRT analyses
DIF analyses
Distractor analysis
Dimensionality analysis
Score creation
SUF creation
Technical report
With the exception of data preparation and descriptive analyses, there is a single main function for each scaling section that automatically performs all necessary calculations and returns results as plots, tables, and messages. However, there are also minor functions that allow individual steps of each section to be performed separately, enabling users to refine the original analyses or adapt them to specific needs. In the corresponding vignettes within this guide, the main functions are presented in detail, while the minor functions are only briefly described.
In addition to the section functions, the package contains general functions (see Section @ref(Helper Functions)) that may be useful for scaling outside the predefined framework.
The output of the main functions usually consists of optionally
generated summary results as an Excel table (function argument
save=TRUE), the original results in RDS format (function argument
save=TRUE), and additional information printed to the console
(function argument print=TRUE). Depending on the function, plots may
also be generated (function argument plots=TRUE). The tables are
typically designed so that they can be used directly for reporting. The
original results are usually stored as a list containing the unmodified
output of the underlying functions (e.g. TAM functions). These results
can also be printed directly in R (function argument return=TRUE). If
they are stored in an R object (by assigning them a name), they can be
further processed directly. For example:
For minor functions that perform analyses, a list of results is always
returned. This output, or parts of it, is usually required for
subsequent functions (e.g. for generating plots or result tables). It is
therefore recommended to store the output in an R object (see above).
When using downstream functions, pay close attention to the instructions
in this guide and in the help files (see vignette getting_started)
regarding which output or which part of the output is actually required!
For minor functions that generate plots or tables from results, users
can choose whether these files should be saved (function argument
save=TRUE).
The functions in the package are partially structured in a very similar way, so that many function arguments are repeated. The most common arguments are presented in this section, but are not explained in further detail in the remainder of the guide.
Input arguments:
| Argument | Type | D escription | Default |
|---|---|---|---|
resp and
vars |
data frame | These function
arguments require
the data frames
resp and vars
introduced in the
previous section.
They can be
passed as they
are and do not
need to be
modified further. |
|
select |
string | The analyses are
almost always
based on only a
subset of the
available items
or only a
specific version
of the items.
This function
argument requires
the name of the
logical variable
in vars that
identifies the
variables
required for this
analysis e.g.
dich for only d
ichotomous items
or scaling for
d ichotomous
and polytomous
items. |
|
valid |
string | Optional in most
functions. This
function argument
requires the name
of the logical
variable in
resp that
identifies all
valid cases see
section Valid
Cases in
vignette
pre p are_data. |
all cases are used |
mvs |
named character vector |
Optional in most
functions
exception:
missing value
analysis.
Requires a named
character vector
containing all u
s er-defined
missing values
see section U s
er-Defined
Missing Values
in vignette
pre p are_data.
Warning: If this
vector is i
ncomplete, severe
problems and
errors in scaling
may occur! |
-999 to -1 |
grouping |
character vector | Optional in some
functions.
Sometimes parts
of the scaling
must be conducted
separately for
different groups
e.g. different
test booklet
versions. In this
case, this
function argument
requires a
character vector
with the names of
the logical
variables in
resp and vars
that identify all
persons or items
belonging to a
group. One name
is specified per
group. Variable
names must match
between resp
and vars e.g.
the variable
easy in vars
identifies all
items a d
ministered in the
easy test
version, and the
variable easy
in resp
identifies all
persons who
completed the
easy test
version. |
NULL |
Process arguments:
| Argument | Type | Description | Default |
|---|---|---|---|
digits |
integer | Number of decimal places to which the results should be rounded. | usually 3 |
warn |
logical | Whether warnings should be printed to the console. | usually
TRUE |
verbose |
logical | Whether the processing steps should be printed to the console. | usually
TRUE |
Output arguments:
| Argument | Type | Description | Default |
|---|---|---|---|
plots |
logical | In main functions, users can choose whether plots of the analyses (e.g. ICC plots and Wright maps in IRT analyses) should be generated. | usually
FALSE |
print |
logical | In main functions, users can choose whether (additional) results should be printed to the console or included in the HTML document. | usually TRUE |
save |
logical | In almost all
functions,
users can
choose whether
results should
be saved (as
Excel and/or
RDS). |
usually TRUE |
return |
logical | In main
functions,
users can
choose whether
results should
be returned in
R (as Rdata). |
usually
FALSE |
path |
string | Path where
results should
be stored. If
the specified
folder does not
exist, it will
be created
automatically.
In main
functions, this
is split into
path_results,
path_plots,
and
path_table. |
"Results",
"Plots",
"Tables" |
overwrite |
logical | Whether Excel tables should be overwritten if they already exist. | FALSE |
name_group |
string | Name of the
group to
which the
analyses refer
(e.g. “easy”,
see 3.1.1
Groupings),
used to label
results,
tables, and
figures
accordingly by
adding a suffix
to the file
name. Do not
confuse this
with
name_grouping
(= name of the
grouping,
e.g. “test
version”) in
mv_person and
mv_item! |
NULL |
In some functions, additional specific arguments are included. These are
described in this guide and can also be accessed at any time via the R
documentation (using the helper functions, see vignette
getting_started).
If parts of the scaling process are carried out without the corresponding package functions, some of the following functions may be helpful.
| Function | Usage |
|---|---|
convert_mv |
Converts user-defined missing values into
NAs. Input arguments are resp, items,
and mvs. The output is resp with
converted missing values. |
prepare_resp |
Prepares resp for subsequent analyses.
Input arguments are resp, vars, items,
valid, mvs, the logical use_only_valid
(whether only valid cases should be used),
and the logical convert (whether
user-defined missing values should be
converted to NAs). The output is resp
containing only the items defined in items
as variables, optionally restricted to valid
cases and with missing values optionally
converted to NAs. |
check_folder |
Checks whether the folder specified in the path exists and creates it if it does not. |
meht |
Performs the Minimum Effect Hypothesis Test. Input arguments are various statistical quantities. |
Step-by-step execution of code helps with debugging.
Since every scaling procedure is slightly different, it is possible that
errors occur when using the NEPSroutines package. However, in many
cases, the cause of error messages is simply issues in the data
themselves. If a function produces an error message, first check whether
your data are correct: Are all item properties in the vars dataset
correctly defined? Does every item in vars also appear in resp and
vice versa? Do the variable classes match the required specifications?
Have the variables been named correctly? The required data structure is
described in detail in the previous section Required Data Structure
and can be partially checked using dedicated functions (see vignette
prepare_data).
If the data are correct and the error still occurs, you can contact skalierung@lifbi.de with the error message, your script, and the data.
NEPSroutines provides example (simulated) data and scaling scripts for
users. The vignette example_data_and_scripts gives an overview and
explains how to access them.