7 Analyzing AMPLab data in Stata
This section provides a practical introduction to analysing AMPLab data
in Stata using the repest package. It is intended for users with a
basic working knowledge of Stata and includes step-by-step examples
covering common AMPLab analyses.
Use the contents below to navigate directly to the topics most relevant to your analysis.
7.1 Loading data in Stata
For hands-on examples we’ll start with the student data.
C:\Users\cash\Downloads\ampl-guideIt is good practice to run your analysis from a do-file rather than the command window, as this makes your work reproducible. All examples in this guide are written as do-file code.
7.2 Add variable and value labels
C:\Users\cash\Downloads\ampl-guideThe numlabel, add command displays the numeric code alongside the
value label in tabulations (for example, 1. Girl rather than just Girl).
This makes it easier to write if conditions using the correct numeric
codes in subsequent commands.
After loading the file, let’s check if we have the most important
variables that repest uses. The five plausible values for language and
mathematics are pv1_read to pv5_read and pv1_math to pv5_math,
respectively. The final weight is fwgt. The replicate weights are
rwgt1 to rwgt135 and countries are indicated by cnt.
C:\Users\cash\Downloads\ampl-guide
Variable Storage Display Value
name type format label Variable label
-------------------------------------------------------------------------------
pv1_read float %9.0g Ability plausible value 1 -
Reading
pv2_read float %9.0g Ability plausible value 2 -
Reading
pv3_read float %9.0g Ability plausible value 3 -
Reading
pv4_read float %9.0g Ability plausible value 4 -
Reading
pv5_read float %9.0g Ability plausible value 5 -
Reading
pv1_math float %9.0g Ability plausible value 1 -
Mathematics
pv2_math float %9.0g Ability plausible value 2 -
Mathematics
pv3_math float %9.0g Ability plausible value 3 -
Mathematics
pv4_math float %9.0g Ability plausible value 4 -
Mathematics
pv5_math float %9.0g Ability plausible value 5 -
Mathematics
fwgt float %9.0g Final Student Weight
rwgt1 float %9.0g Replicate student weight 1
rwgt2 float %9.0g Replicate student weight 2
rwgt3 float %9.0g Replicate student weight 3
rwgt4 float %9.0g Replicate student weight 4
rwgt5 float %9.0g Replicate student weight 5
rwgt6 float %9.0g Replicate student weight 6
rwgt7 float %9.0g Replicate student weight 7
rwgt8 float %9.0g Replicate student weight 8
rwgt9 float %9.0g Replicate student weight 9
rwgt10 float %9.0g Replicate student weight 10All required variables for repest are available in the data.
7.3 The repest package
To get started, you need to install the repest package once. This
package automates the handling of plausible values and replicate
weights, making it easier to analyse AMPLab data correctly in Stata. In
the Stata command window, type:
You only need to run this command once; there is no need to re-install
repest each time you open Stata. The replace option updates the
package if a newer version is available.
7.4 repest command syntax
The basic syntax of the repest command is as follows:
svyname: Either one of the study names supported by the package (e.g., PISA, TIMSS, PIRLS) or SVY which allows you to specify the survey design.estimate(cmd [,cmd_options]): Specifies the statistical command to run.cmdcan be any Stata command that accepts weights — for example,mean,reg,qreg, or the built-in repest commandsmeans,freq,summarize,corrandquantiletable. Command-specific options are passed after a comma within the parentheses.
Once these survey settings have been specified, you can use repest to
estimate means, summary statistics, proficiency levels, group
differences, percentiles, and regression models while correctly
accounting for both plausible values and replicate weights.
7.5 Before you begin: Set up repest for AMPLab
AMPLab is not one of the studies supported by the repest package,
meaning that there aren’t built-in survey specifications. Therefore,
before running any analyses, you need to tell repest how the AMPLab
data are structured.
You will need to use SVY for svyname and specify the survey
parameters directly within the svyparm() option. To analyse AMPLab
data correctly, repest needs to know the following survey
specifications:
AMPLab uses the paired Jackknife method for creating the replicate weights (JK2).
There are 135 replicate weights in the data.
The final weight is given by fwgt.
There are five sets of plausible values for each of mathematics, pv1_math to pv5_math, and reading, pv1_read to pv5_read.
The required parameters are as follows:
| Survey setting | svyparm() suboption |
AMPLab student data |
|---|---|---|
| Final weight | final_weight_name() |
fwgt |
| Replicate weights | rep_weight_name() |
rwgt |
| Variance factor | variancefactor() |
1 |
| Number of replications | NREP() |
135 |
| Number of plausible values | NBpv() |
5 |
Commands for analysing the AMPLab student data will have the following syntax:
7.6 AMPLab Analyses Examples
Before we start analysing proficiency scores, let us get a sense of the countries and grades presented in the data
C:\Users\cash\Downloads\ampl-guide
Country |
3-letter | STF - Grade
code | 3 4 6 7 | Total
-----------+--------------------------------------------+----------
GMB | 4,106 0 0 0 | 4,106
KEN | 0 0 5,238 0 | 5,238
LSO | 0 0 0 3,917 | 3,917
ZMB | 0 4,474 0 4,479 | 8,953
-----------+--------------------------------------------+----------
Total | 4,106 4,474 5,238 8,396 | 22,214 7.6.1 Calculating Mean Age by Country
Let’s start by calculating the mean age of students in Grade 3 in The Gambia. The age variable is called s_age.
C:\Users\cash\Downloads\ampl-guide
-------------------------------------------------------------------------------
s_age Student age at administration period
-------------------------------------------------------------------------------
Type: Numeric (float)
Label: lbl_s_age, but 194 nonmissing values are not labeled
Range: [6,9999] Units: 1.000e-07
Unique values: 195 Missing .: 0/22,214
Examples: 10.833333
12
13
14.25 Note that the ages are not integers, so using tab would not be
appropriate. There is also a value label attached to this variable. We
can inspect it.
C:\Users\cash\Downloads\ampl-guide
lbl_s_age:
9999 9999. MissingMissing age values are coded as \(9999\). Because \(9999\) is a placeholder used to indicate missing data rather than a student’s actual age, these observations should be excluded from analyses involving age. Failure to do so may produce misleading results.
To calculate the average age of students in Grade 3 in The Gambia, the syntax is as follows:
* Calculating average age in The Gambia
repest SVY if cnt=="GMB" & s_age<9999, estimate(mean s_age) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1))C:\Users\cash\Downloads\ampl-guide
_pooled.
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
s_age | 10.74445 .0335423 320.32 0.000 10.6787 10.81019
------------------------------------------------------------------------------Because age is an observed variable rather than a plausible-value
variable, repest uses only the sampling variance derived from the
replicate weights. The reported standard error therefore reflects
uncertainty arising from the sample design but not measurement
uncertainty.
The estimated mean age of Grade 3 students in The Gambia is \(10.74\) years (\(95\%\) CI: \(10.67–10.81\)). This reflects the substantial grade repetition and late entry common in the region, with many students aged \(11–23\) also enrolled in Grade 3.
AMPLab provides replicate weights specifically so that users can
reproduce the official variance estimation procedure. Stata’s svyset
can be configured to use the paired jackknife replicate weights; the
survey package in R can do the same.
The syntax to generate the same result is:
7.6.2 Calculating mean mathematics proficiency
Now we will use \(5\) plausible values to estimate mean mathematics scores in The Gambia.
Plausible values are a set of multiple imputations. The repest package
automatically recognises plausible values when the variable name
contains the \(@\) symbol.
For example:
tells repest to analyse all five reading plausible values, combine
results appropriately, and calculate standard errors that reflect both
sampling and measurement uncertainty. This allows researchers to obtain
valid estimates without having to implement the multiple-imputation
calculations manually.
repest SVY if cnt=="GMB" , estimate(mean pv@_math) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1))C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -2.339032 .0363908 -64.28 0.000 -2.410357 -2.267708
------------------------------------------------------------------------------The estimated mean mathematics proficiency in The Gambia is \(-2.34\) on the Learning Progressions Scale (\(95\%\) CI: \(-2.41\) to \(-2.27\); SE = \(0.036\)). Scores on this scale are centred near the AMPL proficiency cut scores, so a negative mean indicates that Grade 3 students in The Gambia are, on average, below the end-of-primary MPLb threshold. The standard error reflects both sampling uncertainty and uncertainty arising from the plausible values.
Unlike age, mathematics proficiency is represented by five plausible
values. repest estimates the mean separately for each plausible value,
combines the five estimates using multiple-imputation formulas, and then
incorporates the replicate-weight variance to produce the final standard
error.
7.6.3 Proficiency across gender
We can go further and ask how mean proficiency compares between girls and
boys within The Gambia. To do this we first create an indicator variable
for girls (1 = girl, 0 = boy).
C:\Users\cash\Downloads\ampl-guide
Student |
gender | Freq. Percent Cum.
------------+-----------------------------------
1. Female | 11,592 52.18 52.18
2. Male | 10,622 47.82 100.00
------------+-----------------------------------
Total | 22,214 100.00C:\Users\cash\Downloads\ampl-guideWith the girl indicator created, we can compute mean mathematics and
reading proficiency using the by option.
repest SVY if cnt=="GMB", estimate(mean pv@_math pv@_read) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) by(girl)C:\Users\cash\Downloads\ampl-guide
0 1
0.....
girl : 0
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -2.371841 .0393161 -60.33 0.000 -2.4489 -2.294783
pv__read | -1.397026 .031389 -44.51 0.000 -1.458548 -1.335505
------------------------------------------------------------------------------
1.....
girl : 1
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -2.31179 .0459427 -50.32 0.000 -2.401836 -2.221744
pv__read | -1.327546 .0362518 -36.62 0.000 -1.398599 -1.256494
------------------------------------------------------------------------------In The Gambia, girls score slightly higher than boys in both domains. Mean mathematics is \(-2.31\) for girls (SE \(= 0.046\)) versus \(-2.37\) for boys (SE \(= 0.039\)); mean reading is \(-1.33\) for girls (SE \(= 0.036\)) versus \(-1.40\) for boys (SE \(= 0.031\)). The gaps are modest relative to the overall low level of performance on the LPS, but the pattern is consistent across subjects.
7.6.4 Proficiency by Country
Now let us compare mathematics and reading proficiency across countries.
Zambia administered AMPLab in two grades, but repest does not allow
more than one variable in the by option. To handle this we create a
combined cnt_grade grouping below.
C:\Users\cash\Downloads\ampl-guide
group(cnt |
grade_stf) | Freq. Percent Cum.
------------+-----------------------------------
GMB 3 | 4,106 18.48 18.48
KEN 6 | 5,238 23.58 42.06
LSO 7 | 3,917 17.63 59.70
ZMB 4 | 4,474 20.14 79.84
ZMB 7 | 4,479 20.16 100.00
------------+-----------------------------------
Total | 22,214 100.00We can now compute proficiency scores by country (and by grade for Zambia).
repest SVY , estimate(mean pv@_math pv@_read) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) by(cnt_grade)C:\Users\cash\Downloads\ampl-guide
1 2 3 4 5
1.....
cnt_grade : 1
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -2.339032 .0363908 -64.28 0.000 -2.410357 -2.267708
pv__read | -1.359066 .0297572 -45.67 0.000 -1.417389 -1.300743
------------------------------------------------------------------------------
2.....
cnt_grade : 2
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -.4384149 .0339147 -12.93 0.000 -.5048865 -.3719433
pv__read | .1561163 .0389962 4.00 0.000 .0796852 .2325473
------------------------------------------------------------------------------
3.....
cnt_grade : 3
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -.8531881 .0433836 -19.67 0.000 -.9382184 -.7681579
pv__read | -.3905692 .0494259 -7.90 0.000 -.4874422 -.2936962
------------------------------------------------------------------------------
4.....
cnt_grade : 4
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -2.66579 .0352336 -75.66 0.000 -2.734846 -2.596733
pv__read | -1.648457 .031755 -51.91 0.000 -1.710695 -1.586218
------------------------------------------------------------------------------
5.....
cnt_grade : 5
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math | -1.050406 .037421 -28.07 0.000 -1.12375 -.9770622
pv__read | -.5233323 .0418152 -12.52 0.000 -.6052886 -.441376
------------------------------------------------------------------------------Mean performance differs sharply by country–grade group. On mathematics, Kenya Grade 6 has the highest mean (\(-0.44\)), followed by Lesotho Grade 7 (\(-0.85\)) and Zambia Grade 7 (\(-1.05\)), while Zambia Grade 4 (\(-2.67\)) and The Gambia Grade 3 (\(-2.34\)) are much lower. Reading follows a similar ordering, with Kenya the only group whose mean reading score is positive (\(0.16\)). These contrasts partly reflect grade level as well as system differences, which is why separating Zambia’s Grade 4 and Grade 7 samples matters.
7.6.5 Moving beyond means: Summary statistics with repest
repest has some built-in commands that are useful for analysing
large-scale assessments, including summary statistics that focus on the
full distribution.
The summarize command generates point estimates and standard errors
for a range of statistics beyond the mean, such as percentiles and
standard deviations. Below we summarise the mathematics proficiency
distribution for Grade 3 students in The Gambia by gender.
repest SVY if cnt=="GMB" , estimate(summarize pv@_math, stats(mean sd p5 p25 p50 p75 p95)) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) by(girl)C:\Users\cash\Downloads\ampl-guide
0 1
0.....
girl : 0
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math_m~n | -2.371841 .0393161 -60.33 0.000 -2.4489 -2.294783
pv__math_sd | 1.065351 .0334814 31.82 0.000 .9997292 1.130974
pv__math_p5 | -3.850363 .0384245 -100.21 0.000 -3.925673 -3.775052
pv__math_p25 | -3.121192 .0403424 -77.37 0.000 -3.200261 -3.042122
pv__math_p50 | -2.522535 .0619689 -40.71 0.000 -2.643992 -2.401078
pv__math_p75 | -1.790534 .0845723 -21.17 0.000 -1.956293 -1.624775
pv__math_p95 | -.3191085 .1368639 -2.33 0.020 -.5873568 -.0508602
------------------------------------------------------------------------------
1.....
girl : 1
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math_m~n | -2.31179 .0459427 -50.32 0.000 -2.401836 -2.221744
pv__math_sd | 1.090902 .0376727 28.96 0.000 1.017064 1.164739
pv__math_p5 | -3.820493 .0486067 -78.60 0.000 -3.91576 -3.725225
pv__math_p25 | -3.092192 .04298 -71.94 0.000 -3.176431 -3.007953
pv__math_p50 | -2.481752 .0504133 -49.23 0.000 -2.58056 -2.382944
pv__math_p75 | -1.675401 .0712234 -23.52 0.000 -1.814996 -1.535805
pv__math_p95 | -.2163765 .1640926 -1.32 0.187 -.5379921 .1052391
------------------------------------------------------------------------------Looking beyond average performance shows that the gender gap is small across the distribution. Boys and girls have similar spread (SD ≈ \(1.07\) vs \(1.09\)). At the median, girls score \(-2.48\) and boys \(-2.52\); at the 75th percentile the gap is a little larger (\(-1.68\) vs \(-1.79\)). Even the strongest performers remain well below zero on the LPS: the 95th percentile is about \(-0.22\) for girls and \(-0.32\) for boys.
The quantiletable command creates quantile tables.
repest SVY if cnt=="GMB" , estimate(quantiletable pv@_math pv@_read, nquantiles(5)) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) by(girl)C:\Users\cash\Downloads\ampl-guide
0 1
0.....
girl : 0
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math_q1 | -3.669684 .0413036 -88.85 0.000 -3.750637 -3.58873
pv__math_q2 | -3.003046 .0374756 -80.13 0.000 -3.076497 -2.929596
pv__math_q3 | -2.521405 .0431469 -58.44 0.000 -2.605972 -2.436839
pv__math_q4 | -1.945504 .0535702 -36.32 0.000 -2.0505 -1.840509
pv__math_q5 | -.7181873 .0934256 -7.69 0.000 -.9012981 -.5350764
pv__read_q1 | -2.152372 .0303795 -70.85 0.000 -2.211915 -2.09283
pv__read_q2 | -1.835932 .0419015 -43.82 0.000 -1.918057 -1.753806
pv__read_q3 | -1.566249 .0432234 -36.24 0.000 -1.650966 -1.481533
pv__read_q4 | -1.167279 .0517074 -22.57 0.000 -1.268624 -1.065935
pv__read_q5 | -.262206 .0888797 -2.95 0.003 -.4364071 -.0880049
------------------------------------------------------------------------------
1.....
girl : 1
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pv__math_q1 | -3.638227 .0409415 -88.86 0.000 -3.718471 -3.557983
pv__math_q2 | -2.972569 .0404677 -73.46 0.000 -3.051884 -2.893254
pv__math_q3 | -2.474581 .046145 -53.63 0.000 -2.565024 -2.384139
pv__math_q4 | -1.855555 .0616231 -30.11 0.000 -1.976334 -1.734776
pv__math_q5 | -.6151727 .1039124 -5.92 0.000 -.8188373 -.4115082
pv__read_q1 | -2.13194 .0305096 -69.88 0.000 -2.191738 -2.072143
pv__read_q2 | -1.80588 .0413396 -43.68 0.000 -1.886905 -1.724856
pv__read_q3 | -1.488811 .0523999 -28.41 0.000 -1.591513 -1.386109
pv__read_q4 | -1.05992 .0659508 -16.07 0.000 -1.189181 -.9306585
pv__read_q5 | -.1494516 .0851599 -1.75 0.079 -.3163619 .0174588
------------------------------------------------------------------------------The quantile table divides the mathematics and reading distributions into five equal-sized groups. The reported values are the cut-points separating adjacent quintiles. For example, among boys the mathematics cut from the lowest to the second quintile is about \(-3.67\), while the top quintile begins around \(-0.72\). Girls’ corresponding cut-points are slightly higher (about \(-3.64\) and \(-0.62\)), which matches the modest advantage for girls seen in the means and percentiles.
Other useful built-in commands include means, freq, and corr.
7.6.6 Estimating AMPL Proficiency Levels
AMPL established cut scores that correspond to the minimum proficiency levels required at the end of lower primary school (MPLa) and at the end of primary school (MPLb), for both mathematics and reading.
The cut scores for mathematics are defined by AMPL as follows:
| AMPL Proficiency Level | Score |
|---|---|
| Below MPLa | \(< -1.74\) |
| Between MPLa & MPLb | \([-1.74, -0.04)\) |
| Above MPLb | \(\geq -0.04\) |
These cut scores for reading and mathematics were established on the Learning Progressions Scale (LPS) with an international standard setting exercise undertaken in 2022. A pairwise comparison method (PCM) study was then used to confirm and validate the locations of those cut scores.
There is a variable for each plausible value that records this cut-score
information for both mathematics (pl_pv*_math) and reading
(pl_pv*_read). The file also supplies ready-made binary indicators for
reaching MPLa (pl_psa*_math, pl_psa*_read) and MPLb (pl_psb*_math,
pl_psb*_read).
C:\Users\cash\Downloads\ampl-guide
Proficient standard for PV1 - Reading | Freq. Percent Cum.
----------------------------------------+-----------------------------------
1. Below standard a | 11,986 53.96 53.96
2. Above a standard and below standard | 8,049 36.23 90.19
3. Above standard b | 2,179 9.81 100.00
----------------------------------------+-----------------------------------
Total | 22,214 100.00C:\Users\cash\Downloads\ampl-guide
Proficient standard for PV1 - |
Mathematics | Freq. Percent Cum.
----------------------------------------+-----------------------------------
1. Below standard a | 9,281 41.78 41.78
2. Above a standard and below standard | 9,439 42.49 84.27
3. Above standard b | 3,494 15.73 100.00
----------------------------------------+-----------------------------------
Total | 22,214 100.00Next, we estimate the proportion of students reaching the minimum
proficiency level required at the end of lower primary school (MPLa).
The student file already includes binary indicators for reaching MPLa
for each plausible value: pl_psa1_math–pl_psa5_math and
pl_psa1_read–pl_psa5_read (coded \(1\) = at or above MPLa, \(0\) =
below).
C:\Users\cash\Downloads\ampl-guide
Proficient standard |
a for PV1 - |
Mathematics | Freq. Percent Cum.
--------------------+-----------------------------------
0. Below standard a | 9,281 41.78 41.78
1. Above standard a | 12,933 58.22 100.00
--------------------+-----------------------------------
Total | 22,214 100.00Now we estimate the MPLa proportions for reading and mathematics in The Gambia.
repest SVY if cnt=="GMB", estimate(mean pl_psa@_read pl_psa@_math) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1))C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
pl_psa__read | .2150406 .0137092 15.69 0.000 .188171 .2419102
pl_psa__math | .2584249 .0127536 20.26 0.000 .2334284 .2834214
------------------------------------------------------------------------------In The Gambia, about \(22\%\) of Grade 3 students reach reading MPLa (SE = \(0.014\)) and about \(26\%\) reach mathematics MPLa (SE \(= 0.013\)). In other words, roughly three-quarters of students are still below the end-of-lower-primary minimum proficiency threshold in each domain.
7.6.7 Testing for differences between groups
The table below highlights the distinction between two common types of comparisons in AMPLab data: within-country comparisons (e.g., boys versus girls in the same country) and between-country comparisons (e.g., Lesotho versus Kenya). Because these comparisons involve different survey structures, they require slightly different analytical approaches.
The examples that follow demonstrate how to test for differences in
achievement between groups using regression models in repest.
| Comparison Type | What it measures | Survey design impact |
|---|---|---|
| Within-Country (e.g., Boys vs. Girls in Zambia) | The gap between two demographic subgroups who share the same sampling strata, schools, and teachers. | High covariance. Because groups are clustered together in the same schools, their errors are correlated. |
| Between-Country (e.g., Lesotho vs Kenya) | The gap between two entirely independent populations with completely separate sampling frames | Zero covariance. Sampling units in Country A have no mathematical relationship to sampling units in Country B |
In repest you should use over for within-country comparisons and
by for between-country comparisons. You must NOT use over for
countries. When using svyname = SVY, the over() option is designed
for within-country subgroup comparisons only. Using regression achieves
the same goal and works correctly with the SVY option.
7.6.7.1 Testing for differences between boys and girls
Are there differences in the proportion of students reaching reading MPLa
between girls and boys in Kenya? We can use linear regression to test
for this. The girl indicator is already available from earlier in the
chapter (no reload needed).
repest SVY if cnt=="KEN", estimate(reg pl_psa@_read girl) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1))C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
girl | .0670775 .0168496 3.98 0.000 .0340529 .1001021
_cons | .7485114 .0164443 45.52 0.000 .7162811 .7807417
------------------------------------------------------------------------------In Kenya, girls are about \(6.7\) percentage points more likely than boys to reach reading MPLa (coefficient \(= 0.067\); SE \(= 0.017\)). The intercept implies that about \(75\%\) of boys meet reading MPLa, so the girl estimate corresponds to roughly \(82\%\) among girls. The \(95\%\) confidence interval for the girl coefficient is \(0.034\) to \(0.100\).
repest does not always print p-values in the default table, but you
can compute a two-sided p-value from the coefficient and standard error.
* Two-sided p-value from z (large-sample normal approximation)
display 2*normal(-abs(0.0670775/0.0168496))C:\Users\cash\Downloads\ampl-guide
.00006864A p-value near \(0\) means a gap of this size would be highly unusual if there were no true gender difference in the population. Substantively, Kenya shows a clear within-country advantage for girls on reading MPLa, even though a large share of both girls and boys already meet the threshold.
7.6.7.2 Testing for differences between countries
Next, we ask whether mean mathematics scores differ between students enrolled in Grade 7 in Zambia and Lesotho. Create a dummy variable that equals \(1\) for Grade 7 Lesotho, \(0\) for Grade 7 Zambia, and missing for everyone else.
C:\Users\cash\Downloads\ampl-guide
(13,818 missing values generated)repest SVY, estimate(reg pv@_math LSO_7) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
LSO_7 | .1972179 .058168 3.39 0.001 .0832107 .3112251
_cons | -1.050406 .037421 -28.07 0.000 -1.12375 -.9770622
------------------------------------------------------------------------------Lesotho Grade 7 students score about \(0.20\) LPS points higher in mathematics than Zambia Grade 7 students (SE \(= 0.058\); \(95\%\) CI: \(0.08\) to \(0.31\)). The intercept of \(-1.05\) is the Zambia Grade 7 mean. The gap is statistically clear and matches the earlier country–grade means.
We can also compare a lower part of the distribution using quantile regression at the 25th percentile.
repest SVY, estimate(qreg pv@_math LSO_7, quantile(0.25)) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
LSO_7 | .2249767 .0568251 3.96 0.000 .1136016 .3363519
_cons | -1.723347 .032399 -53.19 0.000 -1.786848 -1.659846
------------------------------------------------------------------------------At the 25th percentile, the Lesotho advantage is about \(0.22\) points (SE \(= 0.057\); \(95\%\) CI: \(0.11\) to \(0.34\)). The gap is similar in size to the mean difference, which suggests that Lesotho’s higher mathematics performance is not confined to students near the centre of the distribution.
At Grade 7, the relevant minimum proficiency level is MPLb. The student
file already includes binary indicators for reaching MPLb for each
plausible value: pl_psb1_math–pl_psb5_math and
pl_psb1_read–pl_psb5_read (coded \(1\) = at or above MPLb, \(0\) =
below).
Are there differences in the percentage reaching the end-of-primary minimum proficiency level (MPLb) for mathematics?
repest SVY, estimate(reg pl_psb@_math LSO_7) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1)) C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
LSO_7 | .0359181 .0211479 1.70 0.089 -.0055311 .0773672
_cons | .1611611 .0139389 11.56 0.000 .1338413 .188481
------------------------------------------------------------------------------About \(16\%\) of Zambia Grade 7 students reach mathematics MPLb (intercept \(= 0.161\)). The Lesotho coefficient is \(0.036\) (SE \(= 0.021\)), so Lesotho’s estimated rate is about \(20\%\). The \(95\%\) CI for the difference (\(-0.006\) to \(0.077\)) includes zero, so unlike the mean score gap, the MPLb gap is not statistically clear at conventional levels.
If you prefer an explicit p-value:
* Two-sided p-value from z (large-sample normal approximation)
display 2*normal(-abs(0.0359181/0.0211479))C:\Users\cash\Downloads\ampl-guide
.08942777A p-value around \(0.09\) means this MPLb difference would not be unusual if the two Grade 7 populations had the same proficiency rate. The mean score contrast is clearer than the binary MPLb contrast because many students in both countries remain below the MPLb cut score.
7.6.8 Multivariate regression
In this example we look at student and family factors associated with
learning in Lesotho. We start with the association between reading
scores (pv@_read), gender, and the household wealth index (wealth).
C:\Users\cash\Downloads\ampl-guide
-------------------------------------------------------------------------------
wealth Household wealth index in logits
-------------------------------------------------------------------------------
Type: Numeric (float)
Label: lbl_wealth, but 2112 nonmissing values are not labeled
Range: [-3.7867272,999] Units: 1.000e-11
Unique values: 2,113 Missing .: 0/22,214
Examples: -2.0699382
-.7268303
-.00591791
.96151316
lbl_wealth:
999 999. MissingAlways exclude observations with missing wealth codes (wealth = 999).
Including the placeholder value as if it were a real score would distort
the regression.
repest SVY if cnt=="LSO"& wealth<999, estimate(reg pv@_read girl wealth) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1))C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
girl | .2595522 .0396518 6.55 0.000 .1818362 .3372683
wealth | .1588913 .0287175 5.53 0.000 .1026061 .2151765
_cons | -.4466969 .0517333 -8.63 0.000 -.5480922 -.3453016
------------------------------------------------------------------------------In Lesotho, girls score about \(0.26\) LPS points higher in reading than boys after controlling for wealth (SE \(= 0.040\); \(95\%\) CI: \(0.18\) to \(0.34\)). A one-unit increase in the wealth index is associated with about \(0.16\) points higher reading (SE \(= 0.029\); \(95\%\) CI: \(0.10\) to \(0.22\)). Both associations are statistically clear. The intercept (\(-0.45\)) is the expected reading score for boys with wealth equal to zero on the index.
Next, we add parents’ highest education level and switch the outcome to mathematics MPLb, so the coefficients are on the probability scale.
C:\Users\cash\Downloads\ampl-guide
Highest level of parental |
education | Freq. Percent Cum.
----------------------------------+-----------------------------------
0. Did not complete ISCED level 1 | 1,480 6.66 6.66
1. ISCED level 1 | 3,434 15.46 22.12
2. ISCED level 2 | 3,461 15.58 37.70
3. ISCED level 3 | 5,400 24.31 62.01
4. ISCED level 4 or 5 | 3,328 14.98 76.99
5. ISCED level 6 or higher | 4,304 19.38 96.37
9. Missing/Unknown | 807 3.63 100.00
----------------------------------+-----------------------------------
Total | 22,214 100.00repest SVY if cnt=="LSO"&wealth<999&hisced<9, estimate(reg pl_psb@_math girl wealth i.hisced) svyparm(NBpv(5) final_weight_name(fwgt) rep_weight_name(rwgt) NREP(135) variancefactor(1))C:\Users\cash\Downloads\ampl-guide
_pooled.....
: _pooled
------------------------------------------------------------------------------
| Coefficient Std. err. z P>|z| [95% conf. interval]
-------------+----------------------------------------------------------------
girl | .0623652 .0148212 4.21 0.000 .0333161 .0914143
wealth | .0312661 .0084031 3.72 0.000 .0147964 .0477358
_0b_hisced | 0 (omitted)
_1_hisced | .0115708 .0315911 0.37 0.714 -.0503466 .0734883
_2_hisced | .0617253 .0323853 1.91 0.057 -.0017487 .1251992
_3_hisced | .1139465 .0411526 2.77 0.006 .0332888 .1946041
_4_hisced | .2264876 .0426668 5.31 0.000 .1428622 .3101129
_5_hisced | .1849739 .0499686 3.70 0.000 .0870373 .2829105
_cons | .0913751 .0319593 2.86 0.004 .028736 .1540142
------------------------------------------------------------------------------After controlling for wealth and parental education, girls remain about
\(6.2\) percentage points more likely than boys to reach mathematics MPLb
(SE \(= 0.015\)). Wealth retains a positive association (\(0.031\) per unit;
SE \(= 0.008\)). Relative to parents who did not complete ISCED level 1,
higher parental education is associated with higher MPLb rates,
especially at ISCED levels 3–5 (coefficients from about \(0.11\) to
\(0.23\)). The lowest education contrast (ISCED 1) is not statistically
distinguishable from the reference group. Restricting to hisced < 9
drops missing/unknown parental education codes, which is important for
valid inference.