6. Validation Experiment
In this section, we test the external validity of our
empirical model of price sensitivity. We sent email newsletters about discounts
and rebates to randomly assigned consumers and we checked to see if the
classification determined by the Tobit model in the previous section was
associated with a higher response rate for emails. discount (as opposed to
non-discount) price-sensitive consumers (as opposed to price-insensitive). We
find that consumers whose model predicts that they will be price sensitive are
more receptive to messages that include a discount element. Experimental
Design
We include in this experience the entire mailing list of the
company, which has 246,688 consumers. A consumer signs up for the mailing list
by providing their email address to the business by creating an account,
signing up for updates, or requesting a coupon. Consumers were randomly
assigned to two groups, now Group 1 and Group 2, and each group received a
schedule of newsletters with and without discounts, as shown in Table 6.
18
The control bulletins sent on Sunday were not focused on
discounts and were identical from one group to another, whereas within each
day, only the discount and discount messages were different from one group to
another. For each newsletter sent, we observe if the email has been opened and
which link in the email, if any, has been clicked by the recipient. We also
observe all transactions on the website, which we can link to consumers
participating in the experience by their email address.
The product categories in electronic newsletters varied from
day to day but remained constant between control and treatment groups within a
few days. We have also ensured that all the creative elements of the
newsletters remain constant so that only the messages at a reduced price (for
example «up to -40%») and all the price information vary during
execution. This variation in messages was reflected in the subject of the
message.
Table 6: Schedule of newsletter treatments
|
Group 1 (50%)
|
Group 2 (50%)
|
|
Sunday
|
Control
|
Control
|
|
Monday
|
Discount
|
Full price
|
|
Tuesday
|
Discount
|
Full price
|
|
Wednesday
|
Discount
|
Full price
|
|
Thursday
|
Full price
|
Discount
|
|
Friday
|
Full price
|
Discount
|
The frequency with which customers on the mailing list choose
to receive newsletters varies. 63.34% of subscribers receive them every day,
4.32% three times a week, and 35.35% once a week. The schedule for the
newsletter presented in Table 6 was designed to obtain maximum variation among
consumers, regardless of their frequency, as well as to minimize the effects of
the day of the week.
To establish the validity of the classification on the Tobit
model, we generate predicted values from the model according to the purchase
history of each consumer before the experience of the newsletter. Our model can
be taken as an indication of price sensitivity if consumers we expect to be
more (rather than less) price-sensitive have a greater propensity to open and
click on discount emails. Since the final experience will compare the average
purchasing behavior of one group of consumers to another, price-sensitive, or
insensitive, the
19
classification model should be precise only at the group level
rather than at the level individual. Table 7:
Regressions on newsletter response variables
|
VARIABLES
|
(1)
open
|
(2)
click
|
(3)
click open
|
|
disc
|
-0.0149*
|
-0.00103
|
0.0188
|
|
(0.00811)
|
(0.00417)
|
(0.0181)
|
|
price_sensitivity
|
0.0402
|
0.0248
|
0.0868
|
|
(0.0297)
|
(0.0153)
|
(0.0671)
|
|
disc*price_sensitivity
|
0.125***
|
0.0505**
|
0.0728
|
|
(0.0420)
|
(0.0216)
|
(0.0933)
|
|
Constant
|
0.207***
|
0.0328***
|
0.152***
|
|
(0.00630)
|
(0.00324)
|
(0.0141)
|
|
|
|
|
|
day dummies
|
yes
|
yes
|
yes
|
|
|
|
|
|
Observations
|
106,534
|
106,534
|
22,043
|
|
R-squared
|
0.002
|
0.001
|
0.003
|
Standard errors in parentheses *** p<0.01, ** p<0.05, *
p<0.1
We regress the variables of the results of the bulletin
according to our variables of interest. Each record in the following
regressions is an email-client pair. The dependent variables are binary, where
success is either an open email or a clicked email. The independent variables
are an email discount dummy, the price sensitivity predicted from the empirical
model, the interaction between email discount and price sensitivity, and the
day of the week.
We find that our measure of price sensitivity is positively
correlated with the probability of responding to a discount email compared to
an email without discounts. Table 7 examines the relationship between price
sensitivity and three response variables: (1) if the email was opened, (2) if a
link in the email was clicked, and (3) if a link was clicked. provided the
email is opened. We find that price sensitivity is positively associated with
the first two measures, whose sample size was almost five times that of the
third measure.
The exogenous allocation of newsletters with and without
discounts provides us with an additional means of pre-classifying consumers.
Rather than relying on purchase history to infer price sensitivity, we can
project the probability of responding to a newsletter on consumer attributes
versus a newsletter without discount and thus have a means based on a model
for
20
predicting price sensitivity. Now that we have established the
validity of our price sensitivity classification approach, we can use it to
characterize how purchasing behaviors differ in online retail environments by
price sensitivity.
| 
