where ${\mu }_t$ is the error term.

Before incorporating the indicator of attention, we propose a second model with the monthly devaluation rate as predictor. In this way, estimated expected inflation is conditioned on a larger information set. Let $\Delta {er}_t$ represent the monthly devaluation rate corresponding to month t. Then, the second baseline autoregressive model satisfies:

The predictive ability of social media content is evaluated through extended models that incorporate, as predictor, one of two specifications of the indicator of attention. The first specification is the original indicator described above. The second indicator can be interpreted as an adjusted metric of attention. More specifically, the second indicator is given by: ${\hat{\text{I}}}_t={\text{I}}_t-\frac{{\sum }_{k=1}^{12}{\text{I}}_{t-1}}{12}$. That is, the indicator measures differences between the latest value of the indicator of attention and its average level during the previous year. This alternative specification allows for robustness tests. At the same time, it permits a simple exploration of the appropriateness of alternative specifications. Formally, the forecasting models used to estimate the information content of social media indices are given by the following equation:

where ${\mathit{\text{A}}}_t$ is equal to ${\mathit{\text{I}}}_t$ or ${\widehat{\mathit{\text{I}}}}_t$ and in some model specifications ${\beta }_{er}$ is set equal to 0. The parameter of interest is ${\beta }_{\mathit{\text{I}}}$. ${\mu }_t$ is the error term. Models are estimated for the period 2012-2019. The predictors are standardized to facilitate the comparison of the economic significance of different estimated parameters.

Table 2 shows the estimations of the different specifications of the forecasting models. The baseline models indicate that lagged monthly inflation and lagged devaluation are statistically and economically significant predictors of inflation. Adjusted R²’s suggest that these variables contain substantive information regarding subsequent levels of inflation.¹¹

The estimated extended models indicate that social media content adds information regarding future inflation levels. A one standard deviation increment in the indicator of attention anticipates a mean increment of approximately 0.4% in monthly inflation. Column 1 in Table A.1 shows that the estimated coefficient associated to inflation attention index is similar when we extend the list of terms related to inflation term (i.e., “dollar”, “prices”, “wage”).

Table 2

Forecasting models

Source: own elaboration.Note: standard errors in parenthesis are estimated following Newey and West (1987, 1994). *p<0.1; **p<0.05; ***p<0.01.

On the other hand, it is worth noting that these observations do not depend on the baseline model under consideration or the specification of the social media content indicator. Appendix A evidences that these regularities remain when we incorporate lags on devaluation rate to control for delays in the pass-through from exchange rate to prices. Adjusted R²’s point to noticeable gains in anticipatory ability. For example, in the first baseline model, the adjusted R² increases from 0.427 to more than 0.50 as summaries of social media content are incorporated as predictors.

Having established that social media provides valuable information regarding future levels of inflation, we present additional analyses that characterize this information in more detail. First, we estimate a nonlinear model to explore the value of alternative specifications. Then, we evaluate if the information provided by the attention index overlaps with the information provided by traditional macroeconomic indicators such as interest rates, monetary aggregates or economic activity.

Are increments in attention as informative as drops in attention? To provide an answer to this question we use the adjusted metric of attention, ${\widehat{\mathit{\text{I}}}}_t$, to compute a second indicator of attention that is equal to the adjusted metric of attention if positive and zero otherwise: ${\hat{\text{I}}}_t^{+}=\max \{0,{\hat{\text{I}}}_t\}$ . Then we estimate forecast models in which this new indicator is used as a predictor. In Table 3 the estimated models suggest

Table 3

Nonlinear models

Source: own elaboration.Note: tandard errors are estimated following Newey and West (1987, 1994). *p<0.1; **p<0.05; ***p<0.01.

that increments in attention are particularly informative. More specifically, when the indicator of increments in attention, ${\hat{\text{I}}}_t^{+}$, is used as the predictor, the estimated coefficients and the metrics of model fit increase. Additionally, when the adjusted metric, ${\widehat{\mathit{\text{I}}}}_t$, is incorporated as a second indicator, its coefficient is not significantly different from zero. While our ability to identify the best forecast model is constrained by sample size, these evaluations suggest that there are gains associated to considering more flexible specifications.

Is the information provided by social media content different from the information provided by traditional indicators? We consider three traditional indicators to address this question: economic activity, interest rates, and monetary aggregates. The monthly indicator of economic activity, ${ea}_t$ , is from the national statistics office.¹² The monetary aggregate, $m_t$ , is the monetary base.¹³ Interest rates, ${ir}_t$ , corresponds to 30 through 59-day term deposits.¹⁴

Table 4

Attention indices vs. traditional variables

Source: own elaboration.Note: standard errors are estimated following Newey and West (1987, 1994). *p<0.1; **p<0.05; ***p<0.01.

Table 4 shows the estimated models that incorporate these traditional variables as predictors after standardization. The estimated coefficients associated to these variables have the expected sign. Nevertheless, statistical significance is consistently observed only in the case of interest rates. Importantly, for all model specifications, the coefficient corresponding to the indicator of Twitter content is positive and significant. Additionally, the estimated coefficient remains mostly unaltered as different macroeconomic variables are incorporated to the model. These results suggest that the attention index provides valuable information that is different from the information provided by traditional economic indicators.

Summarizing, the evidence reported above suggests that indices based on social media content have valuable information regarding future levels of inflation. These findings are robust to changes in the set of predictors and the specification of the indicator. Also, the information provided by the attention index is different from that provided by traditional macroeconomic indicators. Appendix B evidences a more distant association between social media content and subsequent levels of inflation rate.

III.1 Out-of-sample forecast

To provide further insights on the information content of social media we implement out of sample forecasts exercises in which models are trained recursively with past information. The performance of forecast generated by the baseline autoregressive model are compared to forecasts produced by models that incorporate an additional predictor. Four predictors are considered: monthly devaluation rate ( $\Delta {er}_t$ ), the inflation attention index ( ${\text{I}}_t$ ), the adjusted level of attention (${\hat{\text{I}}}_t$) and increments in attention (${\widehat{\mathit{\text{I}}}}_t^{+}$) .

Each forecast model is evaluated computing the root-mean-square prediction error (RMSE). For extended models these measure of accuracy is also expressed as a fraction of the RMSE of the baseline model. The performance of the models is assessed using two alternative starting dates for pseudo out-of-sample forecast exercise. The starting dates are selected so that the smallest training subsample represents 60% and 80% of the full sample respectively. Following Faust et al. (2013), resampling techniques are implemented to compute the statistical significance of the differences in accuracy.¹⁵

Table 5 shows the results for out-of-sample forecast exercises. For all extended models, the estimated forecast accuracy is higher than that observed in the case of the baseline model. These differences are statistically significant in all but one case. For forecasts generated by a single predictor, the strongest performance is observed in the case of the original inflation attention index $\left({\mathit{\text{I}}}_t\right)$ . The last row of the table shows that forecast combinations allow for further gains in accuracy. In summary, these out-of-sample forecast exercises provide further support to the idea that social media content provides valuable information regarding future levels of inflation.

Table 5

Out-of-sample forecasts

Source: own elaboration.

IV. Inflation uncertainty

Beyond inflation forecasts, social media can be conjectured to provide information regarding inflation uncertainty. This is a question of interest since inflation uncertainty has important economic consequences (Huizinga, 1993, Heymann and Leijonhufvud, 1995 and Elder, 2004). Our metric seems adequate for this task since higher inflation uncertainty is likely accompanied by a higher level of attention that can be inferred from social media messages. More specifically, under rational inattention, the allocation of attention is adjusted with the value of incoming information (Sims, 2003; Mackowiak and Wiederholt, 2009). Hence, the allocation of more attention to inflation is consistent with the arrival of valuable news regarding the future evolution of inflation. Complementarily, considering the literature on multiple equilibria or sunspots (Benhabib and Farmer, 1999; Ascari et al., 2019), more public discussion of the inflation might be linked to innovations in the coordination of price setting behavior.

Our first evaluation involves estimating quantile regressions that characterize the distribution of inflation shocks as a function of three variables of interest. Formally, shocks are approximated by residuals of an autoregressive model:

Then, quantile $\tau ∊[0,1]$ of the shock ${\hat{e}}_{t+1}$ conditional on the value of indicator $x_t$ is modeled as an affine function:

where the $x_t$ can be one of three indicators: inflation $\left({\mathit{\Delta }cpi}_t\right)$, devaluation rate $\left({\mathit{\Delta }er}_t\right)$ , or the adjusted indicator of inflation attention $\left({\widehat{\mathit{\text{I}}}}_t\right)$.

Figure 2 and Table 6 provide information on the estimated quantiles. The estimations suggest that the attention index is associated to important changes in uncertainty. In particular, the interquartile range of the shock is estimated to increase from 0.47 to 0.83 as the attention index increases from one standard deviation below the mean to one standard deviation above the mean. This strong association between the attention index and uncertainty, proxied by forecast errors, are not observed when quantiles are estimated as a function of lagged inflation or lagged devaluation.

Figure 2

Quantile regressions

Source: own elaboration.Note: Given $\tau \in \{0,1\}$ quantile ${\text{Q}}_{{\hat{e}}_{t+1}|x_t}\left(\tau \right)$ is modeled as ${\mathit{\text{Q}}}_{{\hat{e}}_{t+1}|x_t}\left(\tau \right)={\alpha }_{\tau }+{\beta }_{\tau }{x}_t$. Estimated quantiles are conditioned on $x_t$ with $x_t\in \{{\Delta cpi}_t,{\Delta er}_t,{\hat{\text{I}}}_t\}$. To facilitate comparisons, in each case, the indicator on which quantiles are conditioned, $x_t$, was standardized.

To complement the previous results, we evaluate the association between inflation uncertainty and social media attention estimating simple forecasting models. Following Rossi and Sekhposyan (2015), inflation uncertainty is proxied using the mean absolute forecast error: ${iu}_t=\left|{\hat{e}}_{t+1}\right|$. Then, we propose simple empirical models in which uncertainty is function of indicators $x_t\in \{{\Delta cpi}_t,{\Delta er}_t,{\hat{\text{I}}}_t\}$. Formally, when all indicators are incorporated, the inflation uncertainty forecast model is given by:

Table 6

Inflation uncertainty forecasts

Source: own elaboration.Note: standard errors are estimated following Newey and West (1987, 1994). *p<0.1; **p<0.05; ***p<0.01.

Table 6 confirms that the attention index provides valuable information regarding inflation uncertainty. A one standard deviation increment in the attention index is associated to a 30% increment in the expected mean absolute error. In contrast, no robust association is found when the other two indicators are considered. In the case of the devaluation rate, no statistically significant relationship is found. In the case of lagged inflation, a positive relationship is found in the case of the univariate model. This association loses statistical significance once the attention index is incorporated as a predictor in the model.

V. Evaluation of other subjective indicators

The previous analyses show that social media content provides valuable information regarding future levels of inflation and inflation uncertainty. One open question is whether other indicators of subjective states display similar capacity to provide information regarding future inflation. In this subsection, we estimate inflation forecast models that allow for a comparison of the information content of multiple alternative indicators of subjective states.

Three types of indicators are considered. First, we evaluate search intensity indices from Google Trends. We consider two keywords: “inflacion” (inflation) and “dolar” (dollar). The selection of “dolar” as a keyword responds to the strong association between the exchange rate and the rate of inflation. Also, we consider two indicators of media attention. One index is constructed processing the full text of the economic section of “La Nación” a prominent Argentine newspaper¹⁶. The second mass media index is constructed processing a collection of 1.4M tweets from 6 major news outlets: “ambitocom”, “clarincom”, “cronistacom”, “infobae”, “lanacion” and “perfilcom”. Finally, we also use the index of household inflation expectations from CFI-UTDT. More specifically, in the absence of an indicator of expected inflation for the next month, we use the median expected inflation over the next 12 months.

As in the previous exercises, we estimate simple autoregressive models that incorporate an additional predictor. Table 7 shows the estimated forecasting models for the alternative subjective indicators. To facilitate comparisons, each subjective indicator was standardized. Interestingly, mass media content and mass media tweets fail to provide information regarding future inflation. These results suggest that, in the case under analysis, mass media might not play a central or leading role in the formation of macroeconomic expectations. On other hand this is a tentative statement since we must consider the impact of difference in the size and coverage of each corpus and the sensitivity to adjustments in the methodology used to extract information from each corpus.¹⁷Hence, a more comprehensive analysis is needed to arrive to more informative evaluation of the centrality of each type of source of information.

Negative results are also observed in the case of household surveys. While preliminary, this evidence points to limits in our capacity to extract subjective information through questionnaires. In contrast, search intensity indicators are shown to contain valuable information. In particular, a strong performance is observed in the case of the indicator of search intensity for the keyword “dollar”. In consistence with the previous sections, the evidence linked to search intensity suggest that indicators of online behavior of a large quantity of users constitutes a valuable tool to extract forward-looking macroeconomic information.

Table C.1, in Appendix C, evidences that social media content captures valuable information regarding future inflation rate even when we simultaneity control by google intensity search indexes and traditional macroeconomic indicators. This result remains when we add household inflation expectations as control.

VI. Conclusions

This paper examines the information content of social media messages. More specifically, we analyze “Argentine” tweets for the period 2012-2019. The evidence indicates that Twitter messages provide valuable information regarding expected inflation and inflation uncertainty. The information content is economically significant. An increment of one standard deviation in the index is associated with by an increment of 0.4% in inflation in the following month. The information content of the index is different from that provided by traditional macroeconomic indicators. These findings are robust to changes in the specification of the forecast exercise.

There are several directions in which these exercises can be extended. First, in this work, attention to inflation was approximated using an extremely simple but transparent strategy to summarize unstructured information. The use of natural language processing models could allow for gains in the capacity to extract information from Twitter messages. In a similar direction, models of community detection could be used to discover clusters of users whose content is particularly informative. Finally, this study evaluated regularities using monthly time series. Analyses at higher frequencies can provide further insights regarding the relationship between social media content and inflation dynamics.