2022 Li_A novel text-based framework for forecasting agricultural futures using massive online news headlines

Briefing Document: Novel Text-Based Framework for Forecasting Agricultural Futures Using Massive Online News Headlines

Overview

This document summarizes the research presented in the article titled "A Novel Text-Based Framework for Forecasting Agricultural Futures Using Massive Online News Headlines" published in the International Journal of Forecasting. The study focuses on developing a framework that leverages online news headlines to improve the accuracy of agricultural futures price forecasting, specifically for soybean futures in the Chinese market.

Key Points

Introduction

• Agricultural futures prices are influenced by a complex array of factors, making accurate forecasting challenging.
• Traditional studies have often centered on improving forecasting models without adequately identifying the underlying factors affecting price fluctuations.
• The proposed framework aims to identify and quantify these factors through text mining and sentiment analysis of online news headlines.

Methodology

1. Text-Based Forecasting Framework (TBF):

- The TBF utilizes massive online news data to systematically extract influential factors affecting agricultural futures prices.

- Topic Modeling: A Dependency Parsing Sentence Latent Dirichlet Allocation (DP-Sent-LDA) model is employed to identify topics within news headlines, which correspond to influential factors.

- Sentiment Analysis: A Bi-directional Long Short-Term Memory (Bi-LSTM) model quantifies sentiment related to these factors, particularly those difficult to measure, such as weather and policy changes.

2. Data Collection:

- The dataset comprises 9,715 online news headlines from Hexun.com, covering the period from July 19, 2012, to July 9, 2018.

- Additional numerical data on soybean futures prices were sourced from the Choice database.

3. Influential Factors:

- The study identifies 20 influential factors categorized into eight groups, including market-related factors, policy changes, production-related factors, and external factors (e.g., weather).

4. Proxy Variables:

- A total of 49 proxy variables are established for the identified factors, consisting of 44 traditional market variables and five sentiment-based variables.

5. Forecasting Models:

- Three classic forecasting models are evaluated: Support Vector Regression (SVR), Random Forest (RF), and BackPropagation Neural Networks (BPNN).

- The performance of the proposed TBF is compared against the benchmark ARIMA model.

Results

• The proposed TBF demonstrates superior performance in medium-term and long-term forecasting compared to traditional ARIMA models.
• In short-term forecasting, ARIMA performs slightly better, but as the forecasting horizon increases, the TBF models show greater accuracy.
• Statistical tests (Diebold-Mariano and T-test) confirm the significance of the TBF's superior performance in medium- and long-term predictions.

Implications

• The findings suggest that the proposed TBF framework can effectively leverage online news data for improved forecasting accuracy, potentially aiding investors and policymakers in the agricultural sector.
• The study highlights the importance of identifying and quantifying influential factors in agricultural futures pricing, particularly those that are traditionally difficult to measure.

Limitations and Future Research

• The accuracy of influential factor identification is contingent on the quality of news data; certain factors may remain unreported in online headlines.
• Future research could explore the use of full-text articles rather than headlines to capture a broader range of influential factors.

Conclusion

The study successfully introduces a novel text-based framework that enhances the forecasting of agricultural futures prices by incorporating sentiment analysis and topic modeling of online news headlines. The empirical results underscore the framework's potential as a valuable tool for stakeholders in the agricultural market, particularly in navigating the complexities of price fluctuations. Further exploration of this methodology could lead to more comprehensive insights into agricultural futures forecasting.

Acknowledgments

The research was supported by grants from the National Natural Science Foundation of China and the Youth Innovation Promotion Association of the Chinese Academy of Sciences. The authors express gratitude to the editor and reviewers for their constructive feedback.