Tristan Xiao

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2022-61

May 11, 2022

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-61.pdf

Forecasting future world events is a challenging but fruitful task, especially during times of uncertainty for better decision-making. We introduce a dataset of forecasting questions spanning various categories and topics and a large dataset of news curated from common-crawl. We show the effectiveness of larger models, better retrieval sources and techniques, and temporal architecture for long-range modeling. In order to better measure models’ performance and calibration on questions with numerical outputs, we also introduce another dataset full of numerical questions where we design a baseline algorithm to train models to output confidence intervals at specified confidence levels. With this dataset, we introduce a novel measure of calibration for numerical outputs based on adaptive binning RMS.

Advisors: Dawn Song

---

BibTeX citation:

@mastersthesis{Xiao:EECS-2022-61,
    Author= {Xiao, Tristan},
    Title= {Forecasting Future World Events with Neural Networks},
    School= {EECS Department, University of California, Berkeley},
    Year= {2022},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-61.html},
    Number= {UCB/EECS-2022-61},
    Abstract= {Forecasting future world events is a challenging but fruitful task, especially during times of uncertainty for better decision-making. We introduce a dataset of forecasting questions spanning various categories and topics and a large dataset of news curated from common-crawl. We show the effectiveness of larger models, better retrieval sources and techniques, and temporal architecture for long-range modeling. In order to better measure models’ performance and calibration on questions with numerical outputs, we also introduce another dataset full of numerical questions where we design a baseline algorithm to train models to output confidence intervals at specified confidence levels. With this dataset, we introduce a novel measure of calibration for numerical outputs based on adaptive binning RMS.},
}

EndNote citation:

%0 Thesis
%A Xiao, Tristan 
%T Forecasting Future World Events with Neural Networks
%I EECS Department, University of California, Berkeley
%D 2022
%8 May 11
%@ UCB/EECS-2022-61
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-61.html
%F Xiao:EECS-2022-61