Corpus ID: 237439260

Three fundamental problems in risk modeling on big data: an information theory view

  title={Three fundamental problems in risk modeling on big data: an information theory view},
  author={Jiamin Yu},
Since Claude Shannon founded Information Theory, information theory has widely fostered other scientific fields, such as statistics, artificial intelligence, biology, behavioural science, neuroscience, economics, and finance. Unfortunately, actuarial science has hardly benefited from information theory. So far, only one actuarial paper on information theory can be searched by academic search engines. Undoubtedly, information and risk, both as Uncertainty, are constrained by entropy’s law. Today… Expand

Figures from this paper


Residual and Past Entropy in Actuarial Science and Survival Models
The best policy for an insurance company is that which lasts for a long period of time and is less uncertain with reference to its claims. In information theory, entropy is a measure of theExpand
Beyond Portfolio Theory: The Next Frontier
Logically, investment theory's next frontier is to put into practice the rich set of tools that academia has bestowed on the investment community—starting with Harry Markowitz's seminal article onExpand
When Can Social Media Lead Financial Markets?
This paper uses sentiment analysis techniques and Information Theory measures to demonstrate that social media message sentiment can contain statistically-significant ex-ante information on the future prices of the S&P500 index and a limited set of stocks, in excess of what is achievable using solely message volumes. Expand
Efficiency and Credit Ratings: A Permutation-Information-Theory Analysis
The role of credit rating agencies has been under severe scrutiny after the subprime crisis. In this paper we explore the relationship between credit ratings and informational efficiency of a sampleExpand
A bound on the financial value of information
The bound is shown to be a special case of the result that the increase in exponential growth of wealth achieved with true knowledge of the stock market distribution F over that achieved with incorrect knowledge G is bounded above by the entropy of F relative to G. Expand
Possible aggregation biases in road safety research and a mechanism approach to accident modeling.
  • G. Davis
  • Engineering, Medicine
  • Accident; analysis and prevention
  • 2004
Here, a simple deterministic model of a vehicle/pedestrian encounter is used to illustrate how naïvely applying statistical methods to aggregated data could lead to an ecological fallacy and to Simpson's paradox. Expand
Theoretical Impediments to Machine Learning With Seven Sparks from the Causal Revolution
  • J. Pearl
  • Computer Science, Mathematics
  • WSDM
  • 2018
This paper presents a summary of seven tasks which are beyond reach of current machine learning systems and which have been accomplished using the tools of causal inference, to demonstrate the essential role of such models. Expand
A Student's Guide to Coding and Information Theory
This easy-to-read guide provides a concise introduction to the engineering background of modern communication systems, from mobile phones to data compression and storage, using practical applications to illustrate theoretical definitions. Expand
A Car-Following Model Relating Reaction Times and Temporal Headways to Accident Frequency
This paper deals with a car-following model relating driver reaction time, temporal headway and deceleration response to accident frequency. The central goal is to assess the sensitivity of collisionExpand