Bayesian Learning in Online Decision Making

Online decision-making under uncertainty raises a fundamental question about the relationship between learning and decision-making: when an algorithm begins interacting with an environment, should it rely only on its prior knowledge (e.g., training data), online samples, or combine both in a Bayesian way? Although ''be Bayesian'' is often good advice, across online problems, the value of prior knowledge versus online learning could vary dramatically, and in some settings, one of these sources of information is redundant for making the optimal decision.