The systems deployed over cloud are subject to unpredictable workload conditions that vary from time to time, e.g. an ecommerce website may face higher workloads than normal during festivals or promotional schemes. In order to maintain the performance of such systems, an efficient elastic resource provisioning strategy is required. However, providing such a strategy that determines the right amount of cloud resources that fulfills the Quality of Service (QoS) demand is a challenging task. Over the period, many proposals have been introduced using techniques like threshold based rules, reinforcement learning and control theory, etc. The existing proposals, however, suffer from issues like lack of expertise to appropriately set the quantitative specification of thresholds, online training time overhead of the algorithm, too specific to work well in particular situation like when there is sudden burst in workload or work well in nominal conditions for stable workload, etc. Moreover, the existing approaches do not address uncertainty. Our proposed framework is a step forward to address the mentioned issues for systems that hold time varying workload conditions.