Please use this identifier to cite or link to this item: https://hdl.handle.net/2440/73975
Citations
Scopus Web of Science® Altmetric
?
?
Type: Journal article
Title: Second-order Markov reward models driven by QBD processes
Author: Bean, N.
O'Reilly, M.
Ren, Y.
Citation: Performance Evaluation, 2012; 69(9):440-445
Publisher: Elsevier Science BV
Issue Date: 2012
ISSN: 0166-5316
1872-745X
Statement of
Responsibility: 
Nigel G. Bean, Małgorzata M. O’Reilly, Yong Ren
Abstract: Second-order reward models are an important class of models for evaluating the performance of real-life systems in which the reward measure fluctuates according to some underlying noise. These models consist of a Markov chain driving the evolution of the system, and a continuous reward variable representing its performance. Thus far, only models with a finite number of states have been studied. We consider second-order reward models driven by Quasi-birth-and-death processes, a class of block-structured Markov chains with infinitely many states. We derive the expressions for the Laplace-Stieltjes transforms of the accumulated reward and demonstrate how they can be efficiently evaluated. We use our results to analyse a simple example and, in doing so, show that the second-order feature can make a significant difference to the accumulated reward. The inclusion of the second-order feature also creates new difficulties which require the development of new conditions in the analysis. Crown Copyright © 2012 Published by Elsevier B.V. All rights reserved.
Keywords: Reward model
Quasi-birth-and-death (QBD) process
RG-factorization
Brownian motion
Rights: Crown copyright © 2012
DOI: 10.1016/j.peva.2012.05.002
Grant ID: http://purl.org/au-research/grants/arc/DP110101663
Published version: http://dx.doi.org/10.1016/j.peva.2012.05.002
Appears in Collections:Aurora harvest
Environment Institute publications
Mathematical Sciences publications

Files in This Item:
There are no files associated with this item.


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.