-
H.S. Chang, M.C. Fu, J. Hu, and S.I. Marcus, "Nonstochastic Multi-Armed Bandit Approach to Stochastic Discrete Optimization," submitted for publication. (pdf)
-
H.S. Chang, M.C. Fu, J. Hu, and S.I. Marcus, "A Survey of Some Simulation-based Methods in Markov Decision Processes,” Communications in Information and System, 7, 2007, 59-92. (pdf)
- H.S. Chang, M.C. Fu, J. Hu, and S.I. Marcus, "Recursive Learning Automata Approach to Markov Decision Processes," IEEE Transactions on Automatic Control, 52, July 2007, 1349-1355. (earlier version )
- A. Rawat, H. La, M. Shayman, and S.I. Marcus, “Multicast Traffic Grooming in Unidirectional SONET/WDM Ring”, IEEE Journal on Selected Areas in Communications, 25, August 2007, 70-83. (pdf)
- H.S. Chang, M.C. Fu, J.Q. Hu, and S.I. Marcus, "An Asymptotically Efficient Simulation-Based Algorithm for Finite Horizon Stochastic Dynamic Programming," IEEE Transactions on Automatic Control, 52, January 2007, 89-94. (pdf)
- J. Hu, M.C. Fu, and S.I. Marcus, “A Model Reference Adaptive Search Algorithm for Stochastic Global Optimization," submitted for publication. (pdf)
- J. Hu, M.C. Fu, and S.I. Marcus, “A Model Reference Adaptive Search Algorithm for Global Optimization," Operations Research, 55, May-June 2007, 549-568. (pdf)
- J. Hu, M.C. Fu, V. Ramezani, and S.I. Marcus, “An Evolutionary Random Policy Search Algorithm for Solving Markov Decision Processes,” INFORMS Journal on Computing, 19, Spring 2007, 161-174. (pdf)
-
S.B. Laprise, M.C. Fu, S.I. Marcus, and A.E.B. Lim, “Pricing American-Style Derivatives with European Call Options,” Management Science, 52, January 2006, 95-110. (pdf)
-
H.S. Chang, H.G. Lee, M.C. Fu, and S.I. Marcus, "Evolutionary Policy Iteration for Solving Markov Decision Processes" IEEE Transactions on Automatic Control, 50, November 2005, 1804-1808. (pdf)
-
X. Yao, X. Xie, M.C. Fu, and S.I. Marcus, “Optimal Joint Preventive Maintenance and Production Policies,” Naval Research Logistics, 52, October 2005, 668-681. (earlier version)
-
V. Ramezani and S.I. Marcus, “Risk Sensitive Probability for Markov Chains,” Systems and Control Letters, 54, May 2005, 493-502. (pdf)
-
H.S. Chang, M.C. Fu, J. Hu, and S.I. Marcus, “An Adaptive Sampling Algorithm for Solving Markov Decision Processes,” Operations Research, 53, January-February 2005, 126-139. (earlier version available as TR 2002-19)
-
X. Yao, E. Fernández-Gaucherand, M.C. Fu, and S.I. Marcus, “Optimal Preventive Maintenance Scheduling in Semiconductor Manufacturing,” IEEE Transactions on Semiconductor Manufacturing, 17, August 2004, 345-356. (earlier version)
-
H.S. Chang and S.I. Marcus, "Two-Person Zero-Sum Markov Games: Receding Horizon Approach," IEEE Transactions on Automatic Control, 48, November 2003, 1951-1961. (earlier version)
-
H.S. Chang and S.I. Marcus, " Approximate Receding Horizon Approach for Markov Decision Processes: Average Reward Case," Journal of Mathematical Analysis and Applications, 286, October 2003, 636-651. (earlier version)
-
H.G. Lee, A. Arapostathis, and S.I. Marcus, “Linearization of Discrete-Time Systems via Restricted Dynamic Feedback,” IEEE Transactions on Automatic Control, 48, September 2003, 1646-1650. (earlier version)
-
Y. He, M. C. Fu, and S. I. Marcus, “Convergence of Simultaneous Perturbation Stochastic Approximation for Non-differentiable Optimization,” IEEE Transactions on Automatic Control, 48, August 2003, 1459-1463. (earlier version)
-
H.S. Chang, P. Fard, S.I. Marcus, and M. Shayman, “Multi-Scale Markov Decision Processes,” IEEE Transactions on Automatic Control, 48, June 2003, 976-987. (pdf)
-
S. Bhatnagar, M.C. Fu, S.I. Marcus, and I.-J. Wang, "Two-Timescale Simultaneous Perturbation Stochastic Approximation Using Deterministic Perturbation Sequences," ACM Transactions on Modeling and Computer Simulation, 13, April 2003, 180-209. (pdf)
-
V. Ramezani and S.I. Marcus, "Estimation of Hidden Markov Models: Risk-Sensitive Filter Banks and Qualitative Analysis of Their Sample Paths," IEEE Transactions on Automatic Control, 47, December 2002, 1999-2009. (pdf)
-
X.-R. Cao, Z. Ren, S. Bhatnagar, M.C. Fu, and S.I. Marcus, "A Time Aggregation Approach to Markov Decision Processes," Automatica, 38, 2002, 929-943. (pdf)
-
J. Razavilar, K.J.R. Liu, and S.I. Marcus, "Jointly Optimized Bit-Rate/Delay Control Policy for Wireless Packet Networks with Fading Channels," IEEE Transactions on Communications, 50, No. 3, March 2002, 484-494. (pdf)
- S.P. Coraluppi and S.I. Marcus, "Mixed Risk-Neutral/Minimax Control of Discrete-Time, Finite-State Markov Decision Processes,"IEEE Trans. Automatic Control, 45, June 2000, 528-532. (postscript)
- S.P. Coraluppi and S.I. Marcus, "Risk-Sensitive and Minimax Control of Discrete-Time, Finite-State Markov Decision Processes," Automatica, vol. 35, February 1999, 301-309. (postscript)
- D. Hernandez-Hernandez, S.I. Marcus, and P.J. Fard, "Analysis of a Risk Sensitive Control Problem for Hidden Markov Chains," IEEE Trans. Automatic Control, vol. 44, May 1999, 1093-1100. (postscript)
- D. Hernandez-Hernandez and S.I. Marcus, "Existence of Risk Sensitive Optimal Stationary Policies for Controlled Markov Processes," Applied Mathematics and Optimization, vol. 40, 1999, 273-285. (postscript)
- M.C. Fu, S.I. Marcus and I-J. Wang, "Monotone Optimal Policies for a Transient Queueing Staffing Problem," Operations Research, vol. 48, 2000, 327-331. (postscript)
- V.K. Garg, R. Kumar, and S.I. Marcus, "Probabilistic Language Formalism for Stochastic Discrete Event Systems," IEEE Trans. Automatic Control, vol. 44, February 1999, 280-293. (postscript)
- D. Hernandez-Hernandez and S.I. Marcus, "Corrigendum to Risk Sensitive Control of Markov Processes in Cou
ntable State Space," Systems and Control Letters, vol. 34, 1998, 105-106. (postscript)
- M.K. Ghosh, A. Arapostathis, and S.I. Marcus, "Ergodic Control of Switching
Diffusions," SIAM Journal on Control and Optimization, vol. 35, November 1997, 1952-1988. (postscript)
- E. Fernandez-Gaucherand and S.I. Marcus,
"Risk-Sensitive Optimal Control of Hidden
Markov Models: Structural Results," IEEE Trans. Automatic Control
, vol. 42, October 1997, 1418-1422.(postscript)
- R. Kumar, H.M. Cheung, and S.I. Marcus, "Extension Based Limited Lookahead
Supervision of Discrete Event Systems," Automatica, vol. 34, 1998, 1327-1344.(postscript)
- M.K. Ghosh and S.I. Marcus, "Stochastic Differential Games with Multiple Modes,
" Stochastic Analysis and Applications, vol. 16, 1998, 91-105.(postscript)
- R. Kumar, S. Nelgaval and S.I. Marcus, "A Discrete Event Systems Approach for Protocol Conversion," Discrete Event Dynamic Systems: Theory and Applications, vol. 7, 1997, 295-315. (postscript)
- D. Hernandez-Hernandez and S.I. Marcus, "Risk-Sensitive Control of Markov
Processes in Countable State Space," Systems and Control Letters, vol. 29, November 1996, 147-155. (postscript)
- R. Kumar, V.K. Garg, and S.I. Marcus, "Corrections to "Finite Buffer Realization
of Input-Output Discrete Event Systems," IEEE Trans. Automatic Control, vol. 41, April 1996, 625-627. (postscript)
- M.K. Ghosh, A. Arapostathis and S.I. Marcus, "A Note on an LQG Regulator with Markovian Switching
and Pathwise Average Cost," IEEE Trans. Automatic Control, vol. 40,
November 1995, 1919-1921.
- R. Kumar, V.K. Garg, and S.I. Marcus, "Finite Buffer Realization
of Input-Output Discrete Event Systems," IEEE Trans. Automatic Control,
vol. 40, June 1995, 1042-1053. (postscript)
- E. Fernandez-Gaucherand, M. K. Ghosh, and S.I. Marcus, "Controlled Markov
Processes on the Infinite Planning Horizon: Weighted
and Overtaking Cost Criteria," Zeit. Oper. Res., vol. 39, 1994, 131-155.
- R. Kumar, V.K. Garg, and S.I. Marcus, "Language Stability and Language Stabilization
of Discrete Event Systems," SIAM Journal on Control and Optimization
, vol. 31, September 1993, 1294-1320. (postscript)
- E. Fernandez-Gaucherand, A. Arapostathis, and S.I. Marcus, "Analysis of
an Adaptive Control Scheme for a Partially Observed Controlled Markov Chain,"
IEEE Trans. Automatic Control, vol. 38, June 1993,
987-993.
- R. Kumar, V.K. Garg, and S.I. Marcus, "Predicates and Predicate Transformers for
Supervisory Control of Discrete Event Dynamical Systems," IEEE Trans. Automatic
Control, vol. 38, February 1993, 232-247. (postscript
)
- R. Kumar, V.K. Garg, and S.I. Marcus, "On Supervisory Control of Sequential Behaviors
," IEEE Trans. Automatic
Control, vol. 37, December 1992, 1978-1985. (postscript
)
- E. Fernandez-Gaucherand, A. Arapostathis, and S.I. Marcus, "On the
Average Cost Optimality Equation and the Structure of Optimal Policies for
Partially Observable Markov Processes," Annals of Operations Research,
vol. 29, 1991, 471-512. (postscript)
- R. Kumar, V.K. Garg, and S.I. Marcus, "On Controllability and Normality of DEDS,
" Systems and Control Letters, vol. 17, 1991, 157-168. (postscript)
- R.D. Brandt, V.K. Garg, R. Kumar, F. Lin, S.I. Marcus, and W.M. Wonham, "Formulas
for Calculating Supremal Controllable and Normal Sublanguages," Systems and
Control Letters, vol. 15, 1990, 111-117. (
postscript)