Optimal Policies for Quantum Markov Decision Processes

Citation: M. S. Ying, Y. Feng, S. G. Ying. Optimal policies for quantum markov decision processes. International Journal of Automation and Computing. http://doi.org/10.1007/s11633-021-1278-z doi:  10.1007/s11633-021-1278-z
 Citation: Citation: M. S. Ying, Y. Feng, S. G. Ying. Optimal policies for quantum markov decision processes. International Journal of Automation and Computing . http://doi.org/10.1007/s11633-021-1278-z

## Optimal Policies for Quantum Markov Decision Processes

###### Author Bio: Ming-Sheng Ying is a Distinguished Professor and Research Director of the Center for Quantum Software and Information at the University of Technology Sydney, Australia. He is also Deputy Director for Research (adjunct position) at the Institute of Software at the Chinese Academy of Sciences, and holds the Cheung Kong Chair Professorship at Tsinghua University, China. He has published books: Model Checking Quantum Systems: Principles and Algorithms (2021) (with Yuan Feng), Foundations of Quantum Programming (2016) and Topology in Process Calculus: Approximate Correctness and Infinite Evolution of Concurrent Programs (2001). He received a China National Science Award in Natural Science (2008). He has served on the editorial board of several publications including Artificial Intelligence. He is currently Editor-in-Chief of ACM Transactions on Quantum Computing. His research interests include quantum computation, theory of programming languages, and logics in AI. Email: mingsheng.ying@uts.edu.au (Corresponding author) ORCID iD: 0000-0003-4847-702X Yuan Feng received the B.Sc. degree in mathematics from Department of Applied Mathematics, Tsinghua University, China in 1999, and received the Ph. D. degree in computer science from Department of Computer Science and Technology, Tsinghua University, China in and 2004. He is currently a professor at Centre for Quantum Software and Information (QSI), University of Technology Sydney (UTS), Australia. His research interests include quantum programming theory, quantum information and quantum computation, and probabilistic systems. E-mail: yuan.feng@uts.edu.au ORCID iD: 0000-0002-3097-3896 Sheng-Gang Ying received the B. Sc. degree in physics from Department of Physics, Tsinghua University, China in 2010, and received the Ph. D. degree in computer science from Department of Computer Science and Technology, Tsinghua University, China in 2015, He is currently an associate researcher at State Key Laboratory of Computer Science, Institute of Software, Chinese Academy of Sciences, China. His research interests include quantum programming theory, quantum Markov systems. E-mail: yingsg@ios.ac.cn ORCID iD: 0000-0002-5052-5142
• Figure  1.  A quantum robot walking in a grid (with $n_h=3$ and $n_v=2$

•  [1] M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming, Hoboken, USA: John Wiley, 2005. [2] L. P. Kaelbling, M. L. Littman, A. R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, vol. 101, no. 1−2, pp. 99–134, 1998. DOI:  10.1016/S0004-3702(98)00023-X. [3] J. Barry, D. T. Barry, S. Aaronson. Quantum partially observable Markov decision processes. Physical Review A, vol. 90, no. 3, Article number 032311, 2014. DOI:  10.1103/PhysRevA.90.032311. [4] S. G. Ying, M. S. Ying. Reachability analysis of quantum Markov decision processes. Information and Computation, vol. 263, pp. 31–51, 2018. DOI:  10.1016/j.ic.2018.09.001. [5] M. S. Ying. Foundations of Quantum Programming, Amsterdam, Netherlands: Morgan Kaufmann, 2016. [6] M. S. Ying, N. K. Yu, Y. Feng, R. Y. Duan. Verification of quantum programs. Science of Computer Programming, vol. 78, no. 9, pp. 1679–1700, 2013. DOI:  10.1016/j.scico.2013.03.016. [7] J. Guan, Y. Feng, M. S. Ying. Decomposition of quantum Markov chains and its applications. Journal of Computer and System Sciences, vol. 95, pp. 55–68, 2018. DOI:  10.1016/j.jcss.2018.01.005. [8] M. S. Ying, Y. Feng. Model Checking Quantum Systems: Principles and Algorithms, Cambridge, USA: Cambridge University Press, 2021. [9] S. G. Ying, Y. Feng, N. K. Yu, M. S. Ying. Reachability probabilities of quantum Markov chains. In Proceedings of the 24th International Conference on Concurrency Theory, Springer, Buenos Aires, Argentina, pp.334−348, 2013. DOI:  10.1007/978-3-642-40184-8_24. [10] D. Powell. Quantum boost for artificial intelligence. Nature, to be published. [11] M. S. Ying. Quantum computation, quantum theory and AI). Artificial Intelligence, vol. 174, no. 2, pp. 162–176, 2010. DOI:  10.1016/j.artint.2009.11.009. [12] J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, S. Lloyd. Quantum machine learning. Nature, vol. 549, no. 7671, pp. 195–202, 2017. DOI:  10.1038/nature23474. [13] V. Dunjko, H. J. Briegel. Machine learning & artificial intelligence in the quantum domain: A review of recent progress. Reports on Progress in Physics, vol. 81, no. 7, Article number 074001, 2018. DOI:  10.1088/1361-6633/aab406. [14] S. D. Sarma, D. L. Deng, L. M. Duan. Machine learning meets quantum physics. Physics Today, vol. 72, no. 3, pp. 48–54, 2019. DOI:  10.1063/PT.3.4164. [15] L. P. Kaelbling, M. L. Littman, A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, vol. 4, pp. 237–285, 1996. DOI:  10.1613/jair.301. [16] R. S. Sutton, A. G. Barto. Reinforcement Learning: An Introduction, Cambridge, USA: MIT Press, 1998. [17] D. Y. Dong, C. L. Chen, Z. H. Chen. Quantum reinforcement learning. In Proceedings of the 1st International Conference on Advances in Natural Computation, Springer, Changsha, China, pp.686--689, 2005. [18] D. Y. Dong, C. L. Chen, H. X. Li, T. J. Tarn. Quantum reinforcement learning. IEEE Transactions on Systems,Man,and Cybernetics,Part B (Cybernetics), vol. 38, no. 5, pp. 1207–1220, 2008. DOI:  10.1109/TSMCB.2008.925743. [19] V. Dunjko, J. M. Taylor, H. J. Briegel. Quantum-enhanced machine learning. Physical Review Letters, vol. 117, no. 13, Article number 130501, 2016. DOI:  10.1103/PhysRevLett.117.130501. [20] V. Dunjko, J. M. Taylor, H. J. Briegel. Advances in quantum reinforcement learning. Proceedings of 2017 IEEE International Conference on Systems, Man, and Cybernetics, IEEE, Banff, Canada, pp.282−287, 2017. DOI:  10.1109/SMC.2017.8122616. [21] A. Ambainis, E. Bach, A. Nayak, A. Vishwanath, J. Watrous. One-dimensional quantum walks. In Proceedings of the 33rd ACM Symposium on Theory of Computing, ACM, Heraklion, Greece, pp.37−49, 2001. DOI:  10.1145/380752.380757. [22] P. Benioff. Some foundational aspects of quantum computers and quantum robots. Superlattices and Microstructures, vol. 23, no. 3-4, pp. 407–417, 1998. DOI:  10.1006/spmi.1997.0519. [23] P. Benioff. Quantum robots and environments. Physical Review A, vol. 58, no. 2, pp. 893–904, 1998. DOI:  10.1103/PhysRevA.58.893. [24] D. Y. Dong, C. L. Chen, C. B. Zhang, Z. H. Chen. Quantum robot: Structure, algorithms and applications. Robotica, vol. 24, no. 4, pp. 513–521, 2006. DOI:  10.1017/S0263574705002596. [25] M. Mundhenk, J. Goldsmith, C. Lusena, E. Allender. Complexity of finite-horizon Markov decision process problems. Journal of the ACM, vol. 47, no. 4, pp. 681–720, 2000. DOI:  10.1145/347476.347480. [26] C. H. Papadimitriou, J. N. Tsitsiklis. The complexity of Markov decision processes. Mathematics of Operations Research, vol. 12, no. 3, pp. 441–450, 1987. DOI:  10.1287/moor.12.3.441. [27] N. Ferns, P. S. Castro, D. Precup, P. Panangaden. Methods for computing state similarity in Markov decision processes. In Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence, AUAI, Cambridge, USA, pp.174−181, 2006. [28] N. Ferns, P. Panangaden, D. Precup. Metrics for Markov decision processes with infinite state spaces. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence,Edinburgh,Scotland, pp. 201–208, 2005.
•  [1] Ahmad Alqwadri, Mohammad Azzeh, Fadi Almasalha.  Application of Machine Learning for Online Reputation Systems . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1275-7 [2] Ying Li, De Xu.  Skill Learning for Robotic Insertion Based on One-shot Demonstration and Reinforcement Learning . International Journal of Automation and Computing, doi: 10.1007/s11633-021-1290-3 [3] Bao Xi, Rui Wang, Ying-Hao Cai, Tao Lu, Shuo Wang.  A Novel Heterogeneous Actor-Critic Algorithm with Recent Emphasizing Replay Memory . International Journal of Automation and Computing, doi: 10.1007/s11633-021-1296-x [4] Maryam Aljanabi, Mohammad Shkoukani, Mohammad Hijjawi.  Ground-level Ozone Prediction Using Machine Learning Techniques: A Case Study in Amman, Jordan . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1233-4 [5] Ziheng Chen, Hongshik Ahn.  Item Response Theory Based Ensemble in Machine Learning . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1239-y [6] Zhan Li, Sheng-Ri Xue, Xing-Hu Yu, Hui-Jun Gao.  Controller Optimization for Multirate Systems Based on Reinforcement Learning . International Journal of Automation and Computing, doi: 10.1007/s11633-020-1229-0 [7] G. Dharanibai, Anupama Chandrasekharan, Zachariah C. Alex.  Automated Segmentation of Left Ventricle Using Local and Global Intensity Based Active Contour and Dynamic Programming . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1112-4 [8] Meng-Yang Zhang, Guo-Hui Tian, Ci-Ci Li, Jing Gong.  Learning to Transform Service Instructions into Actions with Reinforcement Learning and Knowledge Base . International Journal of Automation and Computing, doi: 10.1007/s11633-018-1128-9 [9] Abhijit Gosavi, Anish Parulekar.  Solving Markov Decision Processes with Downside Risk Adjustment . International Journal of Automation and Computing, doi: 10.1007/s11633-016-1005-3 [10] De-Rong Liu, Hong-Liang, Li Ding Wang.  Feature Selection and Feature Learning for High-dimensional Batch Reinforcement Learning: A Survey . International Journal of Automation and Computing, doi: 10.1007/s11633-015-0893-y [11] Pavla Bromová, Petr Škoda, Jaroslav Vážný.  Classification of Spectra of Emission Line Stars Using Machine Learning Techniques . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0789-2 [12] Zhong-Liang Zhang, Jie Chen.  Implementable Strategy Research of Brake Energy Recovery Based on Dynamic Programming Algorithm for a Parallel Hydraulic Hybrid Bus . International Journal of Automation and Computing, doi: 10.1007/s11633-014-0787-4 [13] Li-Jie Zhao, Tian-You Chai, De-Cheng Yuan.  Selective Ensemble Extreme Learning Machine Modeling of Effluent Quality in Wastewater Treatment Plants . International Journal of Automation and Computing, doi: 10.1007/s11633-012-0688-3 [14] Appavu Alias Balamurugan Subramanian, S. Pramala, B. Rajalakshmi, Ramasamy Rajaram.  Improving Decision Tree Performance by Exception Handling . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0517-5 [15] Han Xue,  Xun Li,  Hong-Xu Ma.  Random Fuzzy Chance-constrained Programming Based on Adaptive Chaos Quantum Honey Bee Algorithm and Robustness Analysis . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0115-6 [16] Jiang Jiang,  Ying-Wu Chen,  Da-Wei Tang,  Yu-Wang Chen.  TOPSIS with Belief Structure for Group Belief Multiple Criteria Decision Making . International Journal of Automation and Computing, doi: 10.1007/s11633-010-0515-7 [17] Alma Lilia Garcia-Almanza,  Edward P. K. Tsang.  Evolving Decision Rules to Predict Investment Opportunities . International Journal of Automation and Computing, doi: 10.1007/s11633-008-0022-2 [18] W. U. Ahamed,  C. Kambhampati.  Stable Quantum Filters with Scattering Phenomena . International Journal of Automation and Computing, doi: 10.1007/s11633-008-0132-x [19] Eng Siong Chng, Sheng Chen.  Determining the Optimal Decision Delay Parameter for a Linear Equalizer . International Journal of Automation and Computing, doi: 10.1007/s11633-005-0020-6 [20] Jia Tian, Xun Chen, Sheng-Ping Dong.  Multi-Modal Reasoning Medical Diagnosis System Integrated With Probabilistic Reasoning . International Journal of Automation and Computing, doi: 10.1007/s11633-005-0134-x

##### 计量
• 文章访问数:  6
• HTML全文浏览量:  13
• PDF下载量:  5
• 被引次数: 0
##### 出版历程
• 收稿日期:  2020-07-22
• 录用日期:  2021-01-13
• 网络出版日期:  2021-03-20

## Optimal Policies for Quantum Markov Decision Processes

### English Abstract

Citation: M. S. Ying, Y. Feng, S. G. Ying. Optimal policies for quantum markov decision processes. International Journal of Automation and Computing. http://doi.org/10.1007/s11633-021-1278-z doi:  10.1007/s11633-021-1278-z
 Citation: Citation: M. S. Ying, Y. Feng, S. G. Ying. Optimal policies for quantum markov decision processes. International Journal of Automation and Computing . http://doi.org/10.1007/s11633-021-1278-z

/

• 分享
• 用微信扫码二维码

分享至好友和朋友圈