Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

This paper presents a policy-based framework to approach the issue of autonomous reconfiguration management in heterogeneous networks. In contrast to existing policy-based approaches, the proposed framework addresses the management issue from a new perspective through posing it as a problem of learning from current network behavior, while creating and updating policies dynamically in response to changing reconfiguration requirements, and this task is implemented by Reinforcement Learning methodology. A two-layer policy model is used to mapping users and operators' higher level goals into network level objectives. The autonomic reconfiguration procedures for policy creation, storage, evaluation are also presented in detail. Illustrative examples analysis and simulation results demonstrate the performance of the proposed work.

REFERENCES

• 1: M. Dillinger, K. Madami, and N. Alonistioti (Editors), Software Defined Radio: Architectures, Systems and Functions, John Wiley & Sons Ltd, 2003.
• 2: EU IST Project E²R II (End-to-End Reconfigurability -- Phase 2), http://www.e2r.motlabs.com/
• 3: Policy-Based Mobile Ad Hoc Network Management, Proceedings of the Fifth IEEE International Workshop on Policies for Distributed Systems and Networks, p.35, June 07-09, 2004
• 4: B. Moore , E. Ellesson , J. Strassner , A. Westerinen, Policy Core Information Model -- Version 1 Specification, RFC Editor, 2001
• 5: N. Samaan, A. Karmouch, "An Automated Policy-based Management Framework for Differentiated Communication Systems", IEEE Journal on Selected Areas in Communications, vol. 23, pp. 2236--2247, Dec. 2005.
• 6: You-Sun Hwang, Eung-bae Kim, "An architecture of SNMP-based network management of the broadband wireless access system", in Proceedings of the 9th Asia-Pacific Conference, vol.3, pp.1163--1166, Sept.2003
• 7: K. Yoshihara, M. Isomura, and H. Horiuchi, "Distributed Policy-Based Management Enabling Policy Adaptation", IEICE Trans. on Communications, vol. E87-B No.7, pp. 1854--1865, Jul. 2004.
• 8: H. Chaouchi, G. Pujolle, "A New Handover Control in the Current and Future Wireless Networks", IEICE Trans. on Communications, vol. E87-B No. 9, pp. 2537--2547, Sep. 2004.
• 9: A Goal-based Approach to Policy Refinement, Proceedings of the Fifth IEEE International Workshop on Policies for Distributed Systems and Networks, p.229, June 07-09, 2004
• 10: L. P. Kaelbling, M. L. Littman, X. Wang et al., "Reinforcement Learning: A Survey", Journal of Artificial Intelligence Research, vol. 4, pp. 237--285, May 1996.
• 11: J. Nie, S. Haykin, "A Q-learning-based Dynamic Channel Assignment Technique for Mobile Communication Systems", IEEE Trans. on Vehicular Technology, vol. 48, pp. 1676--1687, Sept. 1999.
• 12: S.-M. Senouci, G. Pujolle, "Dynamic Channel Assignment in Cellular Networks: A Reinforcement Learning Solution", 10th Int. Conf. on Telecomm., vol. 1, pp. 302--309, Mar. 2003.
• 13: N. Lilith, K. Dogancay, "Dynamic Channel Allocation for Mobile Cellular Traffic Using Reduced-state Reinforcement Learning", IEEE WCNC 2004, vol. 4, pp. 2195--2200, Mar. 2004.
• 14: Christopher J. C. H. Watkins , Peter Dayan, Technical Note: $\backslash cal\; Q$-Learning, Machine Learning, v.8 n.3-4, p.279-292, May 1992 [doi>10.1007/BF00992698]
• 15: Distributed Management Task Force, "CIM Policy Model, v.2.12," http://www.dmtf.org/standards/cim/cim_schema_v212/cimv212Final-Doc.zip, Apr. 20, 2006.
• 16: J. Vogler, et al., "Equipment Management and Control Architecture", July 2005, E2R White paper, available at http://e2r.motlabs.com/whitepapers/
• 17: J. Shin, J. W. Kim, and C. J. Kuo, "Quality-of-service mapping mechanism for packet video in differentiated services network," IEEE Trans. Multimedia, vol. 3, no. 2, pp. 219--231, Jun. 2001.
• 18: G. Ghinea, J. P. Thomas, and R. S. Fish, "Mapping quality of perception to quality of service: The case for a dynamically reconfigurable communication system," J. Intell. Syst., vol. 10, no. 5/6, pp. 607--632, 2000.
• 19: M. Alanyali , B. Hajek, On simple algorithms for dynamic load balancing, Proceedings of the Fourteenth Annual Joint Conference of the IEEE Computer and Communication Societies (Vol. 1)-Volume, p.230, April 02-06, 1995

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.1 Network Architecture and Design
          Subjects: Network communications

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.3 Network Operations
          Subjects: Network management

General Terms:
Design, Management, Performance

Keywords:
adaptive policies, autonomic communications, policy-based management, reconfigurable systems, reinforcement learning