. Combinaison-directe-de, Un poids par configuration 3. (+) Un poids par configuration et action 4. (+) Un poids par configuration et action -t correction 5, Un poids par configuration 6. (.) Un poids par configuration et action -+ correction

O. Buffet, A. Dutech, and F. Charpillet, Incrementai reinforcement learning for designing multi-agent systems, Proceedings of the fifth international conference on Autonomous agents (Agents'Ol), 2001.
DOI : 10.1145/375735.375826

URL : http://www.loria.fr/~buffet/papiers/agents01-2pages.ps.gz

A. Dutech, O. Buffet, and F. Charpillet, Multi-agent systems by incremental gradient reinforcement learning, Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI'Ol), 2001.
URL : https://hal.archives-ouvertes.fr/inria-00101090

O. Buffet, A. Dutech, and F. Charpillet, Learning to weigh basic behaviors in scalable agents, Proceedings of the first international joint conference on Autonomous agents and multiagent systems part 3, AAMAS '02, 2002.
DOI : 10.1145/545056.545111

URL : https://hal.archives-ouvertes.fr/inria-00100765

O. Buffet, A. Dutech, and F. Charpillet, Adaptive combination of behaviors in an agent, Proceedings of the 15th European Conference on Artificial Intelligence (ECAI'02), 2002.
URL : https://hal.archives-ouvertes.fr/inria-00100766

O. Buffet, A. Dutech, and F. Charpillet, Automatic generation of an agent's basic behaviors, Proceedings of the second international joint conference on Autonomous agents and multiagent systems , AAMAS '03, 2003.
DOI : 10.1145/860575.860716

O. Buffet and A. Dutech, Looking for scalable agents, Proceedings of the fifth European Workshop on Reinforcement Learning (EWRL-5), 2001.
URL : https://hal.archives-ouvertes.fr/inria-00100537

O. Buffet, Apprentissage par renforcement dans un système multi-agents, 2000.

O. Buffet, Apprentissage par renforcement pour la conception de systèmes multi-agents. [rapport d, Laboratoire lorrain de recherche en informatique et ses applications (LORIA), 2002.

. G. Arfken and . Arfken, The Method of Steepest Descents, chapitre 7, pp.428-436, 1985.

. G. Arfken and . Arfken, Successive Applications of \7, chapitre 1, pp.47-51, 1985.

. Asada, Purposive behavior acquisition for a real robot by visionbased reinforcement learning, Machine Learning, pp.279-303, 1996.

. Atkinson, Hilgard's Introduction to Psychology, 1996.

. Aylett, Multi-agent planning: Modelling execution agents, Papers of the 14th Workshop of the UK Planning and Scheduling Special Interest Group, 1995.

M. C. Baird, A. W. Baird, and . Moore, Gradient descent for general reinforcement learning, Advances in Neural Information Processing Systems 11, 1999.

]. L. Baird, Reinforcement Learning Through Gradient Descent, Thèse de doctorat 15213, 1999.

. Bartlett, ]. P. Baxter, J. Bartlett, and . Baxter, Hebbian synaptic modifications in spiking neurons that learn, 1999.

. Barto, Real-time learning and control using asynchronous dynamic programming, 1991.

]. T. Bass, Road to min, Discover, pp.56-61, 1992.

. Baxter, Experiments with infinite-horizon, policy-gradient estimation, Journal of Artificial Intelligence Research, vol.15, pp.351-381, 2001.

. Baxter, ]. J. Bartlett, P. Baxter, and . Bartlett, Infinite-horizon policy-gradient estimation, Journal of Artificial Intelligence Research, vol.15, pp.319-350, 2001.

. Bellman, Dynamic Programming, 1957.

. Bellman, An Introduction to Artificial Intelligence, 1978.

. Benda, On optimal cooperation of knowledge sources -and empirical investigation, 1986.

. Bibliographie and . Bernstein, The complexity of decentralized control of markov decision processes, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (UAI'OO), 2000.

. Bernstein, The Complexity of Decentralized Control of Markov Decision Processes, Mathematics of Operations Research, vol.27, issue.4, pp.819-840, 2002.
DOI : 10.1287/moor.27.4.819.297

]. D. Bertsekas-et-tsitsiklis, J. N. Bertsekas, and . Tsitsiklis, Parallel distributed computation, Numerical methods, 1989.

]. D. Bertsekas, Dynamic Programming : Deterministic and Stochastic Models, 1987.

. Birattari, Toward the formai foundation of ant programming, Ant Algorithms. Third International workshop, 2002.

]. B. Blumberg, Action-selection in hamsterdam : Lessons from ethology, Proceedings of the Third International Conference on Simulation of Adaptive Behavior (SAB'94), 1994.

. Bonabeau, Swarm Intelligence: From Natural to Artificial Systems, 1994.

. Bond and . Gasser, Bond et 1. Gasser. Readings in Distributed Artificial Intelligence, 1988.

. Bourjot, A new swarm mechanism based on social spiders colonies : from web weaving to region detection, Web Intelligence and Agent Systems : An International Journal -WIAS, 2003.
URL : https://hal.archives-ouvertes.fr/inria-00099635

]. C. Boutilier, Planning, learning and coordination in multiagent decision processes The Netherlands, Proceedings of the 6th Conference on Theoretical Aspects of Rationality and Knowledge (TARK '96), De Zeeuwse Stromen, 1996.

]. C. Boutilier, Sequential optimality and coordination in multiagent systems, Proceedings of the 16th International Joint Conference on Artificial Intelligence (IJCAI '99), 1999.

]. C. Brassac-et-pesty, S. Brassac, and . Pesty, Coopération dans les systèmes multi-agents, comportement ou conduite?, Proceedings of the first International Workshop on Decentralized Intelligent and MultiAgent Systems, 1995.

. Brooks, ]. Stein, . Ra, L. A. Brooks, and . Stein, Building brain for bodies, 1994.

]. Ra and . Brooks, A robust layered control system for a mobile robot, IEEE Journal of Robotics and Automation, vol.2, issue.1, pp.14-23, 1986.

]. Ra and . Brooks, A robot that walks; emergent behavior from a carefully evolved network, Neural Computation, vol.1, issue.2, pp.253-262, 1989.

. Brooks, . Ra, and . Brooks, Intelligence without reason, Proceedings of the Twelth International Joint Conference on Artificial Intelligence (IJCAI'91), 1991.

]. Ra and . Brooks, Intelligence without representation, Artificial Intelligence, vol.47, pp.139-159, 1991.

]. O. Buffet, Apprentissage par renforcement dans un système multi-agents, 2000.

]. A. Cassandra, Exact and Approximate Algorithms for Partially Observable Markov Decision Processes, Thèse de doctorat, 1998.

C. Et-mcdermott, ]. E. Charniak, and D. Mcdermott, Introduction to Artificial Intelligence, 1985.

]. W. Clancey, Situated Cognition, 1997.

. Colombetti, Robot shaping : The hamster experiment, Proceedings of the Sixth International Symposium on Robotics and Manufacturing (ISRAM'96), 1996.

C. Et-mahadevan, ]. J. Connell, and S. Mahadevan, Rapid Task Learning for Real Robots, Kluwer Academie publishers, 1993.

]. Y. Demazeau, Steps towards multi-agent oriented programming, First International Workshop on Multi-Agent Systems (IWMAS'97, 1997.

. Deneubourg, The self-organizing exploratory pattern of the argentine ant, Journal of Insect Behavior, vol.3, issue.2, pp.159-168, 1990.
DOI : 10.1093/aesa/79.2.283

]. T. Dietterich, Rierarchical reinforcement learning with the maxq value function decomposition, Journal of Artificial Intelligence Research, vol.13, pp.227-303, 2000.

. Dixon, Incorporating prior knowledge and previously learned information into reinforcement learning agents. Rapport technique, 2000.

. Dorigo, Positive feedback as a search strategy, 1991.

D. Et-colombetti, ]. M. Dorigo, and M. Colombetti, Book precis: Robot shaping : An experiment in behaviour engineering, Special issue on Complete Agent Learning in Complex Environments, pp.3-4391, 1997.

D. Dorigo, ]. M. Caro, G. D. Dorigo, and . Caro, Ant colony optimization : A new metaheuristic, Proceedings of the Congress on Evolutionary Computation, pp.6-9, 1999.

]. C. Bi-bliographie and . Drummond, Accelerating reinforcement learning by composing solutions of automatically identified subtasks, Journal of Artificial Intelligence Research, vol.16, pp.59-104, 2002.

. Dubois, Possibilistic logic, Handbook of Logic in Artificial Intelligence and Logic Programming, pp.439-513, 1994.
URL : https://hal.archives-ouvertes.fr/hal-01123493

]. A. Dury, Modélisation des interactions dans les systèmes multi-agents, Thèse de doctorat, 2000.

. Dutech, Multi-agent systems by incremental gradient reinforcement learning, Proceedings of the l1th International Joint Conference on Artificial Intelligence (IJCAI'Ol), 2001.
URL : https://hal.archives-ouvertes.fr/inria-00101090

. Dutech, ]. A. Samuelides, M. Dutech, and . Samuelides, Un algorithme d'apprentissage par renforcement pour les processus decisionnels de markov partiellement observes: apprendre une rextension selective du passé, 2003.

]. A. Dutech, Apprentissage d'environnement: approches cognitives et comportementales, Thèse de doctorat, 1999.

F. Et-müller, ]. J. Ferber, and J. Müller, Influences and reaction : a model of situated multiagent systems, Proceedings of the 2 nd International Conference on Multi-Agent Systems (ICMAS'96), 1996.

]. J. Ferber, Les Systèmes multi-agents. InterEditions, 1995.

]. I. Ferguson, TouringMachines : An Architecture for Dynamic, Rational, Mobile Agents, Thèse de doctorat, 1992.

]. M. Fox, An Organizational View of Distributed Systems, IEEE Transactions on Systems, Man, and Cybernetics, vol.11, issue.1, pp.70-80, 1981.
DOI : 10.1109/TSMC.1981.4308580

]. S. Gadanho-et-custodio, L. Gadanho, and . Custodio, Asynchronous learning by emotions and cognition, Proceedings of the Seventh International Conference on the Simulation of Adaptive Behavior (SAB '02), 2002.

]. Gardner, Mathematical games : The fantastic combinations of john conway's new solitaire game 'life', Scientific American, pp.120-123, 1970.

]. P. Grassé, La reconstruction du nid et les coordinations interindividuelles chezBellicositermes natalensis etCubitermes sp. la th??orie de la stigmergie: Essai d'interpr??tation du comportement des termites constructeurs, Insectes Sociaux, vol.4, issue.1, pp.41-81, 1959.
DOI : 10.1086/physzool.10.4.30151428

]. S. Harnad, The symbol grounding problem, Physica D: Nonlinear Phenomena, vol.42, issue.1-3, pp.335-346, 1990.
DOI : 10.1016/0167-2789(90)90087-6

]. J. Haugeland, Artificial Intelligence: The Very Idea, 1985.

H. Hayes-roth, A blackboard architecture for control, Artificial Intelligence, vol.3, issue.26, pp.251-321, 1985.

]. D. Hebb, The organisation of behavior, 1949.

]. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proceedings of the National Academy of Sciences, pp.2554-2558, 1982.

. Horvitz, Decision theory in expert systems and artificial intelligence, International Journal of Approximate Reasoning, vol.2, issue.3, pp.247-302, 1988.
DOI : 10.1016/0888-613X(88)90120-X

. Hu, . J. Wellman, M. Hu, and . Wellman, Multiagent reinforcement learning : theoretical framework and an algorithm, Proceedings of the 15th International Conference on Machine Learning (ICML'98), pp.242-250, 1998.

. Hu, . J. Wellman, M. Hu, and . Wellman, Online learning about other agents in a dynamic multiagent system, Proceedings of the second international conference on Autonomous agents , AGENTS '98, pp.239-246, 1998.
DOI : 10.1145/280765.280839

]. M. Humphrys, Action selection methods using reinforcement learning, From AnimaIs to Animats 4 : 4th International Conference on Simulation of Adaptive Behavior (SAB-96), 1996.

]. M. Humphrys, Action Selection methods using Reinforcement Learning, Thèse de doctorat, 1997.

. Jaakkola, On the convergence of stochastic iterative dynamic programming algorithms, Neural Computation, vol.6, issue.6, pp.1186-1201, 1994.

. Jaakkola, Reinforcement learning algorithm for partially observable markov decision problems, Advances in Neural Information Processing Systems, pp.345-352, 1994.

]. J. Jlilrgensen, Psykologi -paa biologisk Grundlag, 1962.

. Joslin, Tileworld users' manual, 1993.

. Kaelbling, Reinforcement learning : A survey, Journal of Artificial Intelligence Research, vol.4, pp.237-285, 1996.

]. R. Kurzweil, The Age of Intelligent Machines, 1990.

]. G. Laurent, Synthèse de comportements par apprentissages par renforcement parallèles, Thèse de doctorat, 2002.

]. Lin, Scaling Up Reinforcement Learning for Robot Control, Proceedings of the Tenth International Conference on Machine Learning (ICML '93), 1993.
DOI : 10.1016/B978-1-55860-307-3.50030-7

R. Littman, M. Sutton, S. Littman, R. Singh, and . Sutton, Predictive representations of state, Advances in Neural Information Processing Systems 14 (NIPS'Dl), 2001.

. M. Littman and . Littman, Markov games as a framework for multi-agent reinforcement learning, Proceedings of the l1th International Conference on Machine Learning (ICML '94), 1994.
DOI : 10.1016/B978-1-55860-335-6.50027-1

. M. Littman and . Littman, The witness algorithm : Solving partially observable markov decision processes. Rapport technique CS-94-40, 1994.

]. W. Lloyd, Two Lectures on the Checks to Population, p.1833, 1833.

. Bibliographie, . Loch, . Singh, S. Loch, and . Singh, Using eligibility traces to find the best memoryless policy in partially observable markov decision processes, Proceedings of the 15th Intemational Conference on Machine Leaming (ICML '98), 1998.

. Luger, ]. G. Stubblefield, W. A. Luger, and . Stubblefield, Artificial Intelligence: Structures and Strategies for Complex Problem Solving, 1993.

. Madani, On the undecidability of probabilistic planning and infinite-horizon partially observable markov decision problems, Sixteenth National Conference on Artificial Intelligence AAAI/IAAI, 1999.

]. P. Maes, How to do the Right Thing, Connection Science, vol.23, issue.2, pp.291-323, 1989.
DOI : 10.2307/1884852

. Sridhar-mahadevan, An average-reward reinforcement learning algorithm for computing bias-optimal policies, Proceedings of the 13th National Conference on Artificial Intelligence (AAAI '96), 1996.

]. E. Malville, L'auto-organisation de groupes pour l'allocation de taches dans les Systèmes Multi-Agents : Application à Corba, Thèse de doctorat, 1999.

J. Maja and . Matarié, Reward functions for accelerated learning, Proceedings of the Eleventh Intemational Conference on Machine Leaming, pp.181-189, 1994.

]. M. Matarié, Reinforcement learning in the multi-robot domain. A utonomous Robots, pp.73-83, 1997.

]. R. Mccallum, Reinforcement Leaming with Selective Perception and Hidden State, Thèse de doctorat, 1995.

. Mcgovern, Hierarchical optimal control of mdps, Proceedings of the 10th Yale Workshop on Adaptive and Leaming systems, 1998.

. Meuleau, Solving very large weakly coupled markov decision processes, AAAI/IAAI, pp.165-172, 1998.

. Meuleau, Learning fini testate controllers for partially observable environments, Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI'99), pp.427-436, 1999.

M. Et-dorigo, ]. N. Meuleau, and M. Dorigo, Ant colony optimization and stochastic gradient descent, Artificial Life, vol.8, issue.2, 2002.

]. M. Minsky, La Société de l, Esprit. InterEditions, 1986.

]. G. Monahan, State of the Art???A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms, Management Science, vol.28, issue.1, pp.1-16, 1982.
DOI : 10.1287/mnsc.28.1.1

]. A. Moore, Efficient memory-based leaming for robot control, Thèse de doctorat, 1990.

]. A. Moore, Variable Resolution Dynamic Programming: Efficiently Learning Action Maps in Multivariate Real-valued State-spaces, Proceedings of the 8th Intemational Conference on Machine Leaming (ICML'91, 1991.
DOI : 10.1016/B978-1-55860-200-7.50069-6

M. Et-pischel, ]. J. Müller, and M. Pischel, The agent architecture interrap : Concept and application. Rapport technique, DFKI Saarbrucken, 1111.

]. J. Müller, A. Müller, and . Rao, Control architectures for autonomous and interacting agents: A survey Wobcke, éditeurs, Intelligent Agents Systems: Theoretical and Practical Issues, Lecture Notes in A.I, 1997.

M. , M. Munos, and A. Moore, Variable resolution discretization in optimal control, Machine Learning, 2002.

]. R. Munos, ]. Neumann, and O. Morgenstern, Apprentissage par renforcement, étude du cas continu Theory of games and economic behavior, Thèse de doctorat , Ecole des Hautes études en Sciences Sociales, 1997. eumann et Morgenstern, 1967.

]. G. Owen, Game Theory : Second Edition, 1982.

. Peshkin, Learning policies with external memory, Proceedings of the Sixteenth International Conference on Machine Learning (ICML '99), pp.307-314, 1999.

. Peshkin, Learning to cooperate via policy search, Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, 2000.

]. R. Pfeifer, the role of morphology and materials in adaptive behavior, From AnimaIs to Animats 6 : Proceedings of the 6th International Conference on Simulation of Adaptive Behavior, 2000.

]. J. Piaget, La Psychologie de l'Intelligence, 1967.

. Precup, . D. Sutton, R. Precup, and . Sutton, Multi-time models for temporally abstract planning, Advances in Neural Information Processing Systems 10 (NIPS'97, p.97

]. M. Puterman, Markov Decision Processes-Discrete Stochastic Dynamic Programming, 1994.

]. J. Randll, /. Lv, P. Alstrl, and /. Lm, Learning to drive a bicycle using reinforcement learning and shaping, Proceedings of the 15th International Conference on Machine LearningICML-98), pp.463-471, 1998.

R. Et-wagner, ]. S. Rescorla, and A. R. Wagner, A theory of pavlovian conditioning : variations in effectiveness of reinforcement and nonreinforcement, Classical Conditioning, pp.64-99, 1972.

R. Et-knight, ]. E. Rich, and K. Knight, Artificial Intelligence, 1991.

]. J. Rosenblatt, DAMN: a distributed architecture for mobile navigation, Proceedings of the 1995 AAAI Spring Symposium on Lessons Learned ]rom Implemented Software Architectures for Physical Agents, pp.27-29, 1995.
DOI : 10.1109/TSMC.1973.5408575

R. Et-norvig, ]. S. Russell, and P. Norvig, Artificial Intelligence: A Modern Approach, 1995.

. Bibliographie, 1. M. Sahota, and . Sahota, Action selection for robots in dynamic environments through interbehaviour bidding, Proceedings of the Third International Conference on Simulation of Adaptive Behavior (SAB '94), 1994.

1. R. Schalkoff and . Schalkoff, Artificial Intelligence: An Engineering Approach, 1990.

2. B. Scherrer and . Scherrer, Apprentissage de représentation et auto-organisation modulaire pour un agent autonome, Thèse de doctorat, 2003.

1. L. Shapley and . Shapley, Stochastic Games, Proceedings of the National Academy of Sciences, vol.39, issue.10, pp.1095-1100, 1953.
DOI : 10.1073/pnas.39.10.1953

]. L. Shapley, Sorne topies in two person games, Annals of Mathematical Studies, vol.5, pp.1-28, 1964.

. Singh, Learning Without State-Estimation in Partially Observable Markovian Decision Processes, Proceedings of the llth International Conference on Machine Learning (ICML '94), 1994.
DOI : 10.1016/B978-1-55860-335-6.50042-8

1. S. Singh-et-cohn, D. Singh, and . Cohn, How to dynamically merge markov decision processes, Advances in Neural Information Processing Systems 10 (NIPS'98, 1998.

1. S. Singh and . Singh, Transfer of learning by composing solutions of elemental sequential tasks, Machine Learning, pp.323-339, 1992.

1. B. Skinner and . Skinner, The Behavior of Organisms : An Experimental Analysis, 1938.

]. B. Skinner, Science and Human Behavior, 1953.

S. Smallwood, E. J. Smallwood, and . Sondik, The Optimal Control of Partially Observable Markov Processes over a Finite Horizon, Operations Research, vol.21, issue.5, pp.1071-1088, 1973.
DOI : 10.1287/opre.21.5.1071

]. Smith, The contract net protocol : High level communication and control in a distributed problem solver, IEEE Transactions on Computers, vol.29, issue.12, pp.1104-1113, 1980.

]. J. Staddon, Adaptative Behavior and Learning, 1983.

]. P. Stone and M. Veloso, Multiagent systems: a survey from a machine learning perspective, Autonomous Robotics, vol.8, issue.3, 2000.
DOI : 10.21236/ADA333248

. Sutton, Between MDPs and Semi-MDPs : Learning , planning, and representing knowledge at multiple temporal scales, 1998.

B. Sutton, G. Sutton, and . Barto, Reinforcement Learning: An Introduction, IEEE Transactions on Neural Networks, vol.9, issue.5, 1998.
DOI : 10.1109/TNN.1998.712192

. Sutton, Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming, Proceedings of the 7th International Conference on Machine Learning (ICML'90), pp.216-224, 1990.
DOI : 10.1016/B978-1-55860-141-3.50030-4

]. G. Tesauro, Practical issues in temporal difference learning, Machine Learning, pp.257-277, 1992.

]. G. Tesauro, I. , T. J. Watson-research, and . Center, Td-gammon, aself-teaching backgammon program, achieves masterlevel play, 1994.
DOI : 10.1162/neco.1994.6.2.215

]. C. Tham-et-prager, R. Tham, and . Pragerthiébaux, A modular q-Iearning architecture for manipulator task decomposition Anytime state-based solution methods for decision processes with non-markovian rewards, Proceedings of the Eleventh International Conference on Machine Learning (ICML '94) Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence (UAI'02), pp.501-510, 1994.

. Thomas, MAS and RATS Multi-agent simulation of social differentiation in rats' groups, International Workshop on Self-Organization and Evolution of Social Behaviour, 2002.
URL : https://hal.archives-ouvertes.fr/inria-00100785

]. S. Thrun, Efficient exploration in reinforcement learning. Rapport technique CMU-CS-92-102, 1992.

T. Et-wolpert, ]. K. Tumer, and D. Wolpert, Collective intelligence and braess' paradox, Proceedings of the Sixteenth National Conference on Artificial Intelligence, 2000.

]. T. Tyrrell, Computational Mechanisms for Action Selection, Thèse de doctorat, 1993.

S. H. Vera, H. A. Vera, and . Simon, Situated Action: A Symbolic Interpretation, Cognitive Science, vol.2, issue.1, pp.7-48, 1993.
DOI : 10.1016/0004-3702(73)90011-8

]. C. Watkins, Learning from delayed rewards. Thèse de doctorat, King's College of Cambridge, 1989.

W. Et-sen, ]. G. Weif6, and S. Sen, Adaptation and learning in multi-agent systems, Lectures Notes in Artificial Intelligence, vol.1042, 1996.

. Weng, Autonomous mental development by robots and animals, Science Magazine, issue.5504, pp.291599-600, 2002.

]. J. Weng, A theory of mentally developing robots, Proceedings of the 2nd International Conference on Development and Learning (ICDL'02), 2002.

W. , B. Williams, and L. Baird, Tight performance bounds on greedy policies based on imperfect value functions, 1993.

]. Rj and . Williams, A class of gradient-estimating algorithms for reinforcement learning in neural networks, Proceedings of the lst International Conference on Neural Networks (ICNN'87), 1987.

]. Rj and . Williams, Simple statistical gradient-following algorithms for connectionnist reinforcement learning, Machine Learning, pp.229-256, 1992.

]. S. Wilson, The animat path to ai, From Animals to Animats : Proceedings of The First International Conference on Simulation of Adaptive Behavior, pp.15-21, 1991.