Unit1 Notes CAT1

1.1 Introduction to AI
1.1.1 What is artificial intelligence?
Artificial Intelligence is the branch of computer science concerned with making computers behave like humans. Major AI textbooks define artificial intelligence as "the study and design of intelligent agents," where an intelligent agent is a system that perceives its environment and takes actions which maximi e its chances of success. John McCarthy, who coined the term in !"#$, defines it as "the science and engineering of making intelligent machines,especially intelligent computer programs." %he definitions of AI according to some text books are categori ed into four approaches and are summari ed in the table below & Systems that think like humans '%he exciting new effort to make computers think ( machines with minds,in the full and literal sense.)*+augeland,!",#Systems that act like humans %he art of creating machines that perform functions that re0uire intelligence when performed by people.)*1ur weil,!""2%he four approaches in more detail are as follows & Systems that think rationally '%he study of mental faculties through the use of computer models.) *.harniak and Mc/ermont,!",#Systems that act rationally .omputational intelligence is the study of the design of intelligent agents.)*3oole et al.,!"",-

!a" Acting humanly # $he $uring $est approach
o %est proposed by Alan %uring in !"#2 o %he computer is asked 0uestions by a human interrogator. %he computer passes the test if a human interrogator,after posing some written 0uestions,cannot tell whether the written responses come from a person or not. 3rogramming a computer to pass ,the computer need to possess the following capabilities &  %atural language processing to enable it to communicate successfully in 4nglish.  &no'ledge representation to store what it knows or hears  Automated reasoning to use the stored information to answer 0uestions and to draw new conclusions.  Machine learning to adapt to new circumstances and to detect and extrapolate patterns %o pass the complete %uring %est,the computer will need  Computer vision to perceive the objects,and  (o)otics to manipulate objects and move about.

!)"$hinking humanly # $he cognitive modeling approach 5e need to get inside actual working of the human mind & *a- through introspection 6 trying to capture our own thoughts as they go by7 *b- through psychological experiments Allen 8ewell and +erbert 9imon,who developed *+S,the '*eneral +ro)lem Solver) tried to trace the reasoning steps to traces of human subjects solving the same problems. %he interdisciplinary field of cognitive science brings together computer models from AI and experimental techni0ues from psychology to try to construct precise and testable theories of the workings of the human mind

!c" $hinking rationally # $he la's of thought approach,
%he :reek philosopher Aristotle was one of the first to attempt to codify 'right thinking),that is irrefuatable reasoning processes. +is syllogism provided patterns for argument structures that always yielded correct conclusions when given correct premises;for example,)9ocrates is a man7all men are mortal7therefore 9ocrates is mortal.). %hese laws of thought were supposed to govern the operation of the mind7their study initiated a field called logic.

!d"Acting rationally # $he rational agent approach
An agent is something that acts. .omputer agents are not mere programs ,but they are expected to have the following attributes also & *a- operating under autonomous control, *b- perceiving their environment, *c- persisting over a prolonged time period, *e- adapting to change. A rational agent is one that acts so as to achieve the best outcome.

1.1.- $he foundations of Artificial Intelligence
%he various disciplines that contributed ideas,viewpoints,and techni0ues to AI are given below & +hilosophy!.-/ 0.C. 1 present" Aristotle *<,=><?? @...- was the first to formulate a precise set of laws governing the rational part of the mind. +e developed an informal system of syllogisms for proper reasoning,which allowed one to generate conclusions mechanically,given initial premises. .omputer +uman @rain ! .3B,!2, gates !2!! neurons !2!2 bits CAM !2!! neurons !! !2 bits disk !2!= synapses >" .ycle time !2 sec !2>< sec @andwidth !2!2 bitsAsec !2!= bitsAsec " Memory updatesAsec !2 !2!= $a)le 1.1 A crude comparison of the raw computational resources available to computers*circa ?22< - and brain. %he computerDs numbers have increased by at least by a factor of !2 every few years. %he brainDs numbers have not changed for the last !2,222 years. .omputational units 9torage units @rains and digital computers perform 0uite different tasks and have different properties. %ablere !.! shows that there are !2222 times more neurons in the typical human brain than there are gates in the .3B of a typical high>end computer. MooreDs Eaw predicts that the .3BDs gate count will e0ual the brainDs neuron count around ?2?2. +sycology!1/23 1 present"

%he origin of scientific psychology are traced back to the wok if :erman physiologist +ermann von +elmholt *!,?!>!,"=- and his student 5ilhelm 5undt*!,<? 6 !"?2In !,F",5undt opened the first laboratory of experimental psychology at the university of Eeip ig. In B9,the development of computer modeling led to the creation of the field of cognitive science. %he field can be said to have started at the workshop in 9eptember !"#$ at MI%.

Computer engineering !13.45present"
Gor artificial intelligence to succeed, we need two things& intelligence and an artifact. %he computer has been the artifact of choice. A1 also owes a debt to the software side of computer science, which has supplied the operating systems, programming languages, and tools needed to write modern programs

Control theory and Cy)ernetics !13./5present"
1tesibios of Alexandria *c. ?#2 @.c.- built the first self>controlling machine& a water clock with a regulator that kept the flow of water running through it at a constant, predictable pace. Modern control theory, especially the branch known as stochastic optimal control, has as its goal the design of systems that maximi e an o)6ective function over time.

7inguistics !13825present"
Modem linguistics and AI, then, were "born" at about the same time, and grew up together, intersecting in a hybrid field called computational linguistics or natural language processing.

1.1.9 $he :istory of Artificial Intelligence
$he gestation of artificial intelligence !13.951388"
%here were a number of early examples of work that can be characteri ed as AI, but it was Alan %uring who first articulated a complete vision of A! in his !"#2 article ".omput> ing Machinery and Intelligence." %herein, he introduced the %uring test, machine learning, genetic algorithms, and reinforcement learning.

$he )irth of artificial intelligence !138;"
Mc.arthy convinced Minsky, .laude 9hannon, and 8athaniel Cochester to help him bring together B.9. researchers interested in automata theory, neural nets, and the study of intelligence. %hey organi ed a two>month workshop at /artmouth in the summer of !"#$. 3erhaps the longest>lasting thing to come out of the workshop was an agreement to adopt Mc.arthyHs new name for the field& artificial intelligence.

<arly enthusiasm= great e>pectations !138-513;3"
%he early years of A! were full of successes>in a limited way.

*eneral +ro)lem Solver **+S- was a computer program created in !"#F by +erbert 9imon and Allen 8ewell to build a universal problem solver machine. %he order in which the program considered
subgoals and possible actions was similar to that in which humans approached the same problems. %hus, :39 was probably the first program to embody the "thinking humanly" approach. At I@M, 8athaniel Cochester and his colleagues produced some of the first A! pro> grams. +erbert :elernter *!"#"- constructed the :eometry %heorem 3rover, which was able to prove theorems that many students of mathematics would find 0uite tricky.

Eisp was invented by Iohn Mc.arthy in !"#, while he was at the Massachusetts Institute of %echnology *MI%-. In !"$<, Mc.arthy started the AI lab at 9tanford.
%om 4vansHs A8AEJ:K program *!"$,- solved geometric analogy problems that appear in IL tests, such as the one in Gigure !.!

?igure 1.1 %he %om 4vanDs A8AEJ:K program could solve geometric analogy problems as shown.

A dose of reality !13;;51329"
Grom the beginning, AI researchers were not shy about making predictions of their coming successes. %he following statement by +erbert 9imon in !"#F is often 0uoted&

'It is not my aim to surprise or shock you>but the simplest way I can summari e is to say that there are now in the world machines that think, that learn and that create. Moreover, their ability to do these things is going to increase rapidly until>in a visible future>the range of problems they can handle will be coextensive with the range to which the human mind has been applied. &no'ledge5)ased systems# $he key to po'er? !13;351323" @endral was an influential pioneer project in artificial intelligence *AI- of the !"$2s, and the computer software e>pert system that it produced. Its primary aim was to help organic chemists in identifying unknown organic molecules, by analy ing their mass spectra and using knowledge of chemistry. It was done at 9tanford Bniversity by 4dward Geigenbaum, @ruce @uchanan, Ioshua Eederberg, and .arl /jerassi.

A1 )ecomes an industry !13/45present"
In !",!, the Iapanese announced the "Gifth :eneration" project, a !2>year plan to build intelligent computers running 3rolog. Jverall, the A! industry boomed from a few million dollars in !",2 to billions of dollars in !",,.

$he return of neural net'orks !13/;5present"
3sychologists including /avid Cumelhart and :eoff +inton continued the study of neural>net models of memory.

A1 )ecomes a science !13/25present"
In recent years, approaches based on hidden Markov models *+MMs- have come to dominate the area. 9peech technology and the related field of handwritten character recognition are already making the transition to widespread industrial and consumer applications. %he 0ayesian net'ork formalism was invented to allow efficient representation of, and rigorous reasoning with, uncertain knowledge.

$he emergence of intelligent agents !13385present"
Jne of the most important environments for intelligent agents is the Internet.

1.1.. $he state of art
What can A1 do today? Autonomous planning and scheduling# A hundred million miles from 4arth, 8A9AHs
Cemote Agent program became the first on>board autonomous planning program to control the scheduling of operations for a spacecraft *Ionsson et al., ?222-. Cemote Agent generated plans from high>level goals specified from the ground, and it monitored the operation of the spacecraft as the plans were executed>detecting, diagnosing, and recovering from problems as they occurred. *ame playing# I@MHs /eep @lue became the first computer program to defeat the world champion in a chess match when it bested :arry 1asparov by a score of <.# to ?.# in an exhibition match *:oodman and 1eene, !""F-. Autonomous control# %he AEMI88 computer vision system was trained to steer a car to keep it following a lane. It was placed in .MBHs 8AMEA@ computer>controlled minivan and used to navigate across the Bnited 9tates>for ?,#2 miles it was in control of steering the vehicle ",N of the time. @iagnosis# Medical diagnosis programs based on probabilistic analysis have been able to perform at the level of an expert physician in several areas of medicine. 7ogistics +lanning# /uring the 3ersian :ulf crisis of !""!, B.9. forces deployed a /ynamic Analysis and Ceplanning %ool, /AC% *.ross and 5alker, !""=-, to do automated logistics planning and scheduling for transportation. %his involved up to #2,222 vehicles, cargo, and people at a time, and had to account for starting points, destinations, routes, and conflict resolution among all parameters. %he AI planning techni0ues allowed a plan to be generated in hours that would have taken weeks with older methods. %he /efense Advanced Cesearch 3roject Agency */AC3A- stated that this single application more than paid back /AC3AHs <2>year investment in AI. (o)otics# Many surgeons now use robot assistants in microsurgery. +ip8av */i:ioia et al., !""$- is a system that uses computer vision techni0ues to create a three>dimensional model of a patientHs internal anatomy and then uses robotic control to guide the insertion of a hip replacement prosthesis. 7anguage understanding and pro)lem solving# 3CJM4C@ *Eittman et al., !"""- is a computer program that solves crossword pu les better than most humans, using constraints on possible word fillers, a large database of past pu les, and a variety of information sources including dictionaries and online databases such as a list of movies and the actors that appear in them.

1.- I%$<77I*<%$ A*<%$S
1.-.1 Agents and environments
An agent is anything that can be viewed as perceiving its environment through sensors and SENSOR acting upon that environment through actuators. %his simple idea is illustrated in Gigure !.?. o o o A human agent has eyes, ears, and other organs for sensors and hands, legs, mouth, and other body parts for actuators. A robotic agent might have cameras and infrared range finders for sensors and various motors for actuators. A software agent receives keystrokes, file contents, and network packets as sensory inputs and acts on the environment by displaying on the screen, writing files, and sending network packets.

?igure 1.- Agents interact with environments through sensors and actuators. +ercept
5e use the term percept to refer to the agentHs perceptual inputs at any given instant.

+ercept SeAuence An agentHs percept seAuence is the complete history of everything the agent has ever perceived. Agent function Mathematically speaking, we say that an agentHs behavior is described by the agent function
that maps any given percept se0uence to an action.

Agent program
Internally, %he agent function for an artificial agent will be implemented by an agent program. It is important to keep these two ideas distinct. %he agent function is an abstract mathematical description7 the agent program is a concrete implementation, running on the agent architecture.

%o illustrate these ideas, we will use a very simple example>the vacuum>cleaner world shown in Gigure !.<. %his particular world has just two locations& s0uares A and @. %he vacuum agent perceives which s0uare it is in and whether there is dirt in the s0uare. It can choose to move left, move right, suck up the dirt, or do nothing. Jne very simple agent function is the following& if the current s0uare is dirty, then suck, otherwise move to the other s0uare. A partial tabulation of this agent function is shown in Gigure !.=.

?igure 1.9 A vacuum>cleaner world with just two locations.

Agent function +ercept SeAuence Action

OA, .leanP Cight OA, /irtyP 9uck O@, .leanP Eeft O@, /irtyP 9uck OA, .leanP, OA, .leanP Cight OA, .leanP, OA, /irtyP 9uck ( ?igure 1.. 3artial tabulation of a simple agent function for the vacuum>cleaner world shown in Gigure !.<.

(ational Agent A rational agent is one that does the right thing>conceptually speaking, every entry in the table for the agent function is filled out correctly. Jbviously, doing the right thing is better than doing the wrong thing. %he right action is the one that will cause the agent to be most successful. +erformance measures A performance measure embodies the criterion for success of an agentHs behavior. 5hen an agent is plunked down in an environment, it generates a se0uence of actions according to the percepts it receives. %his se0uence of actions causes the environment to go through a se0uence of states. If the se0uence is desirable, then the agent has performed well. (ationality 5hat is rational at any given time depends on four things& o %he performance measure that defines the criterion of success. o %he agentHs prior knowledge of the environment. o %he actions that the agent can perform. o %he agentHs percept se0uence to date.
%his leads to a definition of a rational agent#

For each possible percept sequence, a rational agent should select an action that is expected to maximize its performance measure, given the evidence provided by the percept sequence and whatever built-in knowledge the agent has.

Bmniscience= learning= and autonomy An omniscient agent knows the actual outcome of its actions and can act accordingly7 but omniscience is impossible in reality. /oing actions in order to modify future percepts>sometimes called information gathering>is an important part of rationality.

Jur definition re0uires a rational agent not only to gather information, but also to learn as much as possible from what it perceives. %o the extent that an agent relies on the prior knowledge of its designer rather than on its own percepts, we say that the agent lacks autonomy. A rational agent should be autonomous>it should learn what it can to compensate for partial or incorrect prior knowledge. $ask environments 5e must think about task environments= which are essentially the "pro)lems" to which rational agents are
the "solutions."

Specifying the task environment %he rationality of the simple vacuum>cleaner agent, needs specification of o the performance measure o the environment o the agentHs actuators and sensors.

+<AS
All these are grouped together under the heading of the task environment. 5e call this the +<AS *3erformance, 4nvironment, Actuators, 9ensors- description. In designing an agent, the first step must always be to specify the task environment as fully as possible. Agent $ype +erformance <nvironments Actuators Sensors Measure %axi driver 9afe& fast, legal, Coads,other 9teering,accelerator, .ameras,sonar, comfortable trip, traffic,pedestrians, brake, 9peedometer,:39, maximi e profits customers 9ignal,horn,display Jdometer,engine sensors,keyboards, accelerometer ?igure 1.8 34A9 description of the task environment for an automated taxi.

?igure 1.; 4xamples of agent types and their 34A9 descriptions.

+roperties of task environments
o Gully observable vs. partially observable o /eterministic vs. stochastic o 4pisodic vs. se0uential o 9tatic vs. dynamic o /iscrete vs. continuous o 9ingle agent vs. multiagent ?ully o)serva)le vs. partially o)serva)le. If an agentHs sensors give it access to the complete state of the environment at each point in time, then we say that the task environment is fully observable. A task envi> ronment is effectively fully observable if the sensors detect all aspects that are relevant to the choice of action7 An environment might be partially observable because of noisy and inaccurate sensors or because parts of the state are simplly missing from the sensor data. @eterministic vs. stochastic. If the next state of the environment is completely determined by the current state and the action executed by the agent, then we say the environment is deterministic7 other> wise, it is stochastic. <pisodic vs. seAuential In an episodic task environment, the agentHs experience is divided into atomic episodes. 4ach episode consists of the agent perceiving and then performing a single action. .ru> cially, the next episode does not depend on the actions taken in previous episodes.

Gor example, an agent that has to spot defective parts on an assembly line bases each decision on the current part, regardless of previous decisions7 In seAuential environments, on the other hand, the current decision could affect all future decisions. .hess and taxi driving are se0uential& @iscrete vs. continuous. %he discreteAcontinuous distinction can be applied to the state of the environment, to the way time is handled, and to the percepts and actions of the agent. Gor example, a discrete>state environment such as a chess game has a finite number of distinct states. .hess also has a discrete set of percepts and actions. %axi driving is a continuous> state and continuous>time problem& the speed and location of the taxi and of the other vehicles sweep through a range of continuous values and do so smoothly over time. %axi>driving actions are also continuous *steering angles, etc.-. Single agent vs. multiagent. An agent solving a crossword pu le by itself is clearly in a single>agent environment, whereas an agent playing chess is in a two>agent environ> ment. As one might expect, the hardest case is partially observable, stochastic, sequential, dynamic, continuous, and multiagent.
Gigure !.F lists the properties of a number of familiar environments.

?igure 1.2 4xamples of task environments and their characteristics.

Agent programs %he agent programs all have the same skeleton& they take the current percept as input from the sensors and return an action to the actuatom$ 8otice the difference between the agent program, which takes the current percept as input, and the agent function, which takes the entire percept history. %he agent program takes just the current percept as input because nothing more is available from the environment7 if the agentHs actions depend on the entire percept se0uence, the agent will have to remember the percepts.
?unction %A@E4>/CIM48QA:48%*percept- returns an action static& percepts, a se0uence initially empty table, a table of actions, indexed by percept se0uence

append percept to the end of percepts action ¬ EJJ1B3*percepts, tablereturn action
?igure 1./ %he %A@E4>/CIM48>A:48% program is invoked for each new percept and

returns an action each time. /rawbacks& R $a)le lookup of percept>action pairs defining all possible condition>action rules necessary to interact in an environment R +ro)lems 6 %oo big to generate and to store *.hess has about !2S!?2 states, for example6 8o knowledge of non>perceptual parts of the current state 6 8ot adaptive to changes in the environment7 re0uires entire table to be updated if changes occur 6 Eooping& .anHt make actions conditional R R R %ake a long time to build the table 8o autonomy 4ven with learning, need a long time to learn the table entries

Some Agent $ypes R $a)le5driven agents 6 use a percept se0uenceAaction table in memory to find the next action. %hey are implemented by a *large- lookup ta)le. R Simple refle> agents 6 are based on condition5action rules, implemented with an appropriate production system. %hey are stateless devices which do not have memory of past world states. R Agents 'ith memory 6 have internal state, which is used to keep track of past states of the world. R Agents 'ith goals 6 are agents that, in addition to state information, have goal information that describes desirable situations. Agents of this kind take future events into consideration. R Ctility5)ased agents 6 base their decisions on classic a>iomatic utility theory in order to act rationally. Simple (efle> Agent
%he simplest kind of agent is the simple refle> agent. %hese agents select actions on the basis of the current percept, ignoring the rest of the percept history. Gor example, the vacuum agent whose agent function is tabulated in Gigure !.!2 is a simple reflex agent, because its decision is based only on the current location and on whether that contains dirt.

o 9elect action on the basis of only the current percept. 4.g. the vacuum>agent o Earge reduction in possible perceptAaction situations*next page-. o Implemented through condition-action rules If dirty then suck

A Simple (efle> Agent# Schema

?igure 1.3 9chematic diagram of a simple reflex agent.

function 9IM3E4>C4GE4T>A:48%*percept- returns an action static& rules, a set of condition>action rules state ¬ I8%4C3C4%>I83B%*perceptrule ¬ CBE4>MA%.+*state, ruleaction ¬ CBE4>A.%IJ8OruleP return action
?igure 1.14 A simple reflex agent. It acts according to a rule whose condition matches

the current state, as defined by the percept.
function REFLEX-VACUUM-AGENT ([location, status]) return an action if status == Dirty then return Suck el e if location == A then return Right el e if location == B then return Left

?igure 1.11 %he agent program for a simple reflex agent in the two>state vacuum environment. %his program implements the agent function tabulated in the figure !.=.
 Characteristics

o Jnly works if the environment is fully observable. o Eacking history, easily get stuck in infinite loops o Jne solution is to randomi e actions o Model5)ased refle> agents

%he most effective way to handle partial observability is for the agent to keep track of the part of the world it can't see now. %hat is, the agent should maintain some sort of internal state that depends on the percept history and thereby reflects at least some of the unobserved aspects of the current state. Bpdating this internal state information as time goes by re0uires two kinds of knowledge to be encoded in the agent program. Girst, we need some information about how the world evolves independently of the agent>for example, that an overtaking car generally will be closer behind than it was a moment ago. 9econd, we need some information about how the agentHs own actions affect the world>for example, that when the agent turns the steering wheel clockwise, the car turns to the right or that after driving for five minutes northbound on the freeway one is usually about five miles north of where one was five minutes ago. %his knowledge about "how the world working > whether implemented in simple @oolean circuits or in complete scientific theories>is called a model of the world. An agent that uses such a MJ/4E>@A94/ model is called a model5)ased agent.

?igure 1.1- A model based reflex agent function C4GE4T>A:48%>5I%+>9%A%4*percept- returns an action static& rules, a set of condition>action rules state, a description of the current world state action, the most recent action. state ¬ B3/A%4>9%A%4*state, action, perceptrule ¬ CBE4>MA%.+*state, ruleaction ¬ CBE4>A.%IJ8OruleP return action ?igure 1.19 Model based reflex agent. It keeps track of the current state of the world using an internal
model. It then chooses an action in the same way as the reflex agent.

*oal5)ased agents
1nowing about the current state of the environment is not always enough to decide what to do. Gor example, at a road junction, the taxi can turn left, turn right, or go straight on. %he correct decision depends on where the taxi is trying to get to. In other words, as well as a current state description, the agent needs some sort of goal information that describes situations that are desirable>for example, being at the passengerHs destination. %he agent

program can combine this with information about the results of possible actions *the same information as was used to update internal state in the reflex agent- in order to choose actions that achieve the goal. Gigure !.!< shows the goal>based agentHs structure.

?igure 1.1. A goal based agent

Ctility5)ased agents :oals alone are not really enough to generate high>0uality behavior in most environments. Gor example, there are many action se0uences that will get the taxi to its destination *thereby achieving the goal- but some are 0uicker, safer, more reliable, or cheaper than others. :oals just provide a crude binary distinction between "happy" and "unhappy" states, whereas a more general performance measure should allow a comparison of different world states according to exactly how happy they would make the agent if they could be achieved. @ecause "happy" does not sound very scientific, the customary terminology is to say that if one world state is preferred to another, then it has higher utility for the agent.

?igure 1.18 A model>based, utility>based agent. It uses a model of the world, along with
a utility function that measures its preferences among states of the world. %hen it chooses the action that leads to the best expected utility, where expected utility is computed by averaging

over all possible outcome states, weighted by the probability of the outcome.

R R R

.ertain goals can be reached in different ways. 6 9ome are better, have a higher utility. Btility function maps a *se0uence of- state*s- onto a real number. Improves on goals& 6 9electing between conflicting goals 6 9elect appropriately between several goals based on likelihood of success.

?igure 1.1; A general model of learning agents. All agents can improve their performance through learning. A learning agent can be divided into four conceptual components, as shown in Gigure !.!# %he most important distinction is between the learning element, which is responsible for making improvements, and the performance element, which is responsible for selecting external actions. %he performance element is what we have previously considered to be the entire agent& it takes in percepts and decides on actions. %he learning element uses feedback from the critic on how the agent is doing and determines how the performance element should be modified to do better in the future. %he last component of the learning agent is the pro)lem generator. It is responsible for suggesting actions that will lead to new and informative e>periences. @ut if the agent is willing to explore a little, it might discover much better actions for the long run. %he problem generatorHs job is to suggest these e>ploratory actions. %his is what scientists do when they carry out experiments. R

Summary# Intelligent Agents
R R R An agent perceives and acts in an environment, has an architecture, and is implemented by an agent program. %ask environment 6 +<AS !+erformance= <nvironment, Actuators, Sensors" %he most challenging environments are inaccessible, nondeterministic, dynamic, and continuous.

R R

R

An ideal agent always chooses the action which maximi es its expected performance, given its percept se0uence so far. An agent program maps from percept to action and updates internal state. 6 (efle> agents respond immediately to percepts. R simple reflex agents R model>based reflex agents 6 *oal5)ased agents act in order to achieve their goal*s-. 6 Ctility5)ased agents maximi e their own utility function. All agents can improve their performance through learning.

1.9.1 +ro)lem Solving )y Search
An important aspect of intelligence is goal-based problem solving. %he solution of many pro)lems can be described by finding a seAuence of actions that lead to a desirable goal. 4ach action changes the state and the aim is to find the se0uence of actions and states that lead from the initial *start- state to a final *goal- state. A well>defined problem can be described by& • Initial state • Bperator or successor function > for any state x returns s*x-, the set of states reachable from x with one action • State space > all states reachable from initial by any se0uence of actions • +ath > se0uence through state space • +ath cost > function that assigns a cost to a path. .ost of a path is the sum of costs of individual actions along the path • *oal test > test to determine if at goal state

What is Search?
Search is the systematic examination of states to find path from the startDroot state to the goal state. %he set of possible states, together with operators defining their connectivity constitute the search space. %he output of a search algorithm is a solution, that is, a path from the initial state to a state that satisfies the goal test.

+ro)lem5solving agents
A 3roblem solving agent is a goal5)ased agent . It decide what to do by finding se0uence of actions that lead to desirable states. %he agent can adopt a goal and aim at satisfying it. %o illustrate the agentDs behavior ,let us take an example where our agent is in the city of Arad,which is in Comania. %he agent has to adopt a goal of getting to @ucharest. *oal formulation,based on the current situation and the agentDs performance measure,is the first step in problem solving. %he agentDs task is to find out which se0uence of actions will get to a goal state. +ro)lem formulation is the process of deciding what actions and states to consider given a goal.

E!a"#le$ Route fin%in& #ro'le"
Referrin& to fi&ure ()(* Jn holiday in Comania & currently in Arad. Glight leaves tomorrow from @ucharest ?ormulate goal& be in @ucharest ?ormulate pro)lem& states& various cities actions& drive between cities ?ind solution& se0uence of cities, e.g., Arad, 9ibiu, Gagaras, @ucharest

3roblem formulation
A pro)lem is defined by four items& initial state e.g., 'at Arad" successor function 9*x- U set of action>state pairs e.g., 9*Arad- U VOArad >W Xerind7XerindP,(.Y goal test, can be explicit, e.g., x U at @ucharest" implicit, e.g., 8o/irt*xpath cost *additivee.g., sum of distances, number of actions executed, etc. c*x7 a7 y- is the step cost, assumed to be WU 2 A solution is a se0uence of actions leading from the initial state to a goal state. ?igure 1.12 :oal formulation and problem formulation

Search
An agent with several immediate options of unknown value can decide what to do by examining different possible se0uences of actions that leads to the states of known value,and then choosing the best se0uence. %he process of looking for se0uences actions from the current state to reach the goal state is called search. %he search algorithm takes a pro)lem as input and returns a solution in the form of action seAuence. Jnce a solution is found,the e>ecution phase consists of carrying out the recommended action.. Gigure !.!, shows a simple 'formulate,search,execute) design for the agent. Jnce solution has been executed,the agent will formulate a new goal. function 9IM3E4>3CJ@E4M>9JEMI8:>A:48%* percept- returns an action inputs & percept, a percept static& seq, an action se0uence, initially empty state, some description of the current world state goal, a goal, initially null problem, a problem formulation state B3/A%4>9%A%4*state, perceptif se0 is empty then do goal GJCMBEA%4>:JAE*stateproblem GJCMBEA%4>3CJ@E4M*state, goal-

seq 94AC.+* problemaction GIC9%*seq-7 seq C49%*se0return action ?igure 1.1/ A 9imple problem solving agent. It first formulates a goal and a pro)lem,searches for a se0uence of actions that would solve a problem,and executes the actions one at a time. • %he agent design assumes the 4nvironment is R Static & %he entire process carried out without paying attention to changes that might be occurring in the environment. R B)serva)le # %he initial state is known and the agentDs sensor detects all aspects that are relevant to the choice of action R @iscrete # 5ith respect to the state of the environment and percepts and actions so that alternate courses of action can be taken R @eterministic # %he next state of the environment is completely determined by the current state and the actions executed by the agent. 9olutions to the problem are single se0uence of actions An agent carries out its plan with eye closed. %his is called an open loop system because ignoring the percepts breaks the loop between the agent and the environment.

1.9.1.1 Well5defined pro)lems and solutions
A pro)lem can be formally defined by four components& • %he initial state that the agent starts in . %he initial state for our agent of example problem is described by n!"rad# • A Successor ?unction returns the possible actions available to the agent. :iven a state x,9B..499JC>G8*x- returns a set of Vaction,successorY ordered pairs where each action is one of the legal actions in state x,and each successor is a state that can be reached from x by applying the action. Gor example,from the state In*Arad-,the successor function for the Comania problem would return V O:o*9ibiu-,In*9ibiu-P,O:o*%imisoara-,In*%imisoara-P,O:o*Xerind-,In*Xerind-P Y • State Space & %he set of all states reachable from the initial state. %he state space forms a graph in which the nodes are states and the arcs between nodes are actions. • A path in the state space is a se0uence of states connected by a se0uence of actions. • %hr goal test determines whether the given state is a goal state. • • • • A path cost function assigns numeric cost to each action. Gor the Comania problem the cost of path might be its length in kilometers. %he step cost of taking action a to go from state x to state y is denoted by c*x,a,y-. %he step cost for Comania are shown in figure !.!,. It is assumed that the step costs are non negative. A solution to the problem is a path from the initial state to a goal state. An optimal solution has the lowest path cost among all solutions.

?igure 1.13 A simplified Coad Map of part of Comania

1.9.- <EAM+7< +(B07<MS
%he problem solving approach has been applied to a vast array of task environments. 9ome best known problems are summari ed below. %hey are distinguished as toy or real>world problems A toy pro)lem is intended to illustrate various problem solving methods. It can be easily used by different researchers to compare the performance of algorithms. A real 'orld pro)lem is one whose solutions people actually care about.

1.9.-.1 $BF +(B07<MS
Gacuum World <>ample o States& %he agent is in one of two locations.,each of which might or might not contain dirt. %hus there are ? x ?? U , possible world states. o Initial state& Any state can be designated as initial state. o Successor function & %his generates the legal states that results from trying the three actions *left, right, suck-. %he complete state space is shown in figure ?.< o *oal $est & %his tests whether all the s0uares are clean. o +ath test & 4ach step costs one ,so that the the path cost is the number of steps in the path. Gacuum World State Space

?igure 1.-4 %he state space for the vacuum world. Arcs denote actions& E U Eeft,C U Cight,9 U 9uck

$he /5puHHle
An ,>pu le consists of a <x< board with eight numbered tiles and a blank space. A tile adjacent to the balank space can slide into the space. %he object is to reach the goal state ,as shown in figure ?.= <>ample# $he /5puHHle

?igure 1.-1 A typical instance of ,>pu le.

%he problem formulation is as follows & o States & A state description specifies the location of each of the eight tiles and the blank in one of the nine s0uares. o Initial state & Any state can be designated as the initial state. It can be noted that any given goal can be reached from exactly half of the possible initial states. o Successor function & %his generates the legal states that result from trying the four actions*blank moves Eeft,Cight,Bp or down-. o *oal $est & %his checks whether the state matches the goal configuration shown in figure ?.=.*Jther goal configurations are possibleo +ath cost & 4ach step costs !,so the path cost is the number of steps in the path.

o $he />puHHle belongs to the family of sliding5)lock puHHles,which are often used as test problems for new search algorithms in AI. %his general class is known as 83>complete. %he />puHHle has "ZA? U !,!,==2 reachable states and is easily solved. %he 18 puHHle * = x = board - has around !.< trillion states,an the random instances can be solved optimally in few milli seconds by the best search algorithms. %he -.5puHHle *on a # x # board- has around !2?# states ,and random instances are still 0uite difficult to solve optimally with current machines and algorithms.

,>0ueens problem
%he goal of ,>0ueens problem is to place , 0ueens on the chessboard such that no 0ueen attacks any other.*A 0ueen attacks any piece in the same row,column or diagonal-. Gigure ?.# shows an attempted solution that fails& the 0ueen in the right most column is attacked by the 0ueen at the top left. An Incremental formulation involves operators that augments the state description,starting with an empty state.for ,>0ueens problem,this means each action adds a 0ueen to the state. A complete5state formulation starts with all , 0ueens on the board and move them around. In either case the path cost is of no interest because only the final state counts.

?igure 1.-- ,>0ueens problem %he first incremental formulation one might try is the following & o States & Any arrangement of 2 to , 0ueens on board is a state. o Initial state & 8o 0ueen on the board. o Successor function & Add a 0ueen to any empty s0uare. o *oal $est & , 0ueens are on the board,none attacked. In this formulation,we have $=.$<(#F U < x !2!= possible se0uences to investigate. A better formulation would prohibit placing a 0ueen in any s0uare that is already attacked. & o States & Arrangements of n 0ueens * 2 [U n [ U , - ,one per column in the left most columns ,with no 0ueen attacking another are states. o Successor function & Add a 0ueen to any s0uare in the left most empty column such that it is not attacked by any other 0ueen. %his formulation reduces the ,>0ueen state space from < x !2!= to just ?2#F,and solutions are easy to find.

Gor the !22 0ueens the initial formulation has roughly !2=22 states whereas the improved formulation has about !2#? states. %his is a huge reduction,but the improved state space is still too big for the algorithms to handle.

1.9.-.- (<A75WB(7@ +(B07<MS
(BC$<5?I%@I%* +(B07<M Coute>finding problem is defined in terms of specified locations and transitions along links between them. Coute>finding algorithms are used in a variety of applications,such as routing in computer networks,military operations planning,and air line travel planning systems. AI(7I%< $(AG<7 +(B07<M %he airline travel pro)lem is specifies as follows # o States # 4ach is represented by a location*e.g.,an airport- and the current time. o Initial state # %his is specified by the problem. o Successor function # %his returns the states resulting from taking any scheduled flight*further specified by seat class and location-,leaving later than the current time plus the within>airport transit time,from the current airport to another. o *oal $est # Are we at the destination by some prespecified time\ o +ath cost # %his depends upon the monetary cost,waiting time,flight time,customs and immigration procedures,seat 0uality,time of dat,type of air plane,fre0uent>flyer mileage awards, and so on. $BC(I%* +(B07<MS $ouring pro)lems are closely related to route>finding problems,but with an important difference. .onsider for example,the problem,)Misit every city at least once) as shown in Comania map. As with route>finding the actions correspond to trips between adjacent cities. %he state space, however,is 0uite different. %he initial state would be 'In @ucharest7 visitedV@ucharestY). A typical intermediate state would be 'In Maslui7visited V@ucharest,Br iceni,MasluiY). %he goal test would check whether the agent is in @ucharest and all ?2 cities have been visited. $:< $(AG<77I%* SA7<S+<(SB% +(B07<M!$S+" Is a touring problem in which each city must be visited exactly once. %he aim is to find the shortest tour.%he problem is known to be %+5hard. 4normous efforts have been expended to improve the capabilities of %93 algorithms. %hese algorithms are also used in tasks such as planning movements of automatic circuit5)oard drills and of stocking machines on shop floors. G7SI layout A G7SI layout problem re0uires positioning millions of components and connections on a chip to minimi e area ,minimi e circuit delays,minimi e stray capacitances,and maximi e manufacturing yield. %he layout problem is split into two parts & cell layout and channel routing. (B0B$ navigation (B0B$ navigation is a generali ation of the route>finding problem. Cather than a discrete set of routes,a robot can move in a continuous space with an infinite set of possible actions and states. Gor a circular Cobot moving on a flat surface,the space is essentially two>dimensional.

5hen the robot has arms and legs or wheels that also must be controlled,the search space becomes multi>dimensional. Advanced techni0ues are re0uired to make the search space finite. AC$BMA$IC ASS<M07F S<IC<%CI%* %he example includes assembly of intricate objects such as electric motors. %he aim in assembly problems is to find the order in which to assemble the parts of some objects. If the wrong order is choosen,there will be no way to add some part later without undoing somework already done. Another important assembly problem is protein design,in which the goal is to find a se0uence of Amino acids that will be fold into a three>dimensional protein with the right properties to cure some disease. I%$<(%<$ S<A(C:I%* In recent years there has been increased demand for software robots that perform Internet searching.,looking for answers to 0uestions,for related information,or for shopping deals. %he searching techni0ues consider internet as a graph of nodes*pages- connected by links.

1.9.9 S<A(C:I%* ?B( SB7C$IB%S
S<A(C: $(<< +aving formulated some pro)lems,we now need to solve them. %his is done by a search through the state space. A search tree is generated by the initial state and the successor function that together define the state space. In general,we may have a search graph rather than a search tree,when the same state can be reached from multiple paths. Gigure !.?< shows some of the expansions in the search tree for finding a route from Arad to @ucharest.

?igure 1.-9 3artial search trees for finding a route from Arad to @ucharest. 8odes that have been expanded are shaded.7 nodes that have been generated but not yet expanded are outlined in bold7nodes that have not yet been generated are shown in faint dashed line %he root of the search tree is a search node corresponding to the initial state,In*Arad-. %he first step is to test whether this is a goal state. %he current state is expanded by applying the successor function to the current state,thereby generating a new set of states. In this case,we get three new states& In*9ibiu-,In*%imisoara-,and In*Xerind-. 8ow we must choose which of these three possibilities to consider further. %his is the essense of search> following up one option now and putting the others aside for latter,in case the first choice does not lead to a solution.

Search strategy . %he general tree>search algorithm is described informally in Gigure !.?= .

$ree Search

?igure 1.-. An informal description of the general tree>search algorithm

%he choice of which state to expand is determined by the search strategy. %here are an infinite number paths in this state space ,so the search tree has an infinite number of nodes. A node is a data structure with five components & o 9%A%4 & a state in the state space to which the node corresponds7 o 3AC48%>8J/4 & the node in the search tree that generated this node7 o A.%IJ8 & the action that was applied to the parent to generate the node7 o 3A%+>.J9% &the cost,denoted by g*n-,of the path from initial state to the node,as indicated by the parent pointers7 and o /43%+ & the number of steps along the path from the initial state. It is important to remember the distinction between nodes and states. A node is a book keeping data structure used to represent the search tree. A state corresponds to configuration of the world.

?igure 1.-8 8odes are data structures from which the search tree is constructed. 4ach has a parent,a state, Arrows point from child to parent.

Gringe
Gringe is a collection of nodes that have been generated but not yet been expanded. 4ach element of the fringe is a leaf node,that is,a node with no successors in the tree. %he fringe of each tree consists of those nodes with bold outlines. %he collection of these nodes is implemented as a Aueue. %he general tree search algorithm is shown in Gigure ?."

?igure 1.-; %he general %ree search algorithm %he operations specified in Gigure !.?$ on a 0ueue are as follows& o MA&<5IC<C<!element=J" creates a 0ueue with the given element*s-. o <M+$F?!Aueue" returns true only if there are no more elements in the 0ueue. o ?I(S$!Aueue" returns GIC9%*0ueue- and removes it from the 0ueue. o I%S<($!element=Aueue" inserts an element into the 0ueue and returns the resulting 0ueue. o I%S<($5A77!elements=Aueue" inserts a set of elements into the 0ueue and returns the resulting 0ueue. M<ASC(I%* +(B07<M5SB7GI%* +<(?B(MA%C< %he output of problem>solving algorithm is either failure or a solution.*9ome algorithms might struck in an infinite loop and never return an output. %he algorithmDs performance can be measured in four ways & o Completeness & Is the algorithm guaranteed to find a solution when there is one\ o Bptimality & /oes the strategy find the optimal solution o $ime comple>ity & +ow long does it take to find a solution\

o Space comple>ity & +ow much memory is needed to perform the search\

1.9.. C%I%?B(M<@ S<A(C: S$(A$*<S
Cninformed Search Strategies have no additional information about states beyond that provided in the pro)lem definition. Strategies that know whether one non goal state is 'more promising) than another are called Informed search or heuristic search strategies. %here are five uninformed search strategies as given below. o @readth>first search o Bniform>cost search o /epth>first search o /epth>limited search o Iterative deepening search

!.<.=.! @readth>first search
@readth>first search is a simple strategy in which the root node is expanded first,then all successors of the root node are expanded next,then their successors,and so on. In general,all the nodes are expanded at a given depth in the search tree before any nodes at the next level are expanded. @reath>first>search is implemented by calling %C44>94AC.+ with an empty fringe that is a first>in>first>out*GIGJ- 0ueue,assuring that the nodes that are visited first will be expanded first. In otherwards,calling %C44>94AC.+*problem,GIGJ>LB4B4*-- results in breadth>first>search. %he GIGJ 0ueue puts all newly generated successors at the end of the 0ueue,which means that 9hallow nodes are expanded before deeper nodes.

?igure 1.-2 @readth>first search on a simple binary tree. At each stage ,the node to be expanded next is indicated by a marker. +roperties of )readth5first5search

?igure 1.-/ @readth>first>search properties

?igure 1.-3 %ime and memory re0uirements for breadth>first>search. %he numbers shown assume branch factor of b U !2 7 !2,222 nodesAsecond7 !222 bytesAnode $ime comple>ity for 0?S Assume every state has b successors. %he root of the search tree generates b nodes at the first level,each of which generates b more nodes,for a total of b? at the second level. 4ach of these generates b more nodes,yielding b< nodes at the third level,and so on. 8ow suppose,that the solution is at depth d. In the worst case,we would expand all but the last node at level d,generating bd]! > b nodes at level d]!. %hen the total number of nodes generated is b ] b? ] b< ] (] bd ] * bd]! ] b- U J*bd]!-. 4very node that is generated must remain in memory,because it is either part of the fringe or is an ancestor of a fringe node. %he space compleity is,therefore ,the same as the time complexity

1.9...- C%I?B(M5CBS$ S<A(C:
Instead of expanding the shallowest node,uniform5cost search expands the node n with the lowest path cost. uniform>cost search does not care about the number of steps a path has,but only about their total cost.

?igure 1.94 3roperties of Bniform>cost>search

-.8.1.9 @<+$:5?I(S$5S<A(C:
/epth>first>search always expands the deepest node in the current fringe of the search tree. %he progress of the search is illustrated in figure !.<!. %he search proceeds immediately to the deepest level of the search tree,where the nodes have no successors. As those nodes are expanded,they are dropped from the fringe,so then the search 'backs up) to the next shallowest node that still has unexplored successors.

?igure 1.91 /epth>first>search on a binary tree. 8odes that have been expanded and have no descendants in the fringe can be removed from the memory7these are shown in black. 8odes at depth < are assumed to have no successors and M is the only goal node. %his strategy can be implemented by %C44>94AC.+ with a last>in>first>out *EIGJ- 0ueue,also known as a stack. /epth>first>search has very modest memory re0uirements.It needs to store only a single path from the root to a leaf node,along with the remaining unexpanded sibling nodes for each node on the path. Jnce the node has been expanded,it can be removed from the memory,as soon as its descendants have been fully explored*Cefer Gigure ?.!?-. Gor a state space with a branching factor b and maximum depth m,depth>first>search re0uires storage of only bm ] ! nodes.

Bsing the same assumptions as Gigure ?.!!,and assuming that nodes at the same depth as the goal node have no successors,we find the depth>first>search would re0uire !!, kilobytes instead of !2 petabytes,a factor of !2 billion times less space. @ra')ack of @epth5first5search %he drawback of depth>first>search is that it can make a wrong choice and get stuck going down very long*or even infinite- path when a different choice would lead to solution near the root of the search tree. Gor example ,depth>first>search will explore the entire left subtree even if node . is a goal node. 0AC&$(AC&I%* S<A(C: A variant of depth>first search called backtracking search uses less memory and only one successor is generated at a time rather than all successors.7 Jnly J*m- memory is needed rather than J*bm-

1.9.... @<+$:57IMI$<@5S<A(C:
%he problem of unbounded trees can be alleviated by supplying depth>first>search with a pre> determined depth limit l.%hat is,nodes at depth l are treated as if they have no successors. %his approach is called depth5limited5search. %he depth limit soves the infinite path problem. /epth limited search will be nonoptimal if we choose l W d. Its time complexity is J*bl- and its space compleiy is J*bl-. /epth>first>search can be viewed as a special case of depth>limited search with l U oo 9ometimes,depth limits can be based on knowledge of the problem. Gor,example,on the map of Comania there are ?2 cities. %herefore,we know that if there is a solution.,it must be of length !" at the longest,9o l U !2 is a possible choice. +owever,it oocan be shown that any city can be reached from any other city in at most " steps. %his number known as the diameter of the state space,gives us a better depth limit. /epth>limited>search can be implemented as a simple modification to the general tree>search algorithm or to the recursive depth>first>search algorithm. %he pseudocode for recursive depth> limited>search is shown in Gigure !.<?. It can be noted that the above algorithm can terminate with two kinds of failure & the standard failure value indicates no solution7 the cutoff value indicates no solution within the depth limit. /epth>limited search U depth>first search with depth limit l, returns cut off if any path is cut off by depth limit function /epth>Eimited>9earch* problem, limit- returns a solutionAfailAcutoff return Cecursive>/E9*Make>8ode*Initial>9tateOproblemP-, problem, limitfunction Cecursive>/E9*node, problem, limit- returns solutionAfailAcutoff cutoff>occurred\ false if :oal>%est*problem,9tateOnodeP- then return 9olution*nodeelse if /epthOnodeP U limit then return cutoff else for each successor in 4xpand*node, problem- do result Cecursive>/E9*successor, problem, limitif result U cutoff then cutoffQoccurred\ true else if result not U failure then return result if cutoffQoccurred\ then return cutoff else return failure ?igure 1.9- Cecursive implementation of /epth>limited>search&

1.9...8 I$<(A$IG< @<<+<%I%* @<+$:5?I(S$ S<A(C:
Iterative deepening search *or iterative>deepening>depth>first>search- is a general strategy often used in combination with depth>first>search,that finds the better depth limit. It does this by gradually increasing the limit 6 first 2,then !,then ?, and so on 6 until a goal is found. %his will occur when the depth limit reaches d,the depth of the shallowest goal node. %he algorithm is shown in Gigure ?.!=. Iterative deepening combines the benefits of depth>first and breadth>first>search Eike depth>first>search,its memory re0uirements are modest7J*bd- to be precise. Eike @readth>first>search,it is complete when the branching factor is finite and optimal when the path cost is a non decreasing function of the depth of the node. Gigure ?.!# shows the four iterations of I%4CA%IM4>/44348I8:Q94AC.+ on a binary search tree,where the solution is found on the fourth iteration.

?igure 1.99 %he iterative deepening search algorithm ,which repeatedly applies depth>limited> search with increasing limits. It terminates when a solution is found or if the depth limited search resturns failure,meaning that no solution exists.

?igure 1.9. Gour iterations of iterative deepening search on a binary tree Iterative search is not as wasteful as it might seem

Iterative deepening search
S Eimit U 2 S A Eimit U ! S @

S A

S @ 0 A @

S @ A <

Eimit U ?
?igure 1.98

Iterative search is not as wasteful as it might seem

+ro#ertie of iterati,e %ee#enin& earch

Fi&ure ()-.

In general,iterative deepening is the prefered uninformed search method when there is a large search space and the depth of solution is not known.

!.<.=.$ @idirectional 9earch
%he idea behind bidirectional search is to run two simultaneous searches 6 one forward from he initial state and the other backward from the goal, stopping when the two searches meet in the middle *Gigure !.<F%he motivation is that bdA? ] bdA? much less than ,or in the figure ,the area of the two small circles is less than the area of one big circle centered on the start and reaching to the goal.

?igure 1.92 A schematic view of a bidirectional search that is about to succeed,when a @ranch from the 9tart node meets a @ranch from the goal node.

1.9...2 Comparing Cninformed Search Strategies
Gigure !.<, compares search strategies in terms of the four evaluation criteria .

?igure 1.9/ 4valuation of search strategies,b is the branching factor7 d is the depth of the shallowest solution7 m is the maximum depth of the search tree7 l is the depth limit. 9uperscript caveats are as follows& a complete if b is finite7 b complete if step costs WU 4 for positive 47 c optimal if step costs are all identical7 d if both directions use breadth>first search.

1.9.8 AGBI@I%* (<+<A$<@ S$A$<S
In searching,time is wasted by expanding states that have already been encountered and expanded before. Gor some problems repeated states are unavoidable. %he search trees for these problems are infinite. If we prune some of the repeated states,we can cut the search tree down to finite si e. .onsidering search tree upto a fixed depth,eliminating repeated states yields an exponential reduction in search cost. Cepeated states ,can cause a solvable problem to become unsolvable if the algorithm does not detect them.

Cepeated states can be the source of great inefficiency& identical sub trees will be explored many timesZ A A

0 C C C

0 C

0 C

?igure 1.93

?igure 1..4

?igure 1..1 %he :eneral graph search algorithm. %he set closed can be implemented with a hash table to allow efficient checking for repeated states. /o not return to the previous state. R /o not create paths with cycles. R /o not generate the same state twice. > 9tore states in a hash table.

> .heck for repeated states.
o Bsing more memory in order to check repeated state o "lgorithms that forget their history are doomed to repeat it. o Maintain .lose>Eist beside Jpen>Eist*fringeStrategies for avoiding repeated states 5e can modify the general %C44>94AC.+ algorithm to include the data structure called the closed list ,which stores every expanded node. %he fringe of unexpanded nodes is called the open list. If the current node matches a node on the closed list,it is discarded instead of being expanded. %he new algorithm is called :CA3+>94AC.+ and much more efficient than %C44>94AC.+. %he worst case time and space re0uirements may be much smaller than J*bd-.

1.9.; S<A(C:I%* WI$: +A($IA7 I%?B(MA$IB%
o /ifferent types of incompleteness lead to three distinct problem types& o Sensorless pro)lems *conformant-& If the agent has no sensors at all o Contingency pro)lem& if the environment if partially observable or if action are uncertain *adversarialo <>ploration pro)lems& 5hen the states and actions of the environment are unknown. o o o o o o 8o sensor Initial 9tate*!,?,<,=,#,$,F,,After action OCightP the state *?,=,$,,After action O9uckP the state *=, ,After action OEeftP the state *<,FAfter action O9uckP the state *,-

o Answer & OCight,9uck,Eeft,9uckP coerce the world into state F without any sensor o @elief 9tate& 9uch state that agent belief to be there *9EI/4 F- 3artial knowledge of states and actions& – sensorless or conformant problem – Agent may have no idea where it is7 solution *if any- is a se0uence. – contingency problem – 3ercepts provide new information about current state7 solution is a tree or policy7 often interleave search and execution. – If uncertainty is caused by actions of another agent& adversarial problem – exploration problem – 5hen states and actions of the environment are unknown.

Gigure

Gigure .ontingency, start in V!,<Y. MurphyDs law, 9uck can dirty a clean carpet. Eocal sensing& dirt, location only. – 3ercept U OE,/irtyP UV!,<Y – O9uckP U V#,FY – OCightP UV$,,Y – O9uckP in V$YUV,Y *9uccess– @B% O9uckP in V,Y U failure 9olution\\ – @elief>state& no fixed action se0uence guarantees solution Celax re0uirement& – O$uck, %ight, if &%,dirty' then $uckP – 9elect actions based on contingencies arising during execution. %ime and space complexity are always considered with respect to some measure of the problem difficulty. In theoretical computer science ,the typical measure is the si e of the state space. In AI,where the graph is represented implicitly by the initial state and successor function,the complexity is expressed in terms of three 0uantities&

),the )ranching factor or maximum number of successors of any node7 d,the depth of the shallo'est goal node7 and m,the ma>imum length of any path in the state space. Search5cost > typically depends upon the time complexity but can also include the term for memory usage. $otal1cost 1 It combines the search>cost and the path cost of the solution found.

S<A(C:I%* $<C:%IIC<S
55555555555555555555555555555555555555555555555555555555555555555555555555555555555555555 -.1 I%?B(M<@ S<A(C: A%@ <E+7B(A$IB%
-.1.1 Informed!:euristic" Search Strategies -.1.- :euristic ?unctions -.1.9 7ocal Search Algorithms and BptimiHation +ro)lems -.1.. 7ocal Search in Continuous Spaces

-.1.8 Bnline Search Agents and Cnkno'n <nvironments 55555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555555

-.- CB%S$(AI%$ SA$IS?AC$IB% +(B07<MS!CS+"
-.-.1 Constraint Satisfaction +ro)lems -.-.- 0acktracking Search for CS+s -.-.9 $he Structure of +ro)lems

-.1 I%?B(M<@ S<A(C: A%@ <E+7B(A$IB%
-.1.1 Informed!:euristic" Search Strategies

Informed search strategy is one that uses problem>specific knowledge beyond the definition of the problem itself. It can find solutions more efficiently than uninformed strategy. 0est5first search 0est5first search is an instance of general %C44>94AC.+ or :CA3+>94AC.+ algorithm in which a node is selected for expansion based on an evaluation function f*n-. %he node with lowest evaluation is selected for expansion,because the evaluation measures the distance to the goal. %his can be implemented using a priority>0ueue,a data structure that will maintain the fringe in ascending order of f>values. -.1.-. :euristic functions A heuristic function or simply a heuristic is a function that ranks alternatives in various search algorithms at each branching step basing on an available information in order to make a decision which branch is to be followed during a search. %he key component of @est>first search algorithm is a heuristic function,denoted by h*n-& h*n- U extimated cost of the cheapest path from node n to a goal node. Gor example,in Comania,one might estimate the cost of the cheapest path from Arad to @ucharest via a straight5line distance from Arad to @ucharest*Gigure ?.!-.

+euristic function are the most common form in which additional knowledge is imparted to the search algorithm. *reedy 0est5first search *reedy )est5first search tries to expand the node that is closest to the goal,on the grounds that this is likely to a solution 0uickly. It evaluates the nodes by using the heuristic function f*n- U h*n-. %aking the example of (oute5finding pro)lems in Comania , the goal is to reach @ucharest starting from the city Arad. 5e need to know the straight>line distances to @ucharest from various cities as shown in Gigure ?.!. Gor example, the initial state is In*Arad- ,and the straight line distance heuristic h9E/*In*Arad-- is found to be <$$. Bsing the straight5line distance heuristic hS7@ =the goal state can be reached faster.

?igure -.1 Malues of h9E/ > straight line distances to @ucharest

?igure -.- stages in greedy best>first search for @ucharest using straight>line distance heuristic h9E/. 8odes are labeled with their h>values.

Gigure ?.? shows the progress of greedy best>first search using h9E/ to find a path from Arad to @ucharest. %he first node to be expanded from Arad will be 9ibiu,because it is closer to @ucharest than either Xerind or %imisoara. %he next node to be expanded will be Gagaras,because it is closest. Gagaras in turn generates @ucharest,which is the goal.

3roperties of greedy search
o Complete?? 8o6can get stuck in loops, e.g., Iasi Z 8eamt Z Iasi Z 8eamt Z .omplete in finite space with repeated>state checking o $ime?? J*bm-, but a good heuristic can give dramatic improvement o Space?? J*bm-;keeps all nodes in memory o Bptimal?? 8o :reedy best>first search is not optimal,and it is incomplete. %he worst>case time and space complexity is J*bm-,where m is the maximum depth of the search space.

AK Search
AK search ! MinimiHing the total estimated solution cost" %he most widely>known form of best>first search is called AK search *pronounced "A>star search"-. AK search is both complete and optimal. It evaluates nodes by combining g!n#, the cost to reach the node, and h!n.#,the cost to get from the node to the goal f*n- Ug*n- ] h*ng!n# - path cost from the start node to node n h!n# - estimated cost of the cheapest path from n to the goal f *n- > estimated cost of the cheapest solution through n

:euristic function # A good heuristic function for route>finding problems is 9traight>Eine /istance to the goal and it is denoted as h *n-. h *n- U 9traight>Eine distance between n and the goal locatation Galues of h 5straight5line distances to 0ucharest
9E/ 9E/ S7@

Time and space complexity: %ime complexity depends on the heuristic function and the admissible heuristic value. 9pace complexity remains in the exponential order. $he )ehavior of AK search Monotonicity !Consistency- In search tree any path from the root, the f> cost never decreases. %his condition is true for almost all admissible heuristics. A heuristic which satisfies this property is called monotonicity!consistency-. A heuristic h!n# is consistent if, for every node n and every successor n' of n generated by any action a, the estimated cost of reaching the goal from n is no greater than the step cost of getting to n' plus the estimated cost of reaching the goal from n'( f the heuristic is non-monotonic, then we have to make a minor correction that restores monotonicity. Bptimality A^ search is complete, optimal, and optimally efficient among all algorithms A^ using :CA3+>94AC.+ is optimal if h!n# is consistent. Completeness A^ is complete on locally finite graphs *graphs with a finite branching factor- provided there is some constant d such that every operator costs at least d. @ra')ack ") usually runs out of space because it keeps all generated nodes in memory Memory )ounded heuristic search %he simplest way to reduce memory re0uirements for A^ is to adapt the idea of iterative deepening to the heuristic search context, resulting in the iterative>deepening A" *I/A^- algorithm. %he memory re0uirements of A^ is reduced by combining the heuristic function with iterative deepening resulting an I/A^ algorithm. Iterative @eepening AK search !I@AK" /epth first search is modified to use a n f>cost limit rather than a depth limit for I/A^ algorithm. 4ach iteration in the search expands all the nodes inside the contour for the current f>cost and moves to the next contour with new f > cost. Space comple>ity is proportional to the longest path of exploration that is bd is a good estimate of storage re0uirements $ime comple>ity depends on the number of different values that the heuristic function can take on Optimality( yes, because it implies A^ search. Completeness( yes, because it implies A^ search. Disadvantage: It will re0uire more storage space in complex domains *i.e- 4ach contour will include only one state with the previous contour. %o avoid this, we increase the f-cost limit by a fixed amount on each iteration, so that the total number of iteration is proportional to !A. 9uch an algorithm is called admissible. %he two recent memory bounded algorithms are& ∈ ∈∈ R Cecursive @est Girst 9earch *C@f9R Memory bounded A^ search *MA^-

(ecursive 0est5first Search!(0?S"

Cecursive best>first search is a simple recursive algorithm that attempts to mimic the operation of standard best>first search,but using only linear space. %he algorithm is shown in figure ?.=. Its structure is similar to that of recursive depth>first search,but rather than continuing indefinitely down the current path,it keeps track of the f>value of the best alternative path available from any ancestor of the current node. If the current node exceeds this limit,the recursion unwinds back to the alternative path. As the recursion unwinds,C@G9 replaces the f>value of each node along the path with the best f>value of its children.

Gigure ?.# shows how C@G9 reaches @ucharest.

?igure -.9 9tages in A^ 9earch for @ucharest. 8odes are labeled with f U g ] h . %he h>values are the straight>line distances to @ucharest taken from figure ?.!

function RECURSIVE-BEST-FIRST-SEARCH(problem) return a solution or failure return RFBS(problem,MAKE- !"E(I ITIA#-STATE$problem%),&) function RFBS( problem, node, f_limit) return a solution or failure an' a ne( fcost li)it if *!A#-TEST$problem%(STATE$node%) then return node successors ← E+,A "(node, problem) if successors is e)-t. t/en return failure, & for each s in successors do f $s% ← )a0(g(s) 1 h(s), f $node%) repeat best ← t/e lo(est f-2alue no'e in successors if f $best% > f_limit then return failure, f $best% alternative ← t/e se3on' lo(est f-2alue a)on4 successors result, f $best% ← RBFS(problem, best, )in(f_limit, alternative)) if result ≠ failure then return result

?igure -.. %he algorithm for recursive best>first search

?igure -.8 9tages in an C@G9 search for the shortest route to @ucharest. %he f>limit value for each recursive call is shown on top of each current node. *a- %he path via Cimnicu Milcea is followed until the current best leaf *3itesti- has a value that is worse than the best alternative path *Gagaras-. *b- %he recursion unwinds and the best leaf value of the forgotten subtree *=!F- is backed up to Cimnicu Milcea7then Gagaras is expanded,revealing a best leaf value of =#2. *c- %he recursion unwinds and the best leaf value of the forgotten subtree *=#2- is backed upto Gagaras7 then Cimni Micea is expanded. %his time because the best alternative path*through %imisoara- costs atleast ==F,the expansion continues to @ucharest C@G9 <valuation & C@G9 is a bit more efficient than I/A^ – 9till excessive node generation *mind changes-

Eike A^, optimal if h!n# is admissible 9pace complexity is *!bd#. – I/A^ retains only one single number *the current f>cost limit%ime complexity difficult to characteri e – /epends on accuracy if h*n- and how often best path changes. I/A^ en C@G9 suffer from too little memory.

-.1.- :euristic ?unctions
A heuristic function or simply a heuristic is a function that ranks alternatives in various search algorithms at each branching step basing on an available information in order to make a decision which branch is to be followed during a search

?igure -.; A typical instance of the ,>pu le. %he solution is ?$ steps long. $he /5puHHle %he ,>pu le is an example of +euristic search problem. %he object of the pu le is to slide the tiles hori ontally or vertically into the empty space until the configuration matches the goal configuration*Gigure ?.$%he average cost for a randomly generated ,>pu le instance is about ?? steps. %he branching factor is about <.*5hen the empty tile is in the middle,there are four possible moves7when it is in the corner there are two7and when it is along an edge there are three-. %his means that an exhaustive search to depth ?? would look at about <?? approximately U <.! T !2!2 states. @y keeping track of repeated states,we could cut this down by a factor of about !F2,222,because there are only "ZA? U !,!,==2 distinct states that are reachable. %his is a manageable number ,but the corresponding number for the !#>pu le is roughly !2!<. If we want to find the shortest solutions by using A^,we need a heuristic function that never overestimates the number of steps to the goal. %he two commonly used heuristic functions for the !#>pu le are & *!- h! U the number of misplaced tiles. Gor figure ?.$ ,all of the eight tiles are out of position,so the start state would have h! U ,. h! is an admissible heuristic.

*?- h? U the sum of the distances of the tiles from their goal positions. %his is called the city )lock distance or Manhattan distance. h? is admissible ,because all any move can do is move one tile one step closer to the goal. %iles ! to , in start state give a Manhattan distance of h? U < ] ! ] ? ] ? ] ? ] < ] < ] ? U !,. 8either of these overestimates the true solution cost ,which is ?$. $he <ffective 0ranching factor Jne way to characteri e the Auality of a heuristic is the effective )ranching factor )K. If the total number of nodes generated by A^ for a particular problem is %,and the solution depth is d=then b^ is the branching factor that a uniform tree of depth d would have to have in order to contain 8]! nodes. %hus, 8 ] ! U ! ] b^ ] *b^-?](]*b^-d Gor example,if A^ finds a solution at depth # using #? nodes,then effective branching factor is !."?. A well designed heuristic would have a value of b^ close to !,allowing failru large problems to be solved. %o test the heuristic functions h! and h?,!?22 random problems were generated with solution lengths from ? to ?= and solved them with iterative deepening search and with A^ search using both h! and h?. Gigure ?.F gives the averaghe number of nodes expanded by each strategy and the effective branching factor. %he results suggest that h? is better than h!,and is far better than using iterative deepening search. Gor a solution length of !=,A^ with h? is <2,222 times more efficient than uninformed iterative deepening search.

?igure -.2 .omparison of search costs and effective branching factors for the I%4CA%IM4> /44348I8:>94AC.+ and A^ Algorithms with h!,and h?. /ata are average over !22 instances of the ,>pu le,for various solution lengths. Inventing admissi)le heuristic functions • (ela>ed pro)lems o A problem with fewer restrictions on the actions is called a relaxed problem o %he cost of an optimal solution to a relaxed problem is an admissible heuristic for the original problem o If the rules of the ,>pu le are relaxed so that a tile can move anywhere, then h+!n# gives the shortest solution

o If the rules are relaxed so that a tile can move to any ad,acent square, then h-!n# gives the shortest solution

-.1.9 7BCA7 S<A(C: A7*B(I$:MS A%@ B+$IMILA$IB% +(B07<MS
o In many optimi ation problems, the path to the goal is irrelevant7 the goal state itself is the solution o Gor example,in the ,>0ueens problem,what matters is the final configuration of 0ueens,not the order in which they are added. o In such cases, we can use local search algorithms. %hey operate using a single current state*rather than multiple paths- and generally move only to neighbors of that state. o %he important applications of these class of problems are *a- integrated>circuit design, *b-Gactory>floor layout,*c- job>shop scheduling,*d-automatic programming, *e-telecommunications network optimi ation,*f-Mehicle routing,and *g- portfolio management. &ey advantages of 7ocal Search Algorithms *!- %hey use very little memory 6 usually a constant amount7 and *?- they can often find reasonable solutions in large or infinite*continuous- state spaces for which systematic algorithms are unsuitable. B+$IMILA$IB% +(B07<MS Inaddition to finding goals,local search algorithms are useful for solving pure optimiHation pro)lems,in which the aim is to find the )est state according to an o)6ective function. State Space 7andscape %o understand local search,it is better explained using state space landscape as shown in figure ?.,. A landscape has both 'location) *defined by the state- and 'elevation)*defined by the value of the heuristic cost function or objective function-. If elevation corresponds to cost,then the aim is to find the lo'est valley 6 a glo)al minimum7 if elevation corresponds to an o)6ective function,then the aim is to find the highest peak 6 a glo)al ma>imum. Eocal search algorithms explore this landscape. A complete local search algorithm always finds a goal if one exists7 an optimal algorithm always finds a glo)al minimumDma>imum.

?igure -./ A one dimensional state space landscape in which elevation corresponds to the o)6ective function. %he aim is to find the global maximum. +ill climbing search modifies the current state to try to improve it ,as shown by the arrow. %he various topographic features are defined in the text

:ill5clim)ing search
%he hill5clim)ing search algorithm as shown in figure ?.", is simply a loop that continually moves in the direction of increasing value 6 that is,uphill. It terminates when it reaches a 'peak) where no neighbor has a higher value.

function +IEE>.EIM@I8:* problem- return a state that is a local maximum input# problem, a problem local varia)les# current= a node. neighbor= a node. current ¬ MA14>8J/4*I8I%IAE>9%A%4OproblemPloop do neighbor ¬ a highest valued successor of current if MAEB4 OneighborP . MAEB4OcurrentP then return 9%A%4OcurrentP current ¬ neighbor ?igure -.3 %he hill>climbing search algorithm *steepest ascent version-,which is the most basic local search techni0ue. At each step the current node is replaced by the best neighbor7the neighbor with the highest MAEB4. If the heuristic cost estimate h is used,we could find the neighbor with the lowest h. +ill>climbing is sometimes called greedy local search because it grabs a good neighbor state without thinking ahead about where to go next. :reedy algorithms often perform 0uite well.

+ro)lems 'ith hill5clim)ing
+ill>climbing often gets stuck for the following reasons & o 7ocal ma>ima & a local maximum is a peak that is higher than each of its neighboring states,but lower than the global maximum. +ill>climbing algorithms that reach the vicinity

of a local maximum will be drawn upwards towards the peak,but will then be stuck with nowhere else to go o (idges & A ridge is shown in Gigure ?.!2. Cidges results in a se0uence of local maxima that is very difficult for greedy algorithms to navigate. o +lateau> & A plateau is an area of the state space landscape where the evaluation function is flat. It can be a flat local maximum,from which no uphill exit exists,or a shoulder,from which it is possible to make progress.

?igure -.14 Illustration of why ridges cause difficulties for hill>climbing. %he grid of states*dark circles- is superimposed on a ridge rising from left to right,creating a se0uence of local maxima that are not directly connected to each other. Grom each local maximum,all th available options point downhill.

:ill5clim)ing variations
 Stochastic hill5clim)ing o Candom selection among the uphill moves. o %he selection probability can vary with the steepness of the uphill move.  ?irst5choice hill5clim)ing o cfr. stochastic hill climbing by generating successors randomly until a better one is found.  (andom5restart hill5clim)ing o %ries to avoid getting stuck in local maxima.

Simulated annealing search
A hill>climbing algorithm that never makes 'downhill) moves towards states with lower value*or higher cost- is guaranteed to be incomplete,because it can stuck on a local maximum.In contrast,a purely random walk 6that is,moving to a successor choosen uniformly at random from the set of successors 6 is complete,but extremely inefficient. 9imulated annealing is an algorithm that combines hill>climbing with a random walk in someway that yields both efficiency and completeness. Gigure ?.!! shows simulated annealing algorithm. It is 0uite similar to hill climbing. Instead of picking the best move,however,it picks the random move. If the move improves the situation,it is always accepted. Jtherwise,the algorithm accepts the move with some probability less than !. %he probability decreases exponentially with the 'badness) of the move 6 the amount 4 by which the evaluation is worsened.

9imulated annealing was first used extensively to solve ME9I layout problems in the early !",2s. It has been applied widely to factory scheduling and other large>scale optimi ation tasks.

?igure -.11 %he simulated annealing search algorithm,a version of stochastic hill climbing where some downhill moves are allowed.

*enetic algorithms
A :enetic algorithm*or :A- is a variant of stochastic beam search in which successor states are generated by combining two parent states,rather than by modifying a single state. Eike beam search,:as begin with a set of k randomly generated states,called the population. 4ach state,or individual,is represented as a string over a finite alphabet 6 most commonly,a string of 2s and !s. Gor example,an , ,>0uuens state must specify the positions of , 0ueens,each in acolumn of , s0uares,and so re0uires , x log? , U ?= bits.

?igure -.1- %he genetic algorithm. %he initial population in *a- is ranked by the fitness function in *b-,resulting in pairs for mating in *c-. %hey produce offspring in *d-,which are subjected to

mutation in *e-. Gigure ?.!? shows a population of four ,>digit strings representing ,>0ueen states. %he production of the next generation of states is shown in Gigure ?.!?*b- to *e-. In *b- each state is rated by the evaluation function or the fitness function. In *c-,a random choice of two pairs is selected for reproduction,in accordance with the probabilities in *b-. Gigure ?.!< describes the algorithm that implements all these steps. function :484%I.QAE:JCI%+M* population, GI%8499>G8- return an individual input# population, a set of individuals GI%8499>G8, a function which determines the 0uality of the individual repeat new/population ¬ empty set loop for i from ! to 9IX4*population- do x ¬ CA8/JMQ94E4.%IJ8*population, GI%8499QG8y ¬ CA8/JMQ94E4.%IJ8*population, GI%8499QG8child ¬ C43CJ/B.4*x,yif *small random probability- then child ¬ MB%A%4*child add child to new/population population ¬ new/population until some individual is fit enough or enough time has elapsed return the best individual ?igure -.19 A genetic algorithm. %he algorithm is same as the one diagrammed in Gigure ?.!?,with one variation&each mating of two parents produces only one offspring,not two.

-.1.. 7BCA7 S<A(C: I% CB%$I%CBCS S+AC<S
 5e have considered algorithms that work only in discrete environments, but real>world environment are continuous  Eocal search amounts to maximi ing a continuous objective function in a multi>dimensional vector space.  %his is hard to do in general.  .an immediately retreat o /iscreti e the space near each state o Apply a discrete local search strategy *e.g., stochastic hill climbing, simulated annealing Jften resists a closed>form solution o Gake up an empirical gradient o Amounts to greedy hill climbing in discreti ed state space  .an employ 8ewton>Caphson Method to find maxima  .ontinuous problems have similar problems& plateaus, ridges, local maxima, etc.

-.1.8 Bnline Search Agents and Cnkno'n <nvironments
Bnline search pro)lems

 Jffline 9earch *all algorithms so far .ompute complete solution, ignoring environment .arry out action se0uence  Jnline 9earch  Interleave computation and action  .ompute;Act;Jbserve;.ompute;_  Jnline search good  Gor dynamic, semi>dynamic, stochastic domains  5henever offline search would yield exponentially many contingencies  Jnline search necessary for exploration problem  9tates and actions unknown to agent  Agent uses actions as experiments to determine what to do 4xamples Cobot exploring unknown building .lassical hero escaping a labyrinth  Assume agent knows  Actions available in state s 9tep>cost function c*s,a,s`9tate s is a goal state  5hen it has visited a state s previously Admissible heuristic function h*s  8ote that agent doesnDt know outcome state *s ` - for a given action *a- until it tries the action *and all actions from a state s  .ompetitive ratio compares actual cost with cost agent would follow if it knew the search space  8o agent can avoid dead ends in all state spaces  Cobotics examples& 9taircase, ramp, cliff, terrain  Assume state space is safely explorable;some goal state is always reachable Bnline Search Agents  Interleaving planning and acting hamstrings offline search  A^ expands arbitrary nodes without waiting for outcome of action Jnline algorithm can expand only the node it physically occupies @est to explore nodes in physically local order  9uggests using depth>first search  8ext node always a child of the current  5hen all actions have been tried, canDt just drop state Agent must physically backtrack  Jnline /epth>Girst 9earch  May have arbitrarily bad competitive ratio *wandering past goal- Jkay for exploration7 bad for minimi ing path cost  Jnline Iterative>/eepening 9earch  .ompetitive ratio stays small for state space a uniform tree Bnline 7ocal Search  +ill .limbing 9earch  Also has physical locality in node expansions

Is, in fact, already an online search algorithm Eocal maxima problematic& canDt randomly transport agent to new state in effort to escape local maximum  Candom 5alk as alternative  9elect action at random from current state  5ill eventually find a goal node in a finite space  .an be very slow, esp. if 'backward) steps as common as 'forward)  +ill .limbing with Memory instead of randomness  9tore 'current best estimate) of cost to goal at each visited state 9tarting estimate is just h*s  Augment estimate based on experience in the state space %ends to 'flatten out) local minima, allowing progress 4mploy optimism under uncertainty  Bntried actions assumed to have least>possible cost 4ncourage exploration of untried paths  7earning in Bnline Search o Campant ignorance a ripe opportunity for learning Agent learns a 'map) of the environment o Jutcome of each action in each state o Eocal search agents improve evaluation function accuracy o Bpdate estimate of value at each visited state o 5ould like to infer higher>level domain model o 4xample& 'Bp) in ma e search increases y >coordinate Ce0uires o Gormal way to represent and manipulate such general rules *so far, have hidden rules within the successor functiono Algorithms that can construct general rules based on observations of the effect of actions

-.- CB%S$(AI%$ SA$IS?AC$IB% +(B07<MS!CS+"
A Constraint Satisfaction +ro)lem*or .93- is defined by a set of varia)les =T!,T?,(.Tn,and a set of constraints .!,.?,(,.m. 4ach variable Ti has a nonempty domain /,of possible values. 4ach constraint .i involves some subset of variables and specifies the allowable combinations of values for that subset. A State of the problem is defined by an assignment of values to some or all of the variables,VTi U vi,Tj U vj,(Y. An assignment that does not violate any constraints is called a consistent or legal assignment. A complete assignment is one in which every variable is mentioned,and a solution to a .93 is a complete assignment that satisfies all the constraints. 9ome .93s also re0uire a solution that maximi es an o)6ective function. <>ample for Constraint Satisfaction +ro)lem # Gigure ?.!# shows the map of Australia showing each of its states and territories. 5e are given the task of coloring each region either red,green,or blue in such a way that the neighboring regions have the same color. %o formulate this as .93 ,we define the variable to be the regions &5A,8%,L,895,M,9A, and %. %he domain of each variable is the set Vred,green,blueY.%he constraints re0uire neighboring regions to have distinct colors7for example,the allowable combinations for 5A and 8% are the pairs V*red,green-,*red,blue-,*green,red-,*green,blue-,*blue,red-,*blue,green-Y.

%he constraint can also be represented more succinctly as the ine0uality 5A not U 8%,provided the constraint satisfaction algorithm has some way to evaluate such expressions.- %here are many possible solutions such as V 5A U red, 8% U green,L U red, 895 U green, M U red ,9A U blue,% U redY. It is helpful to visuali e a .93 as a constraint graph,as shown in Gigure ?.!#*b-. %he nodes of the graph corresponds to variables of the problem and the arcs correspond to constraints.

?igure -.18 *a- 3rinciple states and territories of Australia. .oloring this map can be viewed as aconstraint satisfaction problem. %he goal is to assign colors to each region so that no neighboring regions have the same color.

?igure -.18 *b- %he map coloring problem represented as a constraint graph. .93 can be viewed as a standard search problem as follows &  Initial state & the empty assignment VY,in which all variables are unassigned.  Successor function & a value can be assigned to any unassigned variable,provided that it does not conflict with previously assigned variables.  *oal test & the current assignment is complete.  +ath cost & a constant cost*4.g.,!- for every step. 4very solution must be a complete assignment and therefore appears at depth n if there are n variables. /epth first search algorithms are popular for .93s

Garieties of CS+s !i" @iscrete varia)les ?inite domains %he simplest kind of .93 involves variables that are discrete and have finite domains. Map coloring problems are of this kind. %he ,>0ueens problem can also be viewed as finite>domain .93,where the variables L!,L?,(..L, are the positions each 0ueen in columns !,(., and each variable has the domain V!,?,<,=,#,$,F,,Y. If the maximum domain si e of any variable in a .93 is d,then the number of possible complete assignments is J*dn- 6 that is,exponential in the number of variables. Ginite domain .93s include 0oolean CS+s,whose variables can be either true or false. Infinite domains /iscrete variables can also have infinite domains 6 for example,the set of integers or the set of strings. 5ith infinite domains,it is no longer possible to describe constraints by enumerating all allowed combination of values. Instead a constraint language of algebric ine0ualities such as 9tartjob! ] # [U 9tartjob<. !ii" CS+s 'ith continuous domains .93s with continuous domains are very common in real world. Gor example ,in operation research field,the scheduling of experiments on the +ubble %elescope re0uires very precise timing of observations7 the start and finish of each observation and maneuver are continuous>valued variables that must obey a variety of astronomical,precedence and power constraints. %he best known category of continuous>domain .93s is that of linear programming problems,where the constraints must be linear ine0ualities forming a convex region. Einear programming problems can be solved in time polynomial in the number of variables. Garieties of constraints # !i" unary constraints involve a single variable. 4xample & 9A a green *ii- @inary constraints involve paris of variables. 4xample & 9A a 5A *iii- +igher order constraints involve < or more variables. 4xample & cryptarithmetic pu les.

?igure -.1; *a- .ryptarithmetic problem. 4ach letter stands for a distinct digit7the aim is to find a substitution of digits for letters such that the resulting sum is arithmetically correct,with the added restriction that no leading eros are allowed. *b- %he constraint hypergraph for the cryptarithmetic problem,showint the "lldiff constraint as well as the column addition constraints. 4ach constraint is a s0uare box connected to the variables it

contains.

-.-.- 0acktracking Search for CS+s
%he term )acktracking search is used for depth>first search that chooses values for one variable at a time and backtracks when a variable has no legal values left to assign. %he algorithm is shown in figure ?.!F.

?igure -.12 A simple backtracking algorithm for constraint satisfaction problem. %he algorithm is modeled on the recursive depth>first search

?igure -.12!)" 3art of search tree generated by simple backtracking for the map coloring problem. +ropagating information through constraints
9o far our search algorithm considers the constraints on a variable only at the time that the variable is chosen by 94E4.%>B8A99I:84/>MACIA@E4. @ut by looking at some of the constraints earlier in the search, or even before the search has started, we can drastically reduce the search space. ?or'ard checking

Jne way to make better use of constraints during search is called for'ard checking. 5henever a variable X is assigned, the forward checking process looks at each unassigned variable / that is connected to X by a constraint and deletes from / Ds domain any value that is inconsistent with the value chosen for X. Gigure #.$ shows the progress of a map>coloring search with forward checking.

Constraint propagation
Although forward checking detects many inconsistencies, it does not detect all of them. Constraint propagation is the general term for propagating the implications of a constraint on one variable onto other variables.

Arc Consistency

k5Consistency

7ocal Search for CS+s

-.-.9 $he Structure of +ro)lems +ro)lem Structure

Independent Su)pro)lems

$ree5Structured CS+s