Download AI Problem-Solving Techniques: State Space Search, Heuristic Search, and Hill Climbing and more Study notes Computer-Aided Analysis of Machine Dynamics in PDF only on Docsity!
Module 1
ARTIFICIAL INTELLIGENCE
What is Artificial Intelligence?
It is a branch of Computer Science that pursues creating the computers or machines as intelligent as human beings. It is the science and engineering of making intelligent machines, especially intelligent computer programs. It is related to the similar task of using computers to understand human intelligence, but AI does not have to confine itself to methods that are biologically observable Definition: Artificial Intelligence is the study of how to make computers do things, which, at the moment, people do better. According to the father of Artificial Intelligence, John McCarthy, it is “The science and engineering of making intelligent machines, especially intelligent computer programs”. Artificial Intelligence is a way of making a computer, a computer-controlled robot, or a software think intelligently , in the similar manner the intelligent humans think. AI is accomplished by studying how human brain thinks and how humans learn, decide, and work while trying to solve a problem, and then using the outcomes of this study as a basis of developing intelligent software and systems. It has gained prominence recently due, in part, to big data, or the increase in speed, size and variety of data businesses are now collecting. AI can perform tasks such as identifying patterns in the data more efficiently than humans, enabling businesses to gain more insight out of their data. From a business perspective AI is a set of very powerful tools, and methodologies for using those tools to solve business problems. From a programming perspective, AI includes the study of symbolic programming, problem solving, and search. AI Vocabulary Intelligence relates to tasks involving higher mental processes, e.g. creativity, solving problems, pattern recognition, classification, learning, induction, deduction, building analogies, optimization, language processing, knowledge and many more. Intelligence is the computational part of the ability to achieve goals. Intelligent behaviour is depicted by perceiving one’s environment, acting in complex environments, learning and understanding from experience, reasoning to solve problems and discover hidden knowledge, applying knowledge successfully in new situations, thinking abstractly, using analogies, communicating with others and more.
Science based goals of AI pertain to developing concepts, mechanisms and understanding biological intelligent behaviour. The emphasis is on understanding intelligent behaviour. Engineering based goals of AI relate to developing concepts, theory and practice of building intelligent machines. The emphasis is on system building. AI Techniques depict how we represent, manipulate and reason with knowledge in order to solve problems. Knowledge is a collection of ‘facts’. To manipulate these facts by a program, a suitable representation is required. A good representation facilitates problem solving. Learning means that programs learn from what facts or behaviour can represent. Learning denotes changes in the systems that are adaptive in other words, it enables the system to do the same task(s) more efficiently next time. Applications of AI refers to problem solving, search and control strategies, speech recognition, natural language understanding, computer vision, expert systems, etc.
Problems of AI:
Intelligence does not imply perfect understanding; every intelligent being has limited perception, memory and computation. Many points on the spectrum of intelligence versus cost are viable, from insects to humans. AI seeks to understand the computations required from intelligent behaviour and to produce computer systems that exhibit intelligence. Aspects of intelligence studied by AI include perception, communicational using human languages, reasoning, planning, learning and memory. The following questions are to be considered before we can step forward:
- What are the underlying assumptions about intelligence?
- What kinds of techniques will be useful for solving AI problems?
- At what level human intelligence can be modelled?
- When will it be realized when an intelligent program has been built?
Branches of AI:
A list of branches of AI is given below. However some branches are surely missing, because no one has identified them yet. Some of these may be regarded as concepts or topics rather than full branches. Logical AI — In general the facts of the specific situation in which it must act, and its goals are all represented by sentences of some mathematical logical language. The program decides what to do by inferring that certain actions are appropriate for achieving its goals.
Heuristics — A heuristic is a way of trying to discover something or an idea embedded in a program. The term is used variously in AI. Heuristic functions are used in some approaches to search or to measure how far a node in a search tree seems to be from a goal. Heuristic predicates that compare two nodes in a search tree to see if one is better than the other, i.e. constitutes an advance toward the goal, and may be more useful. Genetic programming — Genetic programming is an automated method for creating a working computer program from a high-level problem statement of a problem. Genetic programming starts from a high-level statement of ‘what needs to be done’ and automatically creates a computer program to solve the problem.
Applications of AI
AI has applications in all fields of human study, such as finance and economics, environmental engineering, chemistry, computer science, and so on. Some of the applications of AI are listed below: Perception ■ Machine vision ■ Speech understanding ■ Touch ( tactile or haptic ) sensation Robotics Natural Language Processing ■ Natural Language Understanding ■ Speech Understanding ■ Language Generation ■ Machine Translation Planning Expert Systems Machine Learning Theorem Proving Symbolic Mathematics Game Playing AI Technique: Artificial Intelligence research during the last three decades has concluded that Intelligence requires knowledge. To compensate overwhelming quality, knowledge possesses less desirable properties. A. It is huge. B. It is difficult to characterize correctly. C. It is constantly varying. D. It differs from data by being organized in a way that corresponds to its application. E. It is complicated.
An AI technique is a method that exploits knowledge that is represented so that: The knowledge captures generalizations that share properties, are grouped together, rather than being allowed separate representation. It can be understood by people who must provide it—even though for many programs bulk of the data comes automatically from readings. In many AI domains, how the people understand the same people must supply the knowledge to a program. It can be easily modified to correct errors and reflect changes in real conditions. It can be widely used even if it is incomplete or inaccurate. It can be used to help overcome its own sheer bulk by helping to narrow the range of possibilities that must be usually considered. In order to characterize an AI technique let us consider initially OXO or tic-tac-toe and use a series of different approaches to play the game. The programs increase in complexity, their use of generalizations, the clarity of their knowledge and the extensibility of their approach. In this way they move towards being representations of AI techniques.
Example-1: Tic-Tac-Toe
1.1 The first approach (simple) The Tic-Tac-Toe game consists of a nine element vector called BOARD; it represents the numbers 1 to 9 in three rows. An element contains the value 0 for blank, 1 for X and 2 for O. A MOVETABLE vector consists of 19,683 elements (3^9 ) and is needed where each element is a nine element vector. The contents of the vector are especially chosen to help the algorithm. The algorithm makes moves by pursuing the following:
- View the vector as a ternary number. Convert it to a decimal number.
- Use the decimal number as an index in MOVETABLE and access the vector.
- Set BOARD to this vector indicating how the board looks after the move. This approach is capable in time but it has several disadvantages. It takes more space and requires stunning
- What did Rani find that she liked?
- Did Rani buy anything? Method 1 2.3 Data Structures A set of templates that match common questions and produce patterns used to match against inputs. Templates and patterns are used so that a template that matches a given question is associated with the corresponding pattern to find the answer in the input text. For example, the template who did x y generates x y z if a match occurs and z is the answer to the question. The given text and the question are both stored as strings. 2.4 Algorithm Answering a question requires the following four steps to be followed: Compare the template against the questions and store all successful matches to produce a set of text patterns. Pass these text patterns through a substitution process to change the person or voice and produce an expanded set of text patterns. Apply each of these patterns to the text; collect all the answers and then print the answers. 2.5 Example In question 1 we use the template WHAT DID X Y which generates Rani go shopping for z and after substitution we get Rani goes shopping for z and Rani went shopping for z giving z [equivalence] a new coat In question 2 we need a very large number of templates and also a scheme to allow the insertion of ‘find’ before ‘that she liked’; the insertion of ‘really’ in the text; and the substitution of ‘she’ for ‘Rani’ gives the answer ‘a red one’. Question 3 cannot be answered. 2.6 Comments This is a very primitive approach basically not matching the criteria we set for intelligence and worse than that, used in the game. Surprisingly this type of technique was actually used in ELIZA which will be considered later in the course.
Method 2 2.7 Data Structures A structure called English consists of a dictionary, grammar and some semantics about the vocabulary we are likely to come across. This data structure provides the knowledge to convert English text into a storable internal form and also to convert the response back into English. The structured representation of the text is a processed form and defines the context of the input text by making explicit all references such as pronouns. There are three types of such knowledge representation systems: production rules of the form ‘if x then y’, slot and filler systems and statements in mathematical logic. The system used here will be the slot and filler system. Take, for example sentence: ‘She found a red one she really liked’. Event2 Event instance: finding instance: liking tense: past tense: past agent: Rani modifier: much object: Thing1 object: Thing Thing instance: coat colour: red The question is stored in two forms: as input and in the above form. 2.8 Algorithm Convert the question to a structured form using English know how, then use a marker to indicate the substring (like ‘who’ or ‘what’) of the structure, that should be returned as an answer. If a slot and filler system is used a special marker can be placed in more than one slot. The answer appears by matching this structured form against the structured text. The structured form is matched against the text and the requested segments of the question are returned. 2.9 Examples Both questions 1 and 2 generate answers via a new coat and a red coat respectively. Question 3 cannot be answered, because there is no direct response. 2.10 Comments This approach is more meaningful than the previous one and so is more effective. The extra power given must be paid for by additional search time in the knowledge bases. A warning
Answer: Because she was not in. This answer is derived because we have supplied an additional fact that a person cannot be in two places at once. This patch is not sufficiently general so as to work in all cases and does not provide the type of solution we are really looking for.
LEVEL OF THE AI MODEL
‘What is our goal in trying to produce programs that do the intelligent things that people do?’ Are we trying to produce programs that do the tasks the same way that people do? OR Are we trying to produce programs that simply do the tasks the easiest way that is possible? Programs in the first class attempt to solve problems that a computer can easily solve and do not usually use AI techniques. AI techniques usually include a search, as no direct method is available, the use of knowledge about the objects involved in the problem area and abstraction on which allows an element of pruning to occur, and to enable a solution to be found in real time; otherwise, the data could explode in size. Examples of these trivial problems in the first class, which are now of interest only to psychologists are EPAM (Elementary Perceiver and Memorizer) which memorized garbage syllables. The second class of problems attempts to solve problems that are non-trivial for a computer and use AI techniques. We wish to model human performance on these:
- To test psychological theories of human performance. Ex. PARRY [Colby, 1975] – a program to simulate the conversational behavior of a paranoid person.
- To enable computers to understand human reasoning – for example, programs that answer questions based upon newspaper articles indicating human behavior.
- To enable people to understand computer reasoning. Some people are reluctant to accept computer results unless they understand the mechanisms involved in arriving at the results.
- To exploit the knowledge gained by people who are best at gathering information. This persuaded the earlier workers to simulate human behavior in the SB part of AISB simulated behavior. Examples of this type of approach led to GPS (General Problem Solver). Questions for Practice:
- What is intelligence? How do we measure it? Are these measurements useful?
- When the temperature falls and the thermostat turns the heater on, does it act because it believes the room to be too cold? Does it feel cold? What sorts of things can have beliefs or feelings? Is this related to the idea of consciousness?
- Some people believe that the relationship between your mind (a non-physical thing) and your brain (the physical thing inside your skull) is exactly like the relationship between a computational process (a non-physical thing) and a physical computer. Do you agree?
- How good are machines at playing chess? If a machine can consistently beat all the best human chess players, does this prove that the machine is intelligent?
- What is AI Technique? Explain Tic-Tac-Toe Problem using AI Technique.
Analyze the problem – find few important features that may have impact on the appropriateness of various possible techniques for solving the problem Isolate and represent task knowledge necessary to solve the problem Choose the best problem-solving technique(s) and apply to the particular problem Problem definitions A problem is defined by its ‘ elements ’ and their ‘ relations ’. To provide a formal description of a problem, we need to do the following: a. Define a state space that contains all the possible configurations of the relevant objects, including some impossible ones. b. Specify one or more states that describe possible situations, from which the problem- solving process may start. These states are called initial states. c. Specify one or more states that would be acceptable solution to the problem. These states are called goal states. Specify a set of rules that describe the actions ( operators ) available. The problem can then be solved by using the rules , in combination with an appropriate control strategy , to move through the problem space until a path from an initial state to a goal state is found. This process is known as ‘search’. Thus: Search is fundamental to the problem-solving process. Search is a general mechanism that can be used when a more direct method is not known. Search provides the framework into which more direct methods for solving subparts of a problem can be embedded. A very large number of AI problems are formulated as search problems. Problem space A problem space is represented by a directed graph, where nodes represent search state and paths represent the operators applied to change the state. To simplify search algorithms, it is often convenient to logically and programmatically represent a problem space as a tree. A tree usually decreases the complexity of a search at a cost. Here, the cost is due to duplicating some nodes on the tree that were linked numerous times in the graph, e.g. node B and node D. A tree is a graph in which any two vertices are connected by exactly one path. Alternatively, any connected graph with no cycles is a tree.
- Problem solving: The term, Problem Solving relates to analysis in AI. Problem solving may be characterized as a systematic search through a range of possible actions to reach some predefined goal or solution. Problem-solving methods are categorized as special purpose and general purpose.
- A special-purpose method is tailor-made for a particular problem, often exploits very specific features of the situation in which the problem is embedded.
- A general-purpose method is applicable to a wide variety of problems. One General-purpose technique used in AI is ‘means-end analysis’: a step-bystep, or incremental, reduction of the difference between current state and final goal.
Initial condition Goal comment
- (four, three) if four < 4 (4, three) fill four from tap
- (four, three) if three< 3 (four, 3) fill three from tap
- (four, three) If four > 0 (0, three) empty four into drain
- (four, three) if three > 0 (four, 0) empty three into drain
- (four, three) if four + three<4 (four + three, 0) empty three into four
- (four, three) if four + three<3 (0, four + three) empty four into three
- (0, three) If three > 0 (three, 0) empty three into four
- (four, 0) if four > 0 (0, four) empty four into three
- (0, 2) (2, 0) empty three into four
- (2, 0) (0, 2) empty four into three
- (four, three) if four < 4 (4, three-diff) pour diff, 4-four, into four from three
- (three, four) if three < 3 (four-diff, 3) pour diff, 3-three, into three from four and a solution is given below four three rule (Fig. 2.2 Production Rules for the Water Jug Problem) Gallons in Four Jug Gallons in Three Jug Rules Applied 0 0 - 0 3 2 3 0 7 3 3 2 4 2 11 0 2 3 2 0 10 (Fig. 2.3 One Solution to the Water Jug Problem) The problem solved by using the production rules in combination with an appropriate control strategy, moving through the problem space until a path from an initial state to a goal state is found. In this problem solving process, search is the fundamental concept. For simple problems it is easier to achieve this goal by hand but there will be cases where this is far too difficult. 2.4 PRODUCTION SYSTEMS Production systems provide appropriate structures for performing and describing search processes. A production system has four basic components as enumerated below. A set of rules each consisting of a left side that determines the applicability of the rule and a right side that describes the operation to be performed if the rule is applied. A database of current facts established during the process of inference.
A control strategy that specifies the order in which the rules will be compared with facts in the database and also specifies how to resolve conflicts in selection of several rules or selection of more facts. A rule firing module. The production rules operate on the knowledge database. Each rule has a precondition—that is, either satisfied or not by the knowledge database. If the precondition is satisfied, the rule can be applied. Application of the rule changes the knowledge database. The control system chooses which applicable rule should be applied and ceases computation when a termination condition on the knowledge database is satisfied. Example: Eight puzzle (8-Puzzle) The 8-puzzle is a 3 × 3 array containing eight square pieces, numbered 1 through 8, and one empty space. A piece can be moved horizontally or vertically into the empty space, in effect exchanging the positions of the piece and the empty space. There are four possible moves, UP (move the blank space up), DOWN, LEFT and RIGHT. The aim of the game is to make a sequence of moves that will convert the board from the start state into the goal state: This example can be solved by the operator sequence UP, RIGHT, UP, LEFT, DOWN. Example: Missionaries and Cannibals The Missionaries and Cannibals problem illustrates the use of state space search for planning under constraints: Three missionaries and three cannibals wish to cross a river using a two person boat. If at any time the cannibals outnumber the missionaries on either side of the river, they will eat the missionaries. How can a sequence of boat trips be performed that will get everyone to the other side of the river without any missionaries being eaten? State representation:
- BOAT position: original (T) or final (NIL) side of the river.
- Number of Missionaries and Cannibals on the original side of the river.
- Start is (T 3 3); Goal is (NIL 0 0). Operators:
2.4.1 Control Strategies The word ‘ search ’ refers to the search for a solution in a problem space.
- Search proceeds with different types of ‘ search control strategies ’.
- A strategy is defined by picking the order in which the nodes expand. The Search strategies are evaluated along the following dimensions: Completeness, Time complexity, Space complexity, Optimality (the search- related terms are first explained, and then the search algorithms and control strategies are illustrated next). Search-related terms
- Algorithm’s performance and complexity Ideally we want a common measure so that we can compare approaches in order to select the most appropriate algorithm for a given situation. Performance of an algorithm depends on internal and external factors. Internal factors/ External factors Time required to run Size of input to the algorithm Space (memory) required to run Speed of the computer Quality of the compiler Complexity is a measure of the performance of an algorithm. Complexity measures the internal factors, usually in time than space.
- *Computational complexity* It is the measure of resources in terms of Time and Space. If A is an algorithm that solves a decision problem f , then run-time of A is the number of steps taken on the input of length n. Time Complexity T(n) of a decision problem f is the run-time of the ‘best’ algorithm A for f. Space Complexity S(n) of a decision problem f is the amount of memory used by the ‘best’ algorithm A for f.
- ‘ Big - O ’ notation The Big-O , theoretical measure of the execution of an algorithm , usually indicates the time or the memory needed, given the problem size n , which is usually the number of items.
- Big-O notation The Big-O notation is used to give an approximation to the run-time- efficiency of an algorithm; the letter ‘ O ’ is for order of magnitude of operations or space at run-time.
- The Big-O of an Algorithm A If an algorithm A requires time proportional to f(n) , then algorithm A is said to be of order f(n) , and it is denoted as O(f(n)). If algorithm A requires time proportional to n2 , then the order of the algorithm is said to be O(n2). If algorithm A requires time proportional to n , then the order of the algorithm is said to be O(n).
The function f(n) is called the algorithm’s growth-rate function. In other words, if an algorithm has performance complexity O(n) , this means that the run-time t should be directly proportional to n , ie t • n or t = k n where k is constant of proportionality. Similarly, for algorithms having performance complexity O(log2(n)), O(log N), O(N log N), O(2N) and so on. Example 1: Determine the Big-O of an algorithm: Calculate the sum of the n elements in an integer array a[0..n-1]. Line no. Instructions No of execution steps line 1 sum 1 line 2 for (i = 0; i < n; i++) n + 1 line 3 sum += a[i] n line 4 print sum 1 Total 2n + 3 Thus, the polynomial (2n + 3) is dominated by the 1st term as n while the number of elements in the array becomes very large.
- In determining the Big-O , ignore constants such as 2 and 3. So the algorithm is of order n.
- So the Big-O of the algorithm is O(n).
- In other words the run-time of this algorithm increases roughly as the size of the input data n , e.g., an array of size n. Tree structure Tree is a way of organizing objects, related in a hierarchical fashion.
- Tree is a type of data structure in which each element is attached to one or more elements directly beneath it.
- The connections between elements are called branches.
- Tree is often called inverted trees because it is drawn with the root at the top.
- The elements that have no elements below them are called leaves.
- A binary tree is a special type: each element has only two branches below it. Properties
- Tree is a special case of a graph.
- The topmost node in a tree is called the root node.
- At root node all operations on the tree begin.
- A node has at most one parent.
- The topmost node (root node) has no parents.
- Each node has zero or more child nodes , which are below it.
- The nodes at the bottommost level of the tree are called leaf nodes. Since leaf nodes are at the bottom most level, they do not have children.
- A node that has a child is called the child’s parent node.
- The depth of a node n is the length of the path from the root to the node.
- The root node is at depth zero.
