Lecture Notes Alex Aiken CS 264, Spring 95 Topic: Attribute Grammars and Incremental Semantic Analysis ----------------------------------------------------------- I Semantic Actions So far, we have indicated that the job of parsing is to build a parse tree. Actually, this isn't true; building a parse tree is only one of many things that a parser can do. Parsing tools use a generalization of CFG's in which each grammar symbol has one or more values, called "attributes", associated with it. Each production of the grammar may have an associated "action", which can refer to and compute the values of attributes. So we have: terminals & non-terminals ... have attributes productions ... have semantic actions Example: E -> E' + E | E' E' -> int * E' | int For each symbol, let X.val be an integer value associated with X. - For terminal symbols, val is the lexeme provided by the lexical analyzer. - For non-terminals, val should be the integer value of the expression. This attribute is computed from the attributes of sub-expressions. Production Action E -> E' + E1 E.val = E'.val + E1.val | E' E.val = E'.val E' -> int * E1' E'.val = int.val * E1'.val | int E'.val = int.val Note: the attribute of some grammar symbols, such as the terminals + and *, is unused. Example: 5 * 3 + 2 * 4 Parse Tree Equations E1 E1.val = E3'.val + E2.val ------------------ E3'.val = int7.val + E4'.val E3' + E2 E4'.val = int8.val --------- ---- E2.val = E5'.val int7 * E4' E5' E5'.val = int9.val * E6'.val --- ---------- E6'.val = int0.val int8 int9 * E6' int7.val = 5 ---- int8.val = 3 int0 int9.val = 2 int0.val = 4 Working from the leaves to the root, we can compute each val attribute. For example, E6'.val = 4 and E5'.val = 8. Finally, E1.val = 23. Notes: (1) Fresh attributes are associated with every node in the parse tree. (2) The semantic actions specify a system of equations; they don't say in what order the equations are resolved. The user just gives a specification and the parser takes care of the implementation. Warning: you can use side-effects in semantic actions, but in this case you have to understand the order in which attributes get computed or the results will seem unpredictable. (3) In this example, the val attribute can be evaluated bottom-up: the .val attribute for a node of the parse depends only on the .val attributes of its children. (4) The parse tree need not actually be built by the parser. In fact, a parser tool would compile this specification into code that simply traces out the structure of the parse tree without actually building it. (5) Pattern/action parsing can be though of as a systematic translation of the original text into a new form specified by the semantic actions. Because the translation is guided by the syntax, it is called syntax-directed translation. (NB: Book uses SDT in a narrower sense.) (6) Attributes may also be passed top-down: an attribute of a node may depend on an attribute of the parent in the parse tree. Such an attributed is called "inherited". We will talk about inherited attributes eventually, but they will not be used in the course project. II Synthesized and Inherited Attributes - Synthesized: attribute value depends on descendants of the node Example: the val attribute above - Inherited: attribute value depends on parent and siblings of the node Example: symbol table environment - What is an attribute that depends on both? III S-attributed Definitions - An attribute grammar is S-attributed if it consists only of inherited attributes - Can be evaluated bottom-up: - Keep a stack S parallel to parsing stack - consider production A -> XY A.val = X.val + Y.val - When reducing by A -> XY - the top of the S stack has X.val and Y.val - compute A.val - pop X.val and Y.val from S, push A.val - symmetric with reduce action on the parse stack - Tools like Bison/Flex support S-attributed definitions IV. Evaluating Attributes - S attributed definitions are a very special case of attribute grammars - The most general method is to construct an ordering from the parse tree itself: Define a graph as follows. For each attribute E.a to be computed add a node in the graph. If E.a depends on E1.a1,...,En.an then add directed edges from Ei.ai to E.a for 1 <= i <= n. A topological sort of the graph is any ordering n1,...,nk of the nodes such that edges of the graph are all from left-to-right in the ordering; i.e., a node appears in the ordering after all of the nodes it depends on. Any topological sort is a legal evaluation order of the attributes. - Note: for the topological sort to make sense, there can be no cycles in the graph. Cyclically defined attributes are not legal. ==> can make sense even cyclically defined attributes if they are treated as recursive definitions - In practice, computing all of the attribute dependencies from the AST is rarely, if ever, used. Instead, special cases of syntax-directed definitions are used where the attribute evaluation order can be determined once and for all from the actions. - The most important special case is S-attributed grammars: grammars with only synthesized attributes. Building an AST is an example of an S-attributed grammar (i.e., PA3). These attributes can be evaluated bottom-up during parsing. V. Testing For Circularity - If an attribute grammar has a dependence cycle among attributes in some parse tree, then the attribute grammar is said to be circular. - Circular attribute grammars are considered meaningless---that is, erroneous. - It is possible to check whether a given attribute grammar is circular. - First, some intuition. Consider the attribute grammar: --- this example from page 331-332 of Dragon Book --- - NOTICE: - Detecting circularity requires understanding transitive chains of attribute dependencies. - Consider a single production A -> alpha. There are three possible kinds of paths involving multiple attributes of alpha: 1. A path completely among the attributes of alpha 2. A path that goes from some attribute of alpha to attributes of the subtree and back 3. A path that goes from some attribute of alpha to attributes outside the subtree and back - In particular any path originating outside the subtree of A and crossing into the subtree of A must pass through an attribute of A. - This is a key fact exploited by many attribute grammar algorithms. - Now consider the grammar: S -> A A.i := c A -> 1 A.s := f(A.i) A -> 2 A.s := d There is a dependency path A.i -> A.s if and only if the production A -> 1 is used. Thus, the pattern of dependencies can vary depending on the production used, and it is necessary to take this into account in a test for circularity. - Definition: For a production P -> X1...Xn D(P) is the graph of the dependencies between attributes of P,X1,...,Xn - Definition If G1 = (V1,E1) G2 = (V2,E2) then G1 u G2 = (V1 u V2, E1 u E2) - Definition If G1 = (V1,E1) and V <= V1 then G1/V = (V,E') where E' = {(v,w) | v,w \in V and v -*-> w in E1 and only v,w are in V on the path} - Idea for an algorithm: For each production P -> alpha, maintain (a set of) graphs for the dependencies among attributes of alpha. Each such graph has |alpha| nodes and at most |alpha|^2 edges. Note that there are only a finite set of graphs of a certain size. Initially, G(X) = {(Gx,Vx}} where Gx = attributes of X Ex = {} STEP: let P -> X1...Xn be a production let Gi \in G(Xi) let N = D(p) u G1 u ... u Gn if N is cyclic then fail else add N//attributes(P -> X1...Xn) to G(P) ALGORITHM: repeat STEP until no new graphs are generated or until failure Why does this terminate: graphs are all of bounded size => only finitely many graphs can be generated. Why is it correct? Sketch: Every cycle in a parse tree is contained within some subtree of some fixed height k. The initial graphs capture all paths in trees of height 0 (i.e., no paths at all). For a production P -> X1...Xn, if there are graphs G1,...,Gn showing the paths between attributes through trees of height i for X1,...,Xn, then D(P) u G1 u ... u Gn characterizes all paths through trees of height i + 1. Thus, eventually this procedure is guaranteed to detect a cycle if one exists in any parse tree. Note that for a production P -> XY, the step will take the complete cross product of graphs of X and Y. This is correct because the language is context free and X and Y do not constrain each other in any way. VI. Strongly Non-Circular Just like circularity, except we keep only one graph for each non-terminal X. This graph is just the union of all the individual graphs for X in the circularity test. The advantage of strong non-circularity is that it can be tested in polynomial, rather than exponential time. VII. Incremental Analysis - Incremental program analysis is one of the most interesting applications of attribute grammars. - Idea: Consider a parse tree of a program with all of its attributes evaluated. Now modify the parse tree by deleting some subtree and inserting a new one. The problem is to update the attributes with minimum work. - Definition: Let AFFECTED by the set of attribute values that change as result of an edit. The goal is recompute attribute values in time O(|AFFECTED|). - Very naive solution: recompute all attributes. - Less naive: recompute any attribute for which an argument has changed. May not be optimal! A.val := X.val + Y.val Y.val := Z.val * -1 X.val := Z.val Say that an edit causes the value of Z to change from 1 to -1. Attribute dependencies: Z -> X, Z -> Y, X -> A, Y -> A Must update attributes in some order respecting this partial order. Example: X := -1 A := -1 + -1 = -2 -- attributes depending on A updated -- Y := 1 A := -1 + 1 = 0 -- attributes depending on A updated -- Obviously, there is a better order in this case. However, getting an optimal algorithm is complicated by two things: 1. In general, a program edit will introduce some new attributes and remove some old attributes (the set of attributes is not fixed) 2. Computing the entire attribute dependency graph is too expensive for an O(|AFFECTED|) bound. The work spent computing attribute dependencies has to stay within the time bound, which forces us to use a spare representation. VIII. Definitions 1. D(p) = attribute dependency graph for a production p a.val = x.val + y.val x -> a y -> a --> dependencies explicit within the production (not induced by context in a particular parse 2. Program development is incremental; arbitrary subtrees may be missing. To accommodate this, there is a production X -> bottom for every nonterminal X. 3. fully attributed parse tree = every attribute is evaluated and consistent 4. inconsistent: some attribute equation is inconsistent 5. subtree replacement of T by T' at r Let T and T' be two fully attributed trees. Delete subtree rooted at r from T. Assign synthesized attributes of r to attributes of r' (root of T') Replace subtree at r by T'. ==> Any inconsistencies must be in attributes of r' - inherited attributes not consistent with equations at r' - synthesized attributes ... - WHY IS THIS DEFINITION RIGHT? IX Keeping Track of Attribute Dependencies Idea: At each stage, we have a blob of contiguous nodes that we know are in AFFECTED. - within the blob, we can track attribute dependencies explicitly. - The problem is with dependency chains that exist outside of the currently known AFFECTED set. For example: A.a -> B.b -> A.c ==> A.a -*-> A.c even this edge is not explicit. - Two cases: At the bottom of the known AFFECTED region, an inherited attribute contributes to the value of some synthesized attribute. At the top, some synthesized attribute contributes to an inherited attribute. - More formally: Superior Characteristic Graph: For a given node in a parse tree, the set of edges induced on the attributes of that node by everything NOT in the subtree. (D(T) - D(T_r))/V_r Subordinate Characteristic Graph: For a given node in a parse tree, the set of edges induced on the attributes of the subtree. D(T_r)/V_r - The AFFECTED set is a subtree of the parse tree. For each element of the frontier, we need the subordinate dependency graph. For the root, we need the superior graph. - The current set of known changed nodes is the "model". Algorithm for propagating changes: Pick an attribute in the model with in-degree 0 in the model (no edges from other model attributes) Evaluate the attribute If value changed and some dependent attribute is not in the model then EXPAND model to include the production with the missing attribute Remove edges b->x for each successor x of b in the model. EXPAND if new root -> expand superior graph if new leaf -> expand subordinate graph X How Do We Know the Superior and Subordinate Graphs? - easy to compute for empty program - Easy to compute after a modification to the program; superior graphs unchanged on path to root from current edit point; subordinate graphs unchanged outside of graft. INVARIANT: from current cursor position: Maintain superior graphs on path to root Maintain subordinate graphs everywhere else Easy to check that after a modification, the new superior and subordinate graphs can be recomputed from adjacent nodes. XI Efficiency Improvements - bound so far is only asymptotic and attributes may be evaluated more than once. - Consider example from paper: A a b | ^ V | B c ----> d possible order: c,d,expand to get a and b, evaluate b, a, c note the subtle point: c initially has no in-edges within the model, so it gets updated. Then when a is added, c acquires an in-edge. When a is evaluated, the fact that it didn't change is not remembered; c now has 0 in-edges so it gets reevaluated. fix: keep track of attributes for whom an argument has changed (e.g., not a in the above example)