public class PythonCodeGenerator extends CodeGenerator
Modifier and Type | Field and Description |
---|---|
(package private) int |
astVarNumber |
static int |
caseSizeThreshold |
(package private) String |
commonExtraArgs |
(package private) String |
commonExtraParams |
(package private) String |
commonLocalVars |
(package private) String |
currentASTResult
Tracks the rule or labeled subrule being generated.
|
(package private) RuleBlock |
currentRule
Tracks the rule being generated.
|
(package private) Hashtable |
declaredASTVariables
Used to keep track of which AST variables have been defined in a rule
(except for the #rule_name and #rule_name_in var's
|
(package private) String |
exceptionThrown |
protected boolean |
genAST |
static String |
initHeaderAction |
(package private) String |
labeledElementASTType |
(package private) String |
labeledElementInit |
(package private) String |
labeledElementType |
(package private) String |
lexerClassName |
(package private) String |
lt1Value |
static String |
mainHeaderAction |
protected static String |
NONUNIQUE
Special value used to mark duplicate in treeVariableMap
|
(package private) String |
parserClassName |
protected boolean |
saveText |
protected int |
syntacticPredLevel |
(package private) String |
throwNoViable |
(package private) Hashtable |
treeVariableMap
Mapping between the ids used in the current alt, and the
names of variables used to represent their AST values.
|
(package private) String |
treeWalkerClassName |
analyzer, antlrTool, behavior, BITSET_OPTIMIZE_INIT_THRESHOLD, bitsetsUsed, bitsetTestThreshold, charFormatter, currentOutput, DEBUG_CODE_GENERATOR, DEFAULT_BITSET_TEST_THRESHOLD, DEFAULT_MAKE_SWITCH_THRESHOLD, grammar, makeSwitchThreshold, tabs, TokenTypesFileExt, TokenTypesFileSuffix
Constructor and Description |
---|
PythonCodeGenerator() |
Modifier and Type | Method and Description |
---|---|
protected void |
_printAction(String s)
Print an action without leading tabs, attempting to
preserve the current indentation level for multi-line actions
Ignored if string is null.
|
protected void |
_printJavadoc(String s) |
protected int |
addSemPred(String predicate)
Adds a semantic predicate string to the sem pred vector
These strings will be used to build an array of sem pred names
when building a debugging parser.
|
protected void |
checkCurrentOutputStream() |
void |
exitIfError() |
protected String |
extractIdOfAction(String s,
int line,
int column)
Get the identifier portion of an argument-action.
|
protected String |
extractTypeOfAction(String s,
int line,
int column)
Get the type portion of an argument-action.
|
protected void |
flushTokens() |
void |
gen()
Generate the parser, lexer, treeparser, and token types in Java
|
void |
gen(ActionElement action)
Generate code for the given grammar element.
|
void |
gen(AlternativeBlock blk)
Generate code for the given grammar element.
|
void |
gen(BlockEndElement end)
Generate code for the given grammar element.
|
void |
gen(CharLiteralElement atom)
Generate code for the given grammar element.
|
void |
gen(CharRangeElement r)
Generate code for the given grammar element.
|
void |
gen(LexerGrammar g)
Generate the lexer Java file
|
void |
gen(OneOrMoreBlock blk)
Generate code for the given grammar element.
|
void |
gen(ParserGrammar g)
Generate the parser Java file
|
void |
gen(RuleRefElement rr)
Generate code for the given grammar element.
|
void |
gen(StringLiteralElement atom)
Generate code for the given grammar element.
|
void |
gen(TokenRangeElement r)
Generate code for the given grammar element.
|
void |
gen(TokenRefElement atom)
Generate code for the given grammar element.
|
void |
gen(TreeElement t)
Generate code for the given grammar element.
|
void |
gen(TreeWalkerGrammar g)
Generate the tree-parser Java file
|
void |
gen(WildcardElement wc)
Generate code for the given grammar element.
|
void |
gen(ZeroOrMoreBlock blk)
Generate code for the given grammar element.
|
protected void |
genAlt(Alternative alt,
AlternativeBlock blk)
Generate an alternative.
|
protected void |
genASTDeclaration(AlternativeElement el) |
protected void |
genASTDeclaration(AlternativeElement el,
String node_type) |
protected void |
genASTDeclaration(AlternativeElement el,
String var_name,
String node_type) |
protected void |
genBitsets(Vector bitsetList,
int maxVocabulary)
Generate all the bitsets to be used in the parser or lexer
Generate the raw bitset data like "long _tokenSet1_data[] = {...}"
and the BitSet object declarations like "BitSet _tokenSet1 = new BitSet(_tokenSet1_data)"
Note that most languages do not support object initialization inside a
class definition, so other code-generators may have to separate the
bitset declarations from the initializations (e.g., put the initializations
in the generated constructor instead).
|
protected void |
genBlockInitAction(AlternativeBlock blk)
Generate the init action for a block, which may be a RuleBlock or a
plain AlternativeBLock.
|
protected void |
genBlockPreamble(AlternativeBlock blk)
Generate the header for a block, which may be a RuleBlock or a
plain AlternativeBLock.
|
protected void |
genCases(BitSet p)
Generate a series of case statements that implement a BitSet test.
|
PythonBlockFinishingInfo |
genCommonBlock(AlternativeBlock blk,
boolean noTestForSingle)
Generate common code for a block of alternatives; return a
postscript that needs to be generated at the end of the
block.
|
protected void |
genHeader()
Generate a header that is common to all Python files
|
protected void |
genHeaderInit(Grammar grammar) |
protected void |
genHeaderMain(Grammar grammar) |
protected void |
genJavadocComment(Grammar g) |
protected void |
genJavadocComment(RuleSymbol g) |
protected void |
genLexerTest()
Generate an automated test for Python CharScanner (sub)classes.
|
protected void |
genMatch(BitSet b) |
protected void |
genMatch(GrammarAtom atom) |
protected void |
genMatchUsingAtomText(GrammarAtom atom) |
protected void |
genMatchUsingAtomTokenType(GrammarAtom atom) |
void |
genNextToken()
Generate the nextToken() rule.
|
void |
genRule(RuleSymbol s,
boolean startSymbol,
int ruleNum)
Gen a named rule block.
|
protected void |
genSemPred(String pred,
int line) |
protected void |
genSemPredMap()
Write an array of Strings which are the semantic predicate
expressions.
|
protected void |
genSynPred(SynPredBlock blk,
String lookaheadExpr) |
protected void |
genTokenASTNodeMap()
Create and set Integer token type objects that map
to Java Class objects (which AST node to create).
|
void |
genTokenStrings()
Generate a static array containing the names of the tokens,
indexed by the token type values.
|
protected void |
genTokenTypes(TokenManager tm)
Generate the token types Java file
|
String |
getASTCreateString(GrammarAtom atom,
String astCtorArgs)
Get a string for an expression to generate creating of an AST node
|
String |
getASTCreateString(String astCtorArgs)
Get a string for an expression to generate creating of an AST node.
|
String |
getASTCreateString(Vector v)
Get a string for an expression to generate creation of an AST subtree.
|
protected String |
getLookaheadTestExpression(Alternative alt,
int maxDepth)
Generate a lookahead test expression for an alternate.
|
protected String |
getLookaheadTestExpression(Lookahead[] look,
int k) |
protected String |
getLookaheadTestTerm(int k,
BitSet p)
Generate a depth==1 lookahead test expression given the BitSet.
|
String |
getRangeExpression(int k,
int[] elems)
Return an expression for testing a contiguous renage of elements
|
(package private) static boolean |
isEmpty(String s) |
protected boolean |
isspace(char c) |
protected boolean |
lookaheadIsEmpty(Alternative alt,
int maxDepth)
Is the lookahead for this alt empty?
|
String |
mapTreeId(String idParam,
ActionTransInfo transInfo)
Map an identifier to it's corresponding tree-node variable.
|
protected void |
od(String s,
int i,
int end,
String msg) |
protected void |
printAction(String s)
Print an action with leading tabs, attempting to
preserve the current indentation level for multi-line actions
Ignored if string is null.
|
protected void |
printActionCode(String actionStr,
int line) |
protected void |
printGrammarAction(Grammar grammar) |
protected void |
printMainFunc(String s) |
protected void |
printTabs()
Create a Java code-generator using the given Grammar.
|
protected String |
processActionCode(String actionStr,
int line) |
protected String |
processActionForSpecialSymbols(String actionStr,
int line,
RuleBlock currentRule,
ActionTransInfo tInfo)
Lexically process $var and tree-specifiers in the action.
|
void |
setupOutput(String className)
This method exists so a subclass, namely VAJCodeGenerator,
can open the file in its own evil way.
|
(package private) String |
toString(boolean v) |
_print, _println, decodeLexerRuleName, elementsAreRange, encodeLexerRuleName, extractIdOfAction, extractTypeOfAction, genTokenInterchange, getBitsetName, getFIRSTBitSet, getFOLLOWBitSet, markBitsetForGen, print, println, processStringForASTConstructor, removeAssignmentFromDeclaration, reverseLexerRuleName, setAnalyzer, setBehavior, setGrammar, setTool
protected int syntacticPredLevel
protected boolean genAST
protected boolean saveText
String labeledElementType
String labeledElementASTType
String labeledElementInit
String commonExtraArgs
String commonExtraParams
String commonLocalVars
String lt1Value
String exceptionThrown
String throwNoViable
public static final String initHeaderAction
public static final String mainHeaderAction
String lexerClassName
String parserClassName
String treeWalkerClassName
RuleBlock currentRule
String currentASTResult
Hashtable treeVariableMap
Hashtable declaredASTVariables
int astVarNumber
protected static final String NONUNIQUE
public static final int caseSizeThreshold
protected void printTabs()
printTabs
in class CodeGenerator
protected int addSemPred(String predicate)
public void exitIfError()
protected void checkCurrentOutputStream()
protected String extractIdOfAction(String s, int line, int column)
extractIdOfAction
in class CodeGenerator
s
- The action textline
- Line used for error reporting.column
- Line used for error reporting.protected String extractTypeOfAction(String s, int line, int column)
extractTypeOfAction
in class CodeGenerator
s
- The action textline
- Line used for error reporting.protected void flushTokens()
public void gen()
gen
in class CodeGenerator
public void gen(ActionElement action)
gen
in class CodeGenerator
blk
- The {...} action to generatepublic void gen(AlternativeBlock blk)
gen
in class CodeGenerator
blk
- The "x|y|z|..." block to generatepublic void gen(BlockEndElement end)
gen
in class CodeGenerator
blk
- The block-end element to generate. Block-end
elements are synthesized by the grammar parser to represent
the end of a block.public void gen(CharLiteralElement atom)
gen
in class CodeGenerator
blk
- The character literal reference to generateString toString(boolean v)
public void gen(CharRangeElement r)
gen
in class CodeGenerator
blk
- The character-range reference to generatepublic void gen(LexerGrammar g) throws IOException
gen
in class CodeGenerator
IOException
protected void genHeaderMain(Grammar grammar)
protected void genHeaderInit(Grammar grammar)
protected void printMainFunc(String s)
public void gen(OneOrMoreBlock blk)
gen
in class CodeGenerator
blk
- The (...)+ block to generatepublic void gen(ParserGrammar g) throws IOException
gen
in class CodeGenerator
IOException
public void gen(RuleRefElement rr)
gen
in class CodeGenerator
blk
- The rule-reference to generatepublic void gen(StringLiteralElement atom)
gen
in class CodeGenerator
blk
- The string-literal reference to generatepublic void gen(TokenRangeElement r)
gen
in class CodeGenerator
blk
- The token-range reference to generatepublic void gen(TokenRefElement atom)
gen
in class CodeGenerator
blk
- The token-reference to generatepublic void gen(TreeElement t)
CodeGenerator
gen
in class CodeGenerator
public void gen(TreeWalkerGrammar g) throws IOException
gen
in class CodeGenerator
IOException
public void gen(WildcardElement wc)
gen
in class CodeGenerator
wc
- The wildcard element to generatepublic void gen(ZeroOrMoreBlock blk)
gen
in class CodeGenerator
blk
- The (...)* block to generateprotected void genAlt(Alternative alt, AlternativeBlock blk)
alt
- The alternative to generateblk
- The block to which the alternative belongsprotected void genBitsets(Vector bitsetList, int maxVocabulary)
bitsetList
- The list of bitsets to generate.maxVocabulary
- Ensure that each generated bitset can contain at least this value.protected void genBlockInitAction(AlternativeBlock blk)
protected void genBlockPreamble(AlternativeBlock blk)
protected void genCases(BitSet p)
p
- The Bitset for which cases are to be generatedpublic PythonBlockFinishingInfo genCommonBlock(AlternativeBlock blk, boolean noTestForSingle)
protected void genASTDeclaration(AlternativeElement el)
protected void genASTDeclaration(AlternativeElement el, String node_type)
protected void genASTDeclaration(AlternativeElement el, String var_name, String node_type)
protected void genHeader()
protected void genLexerTest()
protected void genMatch(BitSet b)
protected void genMatch(GrammarAtom atom)
protected void genMatchUsingAtomText(GrammarAtom atom)
protected void genMatchUsingAtomTokenType(GrammarAtom atom)
public void genNextToken()
public void genRule(RuleSymbol s, boolean startSymbol, int ruleNum)
rule
- The name of the rule to generatestartSymbol
- true if the rule is a start symbol (i.e., not referenced elsewhere)protected void genSemPred(String pred, int line)
protected void genSemPredMap()
protected void genSynPred(SynPredBlock blk, String lookaheadExpr)
public void genTokenStrings()
protected void genTokenASTNodeMap()
protected void genTokenTypes(TokenManager tm) throws IOException
IOException
public String getASTCreateString(Vector v)
getASTCreateString
in class CodeGenerator
v
- A Vector of String, where each element is an expression in the target language yielding an AST node.public String getASTCreateString(GrammarAtom atom, String astCtorArgs)
getASTCreateString
in class CodeGenerator
atom
- The grammar node for which you are creating the nodestr
- The arguments to the AST constructorpublic String getASTCreateString(String astCtorArgs)
str
- The arguments to the AST constructorprotected String getLookaheadTestExpression(Alternative alt, int maxDepth)
protected String getLookaheadTestTerm(int k, BitSet p)
k
- The lookahead levelp
- The lookahead set for level kpublic String getRangeExpression(int k, int[] elems)
k
- The lookahead levelelems
- The elements representing the set, usually from BitSet.toArray().protected boolean lookaheadIsEmpty(Alternative alt, int maxDepth)
public String mapTreeId(String idParam, ActionTransInfo transInfo)
mapTreeId
in class CodeGenerator
idParam
- The identifier name to mapprotected String processActionForSpecialSymbols(String actionStr, int line, RuleBlock currentRule, ActionTransInfo tInfo)
processActionForSpecialSymbols
in class CodeGenerator
static boolean isEmpty(String s)
protected void printActionCode(String actionStr, int line)
public void setupOutput(String className) throws IOException
IOException
protected boolean isspace(char c)
protected void _printAction(String s)
CodeGenerator
_printAction
in class CodeGenerator
s
- The action string to outputprotected void printAction(String s)
CodeGenerator
printAction
in class CodeGenerator
s
- The action string to outputprotected void printGrammarAction(Grammar grammar)
protected void _printJavadoc(String s)
protected void genJavadocComment(Grammar g)
protected void genJavadocComment(RuleSymbol g)