KiCad PCB EDA Suite
DSNLEXER Class Reference

Class DSNLEXER implements a lexical analyzer for the SPECCTRA DSN file format. More...

#include <dsnlexer.h>

Inheritance diagram for DSNLEXER:
CMP_LIBRARY_LEXER DIALOG_BOM_CFG_LEXER FP_LIB_TABLE_LEXER LIB_TABLE_LEXER NETLIST_LEXER PAGE_LAYOUT_READER_LEXER PCB_LEXER PCB_PLOT_PARAMS_LEXER SPECCTRA_LEXER TEMPLATE_FIELDNAMES_LEXER

Public Member Functions

 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, FILE *aFile, const wxString &aFileName)
 Constructor ( FILE*, const wxString& ) intializes a DSN lexer and prepares to read from aFile which is already open and has aFilename. More...
 
 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, const std::string &aSExpression, const wxString &aSource=wxEmptyString)
 Constructor ( const KEYWORD*, unsigned, const std::string&, const wxString& ) intializes a DSN lexer and prepares to read from aSExpression. More...
 
 DSNLEXER (const std::string &aSExpression, const wxString &aSource=wxEmptyString)
 Constructor ( const std::string&, const wxString& ) intializes a DSN lexer and prepares to read from aSExpression. More...
 
 DSNLEXER (const KEYWORD *aKeywordTable, unsigned aKeywordCount, LINE_READER *aLineReader=NULL)
 Constructor ( LINE_READER* ) intializes a DSN lexer and prepares to read from aLineReader which is already open, and may be in use by other DSNLEXERs also. More...
 
virtual ~DSNLEXER ()
 
bool SyncLineReaderWith (DSNLEXER &aLexer)
 Useable only for DSN lexers which share the same LINE_READER Synchronizes the pointers handling the data read by the LINE_READER Allows 2 DNSLEXER to share the same current line, when switching from a DNSLEXER to an other DNSLEXER. More...
 
void SetSpecctraMode (bool aMode)
 Function SetSpecctraMode changes the behavior of this lexer into or out of "specctra mode". More...
 
void PushReader (LINE_READER *aLineReader)
 Function PushReader manages a stack of LINE_READERs in order to handle nested file inclusion. More...
 
LINE_READERPopReader ()
 Function PopReader deletes the top most LINE_READER from an internal stack of LINE_READERs and in the case of FILE_LINE_READER this means the associated FILE is closed. More...
 
int NextTok () throw ( IO_ERROR )
 Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of file. More...
 
int NeedSYMBOL () throw ( IO_ERROR )
 Function NeedSYMBOL calls NextTok() and then verifies that the token read in satisfies bool IsSymbol(). More...
 
int NeedSYMBOLorNUMBER () throw ( IO_ERROR )
 Function NeedSYMBOLorNUMBER calls NextTok() and then verifies that the token read in satisfies bool IsSymbol() or tok==DSN_NUMBER. More...
 
int NeedNUMBER (const char *aExpectation) throw ( IO_ERROR )
 Function NeedNUMBER calls NextTok() and then verifies that the token read is type DSN_NUMBER. More...
 
int CurTok ()
 Function CurTok returns whatever NextTok() returned the last time it was called. More...
 
int PrevTok ()
 Function PrevTok returns whatever NextTok() returned the 2nd to last time it was called. More...
 
char SetStringDelimiter (char aStringDelimiter)
 Function SetStringDelimiter changes the string delimiter from the default " to some other character and returns the old value. More...
 
bool SetSpaceInQuotedTokens (bool val)
 Function SetSpaceInQuotedTokens changes the setting controlling whether a space in a quoted string is a terminator. More...
 
bool SetCommentsAreTokens (bool val)
 Function SetCommentsAreTokens changes the handling of comments. More...
 
wxArrayString * ReadCommentLines () throw ( IO_ERROR )
 Function ReadCommentLines checks the next sequence of tokens and reads them into a wxArrayString if they are comments. More...
 
void Expecting (int aTok) throw ( IO_ERROR )
 Function Expecting throws an IO_ERROR exception with an input file specific error message. More...
 
void Expecting (const char *aTokenList) throw ( IO_ERROR )
 Function Expecting throws an IO_ERROR exception with an input file specific error message. More...
 
void Unexpected (int aTok) throw ( IO_ERROR )
 Function Unexpected throws an IO_ERROR exception with an input file specific error message. More...
 
void Unexpected (const char *aToken) throw ( IO_ERROR )
 Function Unexpected throws an IO_ERROR exception with an input file specific error message. More...
 
void Duplicate (int aTok) throw ( IO_ERROR )
 Function Duplicate throws an IO_ERROR exception with a message saying specifically that aTok is a duplicate of one already seen in current context. More...
 
void NeedLEFT () throw ( IO_ERROR )
 Function NeedLEFT calls NextTok() and then verifies that the token read in is a DSN_LEFT. More...
 
void NeedRIGHT () throw ( IO_ERROR )
 Function NeedRIGHT calls NextTok() and then verifies that the token read in is a DSN_RIGHT. More...
 
const char * GetTokenText (int aTok)
 Function GetTokenText returns the C string representation of a DSN_T value. More...
 
wxString GetTokenString (int aTok)
 Function GetTokenString returns a quote wrapped wxString representation of a token value. More...
 
const char * CurText ()
 Function CurText returns a pointer to the current token's text. More...
 
const std::string & CurStr ()
 Function CurStr returns a reference to current token in std::string form. More...
 
wxString FromUTF8 ()
 Function FromUTF8 returns the current token text as a wxString, assuming that the input byte stream is UTF8 encoded. More...
 
int CurLineNumber ()
 Function CurLineNumber returns the current line number within my LINE_READER. More...
 
const char * CurLine ()
 Function CurLine returns the current line of text, from which the CurText() would return its token. More...
 
const wxString & CurSource ()
 Function CurFilename returns the current LINE_READER source. More...
 
int CurOffset ()
 Function CurOffset returns the byte offset within the current line, using a 1 based index. More...
 

Static Public Member Functions

static bool IsSymbol (int aTok)
 Function IsSymbol tests a token to see if it is a symbol. More...
 
static const char * Syntax (int aTok)
 

Protected Types

typedef std::vector< LINE_READER * > READER_STACK
 

Protected Member Functions

void init ()
 
int readLine () throw ( IO_ERROR )
 
int findToken (const std::string &aToken)
 Function findToken takes aToken string and looks up the string in the keywords table. More...
 
bool isStringTerminator (char cc)
 

Protected Attributes

bool iOwnReaders
 on readerStack, should I delete them? More...
 
const char * start
 
const char * next
 
const char * limit
 
char dummy [1]
 when there is no reader. More...
 
READER_STACK readerStack
 all the LINE_READERs by pointer. More...
 
LINE_READERreader
 no ownership. ownership is via readerStack, maybe, if iOwnReaders More...
 
bool specctraMode
 if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else not. More...
 
char stringDelimiter
 
bool space_in_quoted_tokens
 blank spaces within quoted strings More...
 
bool commentsAreTokens
 true if should return comments as tokens More...
 
int prevTok
 curTok from previous NextTok() call. More...
 
int curOffset
 offset within current line of the current token More...
 
int curTok
 the current token obtained on last NextTok() More...
 
std::string curText
 the text of the current token More...
 
const KEYWORDkeywords
 table sorted by CMake for bsearch() More...
 
unsigned keywordCount
 count of keywords table More...
 
KEYWORD_MAP keyword_hash
 fast, specialized "C string" hashtable More...
 

Detailed Description

Class DSNLEXER implements a lexical analyzer for the SPECCTRA DSN file format.

It reads lexical tokens from the current LINE_READER through the NextTok() function.

Definition at line 79 of file dsnlexer.h.

Member Typedef Documentation

typedef std::vector<LINE_READER*> DSNLEXER::READER_STACK
protected

Definition at line 89 of file dsnlexer.h.

Constructor & Destructor Documentation

DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
FILE *  aFile,
const wxString &  aFileName 
)

Constructor ( FILE*, const wxString& ) intializes a DSN lexer and prepares to read from aFile which is already open and has aFilename.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aFileis an open file, which will be closed when this is destructed.
aFileNameis the name of the file

Definition at line 76 of file dsnlexer.cpp.

References init(), and PushReader().

77  :
78  iOwnReaders( true ),
79  start( NULL ),
80  next( NULL ),
81  limit( NULL ),
82  reader( NULL ),
83  keywords( aKeywordTable ),
84  keywordCount( aKeywordCount )
85 {
86  FILE_LINE_READER* fileReader = new FILE_LINE_READER( aFile, aFilename );
87  PushReader( fileReader );
88  init();
89 }
void PushReader(LINE_READER *aLineReader)
Function PushReader manages a stack of LINE_READERs in order to handle nested file inclusion...
Definition: dsnlexer.cpp:192
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:111
const char * next
Definition: dsnlexer.h:85
Class FILE_LINE_READER is a LINE_READER that reads from an open file.
Definition: richio.h:180
const char * limit
Definition: dsnlexer.h:86
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:83
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:112
void init()
Definition: dsnlexer.cpp:44
DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
const std::string &  aSExpression,
const wxString &  aSource = wxEmptyString 
)

Constructor ( const KEYWORD*, unsigned, const std::string&, const wxString& ) intializes a DSN lexer and prepares to read from aSExpression.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aSExpressionis text to feed through a STRING_LINE_READER
aSourceis a description of aSExpression, used for error reporting.

Definition at line 92 of file dsnlexer.cpp.

References FMT_CLIPBOARD, init(), and PushReader().

93  :
94  iOwnReaders( true ),
95  start( NULL ),
96  next( NULL ),
97  limit( NULL ),
98  reader( NULL ),
99  keywords( aKeywordTable ),
100  keywordCount( aKeywordCount )
101 {
102  STRING_LINE_READER* stringReader = new STRING_LINE_READER( aClipboardTxt, aSource.IsEmpty() ?
103  wxString( FMT_CLIPBOARD ) : aSource );
104  PushReader( stringReader );
105  init();
106 }
void PushReader(LINE_READER *aLineReader)
Function PushReader manages a stack of LINE_READERs in order to handle nested file inclusion...
Definition: dsnlexer.cpp:192
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:111
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:83
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
#define FMT_CLIPBOARD
Definition: dsnlexer.cpp:39
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:112
Class STRING_LINE_READER is a LINE_READER that reads from a multiline 8 bit wide std::string.
Definition: richio.h:254
void init()
Definition: dsnlexer.cpp:44
DSNLEXER::DSNLEXER ( const std::string &  aSExpression,
const wxString &  aSource = wxEmptyString 
)

Constructor ( const std::string&, const wxString& ) intializes a DSN lexer and prepares to read from aSExpression.

Use this one without a keyword table with the DOM parser in ptree.h.

Parameters
aSExpressionis text to feed through a STRING_LINE_READER
aSourceis a description of aSExpression, used for error reporting.

Definition at line 127 of file dsnlexer.cpp.

References FMT_CLIPBOARD, init(), and PushReader().

127  :
128  iOwnReaders( true ),
129  start( NULL ),
130  next( NULL ),
131  limit( NULL ),
132  reader( NULL ),
134  keywordCount( 0 )
135 {
136  STRING_LINE_READER* stringReader = new STRING_LINE_READER( aSExpression, aSource.IsEmpty() ?
137  wxString( FMT_CLIPBOARD ) : aSource );
138  PushReader( stringReader );
139  init();
140 }
void PushReader(LINE_READER *aLineReader)
Function PushReader manages a stack of LINE_READERs in order to handle nested file inclusion...
Definition: dsnlexer.cpp:192
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:111
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:83
static const KEYWORD empty_keywords[1]
Definition: dsnlexer.cpp:125
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
#define FMT_CLIPBOARD
Definition: dsnlexer.cpp:39
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:112
Class STRING_LINE_READER is a LINE_READER that reads from a multiline 8 bit wide std::string.
Definition: richio.h:254
void init()
Definition: dsnlexer.cpp:44
DSNLEXER::DSNLEXER ( const KEYWORD aKeywordTable,
unsigned  aKeywordCount,
LINE_READER aLineReader = NULL 
)

Constructor ( LINE_READER* ) intializes a DSN lexer and prepares to read from aLineReader which is already open, and may be in use by other DSNLEXERs also.

No ownership is taken of aLineReader. This enables it to be used by other DSNLEXERs also.

Parameters
aKeywordTableis an array of KEYWORDS holding aKeywordCount. This token table need not contain the lexer separators such as '(' ')', etc.
aKeywordCountis the count of tokens in aKeywordTable.
aLineReaderis any subclassed instance of LINE_READER, such as STRING_LINE_READER or FILE_LINE_READER. No ownership is taken.

Definition at line 109 of file dsnlexer.cpp.

References init(), and PushReader().

110  :
111  iOwnReaders( false ),
112  start( NULL ),
113  next( NULL ),
114  limit( NULL ),
115  reader( NULL ),
116  keywords( aKeywordTable ),
117  keywordCount( aKeywordCount )
118 {
119  if( aLineReader )
120  PushReader( aLineReader );
121  init();
122 }
void PushReader(LINE_READER *aLineReader)
Function PushReader manages a stack of LINE_READERs in order to handle nested file inclusion...
Definition: dsnlexer.cpp:192
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:111
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:83
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:112
void init()
Definition: dsnlexer.cpp:44
DSNLEXER::~DSNLEXER ( )
virtual

Definition at line 143 of file dsnlexer.cpp.

References iOwnReaders, and readerStack.

144 {
145  if( iOwnReaders )
146  {
147  // delete the LINE_READERs from the stack, since I own them.
148  for( READER_STACK::iterator it = readerStack.begin(); it!=readerStack.end(); ++it )
149  delete *it;
150  }
151 }
bool iOwnReaders
on readerStack, should I delete them?
Definition: dsnlexer.h:83
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:91

Member Function Documentation

const char* DSNLEXER::CurLine ( )
inline

Function CurLine returns the current line of text, from which the CurText() would return its token.

Definition at line 517 of file dsnlexer.h.

Referenced by PCB_IO::Load(), NextTok(), SYMBOL_LIB_TABLE::Parse(), and FP_LIB_TABLE::Parse().

518  {
519  return (const char*)(*reader);
520  }
int DSNLEXER::CurLineNumber ( )
inline

Function CurLineNumber returns the current line number within my LINE_READER.

Definition at line 507 of file dsnlexer.h.

References LINE_READER::LineNumber().

Referenced by PCB_IO::Load(), NextTok(), SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), and KICAD_NETLIST_PARSER::parseComponent().

508  {
509  return reader->LineNumber();
510  }
virtual unsigned LineNumber() const
Function Line Number returns the line number of the last line read from this LINE_READER.
Definition: richio.h:159
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
int DSNLEXER::CurOffset ( )
inline

Function CurOffset returns the byte offset within the current line, using a 1 based index.

Returns
int - a one based index into the current line.

Definition at line 538 of file dsnlexer.h.

Referenced by PCB_IO::Load(), NextTok(), SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), and KICAD_NETLIST_PARSER::parseComponent().

539  {
540  return curOffset + 1;
541  }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:106
const wxString& DSNLEXER::CurSource ( )
inline

Function CurFilename returns the current LINE_READER source.

Returns
const wxString& - the source of the lines of text, e.g. a filename or "clipboard".

Definition at line 528 of file dsnlexer.h.

References LINE_READER::GetSource().

Referenced by PCB_IO::Load(), NextTok(), SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), and KICAD_NETLIST_PARSER::parseComponent().

529  {
530  return reader->GetSource();
531  }
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
virtual const wxString & GetSource() const
Function GetSource returns the name of the source of the lines in an abstract sense.
Definition: richio.h:130
const std::string& DSNLEXER::CurStr ( )
inline

Function CurStr returns a reference to current token in std::string form.

Definition at line 488 of file dsnlexer.h.

References curText.

489  {
490  return curText;
491  }
std::string curText
the text of the current token
Definition: dsnlexer.h:109
const char* DSNLEXER::CurText ( )
inline
int DSNLEXER::CurTok ( )
inline

Function CurTok returns whatever NextTok() returned the last time it was called.

Definition at line 320 of file dsnlexer.h.

References curTok.

Referenced by DIALOG_BOM_CFG_LEXER::CurTok(), TEMPLATE_FIELDNAMES_LEXER::CurTok(), LIB_TABLE_LEXER::CurTok(), FP_LIB_TABLE_LEXER::CurTok(), PCB_PLOT_PARAMS_LEXER::CurTok(), CMP_LIBRARY_LEXER::CurTok(), NETLIST_LEXER::CurTok(), PAGE_LAYOUT_READER_LEXER::CurTok(), PCB_LEXER::CurTok(), SPECCTRA_LEXER::CurTok(), and scanList().

321  {
322  return curTok;
323  }
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:108
void DSNLEXER::Duplicate ( int  aTok)
throw (IO_ERROR
)

Function Duplicate throws an IO_ERROR exception with a message saying specifically that aTok is a duplicate of one already seen in current context.

Parameters
aTokis the token/keyword type which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 378 of file dsnlexer.cpp.

References Format(), and THROW_PARSE_ERROR.

Referenced by SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), and LIB_TABLE_ROW::Parse().

379 {
380  wxString errText = wxString::Format(
381  _("%s is a duplicate"), GetTokenString( aTok ).GetData() );
383 }
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, CPTREE &aTree)
Function Format outputs a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:205
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507
wxString GetTokenString(int aTok)
Function GetTokenString returns a quote wrapped wxString representation of a token value...
Definition: dsnlexer.cpp:335
void DSNLEXER::Expecting ( int  aTok)
throw (IO_ERROR
)

Function Expecting throws an IO_ERROR exception with an input file specific error message.

Parameters
aTokis the token/keyword type which was expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 354 of file dsnlexer.cpp.

References Format(), GetChars(), and THROW_PARSE_ERROR.

Referenced by NeedLEFT(), NeedRIGHT(), NeedSYMBOL(), NeedSYMBOLorNUMBER(), SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), LIB_TABLE_ROW::Parse(), PCB_PLOT_PARAMS_PARSER::parseBool(), KICAD_NETLIST_PARSER::parseComponent(), PCB_PLOT_PARAMS_PARSER::parseDouble(), PAGE_LAYOUT_READER_PARSER::parseDouble(), PCB_PLOT_PARAMS_PARSER::parseInt(), PAGE_LAYOUT_READER_PARSER::parseInt(), and KICAD_NETLIST_PARSER::parseLibPartList().

355 {
356  wxString errText = wxString::Format(
357  _("Expecting '%s'"), GetChars( GetTokenString( aTok ) ) );
359 }
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
static const wxChar * GetChars(const wxString &s)
Function GetChars returns a wxChar* to the actual wxChar* data within a wxString, and is helpful for ...
Definition: macros.h:92
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, CPTREE &aTree)
Function Format outputs a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:205
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507
wxString GetTokenString(int aTok)
Function GetTokenString returns a quote wrapped wxString representation of a token value...
Definition: dsnlexer.cpp:335
void DSNLEXER::Expecting ( const char *  aTokenList)
throw (IO_ERROR
)

Function Expecting throws an IO_ERROR exception with an input file specific error message.

Parameters
aTokenListis the token/keyword type which was expected at the current input location, e.g. "pin|graphic|property"
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 362 of file dsnlexer.cpp.

References Format(), GetChars(), and THROW_PARSE_ERROR.

363 {
364  wxString errText = wxString::Format(
365  _("Expecting '%s'"), GetChars( wxString::FromUTF8( text ) ) );
367 }
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
static const wxChar * GetChars(const wxString &s)
Function GetChars returns a wxChar* to the actual wxChar* data within a wxString, and is helpful for ...
Definition: macros.h:92
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, CPTREE &aTree)
Function Format outputs a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:205
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507
int DSNLEXER::findToken ( const std::string &  aToken)
inlineprotected

Function findToken takes aToken string and looks up the string in the keywords table.

Parameters
aTokenis a string to lookup in the keywords table.
Returns
int - with a value from the enum DSN_T matching the keyword text, or DSN_SYMBOL if aToken is not in the kewords table.

Definition at line 261 of file dsnlexer.cpp.

References DSN_SYMBOL, and keyword_hash.

Referenced by NextTok().

262 {
263  KEYWORD_MAP::const_iterator it = keyword_hash.find( tok.c_str() );
264  if( it != keyword_hash.end() )
265  return it->second;
266 
267  return DSN_SYMBOL; // not a keyword, some arbitrary symbol.
268 }
KEYWORD_MAP keyword_hash
fast, specialized "C string" hashtable
Definition: dsnlexer.h:113
wxString DSNLEXER::FromUTF8 ( )
inline

Function FromUTF8 returns the current token text as a wxString, assuming that the input byte stream is UTF8 encoded.

Definition at line 498 of file dsnlexer.h.

Referenced by SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), LIB_TABLE_ROW::Parse(), KICAD_NETLIST_PARSER::parseComponent(), and ReadCommentLines().

499  {
500  return wxString::FromUTF8( curText.c_str() );
501  }
std::string curText
the text of the current token
Definition: dsnlexer.h:109
wxString DSNLEXER::GetTokenString ( int  aTok)

Function GetTokenString returns a quote wrapped wxString representation of a token value.

Definition at line 335 of file dsnlexer.cpp.

References GetTokenText().

336 {
337  wxString ret;
338 
339  ret << wxT("'") << wxString::FromUTF8( GetTokenText(aTok) ) << wxT("'");
340 
341  return ret;
342 }
const char * GetTokenText(int aTok)
Function GetTokenText returns the C string representation of a DSN_T value.
Definition: dsnlexer.cpp:316
const char * DSNLEXER::GetTokenText ( int  aTok)

Function GetTokenText returns the C string representation of a DSN_T value.

Definition at line 316 of file dsnlexer.cpp.

References keywordCount, keywords, KEYWORD::name, and Syntax().

Referenced by GetTokenString(), PCB_PARSER::parseBoardUnits(), and PCB_PARSER::parseDouble().

317 {
318  const char* ret;
319 
320  if( aTok < 0 )
321  {
322  return Syntax( aTok );
323  }
324  else if( (unsigned) aTok < keywordCount )
325  {
326  ret = keywords[aTok].name;
327  }
328  else
329  ret = "token too big";
330 
331  return ret;
332 }
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:111
const char * name
unique keyword.
Definition: dsnlexer.h:42
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:112
static const char * Syntax(int aTok)
Definition: dsnlexer.cpp:272
void DSNLEXER::init ( )
protected

Definition at line 44 of file dsnlexer.cpp.

References commentsAreTokens, curOffset, curTok, DSN_NONE, keyword_hash, keywordCount, keywords, KEYWORD::name, prevTok, space_in_quoted_tokens, specctraMode, stringDelimiter, and KEYWORD::token.

Referenced by DSNLEXER().

45 {
46  curTok = DSN_NONE;
47  prevTok = DSN_NONE;
48 
49  stringDelimiter = '"';
50 
51  specctraMode = false;
52  space_in_quoted_tokens = false;
53  commentsAreTokens = false;
54 
55  curOffset = 0;
56 
57 #if 1
58  if( keywordCount > 11 )
59  {
60  // resize the hashtable bucket count
61  keyword_hash.reserve( keywordCount );
62  }
63 
64  // fill the specialized "C string" hashtable from keywords[]
65  const KEYWORD* it = keywords;
66  const KEYWORD* end = it + keywordCount;
67 
68  for( ; it < end; ++it )
69  {
70  keyword_hash[it->name] = it->token;
71  }
72 #endif
73 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:106
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:103
const KEYWORD * keywords
table sorted by CMake for bsearch()
Definition: dsnlexer.h:111
const char * name
unique keyword.
Definition: dsnlexer.h:42
KEYWORD_MAP keyword_hash
fast, specialized "C string" hashtable
Definition: dsnlexer.h:113
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:101
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:94
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:105
unsigned keywordCount
count of keywords table
Definition: dsnlexer.h:112
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:108
Struct KEYWORD holds a keyword string and its unique integer token.
Definition: dsnlexer.h:40
char stringDelimiter
Definition: dsnlexer.h:100
int token
a zero based index into an array of KEYWORDs
Definition: dsnlexer.h:43
bool DSNLEXER::isStringTerminator ( char  cc)
inlineprotected

Definition at line 147 of file dsnlexer.h.

Referenced by NextTok().

148  {
149  if( !space_in_quoted_tokens && cc==' ' )
150  return true;
151 
152  if( cc == stringDelimiter )
153  return true;
154 
155  return false;
156  }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:101
char stringDelimiter
Definition: dsnlexer.h:100
bool DSNLEXER::IsSymbol ( int  aTok)
static

Function IsSymbol tests a token to see if it is a symbol.

This means it cannot be a special delimiter character such as DSN_LEFT, DSN_RIGHT, DSN_QUOTE, etc. It may however, coincidentally match a keyword and still be a symbol.

Definition at line 345 of file dsnlexer.cpp.

References DSN_STRING, and DSN_SYMBOL.

Referenced by NeedSYMBOL(), and NeedSYMBOLorNUMBER().

346 {
347  // This is static and not inline to reduce code space.
348 
349  // if aTok is >= 0, then it is a coincidental match to a keyword.
350  return aTok==DSN_SYMBOL || aTok==DSN_STRING || aTok>=0;
351 }
void DSNLEXER::NeedLEFT ( )
throw (IO_ERROR
)

Function NeedLEFT calls NextTok() and then verifies that the token read in is a DSN_LEFT.

If it is not, an IO_ERROR is thrown.

Exceptions
IO_ERROR,ifthe next token is not a DSN_LEFT

Definition at line 394 of file dsnlexer.cpp.

References DSN_LEFT, Expecting(), and NextTok().

Referenced by SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), and LIB_TABLE_ROW::Parse().

395 {
396  int tok = NextTok();
397  if( tok != DSN_LEFT )
398  Expecting( DSN_LEFT );
399 }
void Expecting(int aTok)
Function Expecting throws an IO_ERROR exception with an input file specific error message...
Definition: dsnlexer.cpp:354
int NextTok()
Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of f...
Definition: dsnlexer.cpp:541
int DSNLEXER::NeedNUMBER ( const char *  aExpectation)
throw (IO_ERROR
)

Function NeedNUMBER calls NextTok() and then verifies that the token read is type DSN_NUMBER.

If not, and IO_ERROR is thrown using text from aExpectation.

Returns
int - the actual token read in.
Exceptions
IO_ERROR,ifthe next token does not satisfy the above test

Definition at line 428 of file dsnlexer.cpp.

References DSN_NUMBER, Format(), and THROW_PARSE_ERROR.

Referenced by PCB_PARSER::parseDouble(), and PCB_PARSER::parseInt().

429 {
430  int tok = NextTok();
431  if( tok != DSN_NUMBER )
432  {
433  wxString errText = wxString::Format(
434  _("need a NUMBER for '%s'"), wxString::FromUTF8( aExpectation ).GetData() );
436  }
437  return tok;
438 }
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, CPTREE &aTree)
Function Format outputs a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:205
int NextTok()
Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of f...
Definition: dsnlexer.cpp:541
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507
void DSNLEXER::NeedRIGHT ( )
throw (IO_ERROR
)

Function NeedRIGHT calls NextTok() and then verifies that the token read in is a DSN_RIGHT.

If it is not, an IO_ERROR is thrown.

Exceptions
IO_ERROR,ifthe next token is not a DSN_RIGHT

Definition at line 402 of file dsnlexer.cpp.

References DSN_RIGHT, Expecting(), and NextTok().

Referenced by SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), LIB_TABLE_ROW::Parse(), KICAD_NETLIST_PARSER::Parse(), KICAD_NETLIST_PARSER::parseComponent(), KICAD_NETLIST_PARSER::parseLibPartList(), and KICAD_NETLIST_PARSER::parseNet().

403 {
404  int tok = NextTok();
405  if( tok != DSN_RIGHT )
406  Expecting( DSN_RIGHT );
407 }
void Expecting(int aTok)
Function Expecting throws an IO_ERROR exception with an input file specific error message...
Definition: dsnlexer.cpp:354
int NextTok()
Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of f...
Definition: dsnlexer.cpp:541
int DSNLEXER::NeedSYMBOL ( )
throw (IO_ERROR
)

Function NeedSYMBOL calls NextTok() and then verifies that the token read in satisfies bool IsSymbol().

If not, an IO_ERROR is thrown.

Returns
int - the actual token read in.
Exceptions
IO_ERROR,ifthe next token does not satisfy IsSymbol()

Definition at line 410 of file dsnlexer.cpp.

References DSN_SYMBOL, Expecting(), IsSymbol(), and NextTok().

Referenced by DIALOG_BOM_CFG_LEXER::NeedSYMBOL(), TEMPLATE_FIELDNAMES_LEXER::NeedSYMBOL(), LIB_TABLE_LEXER::NeedSYMBOL(), FP_LIB_TABLE_LEXER::NeedSYMBOL(), PCB_PLOT_PARAMS_LEXER::NeedSYMBOL(), CMP_LIBRARY_LEXER::NeedSYMBOL(), NETLIST_LEXER::NeedSYMBOL(), PAGE_LAYOUT_READER_LEXER::NeedSYMBOL(), PCB_LEXER::NeedSYMBOL(), and SPECCTRA_LEXER::NeedSYMBOL().

411 {
412  int tok = NextTok();
413  if( !IsSymbol( tok ) )
415  return tok;
416 }
static bool IsSymbol(int aTok)
Function IsSymbol tests a token to see if it is a symbol.
Definition: dsnlexer.cpp:345
void Expecting(int aTok)
Function Expecting throws an IO_ERROR exception with an input file specific error message...
Definition: dsnlexer.cpp:354
int NextTok()
Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of f...
Definition: dsnlexer.cpp:541
int DSNLEXER::NeedSYMBOLorNUMBER ( )
throw (IO_ERROR
)

Function NeedSYMBOLorNUMBER calls NextTok() and then verifies that the token read in satisfies bool IsSymbol() or tok==DSN_NUMBER.

If not, an IO_ERROR is thrown.

Returns
int - the actual token read in.
Exceptions
IO_ERROR,ifthe next token does not satisfy the above test

Definition at line 419 of file dsnlexer.cpp.

References DSN_NUMBER, Expecting(), IsSymbol(), and NextTok().

Referenced by DIALOG_BOM_CFG_LEXER::NeedSYMBOLorNUMBER(), TEMPLATE_FIELDNAMES_LEXER::NeedSYMBOLorNUMBER(), LIB_TABLE_LEXER::NeedSYMBOLorNUMBER(), FP_LIB_TABLE_LEXER::NeedSYMBOLorNUMBER(), PCB_PLOT_PARAMS_LEXER::NeedSYMBOLorNUMBER(), CMP_LIBRARY_LEXER::NeedSYMBOLorNUMBER(), NETLIST_LEXER::NeedSYMBOLorNUMBER(), PAGE_LAYOUT_READER_LEXER::NeedSYMBOLorNUMBER(), PCB_LEXER::NeedSYMBOLorNUMBER(), and SPECCTRA_LEXER::NeedSYMBOLorNUMBER().

420 {
421  int tok = NextTok();
422  if( !IsSymbol( tok ) && tok!=DSN_NUMBER )
423  Expecting( "symbol|number" );
424  return tok;
425 }
static bool IsSymbol(int aTok)
Function IsSymbol tests a token to see if it is a symbol.
Definition: dsnlexer.cpp:345
void Expecting(int aTok)
Function Expecting throws an IO_ERROR exception with an input file specific error message...
Definition: dsnlexer.cpp:354
int NextTok()
Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of f...
Definition: dsnlexer.cpp:541
int DSNLEXER::NextTok ( )
throw (IO_ERROR
)

Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of file.

Users should wrap this function to return an enum to aid in grammar debugging while running under a debugger, but leave this lower level function returning an int (so the enum does not collide with another usage).

Returns
int - the type of token found next.
Exceptions
IO_ERROR- only if the LINE_READER throws it.

Definition at line 541 of file dsnlexer.cpp.

References commentsAreTokens, CurLine(), CurLineNumber(), curOffset, CurOffset(), CurSource(), curText, curTok, DSN_COMMENT, DSN_DASH, DSN_EOF, DSN_LEFT, DSN_NUMBER, DSN_QUOTE_DEF, DSN_RIGHT, DSN_STRING, DSN_STRING_QUOTE, findToken(), isNumber(), isSep(), isSpace(), isStringTerminator(), limit, next, prevTok, readLine(), specctraMode, start, stringDelimiter, and THROW_PARSE_ERROR.

Referenced by PCAD2KICAD::LoadInputFile(), NeedLEFT(), NeedRIGHT(), NeedSYMBOL(), NeedSYMBOLorNUMBER(), DIALOG_BOM_CFG_LEXER::NextTok(), TEMPLATE_FIELDNAMES_LEXER::NextTok(), FP_LIB_TABLE_LEXER::NextTok(), LIB_TABLE_LEXER::NextTok(), PCB_PLOT_PARAMS_LEXER::NextTok(), CMP_LIBRARY_LEXER::NextTok(), NETLIST_LEXER::NextTok(), PAGE_LAYOUT_READER_LEXER::NextTok(), PCB_LEXER::NextTok(), SPECCTRA_LEXER::NextTok(), ReadCommentLines(), and scanList().

542 {
543  const char* cur = next;
544  const char* head = cur;
545 
546  prevTok = curTok;
547 
548  if( curTok == DSN_EOF )
549  goto exit;
550 
551  if( cur >= limit )
552  {
553 L_read:
554  // blank lines are returned as "\n" and will have a len of 1.
555  // EOF will have a len of 0 and so is detectable.
556  int len = readLine();
557  if( len == 0 )
558  {
559  cur = start; // after readLine(), since start can change, set cur offset to start
560  curTok = DSN_EOF;
561  goto exit;
562  }
563 
564  cur = start; // after readLine() since start can change.
565 
566  // skip leading whitespace
567  while( cur<limit && isSpace( *cur ) )
568  ++cur;
569 
570  // If the first non-blank character is #, this line is a comment.
571  // Comments cannot follow any other token on the same line.
572  if( cur<limit && *cur=='#' )
573  {
574  if( commentsAreTokens )
575  {
576  // Grab the entire current line [excluding end of line char(s)] as the
577  // current token. The '#' character may not be at offset zero.
578 
579  while( limit[-1] == '\n' || limit[-1] == '\r' )
580  --limit;
581 
582  curText.clear();
583  curText.append( start, limit );
584 
585  cur = start; // ensure a good curOffset below
587  head = limit; // do a readLine() on next call in here.
588  goto exit;
589  }
590  else
591  goto L_read;
592  }
593  }
594  else
595  {
596  // skip leading whitespace
597  while( cur<limit && isSpace( *cur ) )
598  ++cur;
599  }
600 
601  if( cur >= limit )
602  goto L_read;
603 
604  if( *cur == '(' )
605  {
606  curText = *cur;
607  curTok = DSN_LEFT;
608  head = cur+1;
609  goto exit;
610  }
611 
612  if( *cur == ')' )
613  {
614  curText = *cur;
615  curTok = DSN_RIGHT;
616  head = cur+1;
617  goto exit;
618  }
619 
620  // Non-specctraMode, understands and deciphers escaped \, \r, \n, and \".
621  // Strips off leading and trailing double quotes
622  if( !specctraMode )
623  {
624  // a quoted string, will return DSN_STRING
625  if( *cur == stringDelimiter )
626  {
627  // copy the token, character by character so we can remove doubled up quotes.
628  curText.clear();
629 
630  ++cur; // skip over the leading delimiter, which is always " in non-specctraMode
631 
632  head = cur;
633 
634  while( head<limit )
635  {
636  // ESCAPE SEQUENCES:
637  if( *head =='\\' )
638  {
639  char tbuf[8];
640  char c;
641  int i;
642 
643  if( ++head >= limit )
644  break; // throw exception at L_unterminated
645 
646  switch( *head++ )
647  {
648  case '"':
649  case '\\': c = head[-1]; break;
650  case 'a': c = '\x07'; break;
651  case 'b': c = '\x08'; break;
652  case 'f': c = '\x0c'; break;
653  case 'n': c = '\n'; break;
654  case 'r': c = '\r'; break;
655  case 't': c = '\x09'; break;
656  case 'v': c = '\x0b'; break;
657 
658  case 'x': // 1 or 2 byte hex escape sequence
659  for( i=0; i<2; ++i )
660  {
661  if( !isxdigit( head[i] ) )
662  break;
663  tbuf[i] = head[i];
664  }
665  tbuf[i] = '\0';
666  if( i > 0 )
667  c = (char) strtoul( tbuf, NULL, 16 );
668  else
669  c = 'x'; // a goofed hex escape sequence, interpret as 'x'
670  head += i;
671  break;
672 
673  default: // 1-3 byte octal escape sequence
674  --head;
675  for( i=0; i<3; ++i )
676  {
677  if( head[i] < '0' || head[i] > '7' )
678  break;
679  tbuf[i] = head[i];
680  }
681  tbuf[i] = '\0';
682  if( i > 0 )
683  c = (char) strtoul( tbuf, NULL, 8 );
684  else
685  c = '\\'; // a goofed octal escape sequence, interpret as '\'
686  head += i;
687  break;
688  }
689 
690  curText += c;
691  }
692 
693  else if( *head == '"' ) // end of the non-specctraMode DSN_STRING
694  {
695  curTok = DSN_STRING;
696  ++head; // omit this trailing double quote
697  goto exit;
698  }
699 
700  else
701  curText += *head++;
702 
703  } // while
704 
705  // L_unterminated:
706  wxString errtxt( _( "Un-terminated delimited string" ) );
708  }
709  }
710 
711  else // is specctraMode, tests in this block should not occur in KiCad mode.
712  {
713  /* get the dash out of a <pin_reference> which is embedded for example
714  like: U2-14 or "U2"-"14"
715  This is detectable by a non-space immediately preceeding the dash.
716  */
717  if( *cur == '-' && cur>start && !isSpace( cur[-1] ) )
718  {
719  curText = '-';
720  curTok = DSN_DASH;
721  head = cur+1;
722  goto exit;
723  }
724 
725  // switching the string_quote character
726  if( prevTok == DSN_STRING_QUOTE )
727  {
728  static const wxString errtxt( _("String delimiter must be a single character of ', \", or $"));
729 
730  char cc = *cur;
731  switch( cc )
732  {
733  case '\'':
734  case '$':
735  case '"':
736  break;
737  default:
739  }
740 
741  curText = cc;
742 
743  head = cur+1;
744 
745  if( head<limit && !isSep( *head ) )
746  {
748  }
749 
751  goto exit;
752  }
753 
754  // specctraMode DSN_STRING
755  if( *cur == stringDelimiter )
756  {
757  ++cur; // skip over the leading delimiter: ",', or $
758 
759  head = cur;
760 
761  while( head<limit && !isStringTerminator( *head ) )
762  ++head;
763 
764  if( head >= limit )
765  {
766  wxString errtxt( _( "Un-terminated delimited string" ) );
768  }
769 
770  curText.clear();
771  curText.append( cur, head );
772 
773  ++head; // skip over the trailing delimiter
774 
775  curTok = DSN_STRING;
776  goto exit;
777  }
778  } // specctraMode
779 
780  // non-quoted token, read it into curText.
781  curText.clear();
782 
783  head = cur;
784  while( head<limit && !isSep( *head ) )
785  curText += *head++;
786 
787  if( isNumber( curText.c_str(), curText.c_str() + curText.size() ) )
788  {
789  curTok = DSN_NUMBER;
790  goto exit;
791  }
792 
793  if( specctraMode && curText == "string_quote" )
794  {
796  goto exit;
797  }
798 
799  curTok = findToken( curText );
800 
801 exit: // single point of exit, no returns elsewhere please.
802 
803  curOffset = cur - start;
804 
805  next = head;
806 
807  // printf("tok:\"%s\"\n", curText.c_str() );
808  return curTok;
809 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:106
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:103
bool isSep(char cc)
return true if cc is an s-expression separator character
Definition: dsnlexer.cpp:474
int findToken(const std::string &aToken)
Function findToken takes aToken string and looks up the string in the keywords table.
Definition: dsnlexer.cpp:261
static bool isNumber(const char *cp, const char *limit)
Function isNumber returns true if the next sequence of text is a number: either an integer...
Definition: dsnlexer.cpp:491
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
const char * next
Definition: dsnlexer.h:85
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
const char * limit
Definition: dsnlexer.h:86
static bool isSpace(char cc)
Function isSpace tests for whitespace.
Definition: dsnlexer.cpp:447
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:94
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:105
int readLine()
Definition: dsnlexer.h:117
const char * start
Definition: dsnlexer.h:84
std::string curText
the text of the current token
Definition: dsnlexer.h:109
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507
int curTok
the current token obtained on last NextTok()
Definition: dsnlexer.h:108
bool isStringTerminator(char cc)
Definition: dsnlexer.h:147
char stringDelimiter
Definition: dsnlexer.h:100
LINE_READER * DSNLEXER::PopReader ( )

Function PopReader deletes the top most LINE_READER from an internal stack of LINE_READERs and in the case of FILE_LINE_READER this means the associated FILE is closed.

The most recently used former LINE_READER on the stack becomes the current LINE_READER and its previous position in its input stream and the its latest line number should pertain. PopReader always starts reading from a new line upon returning to the previous LINE_READER. A pop is only possible if there are at least 2 LINE_READERs on the stack, since popping the last one is not supported.

Returns
LINE_READER* - is the one that was in use before the pop, or NULL if there was not at least two readers on the stack and therefore the pop failed.

Definition at line 204 of file dsnlexer.cpp.

References dummy, limit, LINE_READER::Line(), next, reader, readerStack, and start.

Referenced by PCB_PARSER::SetLineReader().

205 {
206  LINE_READER* ret = 0;
207 
208  if( readerStack.size() )
209  {
210  ret = reader;
211  readerStack.pop_back();
212 
213  if( readerStack.size() )
214  {
215  reader = readerStack.back();
216  start = reader->Line();
217 
218  // force a new readLine() as first thing.
219  limit = start;
220  next = start;
221  }
222  else
223  {
224  reader = 0;
225  start = dummy;
226  limit = dummy;
227  limit = dummy;
228  }
229  }
230  return ret;
231 }
Class LINE_READER is an abstract class from which implementation specific LINE_READERs may be derived...
Definition: richio.h:81
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
char * Line() const
Function Line returns a pointer to the last line that was read in.
Definition: richio.h:139
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:91
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
char dummy[1]
when there is no reader.
Definition: dsnlexer.h:87
int DSNLEXER::PrevTok ( )
inline

Function PrevTok returns whatever NextTok() returned the 2nd to last time it was called.

Definition at line 329 of file dsnlexer.h.

References prevTok.

Referenced by DIALOG_BOM_CFG_LEXER::PrevTok(), TEMPLATE_FIELDNAMES_LEXER::PrevTok(), LIB_TABLE_LEXER::PrevTok(), FP_LIB_TABLE_LEXER::PrevTok(), PCB_PLOT_PARAMS_LEXER::PrevTok(), CMP_LIBRARY_LEXER::PrevTok(), NETLIST_LEXER::PrevTok(), PAGE_LAYOUT_READER_LEXER::PrevTok(), PCB_LEXER::PrevTok(), and SPECCTRA_LEXER::PrevTok().

330  {
331  return prevTok;
332  }
int prevTok
curTok from previous NextTok() call.
Definition: dsnlexer.h:105
void DSNLEXER::PushReader ( LINE_READER aLineReader)

Function PushReader manages a stack of LINE_READERs in order to handle nested file inclusion.

This function pushes aLineReader onto the top of a stack of LINE_READERs and makes it the current LINE_READER with its own GetSource(), line number and line text. A grammar must be designed such that the "include" token (whatever its various names), and any of its parameters are not followed by anything on that same line, because PopReader always starts reading from a new line upon returning to the original LINE_READER.

Definition at line 192 of file dsnlexer.cpp.

References limit, next, reader, readerStack, and start.

Referenced by DSNLEXER(), and PCB_PARSER::SetLineReader().

193 {
194  readerStack.push_back( aLineReader );
195  reader = aLineReader;
196  start = (const char*) (*reader);
197 
198  // force a new readLine() as first thing.
199  limit = start;
200  next = start;
201 }
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
READER_STACK readerStack
all the LINE_READERs by pointer.
Definition: dsnlexer.h:91
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
wxArrayString * DSNLEXER::ReadCommentLines ( )
throw (IO_ERROR
)

Function ReadCommentLines checks the next sequence of tokens and reads them into a wxArrayString if they are comments.

Reading continues until a non-comment token is encountered, and such last read token remains as CurTok() and as CurText(). No push back or "un get" mechanism is used for this support. Upon return you simply avoid calling NextTok() for the next token, but rather CurTok().

Returns
wxArrayString* - heap allocated block of comments, or NULL if none; caller owns the allocation and must delete if not NULL.

Definition at line 812 of file dsnlexer.cpp.

References DSN_COMMENT, FromUTF8(), NextTok(), and SetCommentsAreTokens().

813 {
814  wxArrayString* ret = 0;
815  bool cmt_setting = SetCommentsAreTokens( true );
816  int tok = NextTok();
817 
818  if( tok == DSN_COMMENT )
819  {
820  ret = new wxArrayString();
821 
822  do
823  {
824  ret->Add( FromUTF8() );
825  }
826  while( ( tok = NextTok() ) == DSN_COMMENT );
827  }
828 
829  SetCommentsAreTokens( cmt_setting );
830 
831  return ret;
832 }
bool SetCommentsAreTokens(bool val)
Function SetCommentsAreTokens changes the handling of comments.
Definition: dsnlexer.h:372
wxString FromUTF8()
Function FromUTF8 returns the current token text as a wxString, assuming that the input byte stream i...
Definition: dsnlexer.h:498
int NextTok()
Function NextTok returns the next token found in the input file or DSN_EOF when reaching the end of f...
Definition: dsnlexer.cpp:541
int DSNLEXER::readLine ( )
throw (IO_ERROR
)
inlineprotected

Definition at line 117 of file dsnlexer.h.

References LINE_READER::Length(), LINE_READER::Line(), LINE_READER::ReadLine(), and start.

Referenced by NextTok().

118  {
119  if( reader )
120  {
121  reader->ReadLine();
122 
123  unsigned len = reader->Length();
124 
125  // start may have changed in ReadLine(), which can resize and
126  // relocate reader's line buffer.
127  start = reader->Line();
128 
129  next = start;
130  limit = next + len;
131 
132  return len;
133  }
134  return 0;
135  }
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
char * Line() const
Function Line returns a pointer to the last line that was read in.
Definition: richio.h:139
unsigned Length() const
Function Length returns the number of bytes in the last line read from this LINE_READER.
Definition: richio.h:168
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
virtual char * ReadLine()=0
Function ReadLine reads a line of text into the buffer and increments the line number counter...
bool DSNLEXER::SetCommentsAreTokens ( bool  val)
inline

Function SetCommentsAreTokens changes the handling of comments.

If set true, comments are returns as single line strings with a terminating newline, else they are consumed by the lexer and not returned.

Definition at line 372 of file dsnlexer.h.

References commentsAreTokens.

Referenced by ReadCommentLines().

373  {
374  bool old = commentsAreTokens;
375  commentsAreTokens = val;
376  return old;
377  }
bool commentsAreTokens
true if should return comments as tokens
Definition: dsnlexer.h:103
bool DSNLEXER::SetSpaceInQuotedTokens ( bool  val)
inline

Function SetSpaceInQuotedTokens changes the setting controlling whether a space in a quoted string is a terminator.

Parameters
valIf true, means

Definition at line 358 of file dsnlexer.h.

References space_in_quoted_tokens.

359  {
360  bool old = space_in_quoted_tokens;
361  if( specctraMode )
363  return old;
364  }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:101
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:94
void DSNLEXER::SetSpecctraMode ( bool  aMode)

Function SetSpecctraMode changes the behavior of this lexer into or out of "specctra mode".

If specctra mode, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else none of the above are true. The default mode is non-specctra mode, meaning: 1) stringDelimiter cannot be changed 2) Kicad quoting protocol is in effect 3) space_in_quoted_tokens is not functional

Definition at line 153 of file dsnlexer.cpp.

References space_in_quoted_tokens, specctraMode, and stringDelimiter.

Referenced by DSN::SPECCTRA_DB::SPECCTRA_DB().

154 {
155  specctraMode = aMode;
156  if( aMode )
157  {
158  // specctra mode defaults, some of which can still be changed in this mode.
159  space_in_quoted_tokens = true;
160  }
161  else
162  {
163  space_in_quoted_tokens = false;
164  stringDelimiter = '"';
165  }
166 }
bool space_in_quoted_tokens
blank spaces within quoted strings
Definition: dsnlexer.h:101
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:94
char stringDelimiter
Definition: dsnlexer.h:100
char DSNLEXER::SetStringDelimiter ( char  aStringDelimiter)
inline

Function SetStringDelimiter changes the string delimiter from the default " to some other character and returns the old value.

Parameters
aStringDelimiterThe character in lowest 8 bits.
Returns
int - The old delimiter in the lowest 8 bits.

Definition at line 344 of file dsnlexer.h.

References stringDelimiter.

345  {
346  int old = stringDelimiter;
347  if( specctraMode )
348  stringDelimiter = aStringDelimiter;
349  return old;
350  }
bool specctraMode
if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_...
Definition: dsnlexer.h:94
char stringDelimiter
Definition: dsnlexer.h:100
bool DSNLEXER::SyncLineReaderWith ( DSNLEXER aLexer)

Useable only for DSN lexers which share the same LINE_READER Synchronizes the pointers handling the data read by the LINE_READER Allows 2 DNSLEXER to share the same current line, when switching from a DNSLEXER to an other DNSLEXER.

Parameters
aLexer= the model
Returns
true if the sync can be made ( at least the same line reader )

Definition at line 169 of file dsnlexer.cpp.

References curOffset, curText, limit, next, reader, and start.

Referenced by PCB_PARSER::parseSetup().

170 {
171  // Synchronize the pointers handling the data read by the LINE_READER
172  // only if aLexer shares the same LINE_READER, because only in this case
173  // the char buffer is be common
174 
175  if( reader != aLexer.reader )
176  return false;
177 
178  // We can synchronize the pointers which handle the data currently read
179  start = aLexer.start;
180  next = aLexer.next;
181  limit = aLexer.limit;
182 
183  // Sync these parameters is not mandatory, but could help
184  // for instance in debug
185  curText = aLexer.curText;
186  curOffset = aLexer.curOffset;
187 
188  return true;
189 }
int curOffset
offset within current line of the current token
Definition: dsnlexer.h:106
const char * next
Definition: dsnlexer.h:85
const char * limit
Definition: dsnlexer.h:86
LINE_READER * reader
no ownership. ownership is via readerStack, maybe, if iOwnReaders
Definition: dsnlexer.h:92
const char * start
Definition: dsnlexer.h:84
std::string curText
the text of the current token
Definition: dsnlexer.h:109
const char * DSNLEXER::Syntax ( int  aTok)
static

Definition at line 272 of file dsnlexer.cpp.

References DSN_DASH, DSN_EOF, DSN_LEFT, DSN_NONE, DSN_NUMBER, DSN_QUOTE_DEF, DSN_RIGHT, DSN_STRING, DSN_STRING_QUOTE, and DSN_SYMBOL.

Referenced by GetTokenText(), DIALOG_BOM_CFG_LEXER::TokenName(), TEMPLATE_FIELDNAMES_LEXER::TokenName(), FP_LIB_TABLE_LEXER::TokenName(), LIB_TABLE_LEXER::TokenName(), PCB_PLOT_PARAMS_LEXER::TokenName(), CMP_LIBRARY_LEXER::TokenName(), NETLIST_LEXER::TokenName(), PAGE_LAYOUT_READER_LEXER::TokenName(), PCB_LEXER::TokenName(), and SPECCTRA_LEXER::TokenName().

273 {
274  const char* ret;
275 
276  switch( aTok )
277  {
278  case DSN_NONE:
279  ret = "NONE";
280  break;
281  case DSN_STRING_QUOTE:
282  ret = "string_quote"; // a special DSN syntax token, see specctra spec.
283  break;
284  case DSN_QUOTE_DEF:
285  ret = "quoted text delimiter";
286  break;
287  case DSN_DASH:
288  ret = "-";
289  break;
290  case DSN_SYMBOL:
291  ret = "symbol";
292  break;
293  case DSN_NUMBER:
294  ret = "number";
295  break;
296  case DSN_RIGHT:
297  ret = ")";
298  break;
299  case DSN_LEFT:
300  ret = "(";
301  break;
302  case DSN_STRING:
303  ret = "quoted string";
304  break;
305  case DSN_EOF:
306  ret = "end of input";
307  break;
308  default:
309  ret = "???";
310  }
311 
312  return ret;
313 }
void DSNLEXER::Unexpected ( int  aTok)
throw (IO_ERROR
)

Function Unexpected throws an IO_ERROR exception with an input file specific error message.

Parameters
aTokis the token/keyword type which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 370 of file dsnlexer.cpp.

References Format(), GetChars(), and THROW_PARSE_ERROR.

Referenced by SYMBOL_LIB_TABLE::Parse(), FP_LIB_TABLE::Parse(), LIB_TABLE_ROW::Parse(), and scanList().

371 {
372  wxString errText = wxString::Format(
373  _("Unexpected '%s'"), GetChars( GetTokenString( aTok ) ) );
375 }
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
static const wxChar * GetChars(const wxString &s)
Function GetChars returns a wxChar* to the actual wxChar* data within a wxString, and is helpful for ...
Definition: macros.h:92
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, CPTREE &aTree)
Function Format outputs a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:205
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507
wxString GetTokenString(int aTok)
Function GetTokenString returns a quote wrapped wxString representation of a token value...
Definition: dsnlexer.cpp:335
void DSNLEXER::Unexpected ( const char *  aToken)
throw (IO_ERROR
)

Function Unexpected throws an IO_ERROR exception with an input file specific error message.

Parameters
aTokenis the token which was not expected at the current input location.
Exceptions
IO_ERRORwith the location within the input file of the problem.

Definition at line 386 of file dsnlexer.cpp.

References Format(), GetChars(), and THROW_PARSE_ERROR.

387 {
388  wxString errText = wxString::Format(
389  _("Unexpected '%s'"), GetChars( wxString::FromUTF8( text ) ) );
391 }
const wxString & CurSource()
Function CurFilename returns the current LINE_READER source.
Definition: dsnlexer.h:528
#define THROW_PARSE_ERROR(aProblem, aSource, aInputLine, aLineNumber, aByteIndex)
Definition: ki_exception.h:133
int CurOffset()
Function CurOffset returns the byte offset within the current line, using a 1 based index...
Definition: dsnlexer.h:538
const char * CurLine()
Function CurLine returns the current line of text, from which the CurText() would return its token...
Definition: dsnlexer.h:517
static const wxChar * GetChars(const wxString &s)
Function GetChars returns a wxChar* to the actual wxChar* data within a wxString, and is helpful for ...
Definition: macros.h:92
void Format(OUTPUTFORMATTER *out, int aNestLevel, int aCtl, CPTREE &aTree)
Function Format outputs a PTREE into s-expression format via an OUTPUTFORMATTER derivative.
Definition: ptree.cpp:205
int CurLineNumber()
Function CurLineNumber returns the current line number within my LINE_READER.
Definition: dsnlexer.h:507

Member Data Documentation

bool DSNLEXER::commentsAreTokens
protected

true if should return comments as tokens

Definition at line 103 of file dsnlexer.h.

Referenced by init(), NextTok(), and SetCommentsAreTokens().

int DSNLEXER::curOffset
protected

offset within current line of the current token

Definition at line 106 of file dsnlexer.h.

Referenced by init(), NextTok(), and SyncLineReaderWith().

std::string DSNLEXER::curText
protected

the text of the current token

Definition at line 109 of file dsnlexer.h.

Referenced by CurStr(), NextTok(), and SyncLineReaderWith().

int DSNLEXER::curTok
protected

the current token obtained on last NextTok()

Definition at line 108 of file dsnlexer.h.

Referenced by CurTok(), init(), and NextTok().

char DSNLEXER::dummy[1]
protected

when there is no reader.

Definition at line 87 of file dsnlexer.h.

Referenced by PopReader().

bool DSNLEXER::iOwnReaders
protected

on readerStack, should I delete them?

Definition at line 83 of file dsnlexer.h.

Referenced by DSN::SPECCTRA_DB::SPECCTRA_DB(), and ~DSNLEXER().

KEYWORD_MAP DSNLEXER::keyword_hash
protected

fast, specialized "C string" hashtable

Definition at line 113 of file dsnlexer.h.

Referenced by findToken(), and init().

unsigned DSNLEXER::keywordCount
protected

count of keywords table

Definition at line 112 of file dsnlexer.h.

Referenced by GetTokenText(), and init().

const KEYWORD* DSNLEXER::keywords
protected

table sorted by CMake for bsearch()

Definition at line 111 of file dsnlexer.h.

Referenced by GetTokenText(), and init().

const char* DSNLEXER::limit
protected

Definition at line 86 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), and SyncLineReaderWith().

const char* DSNLEXER::next
protected

Definition at line 85 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), and SyncLineReaderWith().

int DSNLEXER::prevTok
protected

curTok from previous NextTok() call.

Definition at line 105 of file dsnlexer.h.

Referenced by init(), NextTok(), and PrevTok().

LINE_READER* DSNLEXER::reader
protected

no ownership. ownership is via readerStack, maybe, if iOwnReaders

Definition at line 92 of file dsnlexer.h.

Referenced by PCB_PLOT_PARAMS_PARSER::GetReader(), PopReader(), PushReader(), and SyncLineReaderWith().

READER_STACK DSNLEXER::readerStack
protected

all the LINE_READERs by pointer.

Definition at line 91 of file dsnlexer.h.

Referenced by PopReader(), PushReader(), and ~DSNLEXER().

bool DSNLEXER::space_in_quoted_tokens
protected

blank spaces within quoted strings

Definition at line 101 of file dsnlexer.h.

Referenced by init(), SetSpaceInQuotedTokens(), and SetSpecctraMode().

bool DSNLEXER::specctraMode
protected

if true, then: 1) stringDelimiter can be changed 2) Kicad quoting protocol is not in effect 3) space_in_quoted_tokens is functional else not.

Definition at line 94 of file dsnlexer.h.

Referenced by init(), NextTok(), and SetSpecctraMode().

const char* DSNLEXER::start
protected

Definition at line 84 of file dsnlexer.h.

Referenced by NextTok(), PopReader(), PushReader(), readLine(), and SyncLineReaderWith().

char DSNLEXER::stringDelimiter
protected

Definition at line 100 of file dsnlexer.h.

Referenced by init(), NextTok(), SetSpecctraMode(), and SetStringDelimiter().


The documentation for this class was generated from the following files: