I'm working on a compiler project requiring two Lexer and Parser structs respectively. My Parser is supposed to be able to control our Lexer struct and call its member functions when needed for things like retrieving tokens to parse statements correctly.
I've written two separate .h files to contain forward declarations to all of the structs, however I'm seeing a bunch of incomplete type "(insert struct name) is not allowed" errors. I believe there's something wrong with my forward declaration's that I don't understand.
Furthermore, there are some errors regarding static local variables in my Parser struct in parser.cpp . I've put comments where the errors occur!
For added context, I'm following a tutorial for this project, which you can find here.
Here is the following logic I'm using, alongside some comments for extra details:
// lexer.h
#ifndef LEXER
#define LEXER
struct Lexer;
struct TokenType;
struct Token;
#endif
// parser.h
#ifndef PARSER
#define PARSER
struct Parser;
#endif
// test.cpp, file where we call the Lexer + Parser
Lexer lex { source }; // incomplete type "Lexer" is not allowedC/C++(70)
lex.init_source(); // incomplete type "Lexer" is not allowedC/C++(70)
Parser parse { std::move(lex) }; // incomplete type "Parser" is not allowedC/C++(70)
parse.init(); // incomplete type "Parser" is not allowedC/C++(70)
parse.program(); // incomplete type "Parser" is not allowedC/C++(70)
// lexer.cpp, lexer component of compiler
struct TokenType // need to access this in parser.cpp since we use the enum values there
{
enum Token
{
ENDOFFILE = -1,
NEWLINE = 0,
NUMBER = 1,
... // bunch of other tokens
};
struct Token
{
std::string tokenText {}; // raw text of token
int tokenKind; // enum value of token from TokenType::Token
};
struct Lexer
{
std::string source {}; // source string
size_t curPos = 0; // current index in string array/source, size_t used to ensure no conversions later
char curChar = ' '; // current character in string
... // bunch of other member functions
};
// parser.cpp, parser component of compiler
struct Parser
{
Lexer lex; // attempt to pass Lexer object to Parser, as seen in test.cpp
// the variable declarations here are bugging out and are marked as undefined everywhere where they're used, even though they should have static duration and thus act somewhat like global variables?
void nextToken()
{
static auto peekToken = lex.getToken(); // accessing Lexer struct member functions here, would be able to work if we could actually send the Lexer's components through the member variable above
static auto curToken = peekToken;
}
bool checkToken(auto tokenKind) // enum value of token(s) in TokenType::Token in lexer.cpp
{
return tokenKind == curToken.tokenKind; // error here cuz curToken is undefined
}
bool checkPeek(auto tokenKind)
{
return tokenKind == peekToken.tokenKind; // same error here with peekToken
}
void match(auto tokenKind)
{
if (!(checkToken(tokenKind)))
{
abort("Expected token enum ", tokenKind, ", got ", curToken.tokenKind); // another error here with curToken
}
nextToken();
}
void nl()
{
std::cout << "NEWLINE\n";
match(TokenType::Token::NEWLINE); // incomplete type "TokenType" is not allowedC/C++(70), also: 'TokenType::Token' has not been declared
while (checkToken(TokenType::Token::NEWLINE)) // same errors here as above
{
nextToken();
}
}
void expression()
{
;
}
void statement()
{
if (checkToken(TokenType::Token::PRINT)) // same errors here as above
{
std::cout << "STATEMENT-PRINT\n";
nextToken();
if (checkToken(TokenType::Token::STRING)) // same errors here as above
{
nextToken();
}
else // expression given otherwise
{
expression();
}
}
nl(); // output newline
}
void program()
{
std::cout << "[TRACE] PROGRAM\n";
// parse all statements in program until EOF is reached in source file
while (!(checkToken(TokenType::Token::ENDOFFILE))) // same errors here as above
{
statement();
}
}
void init() // init initializes peekToken and curToken on Parser object initialization
{
nextToken();
nextToken();
}
};
Lemme know if you guys need additional information or explanation :)
You are using forward declarations incorrectly, and they won't solve your problem anyway.
A forward declaration simply states that a type exists but not what it looks like. You can't declare an instance of an incomplete type, since the compiler won't know how much memory to allocate for it, what constructors it has, etc.
Forward declarations are useful when you have circular references between classes, but that is not the case here, since Lexer doesn't care to know about Parser.
lexer.h needs to declare the full Lexer, Token, and TokenKind struct types.
lexer.cpp needs to define the bodies of the Lexer methods, etc.
parser.h needs to declare the full Parser type. This includes using #include "lexer.h" in order to be able to declare a Lexer member variable, to use TokenType::Token in method parameters, etc.
parser.cpp needs to define the bodies of the Parser methods, etc.
You can then compile+link the .cpp files to make your final executable.
Try something more like this instead:
lexer.h
#ifndef LEXER_H
#define LEXER_H
#include <string>
struct Token
{
std::string tokenText;
int tokenKind;
};
struct TokenType
{
enum Token
{
ENDOFFILE = -1,
NEWLINE = 0,
NUMBER = 1,
// ...
};
};
struct Lexer
{
std::string source;
size_t curPos = 0;
char curChar = ' ';
// bunch of member function DECLARATIONS ...
};
#endif
lexer.cpp
#include "lexer.h"
// DEFINE Lexer's method bodies here...
parser.h
#ifndef PARSER_H
#define PARSER_H
#include "lexer.h"
struct Parser
{
Lexer lex;
void nextToken();
bool checkToken(TokenType::Token tokenKind);
bool checkPeek(TokenType::Token tokenKind);
void match(TokenType::Token tokenKind);
void nl();
void expression();
void statement();
void program();
void init();
};
#endif
parser.cpp
#include "parser.h"
void Parser::nextToken()
{
...
}
bool Parser::checkToken(TokenType::Token tokenKind)
{
...
}
bool Parser::checkPeek(TokenType::Token tokenKind)
{
...
}
void Parser::match(TokenType::Token tokenKind)
{
...
}
void Parser::nl()
{
...
}
void Parser::expression()
{
...
}
void Parser::statement()
{
...
}
void Parser::program()
{
...
}
void Parser::init()
{
...
}
test.cpp
#include <string>
#include "lexer.h"
#include "parser.h"
int main()
{
std::string source = ...;
Lexer lex { source };
lex.init_source();
Parser parse { std::move(lex) };
parse.init()
parse.program();
}
On a separate note: your use of local static variables inside of Parser::nextToken() is also wrong. By initializing the variables at the time they are declared, they will only be assigned values one time - on the very 1st call of nextToken() on any Parser object. Subsequent calls to nextToken() on any object will not update them.
For what nextToken() is trying to do, you need to use non-static class members instead. Especially since Parser::checkToken() needs access to curToken, and Parser::checkPeek() needs access to peekToken.
Try this instead:
struct Parser
{
Lexer lex;
Token peekToken;
Token curToken;
...
};
void Parser::nextToken()
{
peekToken = lex.getToken();
curToken = peekToken;
}
bool Parser::checkToken(TokenKind::Token tokenKind)
{
return tokenKind == curToken.tokenKind;
}
bool Parser::checkPeek(TokenKind::Token tokenKind)
{
return tokenKind == peekToken.tokenKind;
}
...
void Parser::init()
{
nextToken();
}