KJB
|
#include <Tokenizer.h>
Public Member Functions | |
Tokenizer (const std::string &dataPath) | |
void | split (const String &buffer, std::vector< spear::Word > &tokens) |
Public Member Functions inherited from spear::RCObject | |
void | addReference () |
void | removeReference () |
Static Public Member Functions | |
static String | normalize (const String &s) |
Additional Inherited Members | |
Public Attributes inherited from spear::RCObject | |
unsigned short | refCount |
Protected Member Functions inherited from spear::RCObject | |
RCObject () | |
RCObject (const RCObject &rhs) | |
RCObject & | operator= (const RCObject &rhs) |
virtual | ~RCObject ()=0 |
Tokenizer::Tokenizer | ( | const std::string & | dataPath | ) |
void Tokenizer::split | ( | const String & | buffer, |
std::vector< spear::Word > & | tokens | ||
) |