KJB
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Public Member Functions | Static Public Member Functions | List of all members
spear::Tokenizer Class Reference

#include <Tokenizer.h>

Inheritance diagram for spear::Tokenizer:
spear::RCObject

Public Member Functions

 Tokenizer (const std::string &dataPath)
 
void split (const String &buffer, std::vector< spear::Word > &tokens)
 
- Public Member Functions inherited from spear::RCObject
void addReference ()
 
void removeReference ()
 

Static Public Member Functions

static String normalize (const String &s)
 

Additional Inherited Members

- Public Attributes inherited from spear::RCObject
unsigned short refCount
 
- Protected Member Functions inherited from spear::RCObject
 RCObject ()
 
 RCObject (const RCObject &rhs)
 
RCObjectoperator= (const RCObject &rhs)
 
virtual ~RCObject ()=0
 

Constructor & Destructor Documentation

Tokenizer::Tokenizer ( const std::string &  dataPath)

Member Function Documentation

String Tokenizer::normalize ( const String s)
static
void Tokenizer::split ( const String buffer,
std::vector< spear::Word > &  tokens 
)

The documentation for this class was generated from the following files: