All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
Public Member Functions | Static Public Member Functions | List of all members
spear::Tokenizer Class Reference

#include <Tokenizer.h>

Inheritance diagram for spear::Tokenizer:

Public Member Functions

 Tokenizer (const std::string &dataPath)
void split (const String &buffer, std::vector< spear::Word > &tokens)
- Public Member Functions inherited from spear::RCObject
void addReference ()
void removeReference ()

Static Public Member Functions

static String normalize (const String &s)

Additional Inherited Members

- Public Attributes inherited from spear::RCObject
unsigned short refCount
- Protected Member Functions inherited from spear::RCObject
 RCObject ()
 RCObject (const RCObject &rhs)
RCObjectoperator= (const RCObject &rhs)
virtual ~RCObject ()=0

Constructor & Destructor Documentation

Tokenizer::Tokenizer ( const std::string &  dataPath)

Member Function Documentation

String Tokenizer::normalize ( const String s)
void Tokenizer::split ( const String buffer,
std::vector< spear::Word > &  tokens 

The documentation for this class was generated from the following files: