ReverseTokenizer Class Reference

#include <reverseTokenizer.h>

Inheritance diagram for ReverseTokenizer:

Collaboration diagram for ReverseTokenizer:

Public Member Functions
	ReverseTokenizer (std::istream &stream, const std::string blanks, const std::string separs)
virtual	~ReverseTokenizer ()
virtual int	countTokens ()
virtual bool	hasMoreTokens () const
virtual std::string	nextToken ()
virtual double	progress () const
Public Member Functions inherited from Tokenizer
	Tokenizer (std::istream &stream, const std::string blankspaces, const std::string separators)
virtual	~Tokenizer ()
void	blankspaceChars (const std::string)
std::string	blankspaceChars () const
void	separatorChars (const std::string)
std::string	separatorChars () const
void	lowercaseMode (const bool)
bool	lowercaseMode () const
std::string	streamToString () const

Additional Inherited Members
Protected Member Functions inherited from Tokenizer
bool	isBlankspace (const int character) const
bool	isSeparator (const int character) const
Protected Attributes inherited from Tokenizer
std::istream &	stream
std::ios::iostate	sstate
std::streamoff	offbeg
std::streamoff	offend
std::streamoff	offset

Detailed Description

Tokenizes a stream from the end to the beginning.

---

|_|_|_|_|_|_|_| | | | | | offend | offset offbeg

Definition at line 38 of file reverseTokenizer.h.

Constructor & Destructor Documentation

ReverseTokenizer()

ReverseTokenizer::ReverseTokenizer	(	std::istream &	stream,
		const std::string	blanks,
		const std::string	separs
	)

Definition at line 27 of file reverseTokenizer.cpp.

References Tokenizer::offend, Tokenizer::offset, and Tokenizer::stream.

~ReverseTokenizer()

ReverseTokenizer::~ReverseTokenizer ( )

virtual

Definition at line 44 of file reverseTokenizer.cpp.

Member Function Documentation

countTokens()

int ReverseTokenizer::countTokens ( )

virtual

Returns the number of tokens left.

Implements Tokenizer.

Definition at line 47 of file reverseTokenizer.cpp.

References hasMoreTokens(), nextToken(), Tokenizer::offend, Tokenizer::offset, and Tokenizer::stream.

Here is the call graph for this function:

hasMoreTokens()

bool ReverseTokenizer::hasMoreTokens ( ) const

virtual

Tests if there are more tokens.

Implements Tokenizer.

Definition at line 69 of file reverseTokenizer.cpp.

References Tokenizer::offbeg, and Tokenizer::offset.

Referenced by countTokens().

Here is the caller graph for this function:

nextToken()

std::string ReverseTokenizer::nextToken ( )

virtual

Returns the next token.

Implements Tokenizer.

Definition at line 80 of file reverseTokenizer.cpp.

References Tokenizer::isBlankspace(), Tokenizer::isSeparator(), Tokenizer::lowercaseMode(), Tokenizer::offbeg, Tokenizer::offend, Tokenizer::offset, and Tokenizer::stream.

Referenced by ContextChangeDetector::change(), and countTokens().

Here is the call graph for this function:

Here is the caller graph for this function:

progress()

double ReverseTokenizer::progress ( ) const

virtual

Returns progress percentage.

Implements Tokenizer.

Definition at line 174 of file reverseTokenizer.cpp.

References Tokenizer::offbeg, Tokenizer::offend, and Tokenizer::offset.

---

The documentation for this class was generated from the following files:

• F:/msys64/home/MatteoV/git/presage/src/lib/core/tokenizer/reverseTokenizer.h
• F:/msys64/home/MatteoV/git/presage/src/lib/core/tokenizer/reverseTokenizer.cpp