#include <ARPAPredictor.h>

Inheritance diagram for ARPAPredictor:

Collaboration diagram for ARPAPredictor:

Public Member Functions
	ARPAPredictor (Configuration , ContextTracker , const char *)

	~ARPAPredictor ()

virtual Prediction	predict (const size_t size, const char **filter) const
	Generate prediction. More...

virtual void	learn (const std::vector< std::string > &change)

virtual void	update (const Observable *variable)

void	set_vocab_filename (const std::string &value)

void	set_arpa_filename (const std::string &value)

void	set_timeout (const std::string &value)

Public Member Functions inherited from Predictor
	Predictor (Configuration configuration, ContextTracker contextTracker, const char predictorName="Predictor", const char shortDescription="", const char *longDescription="")

virtual	~Predictor ()

const std::string	getName () const

const std::string	getShortDescription () const

const std::string	getLongDescription () const

Public Member Functions inherited from Observer
virtual	~Observer ()

Private Member Functions
void	loadVocabulary ()

void	createARPATable ()

bool	matchesPrefixAndFilter (std::string, std::string, const char **) const

void	addUnigram (std::string)

void	addBigram (std::string)

void	addTrigram (std::string)

float	computeTrigramBackoff (int, int, int) const

float	computeBigramBackoff (int, int) const

Private Attributes
std::string	LOGGER

std::string	ARPAFILENAME

std::string	VOCABFILENAME

std::string	TIMEOUT

std::string	arpaFilename

std::string	vocabFilename

int	timeout

std::map< std::string, int >	vocabCode

std::map< int, std::string >	vocabDecode

std::map< int, ARPAData >	unigramMap

std::map< BigramKey, ARPAData >	bigramMap

std::map< TrigramKey, float >	trigramMap

int	unigramCount

int	bigramCount

int	trigramCount

int	unigramTot

int	bigramTot

int	trigramTot

ProgressBar< char > *	unigramProg

ProgressBar< char > *	bigramProg

ProgressBar< char > *	trigramProg

Dispatcher< ARPAPredictor >	dispatcher

Additional Inherited Members
Protected Member Functions inherited from Predictor
virtual bool	token_satisfies_filter (const std::string &token, const std::string &prefix, const char **filter) const

virtual void	set_logger (const std::string &level)

Protected Member Functions inherited from Observer
	Observer ()

Protected Attributes inherited from Predictor
const std::string	name

const std::string	shortDescription

const std::string	longDescription

const std::string	PREDICTORS

ContextTracker *	contextTracker

Configuration *	configuration

Logger< char >	logger

Detailed Description

Smoothed n-gram statistical predictor.

Definition at line 112 of file ARPAPredictor.h.

Constructor & Destructor Documentation

◆ ARPAPredictor()

ARPAPredictor::ARPAPredictor	(	Configuration *	config,
		ContextTracker *	ct,
		const char *	name
	)

Definition at line 37 of file ARPAPredictor.cpp.

References ARPAFILENAME, config, createARPATable(), dispatcher, loadVocabulary(), LOGGER, Dispatcher< class_t >::map(), Predictor::name, Predictor::PREDICTORS, set_arpa_filename(), Predictor::set_logger(), set_timeout(), set_vocab_filename(), TIMEOUT, and VOCABFILENAME.

Here is the call graph for this function:

◆ ~ARPAPredictor()

ARPAPredictor::~ARPAPredictor ( )

Definition at line 298 of file ARPAPredictor.cpp.

References bigramProg, trigramProg, and unigramProg.

Member Function Documentation

◆ addBigram()

void ARPAPredictor::addBigram ( std::string row )

private

Definition at line 241 of file ARPAPredictor.cpp.

References bigramCount, bigramMap, bigramProg, bigramTot, endl(), Predictor::logger, OOV, ProgressBar< _charT, _Traits >::update(), and vocabCode.

Referenced by createARPATable().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ addTrigram()

void ARPAPredictor::addTrigram ( std::string row )

private

Definition at line 268 of file ARPAPredictor.cpp.

References endl(), Predictor::logger, OOV, trigramCount, trigramMap, trigramProg, trigramTot, ProgressBar< _charT, _Traits >::update(), and vocabCode.

Referenced by createARPATable().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ addUnigram()

void ARPAPredictor::addUnigram ( std::string row )

private

Definition at line 214 of file ARPAPredictor.cpp.

References endl(), Predictor::logger, OOV, unigramCount, unigramMap, unigramProg, unigramTot, ProgressBar< _charT, _Traits >::update(), and vocabCode.

Referenced by createARPATable().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ computeBigramBackoff()

float ARPAPredictor::computeBigramBackoff	(	int	wd1,
		int	wd2
	)		const

inlineprivate

Computes P( wd2 | wd1 )

Definition at line 443 of file ARPAPredictor.cpp.

References bigramMap, and unigramMap.

Referenced by computeTrigramBackoff(), and predict().

Here is the caller graph for this function:

◆ computeTrigramBackoff()

float ARPAPredictor::computeTrigramBackoff	(	int	wd1,
		int	wd2,
		int	wd3
	)		const

inlineprivate

Computes P(wd3 | wd1 wd2)

Definition at line 409 of file ARPAPredictor.cpp.

References bigramMap, computeBigramBackoff(), endl(), Predictor::logger, trigramMap, and vocabDecode.

Referenced by predict().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ createARPATable()

void ARPAPredictor::createARPATable ( )

private

Definition at line 106 of file ARPAPredictor.cpp.

References addBigram(), addTrigram(), addUnigram(), arpaFilename, bigramCount, bigramProg, bigramTot, endl(), Predictor::logger, trigramCount, trigramProg, trigramTot, unigramCount, unigramProg, and unigramTot.

Referenced by ARPAPredictor().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ learn()

void ARPAPredictor::learn ( const std::vector< std::string > & change )

virtual

Implements Predictor.

Definition at line 455 of file ARPAPredictor.cpp.

References endl(), and Predictor::logger.

Here is the call graph for this function:

◆ loadVocabulary()

void ARPAPredictor::loadVocabulary ( )

private

Definition at line 79 of file ARPAPredictor.cpp.

References endl(), Predictor::logger, vocabCode, vocabDecode, and vocabFilename.

Referenced by ARPAPredictor().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ matchesPrefixAndFilter()

bool ARPAPredictor::matchesPrefixAndFilter	(	std::string	word,
		std::string	prefix,
		const char **	filter
	)		const

private

Definition at line 305 of file ARPAPredictor.cpp.

Referenced by predict().

Here is the caller graph for this function:

◆ predict()

Prediction ARPAPredictor::predict	(	const size_t	size,
		const char **	filter
	)		const

virtual

Generate prediction.

Parameters

size	desired prediction size
filter	filter

note if we have not tokens to compute 3-gram probabilities we compute 2-gram or 1-gram probabilities. the following code might be repetitive but more efficient than having the main loop outside.

Implements Predictor.

Definition at line 320 of file ARPAPredictor.cpp.

References Prediction::addSuggestion(), computeBigramBackoff(), computeTrigramBackoff(), Predictor::contextTracker, endl(), ContextTracker::getToken(), Predictor::logger, matchesPrefixAndFilter(), Utility::strtolower(), unigramMap, vocabCode, and vocabDecode.