Research Publications

2010

Barnard E. Maximum Leave-one-out Likelihood for Kernel Density Estimation. Pattern Recognition Association of South Africa (PRASA). 2010.

We investigate the application of kernel density estimators to pattern-recognition problems. These estimators have a number of attractive properties for data analysis in pattern recognition, but the particular characteristics of patternrecognition problems also place some non-trivial requirements on kernel density estimation – especially on the algorithm used to compute bandwidths. We introduce a new algorithm for variable bandwidth estimation, investigate some of its properties, and show that it performs competitively on a wide range of tasks, particularly in spaces of high dimensionality.

@proceedings{352,
  author = {Etienne Barnard},
  title = {Maximum Leave-one-out Likelihood for Kernel Density Estimation},
  abstract = {We investigate the application of kernel density estimators to pattern-recognition problems. These estimators have a number of attractive properties for data analysis in pattern recognition, but the particular characteristics of patternrecognition problems also place some non-trivial requirements on kernel density estimation – especially on the algorithm used to compute bandwidths. We introduce a new algorithm for variable bandwidth estimation, investigate some of its properties, and show that it performs competitively on a wide range of tasks, particularly in spaces of high dimensionality.},
  year = {2010},
  journal = {Pattern Recognition Association of South Africa (PRASA)},
  pages = {19-24},
  address = {Stellenbosch, South Africa},
}
Schlunz G, Barnard E, van Huyssteen G. Part-of-Speech Effects on Text-to-Speech Synthesis. Pattern Recognition Association of South Africa (PRASA). 2010.

t: One of the goals of text-to-speech (TTS) systems is to produce natural-sounding synthesized speech. Towards this end various natural language processing (NLP) tasks are performed to model the prosodic aspects of the TTS voice. One of the fundamental NLP tasks being used is the part-of-speech (POS) tagging of the words in the text. This paper investigates the effects of POS information on the naturalness of a hidden Markov model (HMM) based TTS voice when additional resources are not available to aid in the modeling of prosody. It is found that, when a minimal feature set is used for the HMM context labels, the addition of POS tags does improve the naturalness of the voice. However, the same effect can be accomplished by including segmental counting and positional information instead of the POS tags.

@proceedings{355,
  author = {Georg Schlunz and Etienne Barnard and Gerhard van Huyssteen},
  title = {Part-of-Speech Effects on Text-to-Speech Synthesis},
  abstract = {t:
One of the goals of text-to-speech (TTS) systems is to produce natural-sounding synthesized speech. Towards this end various natural language processing (NLP) tasks are performed to model the prosodic aspects of the TTS voice. One of the fundamental NLP tasks being used is the part-of-speech (POS) tagging of the words in the text. This paper investigates the effects of POS information on the naturalness of a hidden Markov model (HMM) based TTS voice when additional resources are not available to aid in the modeling of prosody. It is found that, when a minimal feature set is used for the HMM context labels, the addition of POS tags does improve the naturalness of the voice. However, the same effect can be accomplished by including segmental counting and positional information instead of the POS tags.},
  year = {2010},
  journal = {Pattern Recognition Association of South Africa (PRASA)},
  pages = {69-74},
  address = {Stellenbosch, South Africa},
  isbn = {978-0-7992-2470-2},
}
Barnard E. Visualizing data in high-dimensional spaces. Pattern Recognition Association of South Africa (PRASA). 2010.

A novel approach to the analysis of feature spaces in statistical pattern recognition is described. This approach starts with linear dimensionality reduction, followed by the computation of selected sections through and projections of feature space. A number of representative feature spaces are analysed in this way; we find linear reduction to be surprisingly successful, and in the real-world data sets we have examined, typical classes of objects are only moderately complicated.

@proceedings{353,
  author = {Etienne Barnard},
  title = {Visualizing data in high-dimensional spaces},
  abstract = {A novel approach to the analysis of feature spaces in statistical pattern recognition is described. This approach starts with linear dimensionality reduction, followed by the computation of selected sections through and projections of feature space. A number of representative feature spaces are analysed in this way; we find linear reduction to be surprisingly successful, and in the real-world data sets we have examined, typical classes of objects are only moderately complicated.},
  year = {2010},
  journal = {Pattern Recognition Association of South Africa (PRASA)},
  pages = {25-32},
  address = {Stellenbosch, South Africa},
}
van Heerden C, Barnard E. Towards Understanding the Influence of SVM Hyperparameters. Pattern Recognition Association of South Africa (PRASA). 2010.

We investigate the relationship between SVM hyperparameters for linear and RBF kernels and classification accuracy. The process of finding SVM hyperparameters usually involves a gridsearch, which is both time-consuming and resource-intensive. On large datasets, 10-fold cross-validation grid searches can become intractable without supercomputers or high performance computing clusters. We present theoretical and empirical arguments as to how SVM hyperparameters scale with N, the amount of learning data. By using these arguments, we present a simple algorithm for finding approximate hyperparameters on a reduced dataset, followed by a focused line search on the full dataset. Using this algorithm gives comparable results to performing a grid search on complete datasets.

@proceedings{356,
  author = {Charl van Heerden and Etienne Barnard},
  title = {Towards Understanding the Influence of SVM Hyperparameters},
  abstract = {We investigate the relationship between SVM hyperparameters for linear and RBF kernels and classification accuracy. The process of finding SVM hyperparameters usually involves a gridsearch, which is both time-consuming and resource-intensive. On large datasets, 10-fold cross-validation grid searches can become intractable without supercomputers or high performance computing clusters. We present theoretical and empirical arguments as to how SVM hyperparameters scale with N, the amount of learning data. By using these arguments, we present a simple algorithm for finding approximate hyperparameters on a reduced dataset, followed by a focused line search on the full dataset. Using this algorithm gives comparable results to performing a grid search on complete datasets.},
  year = {2010},
  journal = {Pattern Recognition Association of South Africa (PRASA)},
  pages = {283-288},
  address = {Stellenbosch, South Africa},
}
Badenhorst J, Davel MH, Barnard E. Analysing co-articulation using frame-based feature trajectories. Pattern Recognition Association of South Africa (PRASA). 2010. doi:10.13140/RG.2.1.2643.4008.

We investigate several approaches aimed at a more detailed understanding of co-articulation in spoken utterances. They find that the Euclidean difference between instantaneous frame-based feature values and the mean values of these features are most useful for these purposes, and that low-order polynomials are able to model the between-phone transitions accurately. Examples of typical transitions are presented, and shown to give useful insights on the measurable effects of co-articulation.

@proceedings{351,
  author = {Jaco Badenhorst and Marelie Davel and Etienne Barnard},
  title = {Analysing co-articulation using frame-based feature trajectories},
  abstract = {We investigate several approaches aimed at a more detailed understanding of co-articulation in spoken utterances. They find that the Euclidean difference between instantaneous frame-based feature values and the mean values of these features are most useful for these purposes, and that low-order polynomials are able to model the between-phone transitions accurately. Examples of typical transitions are presented, and shown to give useful insights on the measurable effects of co-articulation.},
  year = {2010},
  journal = {Pattern Recognition Association of South Africa (PRASA)},
  pages = {13-14},
  address = {Stellenbosch, South Africa},
  doi = {10.13140/RG.2.1.2643.4008},
}
de Waal A, Barnard E. The Influence of Input Matrix Representation on Topic Modelling Performance. Pattern Recognition Association of South Africa (PRASA). 2010.

Topic models explain a collection of documents with a small set of distributions over terms. These distributions over terms define the topics. Topic models ignore the structure of documents and use a bag-of-words approach which relies solely on the frequency of words in the corpus. We challenge the bag-of-word assumption and propose a method to structure single words into concepts. In this way, the inherent meaning of the feature space is enriched by more descriptive concepts rather than single words. We turn to the field of natural language processing to find processes to structure words into concepts. In order to compare the performance of structured features with the bag-of-words approach, we sketch an evaluation framework that accommodates different feature dimension sizes. This is in contrast with existing methods such as perplexity, which depend on the size of the vocabulary modelled and can therefore not be used to compare models which use different input feature sets. We use a stability-based validation index to measure a model’s ability to replicate similar solutions of independent data sets generated from the same probabilistic source. Stability-based validation acts more consistently across feature dimensions than perplexity or information-theoretic measures.

@proceedings{354,
  author = {Alta de Waal and Etienne Barnard},
  title = {The Influence of Input Matrix Representation on Topic Modelling Performance},
  abstract = {Topic models explain a collection of documents with a small set of distributions over terms. These distributions over terms define the topics. Topic models ignore the structure of documents and use a bag-of-words approach which relies solely on the frequency of words in the corpus. We challenge the bag-of-word assumption and propose a method to structure single words into concepts. In this way, the inherent meaning of the feature space is enriched by more descriptive concepts rather than single words. We turn to the field of natural language processing to find processes to structure words into concepts. In order to compare the performance of structured features with the bag-of-words approach, we sketch an evaluation framework that accommodates different feature dimension sizes. This is in contrast with existing methods such as perplexity, which depend on the size of the vocabulary modelled and can therefore not be used to compare models which use different input feature sets. We use a stability-based validation index to measure a model’s ability to replicate similar solutions of independent data sets generated from the same probabilistic source. Stability-based validation acts more consistently across feature dimensions than perplexity or information-theoretic measures.},
  year = {2010},
  journal = {Pattern Recognition Association of South Africa (PRASA)},
  pages = {69-74},
  month = {22/11-23/11},
  address = {Stellenbosch, South Africa},
}
  • CSIR
  • DSI
  • Covid-19