Research Publications

• Complex time series data often encountered in scientific and engineering domains. • Deep learning (DL) is particularly successful here: – large data sets, multivariate input and/or ouput, – highly complex sequences of interactions. • Model interpretability: – Ability to understand a model’s decisions in a given context [1]. – Techniques typically not originally developed for time series data. – Time series interpretations themselves become uninterpretable. • Knowledge Discovery: – DL has potential to reveal interesting patterns in large data sets. – Potential to produce novel insights about the task itself [2, 3]. • ‘know-it’: Collaborative project that studies knowledge discovery in time series data.

@{507,
  author = {Marelie Davel and Stefan Lotz and Marthinus Theunissen and Almaro de Villiers and Chara Grant and Randle Rabe and Stefan Schoombie and Cleo Conacher},
  title = {Knowledge Discovery in Time Series Data},
  abstract = {• Complex time series data often encountered in scientific and engineering domains.
• Deep learning (DL) is particularly successful here:
– large data sets, multivariate input and/or ouput,
– highly complex sequences of interactions.
• Model interpretability:
– Ability to understand a model’s decisions in a given context [1].
– Techniques typically not originally developed for time series data.
– Time series interpretations themselves become uninterpretable.
• Knowledge Discovery:
– DL has potential to reveal interesting patterns in large data sets.
– Potential to produce novel insights about the task itself [2, 3].
• ‘know-it’: Collaborative project that studies knowledge discovery in
time series data.},
  year = {2023},
  journal = {Deep Learning Indaba 2023},
  month = {September 2023},
}

The design of minimum phase finite impulse response (FIR) filters is considered. The study demonstrates that the residual errors achieved by current state-of-the-art design methods are nowhere near the smallest error possible on a finite resolution digital computer. This is shown to be due to conceptual errors in the literature pertaining to what constitutes a factorable linear phase filter. This study shows that factorisation is possible with a zero residual error (in the absence of machine finite resolution error) if the linear operator or matrix representing the linear phase filter is positive definite. Methodology is proposed able to design a minimum phase filter that is optimal—in the sense that the residual error is limited only by the finite precision of the digital computer, with no systematic error. The study presents practical application of the proposed methodology by designing two minimum phase Chebyshev FIR filters. Results are compared to state-of-the-art methods from the literature, and it is shown that the proposed methodology is able to reduce currently achievable residual errors by several orders of magnitude.

@article{506,
  author = {Jan Olivier and Etienne Barnard},
  title = {Minimum phase finite impulse response filter design},
  abstract = {The design of minimum phase finite impulse response (FIR) filters is considered. The study demonstrates that the residual errors achieved by current state-of-the-art design methods are nowhere near the smallest error possible on a finite resolution digital computer. This is shown to be due to conceptual errors in the literature pertaining to what constitutes a factorable linear phase filter. This study shows that factorisation is possible with a zero residual error (in the absence of machine finite resolution error) if the linear operator or matrix representing the linear phase filter is positive definite. Methodology is proposed able to design a minimum phase filter that is optimal—in the sense that the residual error is limited only by the finite precision of the digital computer, with no systematic error. The study presents practical application of the proposed methodology by designing two minimum phase Chebyshev FIR filters. Results are compared to state-of-the-art methods from the literature, and it is shown that the proposed methodology is able to reduce currently achievable residual errors by several orders of magnitude.},
  year = {2023},
  journal = {The Institute of Engineering and Technology},
  volume = {17},
  edition = {7},
  month = {July 2023},
  doi = {https://doi.org/10.1049/sil2.12166},
}

Channel estimation is a critical component of vehicular communications systems, especially in high-mobility scenarios. The IEEE 802.11p standard uses preamble-based channel estimation, which is not sufficient in these situations. Recent work has proposed using deep neural networks for channel estimation in IEEE 802.11p. While these methods improved on earlier baselines they still can perform poorly, especially in very high mobility scenarios. This study proposes a novel approach that uses two independent LSTM cells in parallel and averages their outputs to update cell states. The proposed approach improves normalised mean square error, surpassing existing deep learning approaches in very high mobility scenarios.

@inbook{504,
  author = {Simbarashe Ngorima and Albert Helberg and Marelie Davel},
  title = {Sequence Based Deep Neural Networks for Channel Estimation in Vehicular Communication Systems},
  abstract = {Channel estimation is a critical component of vehicular communications systems, especially in high-mobility scenarios. The IEEE 802.11p standard uses preamble-based channel estimation, which is not sufficient in these situations. Recent work has proposed using deep neural networks for channel estimation in IEEE 802.11p. While these methods improved on earlier baselines they still can perform poorly, especially in very high mobility scenarios. This study proposes a novel approach that uses two independent LSTM cells in parallel and averages their outputs to update cell states. The proposed approach improves normalised mean square error, surpassing existing deep learning approaches in very high mobility scenarios.},
  year = {2023},
  journal = {Artificial Intelligence Research. SACAIR 2023. Communications in Computer and Information Science},
  volume = {1976},
  pages = {176 - 186},
  month = {29 November 2023},
  publisher = {Springer, Cham},
  isbn = {978-3-031-49001-9},
  doi = {https://doi.org/10.1007/978-3-031-49002-6_12},
}

In this study we examine the radial dependence of the inertial and dissipation range indices, as well as the spectral break separating the inertial and dissipation range in power density spectra of interplanetary magnetic field fluctuations using Parker Solar Probe data from the fifth solar encounter between ∼0.1 and ∼0.7 au. The derived break wavenumber compares reasonably well with previous estimates at larger radial distances and is consistent with gyro-resonant damping of Alfvénic fluctuations by thermal protons. We find that the inertial scale power-law index varies between approximately −1.65 and −1.45. This is consistent with either the Kolmogorov (−5/3) or Iroshnikov–Kraichnan (−3/2) values, and has a very weak radial dependence with a possible hint that the spectrum becomes steeper closer to the Sun. The dissipation range power-law index, however, has a clear dependence on radial distance (and turbulence age), decreasing from −3 near 0.7 au (4 days) to −4 [±0.3] at 0.1 au (0.75 days) closer to the Sun.

@inbook{503,
  author = {Stefan Lotz and Amore Nel and Robert Wicks and Owen Roberts and Nicholas Engelbrecht and Roelf Strauss and Gert Botha and Eduard Kontar and Alexander Pitňa and Stuart Bale},
  title = {The Radial Variation of the Solar Wind Turbulence Spectra near the Kinetic Break Scale from Parker Solar Probe Measurements},
  abstract = {In this study we examine the radial dependence of the inertial and dissipation range indices, as well as the spectral break separating the inertial and dissipation range in power density spectra of interplanetary magnetic field fluctuations using Parker Solar Probe data from the fifth solar encounter between ∼0.1 and ∼0.7 au. The derived break wavenumber compares reasonably well with previous estimates at larger radial distances and is consistent with gyro-resonant damping of Alfvénic fluctuations by thermal protons. We find that the inertial scale power-law
index varies between approximately −1.65 and −1.45. This is consistent with either the Kolmogorov (−5/3) or Iroshnikov–Kraichnan (−3/2) values, and has a very weak radial dependence with a possible hint that the spectrum becomes steeper closer to the Sun. The dissipation range power-law index, however, has a clear dependence on radial distance (and turbulence age), decreasing from −3 near 0.7 au (4 days) to −4 [±0.3] at 0.1 au (0.75 days) closer to the Sun.},
  year = {2023},
  journal = {The Astrophysical Journal},
  volume = {942},
  edition = {2},
  month = {01/2023},
  publisher = {The American Astronomical Society},
  doi = {10.3847/1538-4357/aca903},
}

Code-switching is a phenomenon that occurs mostly in multilingual countries where multilingual speakers often switch between languages in their conversations. The unavailability of large scale code-switched corpora hampers the development and training of language models for the generation of code-switched text. In this study, we explore the initial phase of collecting and creating Sepedi-English code-switched corpus for generating synthetic news. Radio news and the frequency of code-switching on read news were considered and analysed. We developed and trained a Transformer-based language model using the collected code-switched dataset. We observed that the frequency of code-switched data in the dataset was very low at 1.1%. We complemented our dataset with the news headlines dataset to create a new dataset. Although the frequency was still low, the model obtained the optimal loss rate of 2,361 with an accuracy of 66%.

@article{502,
  author = {Simon Ramalepe and Thipe Modipa and Marelie Davel},
  title = {The Analysis of the Sepedi-English Code-switched Radio News Corpus},
  abstract = {Code-switching is a phenomenon that occurs mostly in multilingual countries where multilingual speakers often switch between languages in
their conversations. The unavailability of large scale code-switched corpora hampers the development and training of language models for the generation of code-switched text. In this study, we explore the initial phase of collecting and creating Sepedi-English code-switched corpus for generating synthetic news. Radio news and the frequency of code-switching on read news were considered and analysed. We developed and trained a Transformer-based language model using the collected code-switched dataset. We observed that the frequency of code-switched data in the dataset was very low at 1.1%. We complemented our dataset with the news headlines dataset to create a new dataset.
Although the frequency was still low, the model obtained the optimal loss rate of 2,361 with an accuracy of 66%.},
  year = {2023},
  journal = {Journal of the Digital Humanities Association of Southern Africa},
  volume = {4},
  edition = {1},
  month = {2023-01-25},
  issue = {Vol. 4 No. 01 (2022): Proceedings of the 3rd workshop on Resources for African Indigenous Languages (RAIL)},
  doi = {https://doi.org/10.55492/dhasa.v4i01.4444},
}

Code-switched data is rarely available in written form and this makes the development of large datasets required to train codeswitched language models difficult. Currently, available Sepedi-English code-switched corpora are not large enough to train a Transformer-based model for this language pair. In prior work, larger synthetic datasets have been constructed using a combination of a monolingual and a parallel corpus to approximate authentic code-switched text. In this study, we develop and analyse a new Sepedi-English news dataset (SepEnews). We collect and curate data from local radio news bulletins and use this to augment two existing sources collected from Sepedi newspapers and news headlines, respectively. We then develop and train a Transformer-based model for generating historic code-switched news, and demonstrate and analyse the system’s performance.

@{501,
  author = {Simon Ramalepe and Thipe Modipa and Marelie Davel},
  title = {Transformer-based text generation for code-switched Sepedi-English news},
  abstract = {Code-switched data is rarely available in written form and this makes the development of large datasets required to train codeswitched language models difficult. Currently, available Sepedi-English code-switched corpora are not large enough to train a Transformer-based
model for this language pair. In prior work, larger synthetic datasets have been constructed using a combination of a monolingual and a parallel
corpus to approximate authentic code-switched text. In this study, we develop and analyse a new Sepedi-English news dataset (SepEnews). We collect and curate data from local radio news bulletins and use this to augment two existing sources collected from Sepedi newspapers and news headlines, respectively. We then develop and train a Transformer-based model for generating historic code-switched news, and demonstrate and analyse the system’s performance.},
  year = {2023},
  journal = {Southern African Conference for Artificial Intelligence Research (SACAIR)},
  pages = {84 - 97},
  month = {December 2023},
}

Heterogeneous tabular data is a common and important data format. This empirical study investigates how the performance of deep transformer models compares against benchmark gradient boosting decision tree (GBDT) methods, the more typical modelling approach. All models are optimised using a Bayesian hyperparameter optimisation protocol, which provides a stronger comparison than the random grid search hyperparameter optimisation utilized in earlier work. Since feature skewness is typically handled differently for GBDT and transformer-based models, we investigate the effect of a pre-processing step that normalises feature distribution on the model comparison process. Our analysis is based on the Rossmann Store Sales dataset, a widely recognized benchmark for regression tasks.

@{500,
  author = {Coenraad Middel and Marelie Davel},
  title = {Comparing Transformer-based and GBDT models on tabular data: A Rossmann Store Sales case study},
  abstract = {Heterogeneous tabular data is a common and important data format. This empirical study investigates how the performance of deep transformer models compares against benchmark gradient boosting decision tree (GBDT) methods, the more typical modelling approach. All models are optimised using a Bayesian hyperparameter optimisation protocol, which provides a stronger comparison than the random grid search hyperparameter optimisation utilized in earlier work. Since feature skewness is typically handled differently for GBDT and transformer-based models, we investigate the effect of a pre-processing step that normalises feature distribution on the model comparison process. Our analysis is based on the Rossmann Store Sales dataset, a widely recognized benchmark for regression tasks.},
  year = {2023},
  journal = {Southern African Conference for Artificial Intelligence Research (SACAIR)},
  pages = {115 - 129},
  month = {December 2023},
}

The cost and time required by individual organizations to build an effective cyber defence can become overwhelming with the growing number of cyber attacks. Hence, the introduction of platforms that encourage collaborative effort in the fight against cyber attacks is considered advantageous. However, the acceptability and efficiency of the CTI exchange platforms is massively challenged by lack of trust caused by security issues encountered in such communities. This review examines the security and participation cost issues revolving around the willingness of participants to either join or actively participate in CTI exchange communities and proposed solutions to the security issues from the research perspective.

@{499,
  author = {Moses Olaifa and Joey van Vuuren and Deon Plessis and Louise Leenen},
  title = {Security Issues in Cyber Threat Intelligence Exchange: A Review},
  abstract = {The cost and time required by individual organizations to
build an effective cyber defence can become overwhelming with the growing
number of cyber attacks. Hence, the introduction of platforms that
encourage collaborative effort in the fight against cyber attacks is considered
advantageous. However, the acceptability and efficiency of the CTI
exchange platforms is massively challenged by lack of trust caused by
security issues encountered in such communities. This review examines
the security and participation cost issues revolving around the willingness
of participants to either join or actively participate in CTI exchange communities
and proposed solutions to the security issues from the research
perspective.},
  year = {2023},
  journal = {Computing Conference},
  volume = {Lecture Notes in Networks and Systems 739},
  month = {20-21 October 2023},
}

Since the start of the Covid-19 pandemic, blockchain and cryptocurrency adoption has increased significantly. The adoption rate of blockchain-based technologies has surpassed the Internet adoption rate in the 90s and early 2000s. As this industry has grown significantly, so too has the instances of crypto scams. Numerous cryptocurrency scams exist to exploit users. The generally limited understanding of how cryptocurrencies operate has increased the possible number of scams, relying on people’s misplaced sense of trust and desire for making money quickly and easily. As such, investment scams have also been growing in popularity. Mirror Trading International (MTI) has been named South Africa’s biggest crypto scam in 2020, resulting in losses of $1.7 billion. It is also one of the largest reported international crypto investment scams. This paper focuses on a specific aspect of the MTI scam; an analysis on the fund movements on the blockchain from the perpetrators and members who benefited the most from the scam. The authors used various Open-Source Intelligence (OSINT) tools, alongside QLUE, as well as news articles and blockchain explorers. These tools and techniques are used to follow the money-trial on the blockchain, in search of possible mistakes made by the perpetrator. This could include instances where some personal information might have been leaked. With such disclosed personal information, OSINT tools and investigative techniques can be used to identify the criminals. Due to the CEO of MTI having been arrested, and the case currently being dealt with in the court of law in South Africa, this paper also presents investigative processes that could be followed. Thus, the focus of this paper is to follow the money and consequently propose a process for an investigator to investigate crypto crimes and scams on the blockchain. As the adoption of blockchain technologies continues to increase at unprecedented rates, it is imperative to produce investigative toolkits and use cases to help reduce time spent trying to catch bad actors within the generally anonymous realm of cryptocurrencies

@{498,
  author = {Johnny Botha and Thor Pederson and Louise Leenen},
  title = {An Analysis of the MTI Crypto Investment Scam: User Case},
  abstract = {Since the start of the Covid-19 pandemic, blockchain and cryptocurrency adoption has increased significantly. The adoption rate of blockchain-based technologies has surpassed the Internet adoption rate in the 90s and early 2000s. As this industry has grown significantly, so too has the instances of crypto scams. Numerous cryptocurrency scams exist to exploit users. The generally limited understanding of how cryptocurrencies operate has increased the possible number of scams, relying on people’s misplaced sense of trust and desire for making money quickly and easily. As such, investment scams have also been growing in popularity. Mirror Trading International (MTI) has been named South Africa’s biggest crypto scam in 2020, resulting in losses of $1.7 billion. It is also one of the largest reported international crypto investment scams. This paper focuses on a specific aspect of the MTI scam; an analysis on the fund movements on the blockchain from the perpetrators and members who benefited the most from the scam. The authors used various Open-Source Intelligence (OSINT) tools, alongside QLUE, as well as news articles and blockchain explorers. These tools and techniques are used to follow the money-trial on the blockchain, in search of possible mistakes made by the perpetrator. This could include instances where some personal information might have been leaked. With such disclosed personal information, OSINT tools and investigative techniques can be used to identify the criminals. Due to the CEO of MTI having been arrested, and the case currently being dealt with in the court of law in South Africa, this paper also presents investigative processes that could be followed. Thus, the focus of this paper is to follow the money and consequently propose a process for an investigator to investigate crypto crimes and scams on the blockchain. As the adoption of blockchain technologies continues to increase at unprecedented rates, it is imperative to produce investigative toolkits and use cases to help reduce time spent trying to catch bad actors within the generally anonymous realm of cryptocurrencies},
  year = {2023},
  journal = {Proceedings of the 22-nd European Conference on Cyber Warfare and Security (ECCWS)},
  month = {June 2023},
}

In this paper we present a new simulator to investigate consensus within organisations, based on organisational structure, team dynamics, and artefacts. We model agents who can interact with each other and with artefacts, as well as the mathematical models that govern agent behaviour. We show that for a fixed problem size, there is a maximum time within which all agents will reach consensus, independent of number of agents. We present the results from simulating wide ranges of problem sizes and agent group sizes and report on two significant statistics; the time to reach consensus and the effort to reach consensus. The time to reach consensus has implications for project delivery timelines, and the effort relates to project economics.

@{497,
  author = {Johannes Vorster and Louise Leenen},
  title = {Consensus Simulator for Organisational Structures},
  abstract = {In this paper we present a new simulator to investigate consensus within organisations, based on organisational
structure, team dynamics, and artefacts. We model agents who can interact with each other and with artefacts,
as well as the mathematical models that govern agent behaviour. We show that for a fixed problem size, there
is a maximum time within which all agents will reach consensus, independent of number of agents. We present
the results from simulating wide ranges of problem sizes and agent group sizes and report on two significant
statistics; the time to reach consensus and the effort to reach consensus. The time to reach consensus has
implications for project delivery timelines, and the effort relates to project economics.},
  year = {2023},
  journal = {the 13th International Conference on Simulation and Modelling Methodologies, Technologies and Applications (SimulTech).},
  month = {12- 14 2023},
  address = {Rome, Italy},
}

In this paper we explore the effects of subversive agents on the effectiveness of consensus-seeking processes. A subversive agent can try and commit industrial espionage, or, could be a disgruntled employee. The ability of an organisation to effectively execute projects, especially projects within large and complex organisation such as those found in large corporates, governments and military institutions, depend on team members reaching consensus on everything from the project vision through various design phases and eventually project implementation and realisation. What could the effect be of agents trying to subvert such a process in a way that does not raise suspicions? Such an agent cannot openly sabotage the project, but rather tries to influence others in a way that increases the time it takes to reach consensus, thus delaying projects in subtle ways. Here we explore the effect such agents could have on the time and effort to reach consensus though the use of a stochastic Multi-Agent-Simulation (MAS).

@inbook{495,
  author = {Johannes Vorster and Louise Leenen},
  title = {Exploring the Effects of Subversive Agents on Consensus-Seeking Processes Using a Multi-Agent Simulator},
  abstract = {In this paper we explore the effects of subversive agents on the effectiveness of consensus-seeking processes.
A subversive agent can try and commit industrial espionage, or, could be a disgruntled employee. The ability
of an organisation to effectively execute projects, especially projects within large and complex organisation
such as those found in large corporates, governments and military institutions, depend on team members
reaching consensus on everything from the project vision through various design phases and eventually project
implementation and realisation. What could the effect be of agents trying to subvert such a process in a way
that does not raise suspicions? Such an agent cannot openly sabotage the project, but rather tries to influence
others in a way that increases the time it takes to reach consensus, thus delaying projects in subtle ways. Here
we explore the effect such agents could have on the time and effort to reach consensus though the use of a
stochastic Multi-Agent-Simulation (MAS).},
  year = {2023},
  journal = {Proceedings of the 13th International Conference on Simulation and Modelling Methodologies, Technologies and Applications (SimulTech 2023)},
  month = {07/2023},
  publisher = {SCITEPRESS - Science and Technology Publications, Lda},
  address = {Portugal},
}

Blockchain and cryptocurrency adoption has increased significantly since the start of the Covid-19 pandemic. This adoption rate has overtaken the Internet adoption rate in the 90s and early 2000s, but as a result, the instances of crypto scams have also increased. The types of crypto scams reported are typically giveaway scams, rug pulls, phishing scams, impersonation scams, Ponzi schemes as well as pump and dumps. The US Federal Trade Commission (FTC) reported that in May 2021 the number of crypto scams were twelve times higher than in 2020, and the total loss increased by almost 1000%. The FTC also reported that Americans have lost more than $80 million due to cryptocurrency investment scams from October 2019 to October 2020, with victims between the ages of 20 and 39 represented 44% of the reported cases. Social Media has become the go-to place for scammers where attackers hack pre-existing profiles and ask targets’ contacts for payments in cryptocurrency. In 2020, both Joe Biden and Bill Gates’ Twitter accounts were hacked where the hacker posted tweets promising that for all payments sent to a specified address, double the amount will be returned, and this case of fraud was responsible for $100,000 in losses. A similar scheme using Elon Musk’s Twitter account resulted in losses of nearly $2 million. This paper analyses the most significant blockchain and cryptocurrency scams since the start of the Covid-19 pandemic, with the aim of raising awareness and contributing to protection against attacks. Even though the blockchain is a revolutionary technology with numerous benefits, it also poses an international crisis that cannot be ignored.

@inbook{494,
  author = {Johnny Botha and D.P. Botha and Louise Leenen},
  title = {An Analysis of Crypto Scams during the Covid-19 Pandemic: 2020-2022},
  abstract = {Blockchain and cryptocurrency adoption has increased significantly since the start of the Covid-19 pandemic. This adoption rate has overtaken the Internet adoption rate in the 90s and early 2000s, but as a result, the instances of crypto scams have also increased. The types of crypto scams reported are typically giveaway scams, rug pulls, phishing scams, impersonation scams, Ponzi schemes as well as pump and dumps. The US Federal Trade Commission (FTC) reported that in May 2021 the number of crypto scams were twelve times higher than in 2020, and the total loss increased by almost 1000%. The FTC also reported that Americans have lost more than $80 million due to cryptocurrency investment scams from October 2019 to October 2020, with victims between the ages of 20 and 39 represented 44% of the reported cases. Social Media has become the go-to place for scammers where attackers hack pre-existing profiles and ask targets’ contacts for payments in cryptocurrency. In 2020, both Joe Biden and Bill Gates’ Twitter accounts were hacked where the hacker posted tweets promising that for all payments sent to a specified address, double the amount will be returned, and this case of fraud was responsible for $100,000 in losses. A similar scheme using Elon Musk’s Twitter account resulted in losses of nearly $2 million. This paper analyses the most significant blockchain and cryptocurrency scams since the start of the Covid-19 pandemic, with the aim of raising awareness and contributing to protection against attacks. Even though the blockchain is a revolutionary technology with numerous benefits, it also poses an international crisis that cannot be ignored.},
  year = {2023},
  journal = {Proceedings of the 18th International Conference on Cyber Warfare and Security (ICCWS). Maryland USA, 9-10 March 2023},
  month = {2023},
  publisher = {Academic Publishers},
}

Data analysis-based decision-making is performed daily by domain experts. As data grows, getting access to relevant data becomes a challenge. In an approach known as Ontology-based data access (OBDA), AQ1 ontologies are advocated as a suitable formal tool to address complex data access. This technique combines a domain ontology with a data source by using a declarative mapping specification to enable data access using a domain vocabulary.We investigate this approach by studying the theoretical background; conducting a literature review on the implementation of OBDA in production systems; implementing OBDA on a relational dataset using an OBDA tool and; providing results and analysis of query answering.We selected Ontop (https://ontop-vkg.org) to illustrate how this technique enhances the data usage of the GitHub community. AQ2 Ontop is an open-source OBDA tool applied in the domain of relational databases. The implementation consists of the GHTorrent dataset and an extended SemanGit ontology. We perform a set of queries to highlight a subset of the features of this data access approach. The results look positive and can assist various use cases related to GitHub data with a semantic approach. OBDA does provide benefits in practice, such as querying in domain vocabulary and making use of reasoning over the axioms in the ontology. However, the practical impediments we observe are in the “manual” development of a domain ontology and the creation of a mapping specification which requires deep knowledge of a domain and the data. Also, implementing OBDA within the practical context of an information system requires careful consideration for a suitable user interface to facilitate the query construction from ontology vocabulary. Finally, we conclude with a summary of the paper and direction for future research.

@inbook{493,
  author = {Yahlieel Jafta and Louise Leenen and Thomas Meyer},
  title = {Investigating Ontology-based Data Access with GitHub},
  abstract = {Data analysis-based decision-making is performed daily by
domain experts. As data grows, getting access to relevant data becomes a
challenge. In an approach known as Ontology-based data access (OBDA), AQ1
ontologies are advocated as a suitable formal tool to address complex
data access. This technique combines a domain ontology with a data
source by using a declarative mapping specification to enable data access
using a domain vocabulary.We investigate this approach by studying the
theoretical background; conducting a literature review on the implementation
of OBDA in production systems; implementing OBDA on a relational
dataset using an OBDA tool and; providing results and analysis of
query answering.We selected Ontop (https://ontop-vkg.org) to illustrate
how this technique enhances the data usage of the GitHub community. AQ2
Ontop is an open-source OBDA tool applied in the domain of relational
databases. The implementation consists of the GHTorrent dataset and
an extended SemanGit ontology. We perform a set of queries to highlight
a subset of the features of this data access approach. The results look
positive and can assist various use cases related to GitHub data with
a semantic approach. OBDA does provide benefits in practice, such as
querying in domain vocabulary and making use of reasoning over the
axioms in the ontology. However, the practical impediments we observe
are in the “manual” development of a domain ontology and the creation
of a mapping specification which requires deep knowledge of a domain
and the data. Also, implementing OBDA within the practical context
of an information system requires careful consideration for a suitable
user interface to facilitate the query construction from ontology vocabulary.
Finally, we conclude with a summary of the paper and direction
for future research.},
  year = {2023},
  journal = {Lecture Notes in Computer Science 13870 (Proceedings of the 20th Extended Semantic Web Conference)},
  volume = {13870},
  month = {2023},
  publisher = {Springer},
}

Text generation is one of the important sub-tasks of natural language generation (NLG), and aims to produce humanly readable text given some input text. Deep learning approaches based on neural networks have been proposed to solve text generation tasks. Although these models can generate text, they do not necessarily capture long-term dependencies accurately, making it difficult to coherently generate longer sentences. Transformer-based models have shown significant improvement in text generation. However, these models are computationally expensive and data hungry. In this study, we develop a Sepedi text generation model using a Transformerbased approach and explore its performance. The developed model has one Transformer block with causal masking on the attention layers and two separate embedding layers. To train the model, we use the National Centre for Human Language Technology (NCHLT) Sepedi text corpus. Our experimental setup varied the model embedding size, batch size and the sequence length. The final model was able to reconstruct unseen test data with 75% accuracy: the highest accuracy achieved to date, using a Sepedi corpus.

@{511,
  author = {Simon Ramalepe and Thipe Modipa and Marelie Davel},
  title = {The development of a Sepedi text generation model using transformers},
  abstract = {Text generation is one of the important sub-tasks of natural language generation (NLG), and aims to produce humanly readable text given some input text. Deep learning approaches based on neural networks have been proposed to solve text generation tasks. Although these models can generate text, they do not necessarily capture long-term dependencies accurately, making it difficult to coherently generate longer sentences. Transformer-based models have shown significant improvement in text generation. However, these models are computationally expensive and data hungry. In this study, we develop a Sepedi text generation model using a Transformerbased approach and explore its performance. The developed model has one Transformer block with causal masking on the attention layers and two separate embedding layers. To train the model, we use the National Centre for Human Language Technology (NCHLT) Sepedi text corpus. Our experimental setup varied the model embedding size, batch size and the sequence length. The final model was able to reconstruct unseen test data with 75% accuracy: the highest accuracy achieved to date, using a Sepedi corpus.},
  year = {2022},
  journal = {Southern Africa Telecommunication Networks and Applications Conference (SATNAC)},
  pages = {51 - 56},
  month = {August 2022},
}

Traffic speed prediction using deep learning has been the topic of many studies. In this paper, we analyse the performance of Graph Neural Network-based techniques during periods of traffic congestion. We first compare a selection of recently proposed techniques that claim to achieve good results using the METR-LA and PeMS-BAY data sets. We then investigate the performance of three of these approaches – Graph WaveNet, Spacetime Neural Network (STNN) and Spatio-Temporal Attention Wavenet (STAWnet) – during congested periods, using recurrent congestion patterns to set a threshold for general congestion through the entire traffic network. Our results show that performance deteriorates significantly during congested time periods, which is concerning, as traffic speed prediction is usually of most value during times of congestion. We also found that, while the above approaches perform almost equally in the absence of congestion, there are much bigger differences in performance during peri ods of congestion.

@{510,
  author = {Marko Oosthuizen and Alwyn Hoffman and Marelie Davel},
  title = {A Comparative Study of Graph Neural Network Speed Prediction during Periods of Congestion},
  abstract = {Traffic speed prediction using deep learning has been the topic of many studies. In this paper, we analyse the performance of Graph Neural Network-based techniques during periods of traffic congestion. We first compare a selection of recently proposed techniques that claim to achieve good results using the METR-LA and PeMS-BAY data sets. We then investigate the performance of three of these approaches – Graph WaveNet, Spacetime Neural Network (STNN) and Spatio-Temporal Attention Wavenet (STAWnet) – during congested periods, using recurrent congestion patterns to set a threshold for general congestion through the entire traffic network. Our results show that performance deteriorates significantly during congested time periods, which is concerning, as traffic speed prediction is usually of most value during times of congestion. We also found that, while the above approaches perform almost equally in the absence of congestion, there are much bigger differences in performance during peri ods of congestion.},
  year = {2022},
  journal = {In Proceedings of the 14th International Joint Conference on Computational Intelligence (IJCCI 2022) - NCTA},
  volume = {1},
  pages = {331 - 338},
  month = {October 2022},
  isbn = {978-989-758-611-8},
  doi = {10.5220/0011374100003332},
}

Deep neural networks can be utilised for channel state information (CSI) estimation in wireless communications. We aim to decrease the bit error rate of such networks without increasing their complexity, since the wireless environment requires solutions with high performance while constraining implementation cost. For this reason, we investigate the use of adversarial training, which has been successfully applied to image super-resolution tasks that share similarities with CSI estimation tasks. CSI estimators are usually trained in a Single-In Single-Out (SISO) configuration to estimate the channel between two specific antennas and then applied to multi-antenna configurations. We show that the performance of neural networks in the SISO training environment is not necessarily indicative of their performance in multi-antenna systems. The analysis shows that adversarial training does not provide advantages in the SISO environment, however, adversarially trained models can outperform non-adversarially trained models when applying antenna diversity to Long-Term Evolution systems. The use of a feature extractor network is also investigated in this study and is found to have the potential to enhance the performance of Multiple-In Multiple-Out antenna configurations at higher SNRs. This study emphasises the importance of testing neural networks in the context of use while also showing possible advantages of adversarial training in multi-antenna systems without necessarily increasing network complexity.

@{509,
  author = {Andrew Oosthuizen and Albert Helberg and Marelie Davel},
  title = {Adversarial training for channel state information estimation in LTE multi-antenna systems},
  abstract = {Deep neural networks can be utilised for channel state information (CSI) estimation in wireless communications. We aim to decrease the bit error rate of such networks without increasing their complexity, since the wireless environment requires solutions with high performance while constraining implementation cost. For this reason, we investigate the use of adversarial training, which has been successfully applied to image super-resolution tasks that share similarities with CSI estimation tasks. CSI estimators are usually trained in a Single-In Single-Out (SISO) configuration to estimate the channel between two specific antennas and then applied to multi-antenna configurations. We show that the performance of neural networks in the SISO training environment is not necessarily indicative of their performance in multi-antenna systems. The analysis shows that adversarial training does not provide advantages in the SISO environment, however, adversarially trained models can outperform non-adversarially trained models when applying antenna diversity to Long-Term Evolution systems. The use of a feature extractor network is also investigated in this study and is found to have the potential to enhance the performance of Multiple-In Multiple-Out antenna configurations at higher SNRs. This study emphasises the importance of testing neural networks in the context of use while also showing possible advantages of adversarial training in multi-antenna systems without necessarily increasing network complexity.},
  year = {2022},
  journal = {Southern African Conference for Artificial Intelligence Research},
  volume = {1734},
  pages = {3 - 17},
  month = {November 2022},
  publisher = {Springer, Cham},
  isbn = {978-3-031-22320-4},
  doi = {https://doi.org/10.1007/978-3-031-22321-1_1},
}

Passive acoustic monitoring with hydrophones makes it possible to detect the presence of marine animals over large areas. For monitoring to be cost-effective, this process should be fully automated. We explore a new approach to detecting whale calls, using an end-to-end neural architecture and traditional speech features. We compare the results of the new approach with a convolutional neural network (CNN) applied to spectrograms, currently the standard approach to whale call detection. Experiments are conducted using the “Acoustic trends for the blue and fin whale library” from the Australian Antarctic Data Centre (AADC). We experiment with different types of speech features (mel frequency cepstral coefficients and filter banks) and different ways of framing the task. We demonstrate that a time delay neural network is a viable solution for whale call detection, with the additional benefit that spectrogram tuning – required to obtain high-quality spectrograms in challenging acoustic conditions – is no longer necessary. While the initial speech feature-based system (accuracy 96%) did not outperform the CNN (accuracy 98%) when trained on exactly the same dataset, it presents a viable approach to explore further.

@{508,
  author = {Edrich Fourie and Marelie Davel and Jaco Versfeld},
  title = {Neural speech processing for whale call detection},
  abstract = {Passive acoustic monitoring with hydrophones makes it possible to detect the presence of marine animals over large areas. For monitoring to be cost-effective, this process should be fully automated. We explore a new approach to detecting whale calls, using an end-to-end neural architecture and traditional speech features. We compare the results of the new approach with a convolutional neural network (CNN) applied to spectrograms, currently the standard approach to whale call detection. Experiments are conducted using the “Acoustic trends for the blue and fin whale library” from the Australian Antarctic Data Centre (AADC). We experiment with different types of speech features (mel frequency cepstral coefficients and filter banks) and different ways of framing the task. We demonstrate that a time delay neural network is a viable solution for whale call detection, with the additional benefit that spectrogram tuning – required to obtain high-quality spectrograms in challenging acoustic conditions – is no longer necessary. While the initial speech feature-based system (accuracy 96%) did not outperform the CNN (accuracy 98%) when trained on exactly the same dataset, it presents a viable approach to explore further.},
  year = {2022},
  journal = {Southern African Conference for AI Research (SACAIR)},
  volume = {1734},
  pages = {276 - 290},
  month = {November 2022},
  publisher = {Springer, Cham},
  doi = {https://doi.org/10.1007/978-3-031-22321-1_19},
}

Classification margins are commonly used to estimate the generalization ability of machine learning models. We present an empirical study of these margins in artificial neural networks. A global estimate of margin size is usually used in the literature. In this work, we point out seldom considered nuances regarding classification margins. Notably, we demonstrate that some types of training samples are modelled with consistently small margins while affecting generalization in different ways. By showing a link with the minimum distance to a different-target sample and the remoteness of samples from one another, we provide a plausible explanation for this observation. We support our findings with an analysis of fully-connected networks trained on noise-corrupted MNIST data, as well as convolutional networks trained on noise-corrupted CIFAR10 data.

@inbook{505,
  author = {Marthinus Theunissen and Coenraad Mouton and Marelie Davel},
  title = {The Missing Margin: How Sample Corruption Affects Distance to the Boundary in ANNs},
  abstract = {Classification margins are commonly used to estimate the generalization ability of machine learning models. We present an empirical study of these margins in artificial neural networks. A global estimate of margin size is usually used in the literature. In this work, we point out seldom considered nuances regarding classification margins. Notably, we demonstrate that some types of training samples are modelled with consistently small margins while affecting generalization in different ways. By showing a link with the minimum distance to a different-target sample and the remoteness of samples from one another, we provide a plausible explanation for this observation. We support our findings with an analysis of fully-connected networks trained on noise-corrupted MNIST data, as well as convolutional networks trained on noise-corrupted CIFAR10 data.},
  year = {2022},
  journal = {Artificial Intelligence Research (SACAIR 2022), Communications in Computer and Information Science},
  volume = {1734},
  pages = {78 - 92},
  month = {November 2022},
  publisher = {Springer, Cham},
  doi = {https://doi.org/10.48550/arXiv.2302.06925},
}

We propose a new framework to improve automatic speech recognition (ASR) systems in resource-scarce environments using a generative adversarial network (GAN) operating on acoustic input features. The GAN is used to enhance the features of mismatched data prior to decoding, or can optionally be used to fine-tune the acoustic model. We achieve improvements that are comparable to multi-style training (MTR), but at a lower computational cost. With less than one hour of data, an ASR system trained on good quality data, and evaluated on mismatched audio is improved by between 11.5% and 19.7% relative word error rate (WER). Experiments demonstrate that the framework can be very useful in under-resourced environments where training data and computational resources are limited. The GAN does not require parallel training data, because it utilises a baseline acoustic model to provide an additional loss term that guides the generator to create acoustic features that are better classified by the baseline.

@article{492,
  author = {Walter Heymans and Marelie Davel and Charl Van Heerden},
  title = {Efficient acoustic feature transformation in mismatched environments using a Guided-GAN},
  abstract = {We propose a new framework to improve automatic speech recognition (ASR) systems in resource-scarce environments using a generative adversarial network (GAN) operating on acoustic input features. The GAN is used to enhance the features of mismatched data prior to decoding, or can optionally be used to fine-tune the acoustic model. We achieve improvements that are comparable to multi-style training (MTR), but at a lower computational cost. With less than one hour of data, an ASR system trained on good quality data, and evaluated on mismatched audio is improved by between 11.5% and 19.7% relative word error rate (WER). Experiments demonstrate that the framework can be very useful in under-resourced environments where training data and computational resources are limited. The GAN does not require parallel training data, because it utilises a baseline acoustic model to provide an additional loss term that guides the generator to create acoustic features that are better classified by the baseline.},
  year = {2022},
  journal = {Speech Communication},
  volume = {143},
  pages = {10 - 20},
  month = {09/2022},
  doi = {https://doi.org/10.1016/j.specom.2022.07.002},
}

The accurate estimation of channel state information (CSI) is an important aspect of wireless communications. In this paper, a multi-layer perceptron (MLP) is developed as a CSI estimator in long-term evolution (LTE) transmission conditions. The representation of the CSI data is investigated in conjunction with batch normalisation and the representational ability of MLPs. It is found that discontinuities in the representational feature space can cripple an MLP’s ability to accurately predict CSI when noise is present. Different ways in which to mitigate this effect are analysed and a solution developed, initially in the context of channels that are only affected by additive white Guassian noise. The developed architecture is then applied to more complex channels with various delay profiles and Doppler spread. The performance of the proposed MLP is shown to be comparable with LTE minimum mean squared error (MMSE), and to outperform least square (LS) estimation over a range of channel conditions.

@{491,
  author = {Andrew Oosthuizen and Marelie Davel and Albert Helberg},
  title = {Multi-Layer Perceptron for Channel State Information Estimation: Design Considerations},
  abstract = {The accurate estimation of channel state information (CSI) is an important aspect of wireless communications. In this paper, a multi-layer perceptron (MLP) is developed as a CSI estimator in long-term evolution (LTE) transmission conditions. The representation of the CSI data is investigated in conjunction with batch normalisation and the representational ability of MLPs. It is found that discontinuities in the representational feature space can cripple an MLP’s ability to accurately predict CSI when noise is present. Different ways in which to mitigate this effect are analysed and a solution developed, initially in the context of channels that are only affected by additive white
Guassian noise. The developed architecture is then applied to more complex channels with various delay profiles and Doppler spread. The performance of the proposed MLP is shown to be comparable with LTE minimum mean squared error (MMSE), and to outperform least square (LS) estimation over a range of channel conditions.},
  year = {2022},
  journal = {Southern Africa Telecommunication Networks and Applications Conference (SATNAC)},
  pages = {94 - 99},
  month = {08/2022},
  address = {Fancourt, George},
}

Building computational models of agents in dynamic, partially observable and stochastic environments is challenging. We propose a cognitive computational model of sugarcane growers’ daily decision-making to examine sugarcane supply chain complexities. Growers make decisions based on uncertain weather forecasts; cane dryness; unforeseen emergencies; and the mill’s unexpected call for delivery of a different amount of cane. The Belief-Desire-Intention (BDI) architecture has been used to model cognitive agents in many domains, including agriculture. However, typical implementations of this architecture have represented beliefs symbolically, so uncertain beliefs are usually not catered for. Here we show that a BDI architecture, enhanced with a dynamic decision network (DDN), suitably models sugarcane grower agents’ repeated daily decisions. Using two complex scenarios, we demonstrate that the agent selects the appropriate intention, and suggests how the grower should act adaptively and proactively to achieve his goals. In addition, we provide a mapping for using a DDN in a BDI architecture. This architecture can be used for modelling sugarcane grower agents in an agent-based simulation. The mapping of the DDN’s use in the BDI architecture enables this work to be applied to other domains for modelling agents’ repeated decisions in partially observable, stochastic and dynamic environments.

@article{488,
  author = {C. Sue Price and Deshen Moodley and Anban Pillay and Gavin Rens},
  title = {An adaptive probabilistic agent architecture for modelling sugarcane growers’ decision-making},
  abstract = {Building computational models of agents in dynamic, partially observable and stochastic environments is challenging.  We propose a cognitive computational model of sugarcane growers’ daily decision-making to examine sugarcane supply chain complexities.  Growers make decisions based on uncertain weather forecasts; cane dryness; unforeseen emergencies; and the mill’s unexpected call for delivery of a different amount of cane.  The Belief-Desire-Intention (BDI) architecture has been used to model cognitive agents in many domains, including agriculture.  However, typical implementations of this architecture have represented beliefs symbolically, so uncertain beliefs are usually not catered for.  Here we show that a BDI architecture, enhanced with a dynamic decision network (DDN), suitably models sugarcane grower agents’ repeated daily decisions.  Using two complex scenarios, we demonstrate that the agent selects the appropriate intention, and suggests how the grower should act adaptively and proactively to achieve his goals.  In addition, we provide a mapping for using a DDN in a BDI architecture.  This architecture can be used for modelling sugarcane grower agents in an agent-based simulation.  The mapping of the DDN’s use in the BDI architecture enables this work to be applied to other domains for modelling agents’ repeated decisions in partially observable, stochastic and dynamic environments.},
  year = {2022},
  journal = {South African Computer Journal},
  volume = {34},
  pages = {152-191},
  issue = {1},
  url = {https://sacj.cs.uct.ac.za/index.php/sacj/article/view/857},
  doi = {https://doi.org/10.18489/sacj.v34i1.857},
}

Artifcial Intelligence (AI) systems are ubiquitous. From social media timelines, video recommendations on YouTube, and the kinds of adverts we see online, AI, in a very real sense, flters the world we see. More than that, AI is being embedded in agent-like systems, which might prompt certain reactions from users. Specifcally, we might fnd ourselves feeling frustrated if these systems do not meet our expectations. In normal situations, this might be fne, but with the ever increasing sophistication of AI-systems, this might become a problem. While it seems unproblematic to realize that being angry at your car for breaking down is unfitting, can the same be said for AI-systems? In this paper, therefore, I will investigate the so-called “reactive attitudes”, and their important link to our responsibility practices. I then show how within this framework there exist exemption and excuse conditions, and test whether our adopting the “objective attitude” toward agential AI is justifed. I argue that such an attitude is appropriate in the context of three distinct senses of responsibility (answerability, attributability, and accountability), and that, therefore, AI-systems do not undermine our responsibility ascriptions.

@article{487,
  author = {Fabio Tollon},
  title = {Responsibility gaps and the reactive attitudes},
  abstract = {Artifcial Intelligence (AI) systems are ubiquitous. From social media timelines, video recommendations on YouTube, and the kinds of adverts we see online, AI, in a very real sense, flters the world we see. More than that, AI is being embedded in agent-like systems, which might prompt certain reactions from users. Specifcally, we might fnd ourselves feeling frustrated if these systems do not meet our expectations. In normal situations, this might be fne, but with the ever increasing sophistication of AI-systems, this might become a problem. While it seems unproblematic to realize that being angry at your car for breaking down is unfitting, can the same be said for AI-systems? In this paper, therefore, I will investigate the so-called “reactive attitudes”, and their important link to our responsibility practices. I then show how within this framework there exist exemption and excuse conditions, and test whether our adopting the “objective attitude” toward agential AI is justifed. I argue that such an attitude is appropriate in the context of three distinct senses of responsibility (answerability, attributability, and accountability), and that, therefore, AI-systems do not undermine our responsibility ascriptions.},
  year = {2022},
  journal = {AI and Ethics},
  publisher = {Springer},
  url = {https://link.springer.com/article/10.1007/s43681-022-00172-6},
  doi = {https://doi.org/10.1007/s43681-022-00172-6},
}

We report on the development of two reference corpora for the analysis of SepediEnglish code-switched speech in the context of automatic speech recognition. For the first corpus, possible English events were obtained from an existing corpus of transcribed Sepedi-English speech. The second corpus is based on the analysis of radio broadcasts: actual instances of code switching were transcribed and reproduced by a number of native Sepedi speakers. We describe the process to develop and verify both corpora and perform an initial analysis of the newly produced data sets. We find that, in naturally occurring speech, the frequency of code switching is unexpectedly high for this language pair, and that the continuum of code switching (from unmodified embedded words to loanwords absorbed into the matrix language) makes this a particularly challenging task for speech recognition systems.

@article{483,
  author = {Thipe Modipa and Marelie Davel},
  title = {Two Sepedi‑English code‑switched speech corpora},
  abstract = {We report on the development of two reference corpora for the analysis of SepediEnglish code-switched speech in the context of automatic speech recognition. For the first corpus, possible English events were obtained from an existing corpus of transcribed Sepedi-English speech. The second corpus is based on the analysis of radio broadcasts: actual instances of code switching were transcribed and reproduced by a number of native Sepedi speakers. We describe the process to develop and verify both corpora and perform an initial analysis of the newly produced data sets. We find that, in naturally occurring speech, the frequency of code switching is unexpectedly high for this language pair, and that the continuum of code switching (from unmodified embedded words to loanwords absorbed into the matrix language) makes this a particularly challenging task for speech recognition systems.},
  year = {2022},
  journal = {Language Resources and Evaluation},
  volume = {56},
  pages = {https://rdcu.be/cO6lD)},
  publisher = {Springer},
  address = {South Africa},
  url = {https://rdcu.be/cO6lD},
  doi = {https://doi.org/10.1007/s10579-022-09592-6 (Read here: https://rdcu.be/cO6lD)},
}

Mismatched data is a challenging problem for automatic speech recognition (ASR) systems. One of the most common techniques used to address mismatched data is multi-style training (MTR), a form of data augmentation that attempts to transform the training data to be more representative of the testing data; and to learn robust representations applicable to different conditions. This task can be very challenging if the test conditions are unknown. We explore the impact of different MTR styles on system performance when testing conditions are different from training conditions in the context of deep neural network hidden Markov model (DNN-HMM) ASR systems. A controlled environment is created using the LibriSpeech corpus, where we isolate the effect of different MTR styles on final system performance. We evaluate our findings on a South African call centre dataset that contains noisy, WAV49-encoded audio.

@article{480,
  author = {Walter Heymans and Marelie Davel and Charl Van Heerden},
  title = {Multi-style Training for South African Call Centre Audio},
  abstract = {Mismatched data is a challenging problem for automatic speech recognition (ASR) systems. One of the most common techniques used to address mismatched data is multi-style training (MTR), a form of data augmentation that attempts to transform the training data to be more representative of the testing data; and to learn robust representations applicable to different conditions. This task can be very challenging if the test conditions are unknown. We explore the impact of different MTR styles on system performance when testing conditions are different from training conditions in the context of deep neural network hidden Markov model (DNN-HMM) ASR systems. A controlled environment is created using the LibriSpeech corpus, where we isolate the effect of different MTR styles on final system performance. We evaluate our findings on a South African call centre dataset that contains noisy, WAV49-encoded audio.},
  year = {2022},
  journal = {Communications in Computer and Information Science},
  volume = {1551},
  pages = {111 - 124},
  publisher = {Southern African Conference for Artificial Intelligence Research},
  address = {South Africa},
  doi = {https://doi.org/10.1007/978-3-030-95070-5_8},
}

While deep neural networks (DNNs) have become a standard architecture for many machine learning tasks, their internal decision-making process and general interpretability is still poorly understood. Conversely, common decision trees are easily interpretable and theoretically well understood. We show that by encoding the discrete sample activation values of nodes as a binary representation, we are able to extract a decision tree explaining the classification procedure of each layer in a ReLU-activated multilayer perceptron (MLP). We then combine these decision trees with existing feature attribution techniques in order to produce an interpretation of each layer of a model. Finally, we provide an analysis of the generated interpretations, the behaviour of the binary encodings and how these relate to sample groupings created during the training process of the neural network.

@article{479,
  author = {Coenraad Mouton and Marelie Davel},
  title = {Exploring layerwise decision making in DNNs},
  abstract = {While deep neural networks (DNNs) have become a standard architecture for many machine learning tasks, their internal decision-making process and general interpretability is still poorly understood. Conversely, common decision trees are easily interpretable and theoretically well understood. We show that by encoding the discrete sample activation values of nodes as a binary representation, we are able to extract a decision tree explaining the classification procedure of each layer in a ReLU-activated multilayer perceptron (MLP). We then combine these decision trees with existing feature attribution techniques in order to produce an interpretation of each layer of a model. Finally, we provide an analysis of the generated interpretations, the behaviour of the binary encodings and how these relate to sample groupings created during the training process of the neural network.},
  year = {2022},
  journal = {Communications in Computer and Information Science},
  volume = {1551},
  pages = {140 - 155},
  publisher = {Artificial Intelligence Research (SACAIR 2021)},
  doi = {https://doi.org/10.1007/978-3-030-95070-5_10},
}