People
Research Groups:
More Info
Latest Research Publications:
No Results
Research Groups:
More Info
Latest Research Publications:
Haasbroek DG, Davel MH. Exploring neural network training dynamics through binary node activations. In: Southern African Conference for Artificial Intelligence Research. South Africa; 2020. https://sacair.org.za/proceedings/.
Each node in a neural network is trained to activate for a specific region in the input domain. Any training samples that fall within this domain are therefore implicitly clustered together. Recent work has highlighted the importance of these clusters during the training process but has not yet investigated their evolution during training. Towards this goal, we train several ReLU-activated MLPs on a simple classification task (MNIST) and show that a consistent training process emerges: (1) sample clusters initially increase in size and then decrease as training progresses, (2) the size of sample clusters in the first layer decreases more rapidly than in deeper layers, (3) binary node activations, especially of nodes in deeper layers, become more sensitive to class membership as training progresses, (4) individual nodes remain poor predictors of class membership, even if accurate when applied as a group. We report on the detail of these findings and interpret them from the perspective of a high-dimensional clustering process.
@{402,
author = {Daniël Haasbroek, Marelie Davel},
title = {Exploring neural network training dynamics through binary node activations},
abstract = {Each node in a neural network is trained to activate for a specific region in the input domain. Any training samples that fall within this domain are therefore implicitly clustered together. Recent work has highlighted the importance of these clusters during the training process but has not yet investigated their evolution during training. Towards this goal, we train several ReLU-activated MLPs on a simple classification task (MNIST) and show that a consistent training process emerges: (1) sample clusters initially increase in size and then decrease as training progresses, (2) the size of sample clusters in the first layer decreases more rapidly than in deeper layers, (3) binary node activations, especially of nodes in deeper layers, become more sensitive to class membership as training progresses, (4) individual nodes remain poor predictors of class membership, even if accurate when applied as a group. We report on the detail of these findings and interpret them from the perspective of a high-dimensional clustering process.},
year = {2020},
journal = {Southern African Conference for Artificial Intelligence Research},
pages = {304-320},
month = {22/02/2021 - 26/02/2021},
address = {South Africa},
isbn = {978-0-620-89373-2},
url = {https://sacair.org.za/proceedings/},
}
Research Groups:
No Group
More Info
Latest Research Publications:
No Results
Research Groups:
More Info
Latest Research Publications:
No Results
Research Groups:
More Info
Jesse Heyninck is currently a post-doctoral researcher at the University of Cape Town (UCT) and the Technical University of Dortmund. He has completed his PhD in Philosophy at the Faculty of Philosophy and Educational Sciences at the Ruhr University in Bochum, Germany. His research interests are mainly in the area of Knowledge Representation and Reasoning, especially Formal Argumentation, Non-Monotonic Logics and Defeasible Reasoning.
Latest Research Publications:
Heyninck J, Thimm M, Kern-Isberner G, Rienstra T, Skiba K. On the Relation between Possibilistic Logic and Abstract Dialectical Frameworks. 2021. https://sites.google.com/view/nmr2021/home?authuser=0).
Abstract dialectical frameworks (in short, ADFs) are one of the most general and unifying approaches to formal argumentation. As the semantics of ADFs are based on three-valued interpretations, the question poses itself as to whether some and which monotonic three-valued logic underlies ADFs, in the sense that it allows to capture the main semantic concepts underlying ADFs. As an entry-point for such an investigation, we take the concept of model of an ADF, which was originally formulated on the basis of Kleene’s three-valued logic. We show that an optimal concept of a model arises when instead of Kleene’s three-valued logic, possibilistic logic is used. We then show that in fact, possibilistic logic is the most conservative three-valued logic that fulfils this property, and that possibilistic logic can faithfully encode all other semantical concepts for ADFs. Based on this result, we also make some observations on strong equivalence and introduce possibilistic ADFs.
@misc{422,
author = {Jesse Heyninck, Matthias Thimm, Gabriele Kern-Isberner, Tjitze Rienstra, Kenneth Skiba},
title = {On the Relation between Possibilistic Logic and Abstract Dialectical Frameworks},
abstract = {Abstract dialectical frameworks (in short, ADFs) are one of the most general and unifying approaches to formal argumentation. As the semantics of ADFs are based on three-valued interpretations, the question poses itself as to whether some and which monotonic three-valued logic underlies ADFs, in the sense that it allows to capture the main semantic concepts underlying ADFs. As an entry-point for such an investigation, we take the concept of model of an ADF, which was originally formulated on the basis of Kleene’s three-valued logic. We show that an optimal concept of a model arises when instead of Kleene’s three-valued logic, possibilistic logic is used. We then show that in fact, possibilistic logic is the most conservative three-valued logic that fulfils this property, and that possibilistic logic can faithfully encode all other semantical concepts for ADFs. Based on this result, we also make some observations on strong equivalence and introduce possibilistic ADFs.},
year = {2021},
url = {https://sites.google.com/view/nmr2021/home?authuser=0)},
}
Heyninck J, Thimm M, Kern-Isberner G, Rienstra T, Skiba K. Arguing about Complex Formulas: Generalizing Abstract Dialectical Frameworks. 2021. https://sites.google.com/view/nmr2021/home?authuser=0.
Abstract dialectical frameworks (in short, ADFs) are a unifying model of formal argumentation, where argumentative relations between arguments are represented by assigning acceptance conditions to atomic arguments. This idea is generalized by letting acceptance conditions being assigned to complex formulas, resulting in conditional abstract dialectical frameworks (in short, cADFs). We define the semantics of cADFs in terms of a non-truth-functional four-valued
logic, and study the semantics in-depth, by showing existence results and proving that all semantics are generalizations of the corresponding semantics for ADFs.
@misc{421,
author = {Jesse Heyninck, Matthias Thimm, Gabriele Kern-Isberner, Tjitze Rienstra, Kenneth Skiba},
title = {Arguing about Complex Formulas: Generalizing Abstract Dialectical Frameworks},
abstract = {Abstract dialectical frameworks (in short, ADFs) are a unifying model of formal argumentation, where argumentative relations between arguments are represented by assigning acceptance conditions to atomic arguments. This idea is generalized by letting acceptance conditions being assigned to complex formulas, resulting in conditional abstract dialectical frameworks (in short, cADFs). We define the semantics of cADFs in terms of a non-truth-functional four-valued
logic, and study the semantics in-depth, by showing existence results and proving that all semantics are generalizations of the corresponding semantics for ADFs.},
year = {2021},
url = {https://sites.google.com/view/nmr2021/home?authuser=0},
}
Heyninck J, Arieli O. Approximation Fixpoint Theory for Non-Deterministic Operators and Its Application in Disjunctive Logic Programming. In: 18th International Conference on Principles of Knowledge Representation and Reasoning. Online: IJCAI Organization; 2021. doi:10.24963/kr.2021/32.
Approximation fixpoint theory (AFT) constitutes an abstract and general algebraic framework for studying the semantics of nonmonotonic logics. It provides a unifying study of the semantics of different formalisms for nonmonotonic reasoning, such as logic programming, default logic and autoepistemic logic. In this paper we extend AFT to non-deterministic constructs such as disjunctive information. This is done by generalizing the main constructions and corresponding results to non-deterministic operators, whose ranges are sets of elements rather than single elements. The applicability and usefulness of this generalization is illustrated in the context of disjunctive logic programming.
@{420,
author = {Jesse Heyninck, Ofer Arieli},
title = {Approximation Fixpoint Theory for Non-Deterministic Operators and Its Application in Disjunctive Logic Programming},
abstract = {Approximation fixpoint theory (AFT) constitutes an abstract and general algebraic framework for studying the semantics of nonmonotonic logics. It provides a unifying study of the semantics of different formalisms for nonmonotonic reasoning, such as logic programming, default logic and autoepistemic logic. In this paper we extend AFT to non-deterministic constructs such as disjunctive information. This is done by generalizing the main constructions and corresponding results to non-deterministic operators, whose ranges are sets of elements rather than single elements. The applicability and usefulness of this generalization is illustrated in the context of disjunctive logic programming.},
year = {2021},
journal = {18th International Conference on Principles of Knowledge Representation and Reasoning},
pages = {334-344},
month = {03/11-12/11},
publisher = {IJCAI Organization},
address = {Online},
isbn = {978-1-956792-99-7},
url = {https://proceedings.kr.org/2021/32/},
doi = {10.24963/kr.2021/32},
}
Heyninck J, Kern-Isberner G, Rienstra T, Skiba K, Thimm M. Revision and Conditional Inference for Abstract Dialectical Frameworks. In: 18th International Conference on Principles of Knowledge Representation and Reasoning. Online: IJCAI Organization; 2021. doi:10.24963/kr.2021/33.
For propositional beliefs, there are well-established connections between belief revision, defeasible conditionals and
nonmonotonic inference. In argumentative contexts, such connections have not yet been investigated. On the one hand, the exact relationship between formal argumentation and nonmonotonic inference relations is a research topic that keeps on eluding researchers despite recently intensified efforts, whereas argumentative revision has been studied in numerous works during recent years. In this paper, we show that similar relationships between belief revision, defeasible conditionals and nonmonotonic inference hold in argumentative contexts as well. We first define revision operators for abstract dialectical frameworks, and use such revision operators to define dynamic conditionals by means of the Ramsey test. We show that such conditionals can be equivalently defined using a total preorder over three-valued interpretations, and study the inferential behaviour of the resulting conditional inference relations.
@{418,
author = {Jesse Heyninck, Gabriele Kern-Isberner, Tjitze Rienstra, Kenneth Skiba, Matthias Thimm},
title = {Revision and Conditional Inference for Abstract Dialectical Frameworks},
abstract = {For propositional beliefs, there are well-established connections between belief revision, defeasible conditionals and
nonmonotonic inference. In argumentative contexts, such connections have not yet been investigated. On the one hand, the exact relationship between formal argumentation and nonmonotonic inference relations is a research topic that keeps on eluding researchers despite recently intensified efforts, whereas argumentative revision has been studied in numerous works during recent years. In this paper, we show that similar relationships between belief revision, defeasible conditionals and nonmonotonic inference hold in argumentative contexts as well. We first define revision operators for abstract dialectical frameworks, and use such revision operators to define dynamic conditionals by means of the Ramsey test. We show that such conditionals can be equivalently defined using a total preorder over three-valued interpretations, and study the inferential behaviour of the resulting conditional inference relations.},
year = {2021},
journal = {18th International Conference on Principles of Knowledge Representation and Reasoning},
pages = {345-355},
month = {03/11-12/11},
publisher = {IJCAI Organization},
address = {Online},
isbn = {978-1-956792-99-7},
url = {https://proceedings.kr.org/2021/33/},
doi = {10.24963/kr.2021/33},
}
Research Groups:
More Info
Latest Research Publications:
No Results
Research Groups:
More Info
Latest Research Publications:
Jembere E, Rawatlal R, Pillay A. Matrix Factorisation for Predicting Student Performance. In: 2017 7th World Engineering Education Forum (WEEF). IEEE; 2018.
Predicting student performance in tertiary institutions has potential to improve curriculum advice given to students, the planning of interventions for academic support and monitoring and curriculum design. The student performance prediction problem, as defined in this study, is the prediction of a student’s mark for a module, given the student’s performance in previously attempted modules. The prediction problem is amenable to machine learning techniques, provided that sufficient data is available for analysis. This work reports on a study undertaken at the College of Agriculture, Engineering and Science at University of KwaZulu-Natal that investigates the efficacy of Matrix Factorization as a technique for solving the prediction problem. The study uses Singular Value Decomposition (SVD), a Matrix Factorization technique that has been successfully used in recommender systems. The performance of the technique was benchmarked against the use of student and course average marks as predictors of performance. The results obtained suggests that Matrix Factorization performs better than both benchmarks.
@{194,
author = {Edgar Jembere, Randhir Rawatlal, Anban Pillay},
title = {Matrix Factorisation for Predicting Student Performance},
abstract = {Predicting student performance in tertiary institutions has potential to improve curriculum advice given to students, the planning of interventions for academic support and monitoring and curriculum design. The student performance prediction problem, as defined in this study, is the prediction of a student’s mark for a module, given the student’s performance in previously attempted modules. The prediction problem is amenable to machine learning techniques, provided that sufficient data is available for analysis. This work reports on a study undertaken at the College of Agriculture, Engineering and Science at University of KwaZulu-Natal that investigates the efficacy of Matrix Factorization as a technique for solving the prediction problem. The study uses Singular Value Decomposition (SVD), a Matrix Factorization technique that has been successfully used in recommender systems. The performance of the technique was benchmarked against the use of student and course average marks as predictors of performance. The results obtained suggests that Matrix Factorization performs better than both benchmarks.},
year = {2018},
journal = {2017 7th World Engineering Education Forum (WEEF)},
pages = {513-518},
month = {13/11-16/11},
publisher = {IEEE},
isbn = {978-1-5386-1523-2},
}
Dzitiro J, Jembere E, Pillay A. A DeepQA Based Real-Time Document Recommender System. In: Southern Africa Telecommunication Networks and Applications Conference (SATNAC) 2018. South Africa: SATNAC; 2018.
Recommending relevant documents to users in real- time as they compose their own documents differs from the traditional task of recommending products to users. Variation in the users’ interests as they work on their documents can undermine the effectiveness of classical recommender system techniques that depend heavily on off-line data. This necessitates the use of real-time data gathered as the user is composing a document to determine which documents the user will most likely be interested in. Classical methodologies for evaluating recommender systems are not appropriate for this problem. This paper proposed a methodology for evaluating real-time document recommender system solutions. The proposed method- ology was then used to show that a solution that anticipates a user’s interest and makes only high confidence recommendations performs better than a classical content-based filtering solution. The results obtained using the proposed methodology confirmed that there is a need for a new breed of recommender systems algorithms for real-time document recommender systems that can anticipate the user’s interest and make only high confidence recommendations.
@{189,
author = {Joshua Dzitiro, Edgar Jembere, Anban Pillay},
title = {A DeepQA Based Real-Time Document Recommender System},
abstract = {Recommending relevant documents to users in real- time as they compose their own documents differs from the traditional task of recommending products to users. Variation in the users’ interests as they work on their documents can undermine the effectiveness of classical recommender system techniques that depend heavily on off-line data. This necessitates the use of real-time data gathered as the user is composing a document to determine which documents the user will most likely be interested in. Classical methodologies for evaluating recommender systems are not appropriate for this problem. This paper proposed a methodology for evaluating real-time document recommender system solutions. The proposed method- ology was then used to show that a solution that anticipates a user’s interest and makes only high confidence recommendations performs better than a classical content-based filtering solution. The results obtained using the proposed methodology confirmed that there is a need for a new breed of recommender systems algorithms for real-time document recommender systems that can anticipate the user’s interest and make only high confidence recommendations.},
year = {2018},
journal = {Southern Africa Telecommunication Networks and Applications Conference (SATNAC) 2018},
pages = {304-309},
month = {02/09-05/09},
publisher = {SATNAC},
address = {South Africa},
}
More Info
Latest Research Publications:
No Results
Research Groups:
More Info
Dr Steve Kroon obtained MCom (Computer Science) and PhD (Mathematical
Statistics) degrees at Stellenbosch University. He joined the
Stellenbosch University Computer Science department in 2008. His PhD
thesis considered aspects of statistical learning theory, and his
subsequent research has focused on machine learning and decision
making under uncertainty.
He is a member of the Centre for Artificial Intelligence
Research, the Institute of Electrical and Electronics Engineers, the
International Computer Games Association, the South African
Statistical Association, and the South African Institute for Computer
Scientists and Information Technologists.
Statistics) degrees at Stellenbosch University. He joined the
Stellenbosch University Computer Science department in 2008. His PhD
thesis considered aspects of statistical learning theory, and his
subsequent research has focused on machine learning and decision
making under uncertainty.
He is a member of the Centre for Artificial Intelligence
Research, the Institute of Electrical and Electronics Engineers, the
International Computer Games Association, the South African
Statistical Association, and the South African Institute for Computer
Scientists and Information Technologists.
Latest Research Publications:
Pretorius A, Kroon S, Kamper H. Learning Dynamics of Linear Denoising Autoencoders. In: 35th International Conference on Machine Learning. Sweden: Proceedings of Machine Learning Research (PMLR); 2018.
Denoising autoencoders (DAEs) have proven useful for unsupervised representation learning, but a thorough theoretical understanding is still lacking of how the input noise influences learning. Here we develop theory for how noise influences learning in DAEs. By focusing on linear DAEs, we are able to derive analytic expressions that exactly describe their learning dynamics. We verify our theoretical predictions with simulations as well as experiments on MNIST and CIFAR-10. The theory illustrates how, when tuned correctly, noise allows DAEs to ignore low variance directions in the inputs while learning to reconstruct them. Furthermore, in a comparison of the learning dynamics of DAEs to standard regularised autoencoders, we show that noise has a similar regularisation effect to weight decay, but with faster training dynamics. We also show that our theoretical predictions approximate learning dynamics on real-world data and qualitatively match observed dynamics in nonlinear DAEs.
@{202,
author = {Arnu Pretorius, Steve Kroon, H. Kamper},
title = {Learning Dynamics of Linear Denoising Autoencoders},
abstract = {Denoising autoencoders (DAEs) have proven useful for unsupervised representation learning, but a thorough theoretical understanding is still lacking of how the input noise influences learning. Here we develop theory for how noise influences learning in DAEs. By focusing on linear DAEs, we are able to derive analytic expressions that exactly describe their learning dynamics. We verify our theoretical predictions with simulations as well as experiments on MNIST and CIFAR-10. The theory illustrates how, when tuned correctly, noise allows DAEs to ignore low variance directions in the inputs while learning to reconstruct them. Furthermore, in a comparison of the learning dynamics of DAEs to standard regularised autoencoders, we show that noise has a similar regularisation effect to weight decay, but with faster training dynamics. We also show that our theoretical predictions approximate learning dynamics on real-world data and qualitatively match observed dynamics in nonlinear DAEs.},
year = {2018},
journal = {35th International Conference on Machine Learning},
pages = {4141-4150},
month = {10/07-15/07},
publisher = {Proceedings of Machine Learning Research (PMLR)},
address = {Sweden},
isbn = {1938-7228},
}
Kroon S, Heavens A, Fantaye Y, et al. No evidence for extensions to the standard cosmological model. Physical Review Letters. 2017;119(2017). https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.119.101301.
No Abstract
@article{152,
author = {Steve Kroon, A. Heavens, Y. Fantaye, E. Sellentin, H. Eggers, Z. Hosenie, A. Mootoovaloo},
title = {No evidence for extensions to the standard cosmological model},
abstract = {No Abstract},
year = {2017},
journal = {Physical Review Letters},
volume = {119},
pages = {101301-101305},
issue = {2017},
publisher = {American Physical Society},
url = {https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.119.101301},
}
Kroon S, Yoon M, Bekker J. New reinforcement learning algorithm for robot soccer. Orion. 2017;33(1). http://orion.journals.ac.za/pub/article/view/542.
Reinforcement Learning (RL) is a powerful technique to develop intelligent agents in the field of Artificial Intelligence (AI). This paper proposes a new RL algorithm called the Temporal-Difference value iteration algorithm with state-value functions and presents applications of this algorithm to the decision-making problems challenged in the RoboCup Small Size League (SSL) domain. Six scenarios were defined to develop shooting skills for an SSL soccer robot in various situations using the proposed algorithm. Furthermore, an Artificial Neural Network (ANN) model, namely Multi-Layer Perceptron (MLP) was used as a function approximator in each application. The experimental results showed that the proposed RL algorithm had effectively trained the RL agent to acquire good shooting skills. The RL agent showed good performance under specified experimental conditions.
@article{151,
author = {Steve Kroon, M. Yoon, J. Bekker},
title = {New reinforcement learning algorithm for robot soccer},
abstract = {Reinforcement Learning (RL) is a powerful technique to develop intelligent agents in the field of Artificial Intelligence (AI). This paper proposes a new RL algorithm called the Temporal-Difference value iteration algorithm with state-value functions and presents applications of this algorithm to the decision-making problems challenged in the RoboCup Small Size League (SSL) domain. Six scenarios were defined to develop shooting skills for an SSL soccer robot in various situations using the proposed algorithm. Furthermore, an Artificial Neural Network (ANN) model, namely Multi-Layer Perceptron (MLP) was used as a function approximator in each application. The experimental results showed that the proposed RL algorithm had effectively trained the RL agent to acquire good shooting skills. The RL agent showed good performance under specified experimental conditions.},
year = {2017},
journal = {Orion},
volume = {33},
pages = {1-20},
issue = {1},
publisher = {Operations Research Society of South Africa (ORSSA)},
address = {South Africa},
isbn = {2224-0004 (online)},
url = {http://orion.journals.ac.za/pub/article/view/542},
}
Kroon S, Le Roux PB, Bester W. DSaaS: A cloud Service for Persistent Data Structures. In: CLOSER, 6th International Conference on Cloud Computing and Services Science. Portugal; 2016.
No Abstract
@{143,
author = {Steve Kroon, PB Le Roux, Willem Bester},
title = {DSaaS: A cloud Service for Persistent Data Structures},
abstract = {No Abstract},
year = {2016},
journal = {CLOSER, 6th International Conference on Cloud Computing and Services Science},
pages = {37-48},
month = {23/04-25/04},
address = {Portugal},
isbn = {978-989-758-182-3},
}
Kroon S, Nienaber S, Booysen MJ. A Comparison of Low-Cost Monocular Vision Techniques for Pothole Distance Estimation. In: IEEE Symposium Series on Computational Intelligence: IEEE Symposium on Computational Intelligence in Vehicles and Transportation Systems. ; 2015.
No Abstract
@{129,
author = {Steve Kroon, S. Nienaber, M.J. Booysen},
title = {A Comparison of Low-Cost Monocular Vision Techniques for Pothole Distance Estimation},
abstract = {No Abstract},
year = {2015},
journal = {IEEE Symposium Series on Computational Intelligence: IEEE Symposium on Computational Intelligence in Vehicles and Transportation Systems},
pages = {419-426},
month = {08/12-10/12},
}
Research Groups:
More Info
Latest Research Publications:
No Results
