Australian Institute for Machine Learning publications

Permanent URI for this collection

https://hdl.handle.net/2440/115881

Browse

Browsing Australian Institute for Machine Learning publications by Author "Abbasnejad, E."

Now showing 1 - 7 of 7
  • No Thumbnail Available
    ItemMetadata only
    Adaptive neuro-surrogate-based optimisation method for wave energy converters placement optimisation
    (Springer Nature, 2019) Neshat, M.; Abbasnejad, E.; Shi, Q.; Alexander, B.; Wagner, M.; 26th International Conference on Neural Information Processing (ICONIP) (12 Dec 2019 - 15 Dec 2019 : Sydney, Australia); Gedeon, T.; Wong, K.W.; Lee, M.
    Installed renewable energy capacity has expanded massively in recent years. Wave energy, with its high capacity factors, has great potential to complement established sources of solar and wind energy. This study explores the problem of optimising the layout of advanced, three-tether wave energy converters in a size-constrained farm in a numerically modelled ocean environment. Simulating and computing the complicated hydrodynamic interactions in wave farms can be computationally costly, which limits optimisation methods to using just a few thousand evaluations. For dealing with this expensive optimisation problem, an adaptive neuro-surrogate optimisation (ANSO) method is proposed that consists of a surrogate Recurrent Neural Network (RNN) model trained with a very limited number of observations. This model is coupled with a fast meta-heuristic optimiser for adjusting the model’s hyper-parameters. The trained model is applied using a greedy local search with a backtracking optimisation strategy. For evaluating the performance of the proposed approach, some of the more popular and successful Evolutionary Algorithms (EAs) are compared in four real wave scenarios (Sydney, Perth, Adelaide and Tasmania). Experimental results show that the adaptive neuro model is competitive with other optimisation methods in terms of total harnessed power output and faster in terms of total computational costs.
  • No Thumbnail Available
    ItemMetadata only
    Discriminative clustering of high-dimensional data using generative modeling
    (IEEE, 2019) Abdi, M.; Lim, C.; Mohamed, S.; Nahavandi, S.; Abbasnejad, E.; Van Den Hengel, A.; IEEE International Midwest Symposium on Circuits and Systems (MWSCAS) (5 Aug 2018 - 8 Aug 2018 : Windsor, Canada)
    We approach unsupervised clustering from a generative perspective. We hybridize Variational Autoencoder (VAE) and Generative Adversarial Network (GAN) in a novel way to obtain a vigorous clustering model that can effectively be applied to challenging high-dimensional datasets. The powerful inference of the VAE is used along with a categorical discriminator that aims to obtain a cluster assignment of the data, by maximizing the mutual information between the observations and their predicted class distribution. The discriminator is regularized with examples produced by an adversarial generator, whose task is to trick the discriminator into accepting them as real data. We demonstrate that using a shared latent representation greatly helps with discriminative power of our model and leads to a powerful unsupervised clustering model. The method can be applied to raw data in a high-dimensional space. Training can be performed end-to-end from randomly-initialized weights by alternating stochastic gradient descent on the parameters of the model. Experiments on two datasets including the challenging MNIST dataset show that the proposed method performs better than the existing models. Additionally, our method yields an efficient generative model.
  • No Thumbnail Available
    ItemMetadata only
    ForeSI: Success-Aware Visual Navigation Agent
    (IEEE, 2022) Kazemi Moghaddam, M.; Abbasnejad, E.; Wu, Q.; Qinfeng Shi, J.; Van Den Hengel, A.; IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (4 Jan 2022 - 8 Jan 2022 : Waikoloa, Hawaii)
    In this work, we present a method to improve the efficiency and robustness of the previous model-free Reinforcement Learning (RL) algorithms for the task of object-goal visual navigation. Despite achieving state-of-the-art results, one of the major drawbacks of those approaches is the lack of a forward model that informs the agent about the potential consequences of its actions, i.e., being model-free. In this work, we augment the model-free RL with such a forward model that can predict a representation of a future state, from the beginning of a navigation episode, if the episode were to be successful. Furthermore, in order for efficient training, we develop an algorithm to integrate a replay buffer into the model-free RL that alternates between training the policy and the forward model. We call our agent ForeSI; ForeSI is trained to imagine a future latent state that leads to success. By explicitly imagining such a state, during the navigation, our agent is able to take better actions leading to two main advantages: first, in the absence of an object detector, ForeSI presents a more robust policy, i.e., it leads to about 5% absolute improvement on the Success Rate (SR); second, when combined with an off the-shelf object detector to help better distinguish the target object, our method leads to about 3% absolute improvement on the SR and about 2% absolute improvement on Success weighted by inverse Path Length (SPL), i.e., presents higher efficiency.
  • No Thumbnail Available
    ItemMetadata only
    On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
    (Morgan Kaufmann, 2020) Teney, D.; Kafle, K.; Shrestha, R.; Abbasnejad, E.; Kanan, C.; Hengel, A.V.D.; Conference on Neural Information Processing Systems (NeurIPS) (6 Dec 2020 - 12 Dec 2020 : Virtual, Online); Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.F.; Lin, H.
    Out-of-distribution (OOD) testing is increasingly popular for evaluating a machine learning system's ability to generalize beyond the biases of a training set. OOD benchmarks are designed to present a different joint distribution of data and labels between training and test time. VQA-CP has become the standard OOD benchmark for visual question answering, but we discovered three troubling practices in its current use. First, most published methods rely on explicit knowledge of the construction of the OOD splits. They often rely on inverting'' the distribution of labels, e.g. answering mostlyyes'' when the common training answer was no''. Second, the OOD test set is used for model selection. Third, a model's in-domain performance is assessed after retraining it on in-domain splits (VQA v2) that exhibit a more balanced distribution of labels. These three practices defeat the objective of evaluating generalization, and put into question the value of methods specifically designed for this dataset. We show that embarrassingly-simple methods, including one that generates answers at random, surpass the state of the art on some question types. We provide short- and long-term solutions to avoid these pitfalls and realize the benefits of OOD evaluation.
  • No Thumbnail Available
    ItemMetadata only
    Reinforcement learning with attention that works: a self-supervised approach
    (Springer, 2019) Manchin, A.; Abbasnejad, E.; Van Den Hengel, A.; International Conference on Neural Information Processing (ICONIP) (12 Dec 2019 - 15 Dec 2019 : Sydney, Australia); Gedeon, T.; Wong, K.W.; Lee, M.
    Attention models have had a significant positive impact on deep learning across a range of tasks. However previous attempts at integrating attention with reinforcement learning have failed to produce significant improvements. Unlike the selective attention models used in previous attempts, which constrain the attention via preconceived notions of importance, our implementation utilises the Markovian properties inherent in the state input. We propose the first combination of self attention and reinforcement learning that is capable of producing significant improvements, including new state of the art results in the Arcade Learning Environment.
  • No Thumbnail Available
    ItemOpen Access
    Unshuffling Data for Improved Generalization in Visual Question Answering
    (IEEE, 2021) Teney, D.; Abbasnejad, E.; van den Hengel, A.; IEEE/CVF International Conference on Computer Vision (ICCV) (11 Oct 2021 - 17 Oct 2021 : Virtual Online)
    Generalization beyond the training distribution is a core challenge in machine learning. The common practice of mixing and shuffling examples when training neural networks may not be optimal in this regard. We show that partitioning the data into well-chosen, non-i.i.d. subsets treated as multiple training environments can guide the learning of models with better out-of-distribution generalization. We describe a training procedure to capture the patterns that are stable across environments while discarding spurious ones. The method makes a step beyond correlation-based learning: the choice of the partitioning allows injecting information about the task that cannot be otherwise recovered from the joint distribution of the training data. We demonstrate multiple use cases with the task of visual question answering, which is notorious for dataset biases. We obtain significant improvements on VQA-CP, using environments built from prior knowledge, existing meta data, or unsupervised clustering. We also get improvements on GQA using annotations of “equivalent questions”, and on multidataset training (VQA v2 / Visual Genome) by treating them as distinct environments.
  • No Thumbnail Available
    ItemMetadata only
    Wind turbine power output prediction using a new hybrid neuro-evolutionary method
    (Elsevier, 2021) Neshat, M.; Nezhad, M.M.; Abbasnejad, E.; Mirjalili, S.; Groppi, D.; Heydari, A.; Tjernberg, L.B.; Astiaso Garcia, D.; Alexander, B.; Shi, Q.; Wagner, M.
    Abstract not available