|
Allende, H., Salas, R., & Moraga, C. (2003). A robust and effective learning algorithm for feedforward neural networks based on the influence function. Lect. Notes Comput. Sc., 2652, 28–36.
Abstract: The learning process of the Feedforward Artificial Neural Networks relies on the data, though a robustness analysis of the parameter estimates of the model must be done due to the presence of outlying observations in the data. In this paper we seek the robust properties in the parameter estimates in the sense that the influence of aberrant observations or outliers in the estimate is bounded so the neural network is able to model the bulk of data. We also seek a trade off between robustness and efficiency under a Gaussian model. An adaptive learning procedure that seeks both aspects is developed. Finally we show some simulations results applied to the RESEX time series.
|
|
|
Cho, A. D., Carrasco, R. A., & Ruz, G. A. (2022). A RUL Estimation System from Clustered Run-to-Failure Degradation Signals. Sensors, 22(14), 5323.
Abstract: The prognostics and health management disciplines provide an efficient solution to improve a system's durability, taking advantage of its lifespan in functionality before a failure appears. Prognostics are performed to estimate the system or subsystem's remaining useful life (RUL). This estimation can be used as a supply in decision-making within maintenance plans and procedures. This work focuses on prognostics by developing a recurrent neural network and a forecasting method called Prophet to measure the performance quality in RUL estimation. We apply this approach to degradation signals, which do not need to be monotonical. Finally, we test our system using data from new generation telescopes in real-world applications.
|
|
|
de la Cruz, R., Padilla, O., Valle, M. A., & Ruz, G. A. (2021). Modeling Recidivism through Bayesian Regression Models and Deep Neural Networks. Mathematics, 9(6), 639.
Abstract: This study aims to analyze and explore criminal recidivism with different modeling strategies: one based on an explanation of the phenomenon and another based on a prediction task. We compared three common statistical approaches for modeling recidivism: the logistic regression model, the Cox regression model, and the cure rate model. The parameters of these models were estimated from a Bayesian point of view. Additionally, for prediction purposes, we compared the Cox proportional model, a random survival forest, and a deep neural network. To conduct this study, we used a real dataset that corresponds to a cohort of individuals which consisted of men convicted of sexual crimes against women in 1973 in England and Wales. The results show that the logistic regression model tends to give more precise estimations of the probabilities of recidivism both globally and with the subgroups considered, but at the expense of running a model for each moment of the time that is of interest. The cure rate model with a relatively simple distribution, such as Weibull, provides acceptable estimations, and these tend to be better with longer follow-up periods. The Cox regression model can provide the most biased estimations with certain subgroups. The prediction results show the deep neural network's superiority compared to the Cox proportional model and the random survival forest.
|
|
|
Escapil-Inchauspé, P., & Ruz, G. A. (2023). Hyper-parameter tuning of physics-informed neural networks: Application to Helmholtz problems. Neurocomputing, 561, 126826.
Abstract: We consider physics-informed neural networks (PINNs) (Raissiet al., 2019) for forward physical problems. In order to find optimal PINNs configuration, we introduce a hyper-parameter optimization (HPO) procedure via Gaussian processes-based Bayesian optimization. We apply the HPO to Helmholtz equation for bounded domains and conduct a thorough study, focusing on: (i) performance, (ii) the collocation points density r and (iii) the frequency kappa, confirming the applicability and necessity of the method. Numerical experiments are performed in two and three dimensions, including comparison to finite element methods.
|
|
|
Fernandez, C., Valle, C., Saravia, F., & Allende, H. (2012). Behavior analysis of neural network ensemble algorithm on a virtual machine cluster. Neural Comput. Appl., 21(3), 535–542.
Abstract: Ensemble learning has gained considerable attention in different learning tasks including regression, classification, and clustering problems. One of the drawbacks of the ensemble is the high computational cost of training stages. Resampling local negative correlation (RLNC) is a technique that combines two well-known methods to generate ensemble diversity-resampling and error negative correlation-and a fine-grain parallel approach that allows us to achieve a satisfactory balance between accuracy and efficiency. In this paper, we introduce a structure of the virtual machine aimed to test diverse selection strategies of parameters in neural ensemble designs, such as RLNC. We assess the parallel performance of this approach on a virtual machine cluster based on the full virtualization paradigm, using speedup and efficiency as performance metrics, for different numbers of processors and training data sizes.
|
|
|
Goles, E., & Palacios, A. G. (2007). Dynamical complexity in cognitive neural networks. Biol. Res., 40(4), 479–485.
Abstract: In the last twenty years an important effort in brain sciences, especially in cognitive science, has been the development of mathematical tool that can deal with the complexity of extensive recordings corresponding to the neuronal activity obtained from hundreds of neurons. We discuss here along with some historical issues, advantages and limitations of Artificial Neural Networks (ANN) that can help to understand how simple brain circuits work and whether ANN can be helpful to understand brain neural complexity.
|
|
|
Goles, E., & Ruz, G. A. (2015). Dynamics of neural networks over undirected graphs. Neural Netw., 63, 156–169.
Abstract: In this paper we study the dynamical behavior of neural networks such that their interconnections are the incidence matrix of an undirected finite graph G = (V, E) (i.e., the weights belong to {0, 1}). The network may be updated synchronously (every node is updated at the same time), sequentially (nodes are updated one by one in a prescribed order) or in a block-sequential way (a mixture of the previous schemes). We characterize completely the attractors (fixed points or cycles). More precisely, we establish the convergence to fixed points related to a parameter alpha(G), taking into account the number of loops, edges, vertices as well as the minimum number of edges to remove from E in order to obtain a maximum bipartite graph. Roughly, alpha(G') < 0 for any G' subgraph of G implies the convergence to fixed points. Otherwise, cycles appear. Actually, for very simple networks (majority functions updated in a block-sequential scheme such that each block is of minimum cardinality two) we exhibit cycles with nonpolynomial periods. (C) 2014 Elsevier Ltd. All rights reserved.
|
|
|
Henriquez, P. A., & Ruz, G. A. (2018). A non-iterative method for pruning hidden neurons in neural networks with random weights. Appl. Soft. Comput., 70, 1109–1121.
Abstract: Neural networks with random weights have the advantage of fast computational time in both training and testing. However, one of the main challenges of single layer feedforward neural networks is the selection of the optimal number of neurons in the hidden layer, since few/many neurons lead to problems of underfitting/overfitting. Adapting Garson's algorithm, this paper introduces a new efficient and fast non-iterative algorithm for the selection of neurons in the hidden layer for randomization based neural networks. The proposed approach is divided into three steps: (1) train the network with h hidden neurons, (2) apply Garson's algorithm to the matrix of the hidden layer, and (3) perform pruning reducing hidden layer neurons based on the harmonic mean. Our experiments in regression and classification problems confirmed that the combination of the pruning technique with these types of neural networks improved their predictive performance in terms of mean square error and accuracy. Additionally, we tested our proposed pruning method with neural networks trained under sequential learning algorithms, where Random Vector Functional Link obtained, in general, the best predictive performance compared to online sequential versions of extreme learning machines and single hidden layer neural network with random weights. (C) 2018 Elsevier B.V. All rights reserved.
|
|
|
Mora-Rubio, A., Bravo-Ortiz, M. A., Quiñones Arredondo, S., Saborit-Torres, J. M., Ruz, G. A., & Tabares-Soto, R. (2023). Classification of Alzheimers disease stages from magnetic resonance images using deep learning. PeerJ Comput. Sci., 9, e1490.
Abstract: Alzheimers disease (AD) is a progressive type of dementia characterized by loss of memory and other cognitive abilities, including speech. Since AD is a progressive disease, detection in the early stages is essential for the appropriate care of the patient throughout its development, going from asymptomatic to a stage known as mild cognitive impairment (MCI), and then progressing to dementia and severe dementia; is worth mentioning that everyone suffers from cognitive impairment to some degree as we age, but the relevant task here is to identify which people are most likely to develop AD. Along with cognitive tests, evaluation of the brain morphology is the primary tool for AD diagnosis, where atrophy and loss of volume of the frontotemporal lobe are common features in patients who suffer from the disease. Regarding medical imaging techniques, magnetic resonance imaging (MRI) scans are one of the methods used by specialists to assess brain morphology. Recently, with the rise of deep learning (DL) and its successful implementation in medical imaging applications, it is of growing interest in the research community to develop computer-aided diagnosis systems that can help physicians to detect this disease, especially in the early stages where macroscopic changes are not so easily identified. This article presents a DL-based approach to classifying MRI scans in the different stages of AD, using a curated set of images from Alzheimer�s Disease Neuroimaging Initiative and Open Access Series of Imaging Studies databases. Our methodology involves image pre-processing using FreeSurfer, spatial data-augmentation operations, such as rotation, flip, and random zoom during training, and state-of-the-art 3D convolutional neural networks such as EfficientNet, DenseNet, and a custom siamese network, as well as the relatively new approach of vision transformer architecture. With this approach, the best detection percentage among all four architectures was around 89% for AD vs. Control, 80% for Late MCI vs. Control, 66% for MCI vs. Control, and 67% for Early MCI vs. Control.
|
|
|
Ruz, G. A., Zuniga, A., & Goles, E. (2018). A Boolean network model of bacterial quorum-sensing systems. Int. J. Data Min. Bioinform., 21(2), 123–144.
Abstract: There are several mathematical models to represent gene regulatory networks, one of the simplest is the Boolean network paradigm. In this paper, we reconstruct a regulatory network of bacterial quorum-sensing systems, in particular, we consider Paraburkholderia phytofirmans PsJN which is a plant growth promoting bacteria that produces positive effects in horticultural crops like tomato, potato and grape. To learn the regulatory network from temporal expression pattern of quorum-sensing genes at root plants, we present a methodology that considers the training of perceptrons for each gene and then the integration into one Boolean regulatory network. Using the proposed approach, we were able to infer a regulatory network model whose topology and dynamic exhibited was helpful to gain insight on the quorum-sensing systems regulation mechanism. We compared our results with REVEAL and Best-Fit extension algorithm, showing that the proposed neural network approach obtained a more biologically meaningful network and dynamics, demonstrating the effectiveness of the proposed method.
|
|
|
Salas, R., Allende, H., Moreno, S., & Saavedra, C. (2005). Flexible Architecture of Self Organizing Maps for changing environments. Lect. Notes Comput. Sc., 3773, 642–653.
Abstract: Catastrophic Interference is a well known problem of Artificial Neural Networks (ANN) learning algorithms where the ANN forget useful knowledge while learning from new data. Furthermore the structure of most neural models must be chosen in advance. In this paper we introduce a hybrid algorithm called Flexible Architecture of Self Organizing Maps (FASOM) that overcomes the Catastrophic Interference and preserves the topology of Clustered data in changing environments. The model consists in K receptive fields of self organizing maps. Each Receptive Field projects high-dimensional data of the input space onto a neuron position in a low-dimensional output space grid by dynamically adapting its structure to a specific region of the input space. Furthermore the FASOM model automatically finds the number of maps and prototypes needed to successfully adapt to the data. The model has the capability of both growing its structure when novel clusters appears and gradually forgets when the data volume is reduced in its receptive fields. Finally we show the capabilities of our model with experimental results using synthetic sequential data sets and real world data.
|
|