Laboratory work statistica neural networks. STATISTICA Automated Neural Networks Automated neural networks. STATISTICA Neural Networks in neural network computing

Over a certain history of monitoring patients, an array of data has accumulated and is stored in a table in the STATISTICA system. The corresponding Data Table is shown in Figure 6.

Figure 6. Fragment of the source data table

The purpose of the study is to build a neural network model that, based on a given set of initial data (patient examination data, test results, treatment before admission), based on the treatment prescribed in the hospital, would produce a prognosis for his treatment (values ​​of admission to the hospital I-APFARA, BAB, BKK, diuretics, centrally acting drugs) with sufficient accuracy.

The fact that the problem is nonlinear is beyond doubt. Of course, one could try to solve the problem using the STATISTICA Nonlinear Estimation module, namely, using the iterative procedures offered by this module to “grope” for the type of function. However, there are a number of troubles here that significantly extend the procedure for finding a solution. The most important of them is the formulation of a hypothesis about the explicit form of the dependence being studied, which is not at all obvious.

Without additional research, it is difficult to say anything about the obvious type of dependence. Moreover, it should be mentioned that we did not take into account one more factor. In general, solving such a problem using nonlinear estimation methods can take a very long time, or may not lead to anything. In such critical situations, when it is known that

There is a relationship between the variables;

The relationship is definitely nonlinear;

It’s difficult to say anything about the obvious form of dependence,

Neural network algorithms help out. Let's consider a way to solve this problem in the STATISTICA Neural Networks module.

Unfortunately, there are no universal rules indicating which neural network topology should be followed to solve a particular problem. Therefore, a reasonable procedure for finding the right network is necessary.

The Neural Networks module of the STATISTICA system includes a procedure that organizes the search for the desired network configuration. This procedure consists of building and testing a large number of networks with different architectures and then selecting from among them the network that is best suited for solving the given problem. This tool is called Intelligent Problem Solver. To launch the Neural Networks module, you must use the command of the same name in the main menu of the STATISTICA - Statistics system. (Figure 7)

Figure 7. Launching the Neural Networks module

The following thesis is very common: “neural networks are universal structure, allowing you to implement any algorithm." Let's try, blindly believing this statement, to build a neural network that would "catch" the proposed dependence immediately (meaning, without preliminary exploratory analysis).

One of the most important issues that has not yet been resolved by modern science is the question of the structure of a neural network that would be capable of reproducing the desired multidimensional nonlinear dependence. Indeed, Kolmogorov’s theorem on completeness, which he proved back in 1957, states that a neural network is capable of reproducing any (very important - continuous) function. However, it does not offer the researcher a recipe for creating such a network. In 1988, a number of authors generalized Kolmogorov's theorem and showed that any continuous function can be approximated by a three-layer neural network with one hidden layer and a backpropagation algorithm with any degree of accuracy. Thus, in our case, the positive aspect is the knowledge that the network must be three layers, but again there are no rules at hand that establish the relationship between “any degree of accuracy” and the number of neurons in the intermediate, so-called hidden layer.

Summarizing all of the above, we note that there are no universal rules indicating which neural network topology should be followed to solve a particular problem. Therefore, a reasonable procedure for finding the right network is necessary.

The Neural Networks module of the STATISTICA system includes a unique procedure that organizes the search for the desired network configuration. This tool is called Intelligent Problem Solver. Let's use this tool and search for a neural network that will be capable of solving our problem.

Figure 8. Neural Networks module launch pad

In the Quick tab of this dialog box, in the Problem Type section, it is proposed to select the class of problems that we are faced with. Our goal is to build a multivariate relationship or, in other words, multivariate nonlinear regression. This means that in the Problem Type section you should specify Regression.

Having decided on the class of tasks, it is necessary to specify the variables for carrying out the Analysis. To select variables, use the Variables button. When you click this button, the Select input (independent), output (dependent) and selector variables dialog box appears. In this dialog box, you need to specify two lists of variables. Continuous outputs, in our case, are the variables Admission to the hospital ACEI/ARB, Admission to the hospital BAB, Admission to the hospital BKK, Admission to the hospital of diuretics and admission to the hospital of centrally acting drugs. Continuous inputs, in our example, are from 1 to 61 initial parameters.

Figure 9. Selecting variables for analysis

In the Select analysis section, two options are available: Intelligent Problem Solver and Custom Network Designer. To automatically select neural network parameters, the first option is required, which is set by default. To continue the Analysis, click OK.

In the next step, the Intelligent Problem Solver setup dialog box appears.

The Quick section contains a group of options that are responsible for the execution time of the neural network search algorithm. On this tab, you need to specify the number of networks that need to be tested (to find out whether they are suitable for solving the formulated problem), and also indicate how many of these networks will be included in the final report.

In the Networks tested section we indicate 100, Networks retained - 10 (Figure 10)

The Types tab specifies which types of neural networks will be used in the testing process. To solve the problem of nonlinear regression, a multilayer perceptron is most suitable, so we choose the networks marked in Figure 11.

Figure 10. Setting the number of networks to test

Figure 11. Selecting a network architecture

Next, to organize the search and testing process, it is necessary to indicate the range of changes in the number of neurons in each layer of the neural network. The recommended values ​​for this task are shown in Figure 12.

Figure 12. Indication of the sizes of training, control and test samples

Now, after setting all the training parameters, to start the network search procedure, you must click OK.

The status of the search algorithm is displayed in the IPS Training In Progress dialog box.

While the search algorithm for a suitable neural network is running, this dialog box provides information about the execution time of the algorithm, as well as about the neural networks considered. The purpose of the search algorithm is to enumerate a number of neural network configurations and select the best one in terms of the minimum error at the network output and the maximum of its performance.

Many concepts related to neural network methods are best explained using a specific neural network program as an example. Therefore, in this section there will be many links to the package STATISTICA Neural Networks (abbreviated as ST Neural Networks, a neural network package from StatSoft), which is an implementation of the entire set of neural network methods for data analysis.

In the last few years, we have seen an explosion of interest in neural networks, which are successfully used in a variety of fields - business, medicine, engineering, geology, physics. Neural networks have come into practice wherever forecasting, classification or control problems need to be solved. This impressive success is due to several reasons:

Neural networks are an extremely powerful modeling method that can reproduce extremely complex dependencies. In particular, neural networks are nonlinear in nature (the meaning of this concept is explained in detail later in this chapter). For many years, linear modeling has been the dominant modeling method in most fields because optimization procedures are well developed for it. In problems where linear approximation is unsatisfactory (and there are quite a lot of them), linear models perform poorly. In addition, neural networks cope with the “curse of dimensionality”, which does not allow modeling linear dependencies in the case of a large number of variables

Neural networks learn from examples. The neural network user selects representative data and then runs a learning algorithm that automatically learns the structure of the data. In this case, the user, of course, is required to have some set of heuristic knowledge about how to select and prepare data, select the desired network architecture and interpret the results, but the level of knowledge required for the successful use of neural networks is much more modest than, for example, when using traditional statistical methods.

Neural networks are intuitively attractive because they are based on a primitive biological model of nervous systems. In the future, the development of such neurobiological models may lead to the creation of truly thinking computers. Meanwhile, the “simple” neural networks that the ST Neural Networks system builds are a powerful weapon in the arsenal of a specialist in applied statistics (Neural networks. STATISTICA Neural Networks: Methodology and technologies of modern data analysis.).

Neural networks emerged from artificial intelligence research, namely attempts to replicate the ability of biological nervous systems to learn and correct errors by simulating the low-level structure of the brain (Patterson, 1996). The main area of ​​research in artificial intelligence in the 60s - 80s was expert systems. Such systems were based on high-level modeling of the thinking process (in particular, on the idea that our thinking process is built on the manipulation of symbols). It soon became clear that similar systems, although they may be beneficial in some areas, do not capture some key aspects of human intelligence. One view is that the reason for this is that they are unable to reproduce the structure of the brain. To create artificial intelligence, it is necessary to build a system with a similar architecture.

The brain consists of a very large number (approximately 10,000,000,000) of neurons connected by numerous connections (on average several thousand connections per neuron, but this number can fluctuate greatly). Neurons are special cells capable of transmitting electrochemical signals. A neuron has a branched structure of information input (dendrites), a nucleus and a branching output (axon). The axons of a cell connect to the dendrites of other cells using synapses. When activated, a neuron sends an electrochemical signal along its axon. Through synapses, this signal reaches other neurons, which can in turn be activated. A neuron is activated when the total level of signals arriving at its nucleus from the dendrites exceeds a certain level (activation threshold).

The intensity of the signal received by a neuron (and therefore the possibility of its activation) strongly depends on the activity of the synapses. Each synapse has a length, and special chemicals transmit a signal along it. One of the most respected researchers of neurosystems, Donald Hebb, postulated that learning consists primarily of changes in the “strength” of synaptic connections. For example, in Pavlov's classic experiment, a bell rang each time just before the dog was fed, and the dog quickly learned to associate the ringing of the bell with food. Synaptic connections between the areas of the cerebral cortex responsible for hearing and the salivary glands strengthened, and when the cortex was stimulated by the sound of a bell, the dog began to salivate.

Thus, being built from a very large number of very simple elements (each of which takes a weighted sum of input signals and, if the total input exceeds a certain level, transmits a binary signal), the brain is capable of solving extremely complex problems. Of course, we have not touched upon many complex aspects of the brain structure here, but what is interesting is that artificialneural networks can achieve remarkable results using a model that is not much more complex than the one described above.

Statistica Neural Networks (SNN) package

1. Open data file Series_g from the data available in the package. The file contains a single variable that determines the volume of traffic over several years with monthly data recording. (When you open this file, a number of tables appear related to the intelligent solver option, which at this stage must be closed, leaving only the source data table).
2. Set the variable type “input – output” as follows: select the variable by clicking on the table header, right-click and select an option from the menu Input / Output. The variable name will be highlighted in green.
3. Create new network using the dialog box Create Network. To do this, press successively: File – New – Network. On the monitor screen there is a dialog box (Fig. 1).

Rice. 1. Network creation dialog box

In a time series forecasting problem, the network must know how many copies of one variable it should take and how far ahead it should predict the value of the variable. In this task, accept the parameter Steps (Time window) equal to 12 because the data are monthly observations and the parameter Lookahead– equal to 1.

1. Select Multilayer Perceptron as the network type and set the number of network layers to 3. After that, click the Advice button, as a result of which the program will automatically set the number of neurons in all three layers of the network: 12 – 6 – 1 (Fig. 2).

Rice. 2. Dialog box after setting network parameters

After that press the button Create.

1. When you create the network, SNN will automatically assign the first 12 observations in the data file to type Ignore. During further training and operation of the network in the task of time series analysis, each data block supplied to its input contains data related to several observations. The entire block is assigned to the observation that contains the value of the output variable. As a consequence, the first 12 observations are not actually ignored, but are inputs to the first block of time series data, which corresponds to observation #13. In fact, the program builds a transformed data set in which the number of observations is 12 fewer, but the data in each observation is taken from 13 consecutive lines of the source file.

The created network is shown in Fig. 3.

Rice. 3. Three-layer perceptron

1. In the source data window "Data Set Editor" set 66 training (Training) and 66 controls (Verification) observations (Fig. 4), then press the button for shuffling rows as follows: through the menu Edit – Cases – Shuffle – All (Editing – Cases – Shuffle – All).
2. Train the network using the Levenberg-Marquard method, for which you need to click: Train – Multilayer Perceptron – Levenberg-Marquardt (Train – Multilayer Perceptron – Levenberg-Marquardt). The learning procedure takes a few seconds (depending on the processor type). The Levenberg-Marquard method is one of the reliable and fast learning algorithms, but its use is associated with certain limitations:

Rice. 4. Source data window with separated observations

• this method can only be used for networks with one output element.
• The Levenberg-Marquard method requires memory proportional to the square of the number of weights in the network, so the method is not suitable for networks big size(about 1000 weights).
• The method is applicable only for the root mean square error function.

The Levenberg-Marquard algorithm is designed to minimize the root mean square error function. Near the minimum point, this assumption holds with great accuracy, so the algorithm moves very quickly. Far from the minimum, this assumption may not be correct, so the method finds a compromise between the linear model and gradient descent. A step is taken only if it reduces the error, and where necessary, gradient descent with a sufficiently small step is used to ensure progress.

The Levenberg-Marquard method dialog box is shown in Fig. 5.

Rice. 5. Levenberg-Marquard method dialog box

Main elements of the window:

• Epochs (Number of Epochs)– sets the number of epochs during which the algorithm will run. At each epoch, the entire training set is passed through the network, and then the weights are adjusted.
• Cross-Verification– when the position is marked, the quality of the result produced by the network is checked at each epoch against the control set (if it is specified). When turned off, control observations are ignored, even if they are present in the data file.
• Train– Each time the button is pressed, the algorithm runs through the specified number of epochs.
• Reinitialize– before starting training again, you should press the reset button, since in this case the network weights are randomly set again.
• Jog Weights – When the algorithm might get stuck in a local minimum, this option adds a small amount to each weight.

1. Construct a time series projection using Run – Times Series Projection open the corresponding window (Fig. 6).

Rice. 6. Time series projection window

Description of the dialog box

• Start– specifies whether the time series projection should start at some observation number (Case No) in a data file or from an individual observation.
• Case No – When projecting a time series from a data file, the observation number with the output value from which to start is indicated.
• Length– the number of steps for which the forecast will be projected.
• Variable– indicates the variable that will be designed.

1. Using a trained network, you can perform a time series projection. Initially, the network will work on the first 12 input values, resulting in a prediction of the next value. Then the predicted value, together with the previous 11 input values, is again fed to the input of the network, and the latter produces a forecast of the next value.

The only control parameter that needs to be selected is the projection length. In this example there are a total of 144 observations, 12 of which will be removed during preprocessing, so the results can be compared in at most 132 steps. However, it is possible to project a series beyond the boundaries of the available data, but there will be nothing to compare the result with.

View behavior of predicted values ​​at different lengths using the button Run you can observe changes in the target and output values ​​of the series.

In the given fig. Figure 6 shows that the predicted curve (blue on the monitor screen) was not trained very well, since there are significant deviations between the original and predicted series, starting from approximately 70 observations.

1. Carry out series forecasting using an intelligent solver (third button from the left in the top row). In this case, it is necessary to answer a number of questions in dialogue mode:

• Select the main version (Fig. 7) and click Next.

Rice. 7. Selecting the main version

• Determine the type of task (standard or time series). Here it is necessary to note the time series (Fig. 8).

Rice. 8. Selecting a task type

• Set the observation period to 12 months (Fig. 9).

Rice. 9. Setting the observation period

• Select dependent and independent variables, which are the same variable Series.
• Determine the time of the calculation procedure equal to 2 minutes (Fig. 10).

Rice. 10. Setting the time of the settlement procedure

• Indicate the number of networks to be saved and the actions to take when saving them (Fig. 11).

Rice. 11. Actions for selecting networks

• Select forms for presenting results (Fig. 12) and click Finish.

Rice. 12. Selecting a form for presenting results

As a result of using an intelligent solver, the forecast is much more accurate, since the trained network is much closer to the original series (Fig. 13).

Rice. 13. Forecast using an intelligent solver

Exercise

Build a simulated time series from the Statistica package as follows:

• Create new file, consisting of 20 rows and 2 columns.
• Via the menu Data – Variable Specs enter the expression =vnormal(rnd(1);1;3) into the formula window.
• Simulate 20 values ​​of a random normally distributed variable with a mathematical expectation equal to 1 and a standard deviation equal to 3. These 20 values ​​define the variable Var 1. Convert them to an integer data type, setting it in the variable description window as Type meaning Integer.
• Go to the variable Var 2 as follows: the first value of Var 2 is equal to the first value of the variable Var 1; the second value Var 2 is equal to the sum of the first two values ​​of the variable Var 1; the third value of the variable Var 2 is equal to the sum of the first three values ​​of the variable Var 1, etc.
• Copy the Var 2 variable and go to the SNN package, placing the copied data in the new file created.
• Carry out forecasting of the resulting series using a neural network.

print version

Neural network methods for data analysis are outlined, based on the use of the Statistica Neural Networks package (manufactured by StatSoft), fully adapted for the Russian user. The basics of the theory of neural networks are given; Much attention is paid to solving practical problems, the methodology and technology of conducting research using the Statistica Neural Networks package - a powerful data analysis and forecasting tool that has wide applications in business, industry, management, and finance - is comprehensively reviewed. The book contains many examples of data analysis, practical recommendations for analysis, forecasting, classification, pattern recognition, management production processes using neural networks. For a wide range of readers engaged in research in banking, industry, economics, business, geological exploration, management, transport and other areas. ContentsPreface to the second editionIntroduction. An Invitation to Neural NetworksChapter 1. BASIC CONCEPTS OF DATA ANALYSISChapter 2. INTRODUCTION TO PROBABILITY THEORYChapter 3. INTRODUCTION TO NEURAL NETWORKS THEORYChapter 4. GENERAL OVERVIEW OF NEURAL NETWORKSParallels from biologyBasic artificial modelApplication of neural networksPre- and postprocessing.Multilayer perceptronRadial basis function Probabilistic neural networkGeneralized regression neural networkLinear networkNetwork Kohonen Classification problemsRegression problemsTime series forecastingSelection of variables and dimensionality reductionChapter 5. FIRST STEPS IN STATISTICA NEURAL NETWORKS.Getting startedCreating a data setCreating a new networkCreating a data set and networkTraining a networkRunning a neural networkCarrying out classificationChapter 6. FURTHER CAPABILITIES OF NEURAL NETWORKSClassic example: Fisher's IrisesTraining with cross-validationStopping conditionsSolving regression problemsRadial basis functions Linear models. Kohonen networks Probabilistic and generalized regression networks Network constructor Genetic algorithm for selecting input data Time series Chapter 7. PRACTICAL TIPS FOR SOLVING PROBLEMS Data presentation Isolating useful input variables Reducing dimensionality Selection of network architecture Custom network architectures Time series Chapter 8. CASE STUDIES Example 1. Dimensionality reduction in geological research Example 2 Pattern recognitionExample 3. Nonlinear classification of two-dimensional setsExample 4. Segmentation of various fuel samples according to laboratory researchExample 5. Construction of a behavioral scoring modelExample 6. Function approximationExample 7. Forecasting oil salesExample 8. Monitoring and predicting temperature conditions at an installationExample 9. Determining the reliability of a digital signatureChapter 9 QUICK GUIDEDataNetworksTraining networksOther types of networksWorking with a networkSending results to the STATISTICA systemChapter 10. CLASSICAL METHODS ALTERNATIVE TO NEURAL NETWORKSClassical discriminant analysis in STATISTICAClassificationLogit regressionFactor analysis in STATISTICAChapter 11.MINING DATA IN STATISTICAAppendix 1. Code generatorAppendix 2. Integration of STATISTICA with ERP systemsBibliographyIndex

• Pre- and post-processing, including data selection, nominal coding, scaling, normalization, removal of missing data with interpretation for classification, regression and time series problems;
• Exceptional ease of use plus unrivaled analytical power; for example, has no analogues Solution Wizard will guide the user through all the stages of creating various neural networks and select the best one (this task, otherwise, is solved through a long process of “trial and error” and requires a thorough knowledge of the theory);
• Powerful exploration and analytical technologies, including Principal Component Analysis And Dimensionality Reduction for selecting the necessary input variables in exploratory (neural network) data analysis (selecting the necessary input variables for neural networks often takes a long time; the system STATISTICA Neural Networks can do this work for the user);
• The most advanced, optimized and powerful network training algorithms (including conjugate gradient and Levenberg-Marquard); full control over all parameters affecting network quality, such as activation and error functions, network complexity;
• Support for ensembles of neural networks and neural network architectures of almost unlimited size, created in Network sets - Network Sets; selective training of neural network segments; combining and saving sets of networks in separate files;
• Full integration with the system STATISTICA; all results, graphs, reports, etc. can be further modified using powerful graphical and analytical tools STATISTICA(for example, to analyze predicted residuals, create a detailed report, etc.);
• Seamless integration with powerful automated tools STATISTICA; recording full-fledged macros for any analysis; creating your own neural network analyzes and applications using STATISTICA Visual Basic, challenge STATISTICA Neural Networks from any application that supports COM technology(for example, automatic neural network analysis in a table MS Excel or combining several custom applications written in C++, C#, Java etc.).

STATISTICA Neural Networks in neural network computing:

• The use of neural networks involves much more than just processing data using neural network methods.
• STATISTICA Neural Networks provides a variety of functionality, for working with very complex tasks, including not only the latest Neural Network Architectures And Learning algorithms, but also new approaches in Selection of Input Data And Building a Network. In addition, the developers software and users experimenting with application settings will appreciate the fact that after conducting specified experiments in a simple and intuitive interface STATISTICA Neural Networks,neural network analyzes can be combined in a custom ,application. This is achieved either using the library STATISTICA COM functions, which fully reflects all the functionality of the program, or using code in the language C (C++, C#) or Visual Basic, which is generated by the program and helps run a fully trained neural network or network ensemble.

Initial data

Module STATISTICA Neural Networks fully integrated with the system STATISTICA Thus, a huge selection of tools for editing (preparing) data for analysis (transformations, conditions for selecting observations, data checking tools, etc.) is available. Like all tests STATISTICA, the program can be "attached" to a remote database using in-place processing tools, or linked to live data so that models are trained or run (eg, to calculate predicted values ​​or classify) automatically whenever the data changes.

Input selection and dimensionality reduction

After the data is prepared, you have to decide which variables should be used when working with the neural network. The greater the number of variables, the more complex the neural network will be, and therefore it will require more memory and training time, as well as a larger number of training examples (observations). When there is insufficient data and/or correlations between variables, the issues of selecting meaningful input variables and compressing information into a smaller number of variables become of utmost importance in many neural network applications.

Dimensionality reduction algorithms:

• IN STATISTICA Neural Networks Inverse and direct step-by-step selection algorithms have been implemented. In addition, the neuro-genetic algorithm for selecting input data combines the capabilities of genetic algorithms and PNN/GRNN (PNN - probabilistic neural networks, GRNN - generalized regression neural networks) for automatic search for optimal combinations of input variables, including in cases where there are correlations and nonlinear dependencies between them. Almost instantaneous learning speed PNN/GRNN algorithm not only makes it possible to apply Neuro-Genetic algorithm for input data selection, but also allows you (using the available Editor system data STATISTICA Neural Networks convenient means of suppressing insignificant variables) conduct your own experiments on data sensitivity in real time. STATISTICA Neural Networks also contains a built-in system Principal Component Analysis (PCA, and associative networks for "nonlinear PCA"), which allows you to reduce the dimension of the source data. Note that a huge variety of statistical methods for reducing the dimensionality of data are available in the basic system STATISTICA.

Data scaling and nominal value conversion:

• Before data is entered into the network, it must be prepared in a certain way. It is equally important that the output data can be interpreted correctly. IN STATISTICA Neural Networks there is the possibility of automatic scaling of input and output data (including scaling by minimum/maximum values ​​and by average/standard deviation); Variables with nominal values ​​can also be automatically recoded (for example, Gender=(Male,Female)), including using the 1-of-N coding method. STATISTICA Neural Networks also contains tools for working with missing data. Implemented normalization functions such as "single amount", "winner takes all" And "vector of unit length". There are data preparation and interpretation tools specifically designed for time series analysis. A wide variety of similar tools are also implemented in the basic system STATISTICA.
• In classification problems, it is possible to set confidence intervals that STATISTICA Neural Networks is then used to assign observations to one class or another. In combination with a special implemented in STATISTICA Neural Networks activation function Softmax and cross-entropy error functions, this provides a fundamental probability-theoretic approach to classification problems.

Selecting a neural network model, Network ensembles

The variety of neural network models and the many parameters that need to be set (network size, learning algorithm parameters, etc.) can confuse some users (that’s why there is Solution Wizard, which can automatically search for a suitable network architecture of any complexity).

The STATISTICA Neural Networks system implements all the main types of neural networks used in solving practical problems, including:

• multilayer perceptrons (networks with direct signal transmission);
• networks based on radial basis functions;
• self-organizing Kohonen maps;
• probabilistic (Bayesian) neural networks;
• generalized regression neural networks;
• main component networks;
• networks for clustering;
• linear networks.

Also, in the system STATISTICA Neural Networks implemented Network ensembles, formed from random (but significant) combinations of the above networks. Another handy feature is that you can link networks together so they run sequentially. This is useful in preprocessing to find solutions with minimal cost.

In the package STATISTICA Neural Networks Numerous tools are available to help the user select an appropriate network architecture. The system's statistical and graphical tools include histograms, matrices and error plots for the entire population and for individual observations, final data on correct/incorrect classification, and all important statistics - for example, explained proportion of variance - are calculated automatically.

To visualize data in a package STATISTICA Neural Networks Scatterplots and 3D response surfaces are implemented to help the user understand the “behavior” of the network.
Of course, you can use any information obtained from the listed sources for further analysis by other means. STATISTICA, as well as for subsequent inclusion in reports or for customization.

STATISTICA Neural Networks automatically remembers the best network option from those that you received while experimenting with the task, and you can refer to it at any time. The usefulness of the network and its predictive ability are automatically tested on a special test set of observations, as well as by estimating the size of the network, its efficiency, and the cost of misclassification. Implemented in STATISTICA Neural Networks automatic cross-validation and regularization procedures Wigend scales allow you to quickly find out whether your network is insufficient or, on the contrary, too complex for a given task.

To improve performance in the package STATISTICA Neural Networks Numerous network configuration options are presented. Thus, you can specify a linear output network layer in regression problems or a softmax activation function in probabilistic estimation and classification problems. If your data has a lot of outliers, then when training the network, you can replace the standard error function with a less sensitive function "city blocks". The system also implements cross-entropy error functions based on information theory models and a number of special functions activations, including: step, sawtooth and sine.

Solution Wizard (automatically evaluates the problem and selects several networks of different architectures):

• Part of the package STATISTICA Neural Network s is Solution Wizard - Intelligent Problem Solver, which evaluates many neural networks of various architectures and complexity and selects the networks of the best architecture for a given task.
• Master is able to build networks for data with independent observations (standard regression networks, classification networks, or mixed ones), as well as networks designed to predict future values ​​of a certain variable based on existing values ​​of the same variable (time series networks).
• When creating a neural network, considerable time is spent on selecting appropriate variables and optimizing the network architecture using heuristic search. STATISTICA Neural Networks takes over this work and automatically performs the heuristic search for you. This procedure takes into account the input dimension, network type, network dimensions, and required output encoding functions.
• During the search, you can set the number of responses received during the training process. When setting the maximum detail mode Solution Wizard It will display the architecture and quality levels for each network tested.
• Solution Wizard is an extremely effective tool when using complex techniques, allowing you to automatically find the best network architecture. Instead of spending many hours sitting in front of the computer, let the system STATISTICA Neural Networks do this work for you.
• Automatic network constructor can also be used during model development when the module STATISTICA Neural Networks, together with other modules of the base system STATISTICA, is used to identify the most significant variables (for example, the best predictors for their subsequent inclusion and testing in any model Nonlinear Estimation).

Neural network training:

• The success of your experiments to find the best network type and architecture depends significantly on the quality and speed of network learning algorithms. In system STATISTICA Neural Networks The best training algorithms to date have been implemented.
• To train multilayer perceptrons in the system STATISTICA Neural Networks First of all, the backpropagation method is implemented - with a time-varying learning rate and inertia coefficient, mixing observations before the next step of the algorithm and adding additive noise for robust generalization. In addition, in the system STATISTICA Neural Networks two fast second-order algorithms have been implemented - conjugate gradient methods and Levenberg-Marquard. The latter is an extremely powerful modern nonlinear optimization algorithm, and experts highly recommend using it. At the same time, the scope of application of this method is limited to cases of relatively small-sized networks with one output neuron, and for more cumbersome tasks in the package STATISTICA Neural Networks There is a conjugate gradient method. Typically, both algorithms converge faster than backpropagation and usually produce a better solution.
• Iterative process of network training in the system STATISTICA Neural Networks is accompanied by an automatic display of the current training error and the error calculated independently on the testing set, and a graph of the total error is also shown. You can interrupt training at any time by simply pressing a button. In addition, it is possible to set stopping conditions, under which the training will be interrupted; such a condition may be, for example, the achievement of a certain error level, or a stable increase in the test error over a given number of passes - “epochs” (which indicates the so-called retraining of the network). If overfitting occurs, the user should not care: STATISTICA Neural Networks automatically remembers the instance best network obtained during the training process, and this network option can always be accessed by clicking the corresponding button. After network training is completed, you can check the quality of its work on a separate test set.
• In the package STATISTICA Neural Networks A number of learning algorithms have also been implemented for other networks of other architectures. The parameters of radial splines and smoothing coefficients for networks based on a radial basis function and generalized regression networks can be selected using algorithms such as: Kohonen training, subsample, K-means method, isotropy and nearest neighbor methods. The neurons of the linear output layer of networks based on a radial basis function, like those of linear networks, are fully optimized singular value decomposition (SVD) method.
• Creation of hybrid network structures. In system STATISTICA Neural Networks It is possible to create networks of mixed structure. For example, in a modified network based on a radial basis function, the first layer of neurons can be trained by Kohonen algorithm ah, and the second one - nonlinear layer - Levenberg-Marquard method.

Neural network testing:

• After the network is trained, you need to check the quality of its work and determine its characteristics. For this purpose in the package STATISTICA Neural Networks There is a set of on-screen statistics and graphical tools.
• In the event that several models (networks and ensembles) are specified, then (if possible) STATISTICA Neural Network s will display comparative results (for example, plot response curves of several models in one graph, or present predictors of several models in one table). This property is very useful for comparing different models trained on the same data set.
• All statistics are calculated separately for the training, validation and test sets. All weights and activation parameters are available as a convenient text file, which with one click can be converted into a table of system results STATISTICA. Experimental results for individual observations or for the entire data set can also be viewed in tabular form STATISTICA and use in further analyzes or graphs.
• The following summary statistics are automatically calculated: root mean square error of the network, the so-called discrepancy matrix (confusion matrix) for classification problems (where all cases of correct and incorrect classification are summed up) and the proportion of explained regression for regression problems. Kohonen Network has a window Topological map, in which you can visually observe the activation of network elements, as well as change the labels of observations and nodes in the process of data analysis. There is also a Win Frequency window that allows you to instantly localize clusters in a topological map. Cluster analysis can be performed using a combination of a standard architecture network with a special system cluster diagram STATISTICA Neural Networks. For example, you can train a network for principal component analysis and graph the data as projected onto the first two components.

Editing, modification and serial connection of neural networks

In system STATISTICA Neural Networks There are intelligent tools that allow you to cut off pieces from existing networks and connect multiple networks together. Thus, you can remove or add individual neurons, remove an entire layer from the network, and networks that are consistent in the number of inputs/outputs can be sequentially connected to each other. Thanks to these features, the package STATISTICA Neural Networks allows you to use tools such as dimensionality reduction (during pre-processing) using associative networks and a loss matrix (for making decisions with the least losses). The loss matrix is ​​automatically used when working with probabilistic neural networks.

Ready-made solutions (custom applications using STATISTICA Neural Networks):

• Simple and convenient system interface STATISTICA Neural Networks allows you to quickly create neural network applications to solve your problems.
• There may be a situation where it is necessary to integrate these solutions into an existing system, for example, to make them part of a wider computing environment (these may be procedures developed separately and built into the corporate computing system).
• Trained neural networks can be applied to new datasets (for prediction) in several ways: You can save the trained network or ensemble of networks (for example, to calculate an average prediction based on multiple architectures) and then apply it to a new dataset (for prediction, predicted classification or forecasting); You can use a code generator to automatically generate program code in a language C (C++, C#) or Visual Basic and further use it to predict new data in any software environment visual basic or C++ (C#), i.e. implement a fully trained neural network into your application. In conclusion, all system functionality STATISTICA, including STATISTICA Neural Networks can be used as COM objects (Component Object Model) in other applications (for example, Java, MS Excel etc.). For example, you can implement automated analyzes created in STATISTICA Neural Networks to tables MS Excel.

List of learning algorithms:

• Backpropagation;
• Levenberg-Marquard;
• Conjugate gradients;
• Quasi-Newtonian;
• Rapid spread;
• Delta-delta-with-bar;
• Pseudo-inverse;
• Kohonen Training;
• Marking nearby classes;
• Training vector quantizer;
• Radial (sub)sampling;
• K-means method;
• K-Nearest Neighbors (KNN) method;
• Setting isotropic deviations;
• Setting obvious deviations;
• Probabilistic neural network;
• Generalized regression neural network;
• Genetic algorithm for selecting input data;
• Step-by-step direct or reverse selection of input data.

System requirements

System STATISTICA Neural Networks can work even on relatively weak or old computers. However, since many of the package's procedures are computationally intensive, it is strongly recommended to use Pentium processor with 32 megabytes of RAM.

Network size restrictions:

• A neural network can be of almost any size (that is, its dimensions can be taken many times larger than is actually necessary and reasonable); Up to 128 layers are allowed without restrictions on the number of neurons. In fact, for any practical tasks the program is limited only by the hardware capabilities of the computer.

E-Manual:

• As part of the system STATISTICA Neural Networks there is a well-illustrated textbook that provides a complete and clear introduction to neural networks, as well as examples. A system of detailed, context-sensitive help is available from any dialog box.

Source code generator:

• Generator source code is an additional product that allows users to easily create their own applications based on the system STATISTICA Neural Networks. This add-on product creates the source system code for the neural network model (as a file in C, C++, C#), which can be separately compiled and integrated into your program for free distribution. This product is designed specifically for enterprise systems developers, as well as those users who need to transform highly optimized procedures created in STATISTICA Neural Networks into external applications to solve complex analytical problems.

Neural network methods are becoming increasingly widespread in a variety of fields.

Industry:

• Process management (in particular, monitoring of production processes with continuous regulation of control parameters).
• Classification of fuel samples (segmentation of fuel grades based on analysis of their spectra).
• Technical diagnostics (use vibration and noise to identify faults in the mechanism at an early stage and carry out preventive repairs).
• Engine control systems (assessing and controlling fuel consumption using sensor data).
• Real-time switching detector systems in physics. Neural networks are noise-resistant and allow the use of robust patterns in physical data with large statistical noise.

Marketing:

• Gold price forecasting;
• Forecasting prices for raw materials;
• Trade by direct mail.

Finance:

• Assessment of creditworthiness (the classic task is to determine from personal data whether a given borrower is reliable).
• Forecasting financial time series.

Geological exploration:

• Increasing the efficiency of the mining process (highlighting significant factors affecting mining efficiency indicators).

Other industries:

• Optical character recognition, including signature recognition;
• Image processing;
• Forecasting chaotic time series;
• Medical diagnostics;
• Speech synthesis;
• Linguistic analysis.