Some of my readers asked me about the inputs that worked best for applying NN to the eur/usd. The answer would be:
--> raw prices OHCL performed poor in general even if lags were introduced
--> technical indicators enhanced the explanatory power of the models ( the best result so far has been 84% trend accuracy)
--> financial ratios, we need to try to use them; a recent article for example included 24 ratios about the Canadian stock market
Thursday, May 25, 2006
Friday, May 19, 2006
NN results...
Someone asked me to post the results... i want to give you an example of the potential profitability of NN if applied correctly... look at my post on moneytec:
link to the results
link to the results
Sunday, May 14, 2006
Reply to Zendini comments
Thanks a lot for you comment Zendini.. i really appreciate the fact that you decided to express your view on the subject...I still believe that NN have huge potential... i have recently read dozens of papers about empirical results starting from 1988... and i'm doing research on s&p500 data with very good results... I have also written a paper for a university project that i can't, unfortunately post on the blog for obvious reasons... so if you have doubts about the subject please refere to the following sources:
J. Loofbourrow & T. Loofbourrow, “Neural Networks for Trading”, AI in Finance (Special Report), 42-51
J. Zirilli, Financial Prediction Using Neural Networks, London, 1997, International Thompson Computer Press
A. Skabar & I. Cloete, Neural Networks, Financial Trading and the Efficient Markets Hypothesis, XXV Australasian Computer Science Conference 2002, Australian Computer Society, Inc.
Shaikh A. Hamid & Z. Iqbal, Using neural networks for forecasting volatility of S&P 500 Index futures prices, Journal of Business Research 57 (2004) 1116-1125.
K. Kohara, T.Ishikawa, Y.Fukuhara, Yukihiro Nakamura, Stock Price Prediction Using Prior Knowledge and Neural Networks, Intelligent Systems in Accounting, Finance and Management vol 6: 11-22, 1997
J. Kamruzzaman & R. Sarker, Comparing ANN Based Models with ARIMA for Prediction of Forex Rates.
G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control Holden-Day, San Francisco, CA
G. Zhang and M.Y. Hu, Neural Network Forecasting of the British Pound/USD Dollar Exchange Rate, “OMEGA; Int. Journal of Management Science, 26, pp 495-506, 1998
G. Deboeck, Trading on the Edge: Neural, Genetic and Fuzzy Systems for Chaotic Financial Markets
An-Sing Chen, Hazem Daouk, Mark T. Leung, Application of Neural Networks to an Emerging Financial Market: Forecasting and Trading the Taiwan Stock Index 2001
Wei Cheng, Lorry Wagner, Chien-Hua Lin, Forecasting the 30-year U.S. Treasury Bond with a System of Neural Networks,NeuroVe$t Journal, January/February 1996
S. Dutta and S. Sheckhar, Bond Rating: A Non-Conservative Application of Neural Networks, Proceedings of the IEEE International Conference on Neural Networks,1998, 2, 443-450
J. Moody and J. Utans, Architecture Selection Strategies for Neural Networks, In. A. P. Refenes, Ed., Neural Networks in the Capital Markets, Chichester, England, 1995, John Wiley & Sons, 277-300.
T. Lubecke, Kyung Doo Nam, R. Markland, C. Kwok, Combining Foreign Exchange Forecasts using Neural Networks, 1998, Global Finance Journal 9(1): 5-27
A. Fadlalla & Chien-Hua Lin, An Analysis of the Applications of Neural Networks in Finance, 2001, Interfaces 31: 4 July-August 2001 (pp. 112-122), Informs.
Have a nice day!
J. Loofbourrow & T. Loofbourrow, “Neural Networks for Trading”, AI in Finance (Special Report), 42-51
J. Zirilli, Financial Prediction Using Neural Networks, London, 1997, International Thompson Computer Press
A. Skabar & I. Cloete, Neural Networks, Financial Trading and the Efficient Markets Hypothesis, XXV Australasian Computer Science Conference 2002, Australian Computer Society, Inc.
Shaikh A. Hamid & Z. Iqbal, Using neural networks for forecasting volatility of S&P 500 Index futures prices, Journal of Business Research 57 (2004) 1116-1125.
K. Kohara, T.Ishikawa, Y.Fukuhara, Yukihiro Nakamura, Stock Price Prediction Using Prior Knowledge and Neural Networks, Intelligent Systems in Accounting, Finance and Management vol 6: 11-22, 1997
J. Kamruzzaman & R. Sarker, Comparing ANN Based Models with ARIMA for Prediction of Forex Rates.
G. E. P. Box and G. M. Jenkins, Time Series Analysis: Forecasting and Control Holden-Day, San Francisco, CA
G. Zhang and M.Y. Hu, Neural Network Forecasting of the British Pound/USD Dollar Exchange Rate, “OMEGA; Int. Journal of Management Science, 26, pp 495-506, 1998
G. Deboeck, Trading on the Edge: Neural, Genetic and Fuzzy Systems for Chaotic Financial Markets
An-Sing Chen, Hazem Daouk, Mark T. Leung, Application of Neural Networks to an Emerging Financial Market: Forecasting and Trading the Taiwan Stock Index 2001
Wei Cheng, Lorry Wagner, Chien-Hua Lin, Forecasting the 30-year U.S. Treasury Bond with a System of Neural Networks,NeuroVe$t Journal, January/February 1996
S. Dutta and S. Sheckhar, Bond Rating: A Non-Conservative Application of Neural Networks, Proceedings of the IEEE International Conference on Neural Networks,1998, 2, 443-450
J. Moody and J. Utans, Architecture Selection Strategies for Neural Networks, In. A. P. Refenes, Ed., Neural Networks in the Capital Markets, Chichester, England, 1995, John Wiley & Sons, 277-300.
T. Lubecke, Kyung Doo Nam, R. Markland, C. Kwok, Combining Foreign Exchange Forecasts using Neural Networks, 1998, Global Finance Journal 9(1): 5-27
A. Fadlalla & Chien-Hua Lin, An Analysis of the Applications of Neural Networks in Finance, 2001, Interfaces 31: 4 July-August 2001 (pp. 112-122), Informs.
Have a nice day!
Tuesday, May 09, 2006
Discussion board...
I'm thinking of creating a discussion board about NN and their application to financial markets... even though i got great feedback from readers of this blog i think we could try to discuss the issue together more often... could you please comment on this..
take care...
take care...
Monday, May 08, 2006
Limits and application of NN
"In principle, NNs can compute any computable function, i.e., they can do everything a normal digital computer can do (Valiant, 1988; Siegelmann and Sontag, 1999; Orponen, 2000; Sima and Orponen, 2001), or perhaps even more, under some assumptions of doubtful practicality (see Siegelmann, 1998, but also Hadley, 1999).
Practical applications of NNs most often employ supervised learning. For supervised learning, you must provide training data that includes both the input and the desired result (the target value). After successful training, you can present input data alone to the NN (that is, input data without the desired result), and the NN will compute an output value that approximates the desired result. However, for training to be successful, you may need lots of training data and lots of computer time to do the training. In many applications, such as image and text processing, you will have to do a lot of work to select appropriate input data and to code the data as numeric values.
In practice, NNs are especially useful for classification and function approximation/mapping problems which are tolerant of some imprecision, which have lots of training data available, but to which hard and fast rules (such as those that might be used in an expert system) cannot easily be applied. Almost any finite-dimensional vector function on a compact set can be approximated to arbitrary precision by feedforward NNs (which are the type most often used in practical applications) if you have enough data and enough computing resources.
To be somewhat more precise, feedforward networks with a single hidden layer and trained by least-squares are statistically consistent estimators of arbitrary square-integrable regression functions under certain practically-satisfiable assumptions regarding sampling, target noise, number of hidden units, size of weights, and form of hidden-unit activation function (White, 1990). Such networks can also be trained as statistically consistent estimators of derivatives of regression functions (White and Gallant, 1992) and quantiles of the conditional noise distribution (White, 1992a). Feedforward networks with a single hidden layer using threshold or sigmoid activation functions are universally consistent estimators of binary classifications (Faragó and Lugosi, 1993; Lugosi and Zeger 1995; Devroye, Györfi, and Lugosi, 1996) under similar assumptions. Note that these results are stronger than the universal approximation theorems that merely show the existence of weights for arbitrarily accurate approximations, without demonstrating that such weights can be obtained by learning.
Unfortunately, the above consistency results depend on one impractical assumption: that the networks are trained by an error (L_p error or misclassification rate) minimization technique that comes arbitrarily close to the global minimum. Such minimization is computationally intractable except in small or simple problems (Blum and Rivest, 1989; Judd, 1990). In practice, however, you can usually get good results without doing a full-blown global optimization; e.g., using multiple (say, 10 to 1000) random weight initializations is usually sufficient.
One example of a function that a typical neural net cannot learn is Y=1/X on the open interval (0,1). An open interval is not a compact set. With any bounded output activation function, the error will get arbitrarily large as the input approaches zero. Of course, you could make the output activation function a reciprocal function and easily get a perfect fit, but neural networks are most often used in situations where you do not have enough prior knowledge to set the activation function in such a clever way. There are also many other important problems that are so difficult that a neural network will be unable to learn them without memorizing the entire training set, such as:
* Predicting random or pseudo-random numbers.
* Factoring large integers.
* Determing whether a large integer is prime or composite.
* Decrypting anything encrypted by a good algorithm.
And it is important to understand that there are no methods for training NNs that can magically create information that is not contained in the training data.
Feedforward NNs are restricted to finite-dimensional input and output spaces. Recurrent NNs can in theory process arbitrarily long strings of numbers or symbols. But training recurrent NNs has posed much more serious practical difficulties than training feedforward networks. NNs are, at least today, difficult to apply successfully to problems that concern manipulation of symbols and rules, but much research is being done.
There have been attempts to pack recursive structures into finite-dimensional real vectors (Blair, 1997; Pollack, 1990; Chalmers, 1990; Chrisman, 1991; Plate, 1994; Hammerton, 1998). Obviously, finite precision limits how far the recursion can go (Hadley, 1999). The practicality of such methods is open to debate.
As for simulating human consciousness and emotion, that's still in the realm of science fiction. Consciousness is still one of the world's great mysteries. Artificial NNs may be useful for modeling some aspects of or prerequisites for consciousness, such as perception and cognition, but ANNs provide no insight so far into what Chalmers (1996, p. xi) calls the "hard problem":
Many books and articles on consciousness have appeared in the past few years, and one might think we are making progress. But on a closer look, most of this work leaves the hardest problems about consciousness untouched. Often, such work addresses what might be called the "easy problems" of consciousness: How does the brain process environmental stimulation? How does it integrate information? How do we produce reports on internal states? These are important questions, but to answer them is not to solve the hard problem: Why is all this processing accompanied by an experienced inner life?"
(source)
Practical applications of NNs most often employ supervised learning. For supervised learning, you must provide training data that includes both the input and the desired result (the target value). After successful training, you can present input data alone to the NN (that is, input data without the desired result), and the NN will compute an output value that approximates the desired result. However, for training to be successful, you may need lots of training data and lots of computer time to do the training. In many applications, such as image and text processing, you will have to do a lot of work to select appropriate input data and to code the data as numeric values.
In practice, NNs are especially useful for classification and function approximation/mapping problems which are tolerant of some imprecision, which have lots of training data available, but to which hard and fast rules (such as those that might be used in an expert system) cannot easily be applied. Almost any finite-dimensional vector function on a compact set can be approximated to arbitrary precision by feedforward NNs (which are the type most often used in practical applications) if you have enough data and enough computing resources.
To be somewhat more precise, feedforward networks with a single hidden layer and trained by least-squares are statistically consistent estimators of arbitrary square-integrable regression functions under certain practically-satisfiable assumptions regarding sampling, target noise, number of hidden units, size of weights, and form of hidden-unit activation function (White, 1990). Such networks can also be trained as statistically consistent estimators of derivatives of regression functions (White and Gallant, 1992) and quantiles of the conditional noise distribution (White, 1992a). Feedforward networks with a single hidden layer using threshold or sigmoid activation functions are universally consistent estimators of binary classifications (Faragó and Lugosi, 1993; Lugosi and Zeger 1995; Devroye, Györfi, and Lugosi, 1996) under similar assumptions. Note that these results are stronger than the universal approximation theorems that merely show the existence of weights for arbitrarily accurate approximations, without demonstrating that such weights can be obtained by learning.
Unfortunately, the above consistency results depend on one impractical assumption: that the networks are trained by an error (L_p error or misclassification rate) minimization technique that comes arbitrarily close to the global minimum. Such minimization is computationally intractable except in small or simple problems (Blum and Rivest, 1989; Judd, 1990). In practice, however, you can usually get good results without doing a full-blown global optimization; e.g., using multiple (say, 10 to 1000) random weight initializations is usually sufficient.
One example of a function that a typical neural net cannot learn is Y=1/X on the open interval (0,1). An open interval is not a compact set. With any bounded output activation function, the error will get arbitrarily large as the input approaches zero. Of course, you could make the output activation function a reciprocal function and easily get a perfect fit, but neural networks are most often used in situations where you do not have enough prior knowledge to set the activation function in such a clever way. There are also many other important problems that are so difficult that a neural network will be unable to learn them without memorizing the entire training set, such as:
* Predicting random or pseudo-random numbers.
* Factoring large integers.
* Determing whether a large integer is prime or composite.
* Decrypting anything encrypted by a good algorithm.
And it is important to understand that there are no methods for training NNs that can magically create information that is not contained in the training data.
Feedforward NNs are restricted to finite-dimensional input and output spaces. Recurrent NNs can in theory process arbitrarily long strings of numbers or symbols. But training recurrent NNs has posed much more serious practical difficulties than training feedforward networks. NNs are, at least today, difficult to apply successfully to problems that concern manipulation of symbols and rules, but much research is being done.
There have been attempts to pack recursive structures into finite-dimensional real vectors (Blair, 1997; Pollack, 1990; Chalmers, 1990; Chrisman, 1991; Plate, 1994; Hammerton, 1998). Obviously, finite precision limits how far the recursion can go (Hadley, 1999). The practicality of such methods is open to debate.
As for simulating human consciousness and emotion, that's still in the realm of science fiction. Consciousness is still one of the world's great mysteries. Artificial NNs may be useful for modeling some aspects of or prerequisites for consciousness, such as perception and cognition, but ANNs provide no insight so far into what Chalmers (1996, p. xi) calls the "hard problem":
Many books and articles on consciousness have appeared in the past few years, and one might think we are making progress. But on a closer look, most of this work leaves the hardest problems about consciousness untouched. Often, such work addresses what might be called the "easy problems" of consciousness: How does the brain process environmental stimulation? How does it integrate information? How do we produce reports on internal states? These are important questions, but to answer them is not to solve the hard problem: Why is all this processing accompanied by an experienced inner life?"
(source)
Monday, May 01, 2006
Book about NN by Smith
Today i will start reading a new book by Murray Smith called "NN for statistical modeling". The book is about backpropagation, which is supposed to be the most widely used and studied method in the NN community.The book was published in 1996 and seems to be a rather understandable one combining theoretical and practical issues. I will keep you informed about it.
Sunday, April 30, 2006
A second application of NN
The HR-2 robot was constructed during a period of three months at Chalmers University in Sweden. It has 22 degrees of freedom which enables it to easily move around imitating human motions. The robot is also equipped with stereovision giving it possibilities to perform hand-eye coordination. For that task an artificial neural network is evolved. Furthermore, the artificial brain is capable of tracking faces as well as recognising them. The HR-2 is also able to speak.
A non financial application of NN
This simulated car is controlled by a neural network that has been trained by evolutionary algorithms (aka genetic algorithms). Its input are several rangefinder sensors, and outputs are speed and steering commands. No human knowledge has gone into designing the driving behaviour.
John Kenneth Galbraith died
"John Kenneth Galbraith, the iconoclastic economist, teacher and diplomat and an unapologetically liberal member of the political and academic establishment that he needled in prolific writings for more than half a century, died yesterday at a hospital in Cambridge, Mass. He was 97.
Galbraith lived in Cambridge and at an "unfarmed farm" near Newfane, Vt. His death was confirmed by his son J. Alan Galbraith.
Galbraith was one of the most widely read authors in the history of economics; among his 33 books was "The Affluent Society" (1958), one of those rare works that forces a nation to re-examine its values. He wrote fluidly, even on complex topics, and many of his compelling phrases - among them "the affluent society," "conventional wisdom" and "countervailing power" - became part of the language. An imposing presence, lanky and angular at 6 feet 8 inches tall, Galbraith was consulted frequently by national leaders, and he gave advice freely, though it may have been ignored as often as it was taken. Galbraith clearly preferred taking issue with the conventional wisdom he distrusted".
(source)
Galbraith lived in Cambridge and at an "unfarmed farm" near Newfane, Vt. His death was confirmed by his son J. Alan Galbraith.
Galbraith was one of the most widely read authors in the history of economics; among his 33 books was "The Affluent Society" (1958), one of those rare works that forces a nation to re-examine its values. He wrote fluidly, even on complex topics, and many of his compelling phrases - among them "the affluent society," "conventional wisdom" and "countervailing power" - became part of the language. An imposing presence, lanky and angular at 6 feet 8 inches tall, Galbraith was consulted frequently by national leaders, and he gave advice freely, though it may have been ignored as often as it was taken. Galbraith clearly preferred taking issue with the conventional wisdom he distrusted".
(source)
Saturday, April 29, 2006
Help us to promote the site
If you like this blog please add it to:
http://del.icio.us/
or link it from other sites...
thanks a lot
http://del.icio.us/
or link it from other sites...
thanks a lot
Wednesday, April 26, 2006
ANN and ARIMA article summary
I would like to draw your attention to a paper published on the ASOR Bulletin in 2003. It's titled "comparing ANN Based Models with ARIMA for Prediction of Forex Rates". It's an interesting paper written in plain language. Let me give you a summary of the main ideas:
- the ANN models used (including SB, SCG and BR)all outperformed the ARIMA models (Box-Jenkins);
- the analysis was about the forex market, six currency pairs in total;
- "ANN proved to be very effective in describing the dynamics of non-stationary time series due to its unique non-parametric, non-assumable, noise-tolerant, and adaptive approximators that can map any nonlinear function without a priori assumptions about the data";
- details about ARIMA are available in Jarret "Business Forecasting Methods";
- the idea of NN is explained;
- multilayer feedforward network (MFN) is described briefly;
- no study has been reported to analytically determine the generalization performance of each algorithm;
- the author uses MA5, MA10, MA20, MA60, MA120 (Moving averages of given lengths);
- the NN used had 6 inputs, 1 layer and one output;
- the statistical metrics used to evaluate the performance are: Mean Square Error, Mean Absolute Error, Directional Symmetry;
- neural networks models produce much better performance than the conventional ARIMA models for both shorter and longer term forecasting;
- the ANN models used (including SB, SCG and BR)all outperformed the ARIMA models (Box-Jenkins);
- the analysis was about the forex market, six currency pairs in total;
- "ANN proved to be very effective in describing the dynamics of non-stationary time series due to its unique non-parametric, non-assumable, noise-tolerant, and adaptive approximators that can map any nonlinear function without a priori assumptions about the data";
- details about ARIMA are available in Jarret "Business Forecasting Methods";
- the idea of NN is explained;
- multilayer feedforward network (MFN) is described briefly;
- no study has been reported to analytically determine the generalization performance of each algorithm;
- the author uses MA5, MA10, MA20, MA60, MA120 (Moving averages of given lengths);
- the NN used had 6 inputs, 1 layer and one output;
- the statistical metrics used to evaluate the performance are: Mean Square Error, Mean Absolute Error, Directional Symmetry;
- neural networks models produce much better performance than the conventional ARIMA models for both shorter and longer term forecasting;
Sunday, April 23, 2006
Saturday, April 22, 2006
Learning paradigms
we can distinguish three main learning paradigms:
- supervised learning
- nonsupervised learning
- reinforcement learning
- supervised learning
- nonsupervised learning
- reinforcement learning
Artificial intelligence vs. cognitive modelling
"Artificial intelligence and cognitive modeling try to simulate some properties of neural networks. While similar in their techniques, the former has the aim of solving particular tasks, while the latter aims to build mathematical models of biological neural systems.
In the artificial intelligence field, artificial neural networks have been applied successfully to speech recognition, image analysis and adaptive control, in order to construct software agents (in computer and video games) or autonomous robots. Most of the currently employed artificial neural networks for artificial intelligence are based on statistical estimation, optimisation and control theory.
The cognitive modelling field is the physical or mathematical modelling of the behaviour of neural systems; ranging from the individual neural level (e.g. modelling the spike response curves of neurons to a stimulus), through the neural cluster level (e.g. modelling the release and effects of dopamine in the basal ganglia) to the complete organism (e.g. behavioural modelling of the organism's response to stimuli)".
(source)
In the artificial intelligence field, artificial neural networks have been applied successfully to speech recognition, image analysis and adaptive control, in order to construct software agents (in computer and video games) or autonomous robots. Most of the currently employed artificial neural networks for artificial intelligence are based on statistical estimation, optimisation and control theory.
The cognitive modelling field is the physical or mathematical modelling of the behaviour of neural systems; ranging from the individual neural level (e.g. modelling the spike response curves of neurons to a stimulus), through the neural cluster level (e.g. modelling the release and effects of dopamine in the basal ganglia) to the complete organism (e.g. behavioural modelling of the organism's response to stimuli)".
(source)
NN definition
Let's start by some basic issues... let's define neural networks:
" A neural network is an interconnected group of biological neurons. In modern usage the term can also refer to artificial neural networks, which are constituted of artificial neurons. Thus the term 'Neural Network' specifies two distinct concepts:
1. A biological neural network is a plexus of connected or functionally related neurons in the peripheral nervous system or the central nervous system. In the field of neuroscience, it most often refers to a group of neurons from a nervous system that are suited for laboratory analysis.
2. Artificial neural networks were designed to model some properties of biological neural networks, though most of the applications are of technical nature as opposed to cognitive models."
(source: http://en.wikipedia.org/wiki/Neural_networks ).
" A neural network is an interconnected group of biological neurons. In modern usage the term can also refer to artificial neural networks, which are constituted of artificial neurons. Thus the term 'Neural Network' specifies two distinct concepts:
1. A biological neural network is a plexus of connected or functionally related neurons in the peripheral nervous system or the central nervous system. In the field of neuroscience, it most often refers to a group of neurons from a nervous system that are suited for laboratory analysis.
2. Artificial neural networks were designed to model some properties of biological neural networks, though most of the applications are of technical nature as opposed to cognitive models."
(source: http://en.wikipedia.org/wiki/Neural_networks ).
Thursday, April 20, 2006
Introduction to the blog
Welcome!
This is a blog that will allow me to introduce the idea of NN and their application for financial markets... this mix will represent a mix of theory, empirical findings, forecasts and opionions... it will cover the topic from a newbie to a professional level... you are welcome to add comments and write anything you want on the topic...
This is a blog that will allow me to introduce the idea of NN and their application for financial markets... this mix will represent a mix of theory, empirical findings, forecasts and opionions... it will cover the topic from a newbie to a professional level... you are welcome to add comments and write anything you want on the topic...
Subscribe to:
Posts (Atom)