In this article, we will discuss how to use the “Search for Patterns in Statistics” technology to easily and quickly find and use patterns in massive statistical data.
Nowadays, a lot of statistics have accumulated, and you need to find patterns in them. That’s what this article is about. The technique we’re talking about is called “finding patterns in statistics.” Some aspects of the technology overlap with neural networks, such as preparing data for processing.
You can familiarize yourself with the operation of this technology by downloading the program from the link provided at the end of this article.
The video details how the technology works through a concrete example. The process is demonstrated from start to finish and explained in detail.
As an illustration, the task of finding patterns in 16 years of exchange trading statistics was chosen.
Now let’s talk about the technology itself. The solution to this problem can be divided into three stages.
- Data collection.
- Data analysis and processing, pattern algorithm recognition.
- Apply off-the-shelf algorithms to solve specific problems.
data collection
We have statistics that must be prepared before processing. This technology works in a binary system, i.e. it uses the principles of “yes” or “no”, “0” or “1”. To do this, you set various conditions in your data and determine whether they are met. If the condition is met, the value is 1. As a result, we get a table to work with.
Data analysis and processing, pattern recognition algorithms
We then adjust the parameters and process the table to identify patterns, which is further called an algorithm. This video shows how to analyze each algorithm and understand how it works. Depending on the statistics we set and the quantities we set, we can get thousands of algorithms.
Apply ready-made algorithms to solve specific problems
There are many ways to use off-the-shelf algorithms.
You can use a single algorithm, or create a suite of multiple algorithms and instantly apply all found algorithms.
In the video you will see all the options available for using the algorithm.
There are many ways to use algorithms.
Generate algorithm package
You can build a working algorithm package based on all the algorithms found. As shown in the previous video, such a package will work efficiently for a given probability in the studied statistics.
The video explains in detail how such packets come into being.
Use the complete draft algorithm package
This approach is similar to how neural networks work: the inputs are the parameters of a given object, and at the output we get the true probability of the object.
This method uses nonlinear regression with natural logarithm. After processing the entire algorithm, we get the formula:
Y = A + B * ln(X),
where the coefficients A and B are constant for a given set of algorithms,
X is the number of algorithms running in this object.
By plugging the values A, B, and X into this formula, we can determine the probability of how the object will perform in the statistics.
This approach is based on the following principles:
1. Each object can run a different number of algorithms.
2. The more algorithms running in an object, the higher the chance of getting a positive result.
This video explains in detail how this method works.
Generate data package from block
This approach is based on the following principles.
If you have a large amount of statistics, you can break it into small pieces and search for algorithms in them. You can then test each algorithm on all statistics and select only those that show good results.
This video explains in detail how this method works.
Algorithm analysis
Depending on the settings we set, you can get a number of different algorithms.
The video analyzes how different settings affect the search algorithm and its impact.
Use this link to download the program to evaluate the capabilities of this technology.
https://ru.files.fm/filebrowser#/Search%20for%20patterns