On the applied use of large language models...
On the applied use of large language models...
Straight to the point - is there any positive experience of integration into research projects in the field of economics and finance? No, not a single model is functional, nothing works.
There are two critical and so far unsolvable problems at the architectural level of the GII models themselves.
First, there is no built-in control for verification of output data and correctness of interpretation. In other words, GII is not capable of assessing the correctness and adequacy of the generated content; there is no built-in criterion of truth.
Modern GII models do not have critical thinking and a verifier of results, which in the context of LLM work means: identifying logical connections and contradictions, evaluating arguments and evidence, analyzing data and sources, adapting the output result to the context of conditions.
LLMs available today:
• They do not check the reliability of information sources and do not distinguish reliable data from unreliable ones.
• Are not able to independently identify logical errors or contradictions in their answers.
• Cannot critically evaluate the arguments and evidence presented.
• Cannot adequately adapt their responses to the specific conditions or context of the task.
LLMs are trained on extremely large amounts of data, the initial reliability of which is in doubt, and in this set of information garbage, data compression and weights are determined.
The data on which LLMs were trained initially may contain errors, bias and unreliable information, and therefore training is often based on false information.
In a sense, weights in large language models (LLMs) define the hierarchy of information interpretation, allowing the model to recognize hierarchical and contextual dependencies in the data. In other words, weights determine the degree of connectivity of information blocks, how one piece of information affects another piece of information.
What does this mean in practice? LLMs are extremely ineffective at developing innovative meaning constructs and interpreting initially contradictory information, producing complex multi-level assessments of factors, circumstances and dependencies.
GIIs can be effective in interpreting the generally accepted most popular facts of a regular nature, but not methods for building a hierarchy of priorities and a multi-level composition of risk factors in an ambiguous and unstructured set of data, the distribution vector of which is not predictable.
Consequently, complex analytics of processes and events is not subject to GII, therefore, in GII there is no intelligence in the broad sense. This is a highly erudite system that is quite stupid in understanding the connections and dependencies of complex systems, and sociology, psychology, political science, economics are precisely those areas where there is no strict structuring of data and no unambiguity in interpretation.
You can formalize mathematics or physics (here GII can achieve success in the next 3-5 years), but it is impossible to formalize the motives and actions of society, therefore GII cannot manage business processes, cannot predict and evaluate all those areas where a person is involved (finance , economics, sociology, politics, etc.).
What does this lead to? GIIs generate a huge amount of content, which is almost impossible to apply to applied problems due to lack of reliability.
Ideally, the system should work like a low-level program in a processor, where repeating experiments always gives the same result - there is unambiguity and predictability. The GII has too wide a range of tolerances.
As a result, the time and resources to verify the results of the GII work exceed any potential benefits. Simply put, GIIs are too mesmerizingly fake to be used in serious research and business processes.
Low reliability of the output content is built into the LLM architecture level, so the problem is not correctable either now or in the near future.
The second problem is the lack of learning and the limited length of the context window. This topic requires a separate review.