Introduction
In this era, bold proclamations such as "Cancer will be conquered!", "All jobs will be obsolete", "AI will usher in a utopia", and "This technology will be a game-changer", are fervently echoed by intellectuals and laypeople alike. The cause for this fervor? The introduction of a groundbreaking product by the titans of technology was heralded as the pinnacle of engineering.
This scenario played out when ChatGPT, a creation of OpenAI, was made freely available to the public. The launch was a groundbreaking moment, akin to witnessing modern-day sorcery. For some, it was too extraordinary to believe, while others perceived it as a golden opportunity.
However, OpenAI has not simply rested on its laurels following this achievement. On the contrary, it continues to strive relentlessly, innovating and enhancing its creation through advancements in mathematics, ingenious solutions, and cutting-edge engineering.
It is also absolutely harvesting the data which it records while all the customers are using their product be it ChatGPT, API calls, GPT-4, Dall-E 2, and so on. It’s a gain of function where using the customers’ data creates an efficient end-to-end system that directly serves as a gradient of negentropy.
Contents
A Deeper Dive
a. How Can One Encode Seemingly Intangible Data
b. How Can One Encode Seemingly Intangible Data
There are various architectural design choices for a team to parse and convert unstructured useful data to structured data. This structured data is then fed into the state-of-the-art deep learning and clustering algorithms to group users and learn more about what the customers are thinking, what they want, and what they would engage in.
c. How the Gain of Function
"Gain of function" is a term used in various scientific fields, but it is most often associated with virology and genetics. In both fields, it refers to any change in an organism's or a virus's genetic structure that enhances its capabilities or characteristics in some way.
Virology: Gain-of-function research is a type of virological research wherein viruses are genetically manipulated to give them new abilities or enhance existing ones, such as increased transmissibility or pathogenicity. This type of research is done to anticipate and prepare for potential disease outbreaks. However, it's controversial due to the risk of accidental release of these enhanced viruses.
Genetics and Molecular Biology: In genetics, a gain of function mutation is a type of mutation that changes the gene product (the protein that the gene makes) in a way that gives it a new function or increases its original function. This can be contrasted with a loss of function mutation, which impairs or completely eliminates the normal function of a gene.
Let's break this down:
Learning: Deep learning models are a type of machine learning model. Machine learning, in a nutshell, is the process by which a model learns to perform a task by being exposed to examples (data). It's akin to how a child might learn to identify dogs by being shown many pictures of dogs. The child, or in our case the deep learning model, gets better at the task the more examples it sees.
Gain of function: When a deep learning model is trained on a dataset, it learns to identify patterns in that data, which it can then use to make predictions or decisions. This could be considered a "gain of function" because the model is gaining the ability to perform a task it couldn't perform before training. This is similar to a biological gain of function, where an organism acquires a new ability due to a change in its genes.
Improvement: Just as a gain of function in biology might enhance an organism's abilities, training a deep learning model enhances its ability to perform its task. The more high-quality data it is exposed to, the better it can become at making accurate predictions or decisions.
However, it's important to note that this analogy has its limitations. Biological "gain of function" typically refers to a single change (e.g., a mutation) that gives a new ability or enhances an existing one. In contrast, a deep learning model is typically trained on many examples and iteratively adjusts its internal parameters to improve its performance, which is a somewhat different process.
Also, in biology, a gain of function often comes about through random chance (e.g., a random mutation), while in machine learning, the learning process is driven by a systematic algorithm (e.g., gradient descent).
Overall, while the comparison isn't perfect, thinking of deep learning as a form of "gain of function" system can be a useful way to conceptualize how these models improve and adapt to their tasks over time.
How All of This Fits
When all this metadata is parsed and encoded to a defined computable structure it is then fed to a state-of-the-art deep learning system.
The Steps
Tokenization: All encoded metadata is converted to tokens using an in-house custom tokenizer
Embedding: The tokens are then converted to high-dimensional embeddings which are then fed to a transformer architecture
Output Generation: After a successful forward pass and de-tokenization of the outputs we get the generations of the AI models which could be a cluster of similar users.
Hidden States: Hidden states are intermediate and latent high-dimensional computed vectors of the inputs. They are extremely rich which is computed via some alien semantics in the algorithm.
Gain of Function: Now the high-fidelity outputs are an attractor for more consumers and more consumers bring in more prompt-output pairs and this is where the gain is harvested.
Closing Thoughts
It's important to note that this gain-of-function analogy has its limitations. Biological "gain of function" typically refers to a single change (mutation/bug)
that gives a new ability or enhances an existing one. In contrast, a deep learning model is typically trained on many examples and iteratively adjusts its internal parameters to improve its performance, which is a somewhat different process.
Also, in biology, a gain of function often comes about through random chance (a random mutation)
, while in machine learning, the learning process is driven by a systematic algorithm (gradient descent)
.
Overall, while the comparison isn't perfect, thinking of tech giants as a gain of function-driven products is a bit too far right now but I won’t be surprised if it converges to that.