Dual Process Theory

The concept of dual process theory, which gained popularity through the work of Nobel laureate Daniel Kahneman, offers a clear framework for understanding the limitations of contemporary AI. This theory posits two distinct cognitive processing systems within our brains, referred to as System 1 and System 2.

System 1

System 1 encompasses the automatic, subconscious processing of information, akin to instinctive pattern recognition. It allows for quick decisions based on immediate needs, which is both energy-efficient and necessary for survival in urgent situations.

• Automating responses conserves energy, making life more efficient.
• In dangerous scenarios, like encountering a predator, rapid responses are essential.

Think of System 1 as a parallel to human intuition. While this intuition can be powerful, it can also lead to incorrect conclusions when relevant knowledge is lacking. Such misjudgments can mirror the flaws seen in AI systems like GPT, which may present inaccurate approximations as facts.

How can we transcend System 1? Let's delve deeper.

System 2

In unfamiliar situations, or when intuition seems off, organisms can engage System 2, characterized by slower, more deliberate thinking. Learning to drive or mastering a new musical piece requires this step-by-step exploration to identify effective actions.

To discover the optimal actions, a solid world model is crucial. A world model simplifies and abstracts reality, allowing for better planning and prediction of outcomes based on different actions.

System 2 processing is resource-intensive, which is why it cannot be engaged constantly. Once skills are mastered, they can shift into System 1 for automatic execution, such as driving or playing an instrument without conscious thought.

In summary, effective System 2 abilities enhance reasoning and planning.

Current AI systems demonstrate exceptional System 1 capabilities but lack robust System 2 functionalities. Experts generally agree that while AI systems like GPT may harbor some form of abstract representation of the world, these representations are often fragile and not explicit. For AI to plan and reason effectively, it requires well-structured world models built through explicit training.

The Journey Toward ASI

The combination of System 1 and System 2 defines human-like intelligence, often referred to as AGI (artificial general intelligence). However, many experts argue that intelligence tends to be specialized rather than broadly applicable, tailored to the survival needs of an organism. Though AGI has become a widely accepted term, alternatives like AHI (artificial human intelligence) may be more accurate.

What does the future hold?

• Currently, we have ANI (artificial narrow intelligence).
• As we enhance System 2 capabilities, we may reach stages like ARI (artificial rat intelligence), AMI (artificial mouse intelligence), or ACI (artificial cat intelligence).
• Eventually, we may achieve AGI or AHI.
• Beyond this, the goal would be to develop ASI (artificial superintelligence).

What potential avenues exist for enhancing System 2 capabilities essential for advancing beyond ANI? Let's explore some options.

But Isn't It Here Already?

A small faction within the AI community posits that AI might already possess sentience and nearly developed System 2 capabilities. This view is not widely accepted among experts.

Deep Learning as a Dominant Force

This perspective, championed by organizations like OpenAI and Google, suggests that deep learning will be pivotal in achieving AGI. This progress will likely stem from scaling and evolving deep learning architectures, supplemented by techniques such as reinforcement learning and discrete search.

Potential outcomes from this approach include:

• Challenges: Concerns about the robustness of world models and System 2 capabilities in primarily deep learning-based solutions.
• Cost: High energy expenditure compared to human brain functionality.
• Future Directions: Anticipated scaling and architectural improvements in deep learning as planning capabilities evolve.
• Examples: Approaches by OpenAI and Google (GPT, Gemini).

Hybrid and Multimodal Architectures

Hybrid and multimodal systems might integrate various methodologies, maintaining deep learning at their core. These systems would differ from traditional monolithic models, resembling the brain's functional partitions.

For instance, the hippocampus manages memory, while the basal ganglia oversee automatic behaviors. Yann LeCun has proposed a multimodal architecture that includes several modules to facilitate System 1 and System 2 processing.

Some key modules in this architecture include:

• Configurator Module: A master module that connects and adjusts parameters of other modules to align with current goals.
• Perception Module: Abstracts sensory input into a latent representation of the world state.
• World Model: Estimates missing information and predicts future states based on actions. The JEPA architecture, proposed by LeCun, is one such implementation.
• Cost Module: Measures the agent's "energy," indicating how far it is from its goals.
• Memory Module: Stores information about world states and associated costs.
• Actor Module: Generates action sequences to tackle new scenarios, incorporating both policy mapping (System 1) and model-predictive control (System 2).

For further details, refer to his academic paper and additional resources.

Active Inference

Active Inference stands out as a promising alternative to deep learning, grounded in Bayesian mathematics. This approach focuses on real-time belief updates about an agent's environment, striving to minimize "free energy," or surprise.

By balancing belief updates and actions, active inference allows agents to navigate their surroundings intelligently.

Key advantages of active inference include:

• Continual learning without the need for retraining.
• Transparency and explainability of decision-making processes.
• Significantly lower data and energy requirements.
• Capability to implement advanced System 2 functions.
• Transferable learnings between agents.
• A foundation in robust scientific principles.

However, active inference has faced challenges, particularly the "Bayesian Wall," which limited scalability. Recently, Verses has announced advancements in overcoming this barrier, which could revolutionize the approach.

Summary of Active Inference:

• Challenges: Questions remain regarding the scalability of active inference implementations.
• Cost: Low energy consumption compared to deep learning.
• Future Directions: Keep an eye on Verses' forthcoming academic papers on active inference.
• Examples: Verses' implementation.

Mixture of Approaches

The convergence of the various approaches discussed above is likely. Should architectures like JEPA prove effective, they may influence the broader deep learning community, leading to the emergence of new multimodal and hybrid systems.

If Verses' claims about breaking the Bayesian Wall hold true, elements of the deep learning community may start exploring ways to merge these methodologies. The combination of deep learning's System 1 strengths and the transparency of probabilistic methods could pave the way for innovative solutions on the path to AGI.

Summary of Mixture of Approaches:

• Challenges: Technical hurdles related to combining different methodologies.
• Cost: Potentially more efficient than approaches relying solely on deep learning.
• Future Directions: Watch for advancements in the highlighted approaches.

Exploring Additional Opportunities

Beyond the previously discussed strategies, other possibilities exist:

• Some in the AI community believe that advanced technologies like quantum computing may be crucial for achieving true AGI.
• Others argue that embodiment—AI interacting with its environment through sensors and actuators—will be vital for AGI development.
• As noted by professor Kenneth Stanley, the path to AGI may be unpredictable, with surprising new architectures and algorithms emerging from researchers' exploratory work.

In conclusion, this exploration highlights various pathways toward AGI, emphasizing that we are at a pivotal moment in AI development. The future promises to be filled with excitement, challenges, and significant advancements that will require collective efforts to ensure technology benefits humanity.

In Plain English?

Thank you for being a part of the **In Plain English* community! Before you leave:*

• Be sure to clap and follow the writer.
• Connect with us: X | LinkedIn | YouTube | Discord | Newsletter
• Explore our other platforms: Stackademic | CoFeed | Venture | Cubed
• Discover more content at PlainEnglish.io