Can ChatGPT Pass the CPA Exam?

‍

Artificial Intelligence (AI) has been making significant strides in various industries, and the field of accounting is no exception. The emergence of advanced language models like ChatGPT has sparked debates about the future of accounting jobs. Can ChatGPT, with its ability to process vast amounts of information and generate human-like responses, actually pass the Certified Public Accountant (CPA) exam? In this article, we will explore the results of recent experiments and examine the implications for the accounting profession.

The Journey of ChatGPT

ChatGPT is an AI language model developed by OpenAI. It has undergone several iterations, with each version aiming to improve its performance in specific tasks. The journey towards ChatGPT’s success in the CPA exam involved multiple attempts, learning tools, and upgrades.

ChatGPT 3.5: A Failed Attempt

In a previous experiment by Accounting Today, ChatGPT 3.5 was put to the test, and the results were less than impressive. The chatbot scored poorly across all sections of the CPA exam, with its strongest category being business environment and concepts (BEC) at only 48 percent. This failure highlighted the limitations of the earlier version and the need for improvements.

ChatGPT 4.0: A Resounding Success

Researchers were determined to enhance ChatGPT’s performance, leading to the development of ChatGPT 4.0. This upgraded version incorporated better training methods and reasoning capabilities. When subjected to the CPA exam, ChatGPT 4.0 demonstrated remarkable progress, passing all four sections with flying colors.

The scores achieved by ChatGPT 4.0 in the CPA exam were as follows:

• Auditing and attestation (AUD): 91.5 percent
• Business environment and concepts (BEC): 85.7 percent
• Financial accounting and reporting (FAR): 78 percent
• Regulation (REG): 82 percent

These scores indicated a significant improvement over the previous version and raised intriguing questions about the potential of AI in accounting.

Unveiling the Experiment

To understand the factors contributing to ChatGPT’s success in the CPA exam, it is essential to delve into the details of the experiment conducted by researchers.

Zero Shot Scenario: A Test of Pre-existing Knowledge

In the initial “zero shot” scenario, ChatGPT 4.0 was provided with prompts and high-level descriptions of the exam questions. However, it received no explicit training on the specific task at hand. The chatbot relied solely on its pre-existing knowledge and general understanding to generate responses. While the performance was better than the earlier version, ChatGPT 4.0 still fell short of passing the exam, with an average score of 67.8 percent.

The Power of Training: The 10-Shot Scenario

To enhance ChatGPT’s performance, researchers implemented a “10-shot” scenario. In this approach, the AI chatbot was trained using ten sample accounting questions. The purpose of this training was to provide subject matter expertise and help the AI think like an accountant.

The results of the 10-shot scenario were encouraging, with ChatGPT 4.0 achieving an average score of 74.4 percent across all sections. Although it was just 0.6 percent short of passing, this experiment showcased the potential of training in improving the AI’s performance.

Chain of Thought Prompting: The Final Push

To further refine ChatGPT’s capabilities, researchers employed “chain of thought” prompting. This technique involved breaking down complex problems into smaller, manageable steps to arrive at the final answer. It mimicked the process of studying before an exam, equipping the AI with the ability to tackle intricate accounting tasks.

By combining the 10-shot training with chain of thought prompting, ChatGPT 4.0 successfully passed the CPA exam, achieving an average score of 84.3 percent across all four sections. This breakthrough has significant implications for the future of accounting and the interaction between humans and machines.

The Implications for the Accounting Profession

The successful performance of ChatGPT 4.0 in the CPA exam raises thought-provoking questions about the role of AI in the accounting profession. While there is no denying that AI has the potential to automate certain accounting tasks, the human accountant’s competitive advantage is being challenged.

Redefining the Competitive Advantage

Traditionally, human accountants held a competitive advantage over machines due to their expertise, experience, and ability to navigate complex accounting challenges. However, with AI now achieving comparable results to human accountants on real-world accounting tasks, the notion of a competitive advantage is being reconsidered.

The study conducted by researchers explicitly questions the competitive advantage of human accountants relative to AI. For the first time, AI has performed as well as a majority of human accountants on a real-world accounting task. This realization prompts important discussions about the future collaboration between machines and accountants.

Shifting Roles and Responsibilities

The rise of AI in accounting does not necessarily imply the replacement of human accountants. Instead, it signifies a shift in roles and responsibilities. AI can automate routine and repetitive tasks, allowing accountants to focus on higher-level analysis, strategic decision-making, and providing valuable insights to clients.

While AI excels at processing vast amounts of data and generating responses, human accountants possess critical thinking skills, professional judgment, and the ability to interpret complex financial information in a broader context. The synergy between AI and accountants has the potential to transform the profession, enabling accountants to add greater value to their organizations and society.

Debunking Myths About AI and Accounting

Despite the promising results of ChatGPT’s performance in the CPA exam, there are common misconceptions that need to be addressed when it comes to the impact of AI on accounting jobs.

Myth #1: Artificial Intelligence is Actually Intelligent

It is crucial to understand that AI, including ChatGPT, is not sentient or conscious. It lacks true intelligence in the traditional sense and is merely a series of code designed to perform specific tasks. While AI models like ChatGPT can process and generate human-like responses, they rely on pre-existing knowledge and training provided by humans. AI is a tool to augment human capabilities rather than a replacement for human expertise.

Myth #2: AI Will Take Your Job

The fear of AI taking over accounting jobs has been a topic of discussion for years. However, the reality is that AI will not eliminate the need for human accountants but rather transform their roles. AI can automate repetitive and mundane tasks, freeing up accountants’ time to focus on higher-level analysis, complex problem-solving, and client advisory services. Instead of job replacement, AI offers the potential for job enhancement and increased efficiency.

Myth #3: AI Will Eliminate Grunt Work

Grunt work, characterized by repetitive and low-value tasks, is often associated with entry-level accounting positions. While AI can automate certain aspects of grunt work, it does not eliminate it entirely. As AI evolves, new forms of grunt work may arise, and accountants will need to adapt and master these tasks. The dynamic nature of accounting ensures that there will always be a need for human involvement and expertise.

Myth #4: AI Will Transform Accounting, Change the World, and Save Humanity

The hype surrounding AI often leads to exaggerated claims and doomsday scenarios. While AI will undoubtedly bring about changes in the accounting profession, it is unlikely to transform it entirely. Accounting fundamentals, such as debits, credits, financial statements, and audits, will remain integral to the field. AI is a tool that can optimize processes, improve efficiency, and enhance decision-making, but it is not a magic solution that will solve all problems or save humanity.

Embracing the Future of Accounting

As AI continues to advance, it is crucial for accountants to embrace the technology and adapt to the changing landscape. Here are some key steps to prepare for the future of accounting:

Continuous Learning: Stay updated with the latest advancements in AI and accounting technologies. Invest in professional development opportunities to enhance your skills and knowledge.

Collaboration with AI: Embrace the collaboration between humans and machines. Understand the strengths and limitations of AI and leverage its capabilities to augment your own expertise.

Focus on Value-Added Tasks: As routine tasks are automated, shift your focus towards higher-value activities that require critical thinking, analysis, and strategic decision-making. Use AI as a tool to enhance your productivity and efficiency in these areas.

Adaptability and Resilience: Develop a mindset of adaptability and resilience to navigate the evolving accounting landscape. Embrace change, learn new technologies, and be open to exploring innovative ways to add value to your clients and organization.

Conclusion

The successful performance of ChatGPT 4.0 in the CPA exam has ignited discussions about the role of AI in accounting. While AI has the potential to automate certain tasks, it does not eliminate the need for human accountants. Instead, it redefines their roles and responsibilities, allowing them to focus on higher-level analysis and strategic decision-making. By embracing the collaboration between humans and machines, accountants can leverage AI to enhance their productivity, efficiency, and value-added services. The future of accounting lies in the synergy between human expertise and AI capabilities, creating a dynamic and transformative profession.