For certain technology leaders at large accounting firms, ChatGPT holds much promise for the profession in the future, but it won't be fully realized until some major issues are addressed in the program. Until then, the infamous talking bot is being used largely for support work and kept far away from client engagements.
To start, there is the security issue, according to Peter Scavuzzo, chief information officer for Top 25 Firm Marcum as well as the CEO of Marcum Technology. In general, anything entered into ChatGPT is stored on the servers of OpenAI, the company that created it. This data is then used to further train the model for future improvements (ChatGPT 4.0, for instance, was trained partially off people's interactions with Version 3.5). This means that entering any client information into the AI could represent a serious breach of confidentiality.
"It collects your data and feeds it back into its data set, which prohibits your ability to do anything with real client data. That is a risk we've theorized. 'Let's take a trial balance and throw it into ChatGPT and ask it what you can identify there.' We're not going to do that because we're not going to put client data into the ChatGPT engine. That can be dangerous," he said.
For another, there is the fact that, as authoritative as ChatGPT can sound, it is prone to what some call "hallucinations," which basically means the AI makes things up out of whole cloth. While subject matter experts are usually able to detect this, those less familiar with a topic may have more difficulty. This is why Marcum is not deploying it on client engagements like audits. "It's concerning … When you think of practical uses, immediately you have to excuse it completely from the audit side. It absolutely cannot substantiate audit analysis. ChatGPT will never do that at this point. So we've in essence dismissed it from there," Scavuzzo said.
The long-term potential
James Bourke, managing director of the advisory practice at Top 100 Firm Withum, echoed many of Scavuzzo's concerns regarding ChatGPT's security and tendency to make stuff up. Similarly, this is why no one at his own firm is using the program for serious client work.
This does not diminish his belief that the technology can have serious potential for the accounting profession — not today but in the future, when these issues have been sufficiently mitigated. For instance, he said he recently witnessed someone feeding the entire Internal Revenue Code into ChatGPT 4.0 and then begin asking specific questions about tax law. He noted that, as a human, he certainly cannot read the entire Tax Code and analyze it within just a few seconds, but that's exactly what ChatGPT did. He conceded it sometimes would draw the wrong conclusion, but no more often than a human staffer might.
"To me, it will probably be as good as a staff accountant reading the code," he said. "If I gave that staff accountant [the IRC] and I [had ChatGPT with the IRC] as its knowledge base, I can guarantee you will get a faster response, much faster than the staff accountant," he said.
Of course, putting it to work on actual tax research, versus a novelty demonstration, is contingent on a lot of other conditions. He said, ideally, the firm would need not so much the base ChatGPT, but a version trained on the firm's own data — which can be controlled both for security and accuracy — before ChatGPT is ready for this kind of work.
But when it is ready, according to Bourke, the impact will be revolutionary.
"If you control the foundation, you control the knowledge, and then you can have it research, draw conclusions, do things based on what you've taught it to learn — that is a massive power," he said.
Once that point is reached, ChatGPT can play a key role in client engagements, particularly where it concerns automation. He noted that the tax forms tend to be very rule-driven, to the point where the IRS includes step-by-step instructions on how to fill them out. By drawing on these instructions on processing a tax return, he said, ChatGPT could theoretically be used to help fill out some returns under the supervision of a human. Similarly, Bourke said he could envision a future where accountants could use it to test data in client books and records, as well as other audit procedures.
"Don't look at it for what it is today, but for its potential in the industry. The potential is manual task automation services by this technology … ChatGPT will never be able to replace the entire profession, but it will absolutely play a critical role in accounting as a tool to help us through," he said.
What it's doing now
This is not to say that all of its best uses are in the future. Accounting is more than just client engagements — there is a great deal of support work that a firm relies on, and it has been on these sorts of tasks that the AI has found the most use.
Scavuzzo said that coding has been one of the major applications they've found, as ChatGPT is capable of generating computer code based on plain language prompts. Coders in the firm's app development team are already using it to expedite solution production, to refine code on existing projects and to test out new ideas. He noted, though, that doing so efficiently requires a higher degree of expertise than most staff members possess.
"It will always generate code that theoretically works. The problem is whether it's working efficiently. A junior code developer might not know the difference. They ask the engine to provide, please, code to do XYZ and it gives them 25 lines to do that. That developer takes that as adequate, as it is technically functional. But a senior developer does the exact same task but uses [the output] as a starting point as opposed to an end game. They take those 25 lines of code and make them ultra-efficient. What we've found is, as a way to get things done efficiently, it's not there yet but it can be used as a starting block," he said.
People at Marcum have also used it to summarize and analyze white papers, as well as to refine their own content. They've also integrated it into their Microsoft Teams application to make it act as a real-time FAQ that team members can query. Scavuzzo said he would also like to apply this same capacity to product information, letting ChatGPT participate in an interactive dialogue with prospective customers who have questions about a solution.
Bourke added that he himself uses ChatGPT to create presentations. As someone who gives a lot of presentations at conferences, he said his normal process is to think about the topic, make an outline, then compose the slides. Using ChatGPT has sped up this process immensely.
"Whenever I do anything, I put a couple of dots on paper of what I want to cover, then make a PowerPoint based on my agenda. … It is a powerful tool to help the thought process. It quickly gives me ideas of things I should be talking about in this subject area that I probably would not have covered had I not gone to ChatGPT," he said.
Bourke acknowledged that the rise of ChatGPT has caused some anxiety among his peers, to the point where some are rejecting it out of hand. This, he said, would be a grave error. "Some firms are blocking it. But I say get over it! You can't avoid it! Embrace it! Don't block it!" he said, before adding, "Hopefully they're not using it for critical tax research or audit research. We want to make sure they're using it intelligently."
