Adobe Acrobat AI Review (vs ChatGPT 4o)

I ran the same document and prompts through Adobe Acrobat AI Assistant Assistant and ChatGPT 4o, and I concluded that I’ll stick to ChatGPT 4. Here’s why:

Aspect	ChatGPT 4o	Adobe Acrobat AI Assistant
Model Quality	One of the best models	I don’t know what model it uses
Speed	Faster	Slower
Price	US$20/month	~US$6/month on annual subscription

I don’t know which GPT version Acrobat uses. Information is scant beyond them using “Microsoft Azure OpenAI Service“.

Output Quality

An AI product has to impress on the first try.

I put a few long insurance documents through both Acrobat AI and ChatGPT. For the content below, I experimented on Progressive’s New York Direct Auto Policy PDF.

Let’s take a look at the answers. It’ll give you insights into why I’m sticking with ChatGPT.

My methodology: I look for a paragraph and craft a question. I test whether the AI answers the question correctly. Bonus points if it points out other relevant sections that I have missed.

Acrobat AI Analyzes New York Auto Policy

This is a car insurance policy and this section speaks to the appraisal process.

I asked Acrobat AI and ChatGPT the same questions. I struck out the sections which I felt were a waste of time to read.

My question: “My car was in an accident and I don’t agree with the insurer’s valuation of my car. What’s next?”

To address a disagreement with the insurer’s valuation of your car after an accident, you can take the following steps:

1. Review your insurance policy to understand the terms and conditions related to the valuation of your car and the process for disputing the insurer’s valuation.
2. Gather evidence to support your claim, such as photographs of your car before the accident, receipts for recent repairs or upgrades, and maintenance records.
3. Contact your insurer and express your disagreement with their valuation, providing them with the evidence you have gathered.
4. Request an independent appraisal by hiring a professional appraiser who specializes in valuing cars.
5. Present the independent appraisal to your insurer and engage in negotiations to reach a fair settlement.
6. If negotiations are unsuccessful, consider seeking legal advice from a lawyer who specializes in insurance claims.
7. If all else fails, you may have the option to submit the disagreement to arbitration, where an impartial third party will review the case and make a binding decision. ​

It’s important to note that the specific steps and options available to you may vary depending on the terms of your insurance policy and the laws and regulations in your jurisdiction. It is recommended to carefully review your policy and consult with professionals for personalized advice in your situation.

Hence, only 93 out of 211 words were relevant and not completely accurate as to the appraisal steps.

And ChatGPT 4’s answer:

To dispute the insurer’s valuation of your car, you can follow these steps as outlined in your policy:

1. Request an Appraisal: If you disagree with the amount of loss determined by the insurer, you can demand an appraisal of the loss. Both you and the insurer will each appoint a competent appraiser. If these appraisers cannot agree, they will select a qualified umpire to resolve the disagreement. The decision agreed upon by any two of these three (the two appraisers and the umpire) will be binding​​.
2. Submit Relevant Documentation: Gather and provide any additional documentation that supports your valuation, such as repair estimates, receipts for recent work, or valuations from other sources (e.g., Kelley Blue Book, NADA Guides).
3. Communicate Clearly: Clearly state why you believe the valuation is incorrect. This might include pointing out discrepancies in the condition assessment, mileage, or recent improvements that were not considered.
4. Legal Recourse: If the appraisal process does not resolve the issue to your satisfaction, you may consider seeking legal advice to explore further options such as mediation or small claims court.

These steps should help you in disputing the valuation effectively.

ChatGPT answers the appraisal process correctly in point one.

One thing: both AIs didn’t include time limits for disputes. One important deadline is how much time you have when you file a demand for appraisal — it’s 30 days to find an appraiser and notify Progressive of who they are.

So, I asked both AIs:

“I got into an accident. I don’t agree with my insurer’s valuation of my vehicle. What are the timelines I need to know about?”

For some reason, both outputs were longwinded. Maybe it’s because I wrote the prompt in a roundabout way on purpose, to make it harder for the AI. I could have used “appraisal” or “loss” in my prompt, making it easier for the AI, but I am trying to imagine what the layperson would ask.

Acrobat AI says 20-30 days. It’s sort of accurate but not exactly.

ChatGPT accurately says within 30 days. This is correct.

Verifying AI Output With Acrobat AI and ChatGPT

The next step involves verifying that the AI isn’t hallucinating.

Picture this: you haven’t read the document, but want the AI to tell you where it got its info.

I asked both AIs, “Can you point me to the relevant section? For example, point out a relevant section or page number.”

The correct answer is page 45.

Here’s what I got from Acrobat AI:

And ChatGPT:

Speed: ChatGPT wins

When you press the “AI Assistant” button, it automatically tries to generate an overview for you. In that time, you can’t do any AI prompting.

To analyze this insurance policy document from the AAA, it took 33 seconds before it allowed me to prompt. On the second try, I recorded the video above, and it took 43 seconds.

ChatGPT works a bit differently. You upload the file and then type your first prompt in. It took a few seconds — so little time that I couldn’t time it.

Pricing

ChatGPT Plus, which gives you access to GPT-4 models, cost US$20 a month.

Adobe Acrobat AI Assistant costs significantly less at ~$6/month for an annual subscription.

What’s Acrobat AI Good For?

“Insights you can trust, fast” is the selling point Adobe has on their landing page.

I see it a bit differently. Acrobat AI exists for convenience. By clicking a little button, you can get your document analyzed in the Acrobat app.

But I made a mistake. I ran a few documents through Acrobat AI thinking it would be as good as ChatGPT’s results, but the test results show otherwise.

Acrobat AI is probably still in its early stages, so it might evolve further in the future. We will see.