ChatPDF Review
One of my peeves when working with ChatGPT is the inability to grant it access to files that are larger than the 6K character chat input. What is needed is the ability to upload files to it, or to free GPT to follow links to files or pages online. I have been looking for a way to do this. Today I tried ChatPDF.
The first thing to point out is that it only works for PDF documents, a completely dumb decision on the part of the developers. PDF is arguably the worst format for documents in existence. What this means for my experiment is that I started with a ripped video file, fed that into Trint (an audio transcription AI) that has a proprietary time coded document format, then downloaded the uncorrected file exported as a PDF, then uploaded it to ChatPDF. Quite an annoying process.
As an aside, I have to say that Trint has improved significantly. It used to blow up on video files that are longer than one hour (after processing for an hour, of course) and I complained loudly about this. They had told me that I need to chop the video file into pieces less than one hour in length. I told them that this policy is arbitrarily stupid and unnecessary and I said ‘so, if I transcode the file to 1.5X speed, a 1.5 hour video becomes a one hour video and that’s just fine. You do see the stupidity of this.’ Yes, I get that there must be some practical limit — perhaps 8 hours is reasonable? So today I was pleased to find that today it processed a 4-hour podcast without a hitch. It looks like they heard their customers’ concerns.
I uploaded an uncorrected and unreviewed transcript to test whether a system like ChatPDF is able to produce a digest or summary of the podcast the way Eighify (sometimes) will do. This would be extremely useful, as podcasts tend to be littered with fluff and waste a lot of valuable time. For the same reason, I stopped listening to commercial radio years ago.
Following is my dialogue with ChatPDF:
Chat with Eye_of_the_Storm_Ep_31_v2se29c UNPROOFED timecodes.docx.pdf
Welcome to this helpful PDF file! In this file, you'll find the first paragraph of the Eye of the Storm Ep 31 v2se29c.mp4 video, which includes a special guest appearance by Kyle from Just Human. We're excited to share this with you and hope you enjoy reading it. Here are three example questions you might have about this file:
Who is Kyle from Just Human and what shows does he appear on?
What is the Eye of the Storm Ep 31 v2se29c.mp4 video about?
Where can I find more information about the podcast that Kyle hosts every morning?
Please list the major topics of this transcript, omitting sponsor advertising.
I'm sorry, but I cannot provide an answer without knowing which transcript you are referring to. Please provide more information or context.
This file
I apologize for the confusion earlier. This file does not contain a transcript, but rather three pages of text with no clear topic or theme. Page 17 contains a short clip from April 4th, while page 25 discusses a bunker of information located in Utah and mentions Kim's analysis of timestamp 755. Page 1 welcomes viewers to the Eye of the Storm Ep 31 v2se29c.mp4 video and introduces special guest Kyle from Just Human, listing the shows he appears on and his social media handle.
I will refer to this document as Eye of the Storm episode 31. It is an AI-generated transcript that has not been checked by a human for errors. On page 3, what is the main topic of discussion that begins?
I'm sorry, but there is no page 3 in the document you provided earlier. The document only contains three pages, which are pages 1, 17, and 25. Please provide more information or context so I can assist you better.
On what page does discussion about the Durham Report begin?
The discussion about the Durham Report begins on page 2 of the document.
This file contains timestamps. Please list the times that discuss John Durham or the Durham Report
I apologize for the confusion earlier. There are no timestamps in the document you provided earlier. The document only contains three pages, which are pages 1, 17, and 25. Please provide more information or context so I can assist you better.
Please identify which GPT version is chatting with me
I am a GPT-3 language model developed by OpenAI.
I abandoned the experiment after confirming that the Language Model is GPT-3, which definitely gives the experience of chatting with a dumb robot. GPT-4 is light years ahead.
Here are a few observations:
The dumb robot did not infer what transcript I was referring to, which was the only file I connected.
The dumb robot continues to insist that the file is not a transcript after being told that it is a transcript.
The dumb robot’s attention span is limited to three pages, and in that span, it is true that there is no clear topic. Therefore it is apparent that GPT-3 is not going to be useful to query a 61-page transcript. It does manage to vaguely refer to topics throughout the document (in random order) but is not able to discern what topics might be important or which parts are just irrelevant chatter.
The dumb robot denied several times that there are timestamps even after being told that there are. Nevertheless it correctly finds that page 2 is the start of the topic of the Durham Report.
The most comical aspect of this chat is its insistence that this document contains only three pages and those are pages 1, 17, and 25.
Overall, this experience was completely useless, but if someone can just get files to connect to GPT-4, I think it would be a very powerful tool.
Below are the website's FAQs which indicate that they are having difficulty doing this with GPT-4. I cannot imagine the reason.
I’m going to keep an eye out for similar projects but I don’t think this one is ready. I certainly will not be subscribing to this version but if they succeed in getting GPT-4, it will probably be worth the money.
ChatPDF FAQ
Can ChatPDF speak my language?
Yes, ChatPDF can read PDFs and answer questions in any language. You can upload a PDF in one language and ask questions in another. The greeting message will be in the PDF’s language. After that, ChatPDF will answer in the language you ask. If a message isn’t in the language you want, just ask ChatPDF to change it.
Is ChatPDF free?
ChatPDF allows you to use it for free with 3 PDFs every day, each up to 120 pages. For more, you can upgrade to the Plus plan for $5 per month. For additional information, check the pricing page.
Is there an API to integrate ChatPDF?
Do you want to integrate ChatPDF into your website, mobile app or use it via an API? Please quickly fill out the API registration form and we’ll get back to you shortly.
Are my files secure?
ChatPDF will never share your files with anyone. They are stored on a secure cloud storage and can be deleted at any time.
How do I delete a PDF from ChatPDF?
From Chat view: click gray Delete Chat button on the top right. From the Home page: click the X next to the chat title in the My Chats section. This will permanently delete the chat and PDF file content from ChatPDF servers.
Why can't ChatPDF see all PDF pages?
For each answer, ChatPDF can look at only a few paragraphs from the PDF at once. These paragraphs are the most related to the question. ChatPDF might say it can't see the whole PDF or mention just a few pages because it can view only paragraphs from those pages for the current question.
How does ChatPDF work?
In the analyzing step, ChatPDF creates a semantic index over all paragraphs of the PDF. When answering a question, ChatPDF finds the most relevant paragraphs from the PDF and uses the ChatGPT API from OpenAI to generate an answer.
Can ChatPDF understand images and tables in PDFs?
ChatPDF can not yet read images in the PDF, including images that consist of scanned text. Text in tables is read by ChatPDF, but it might have problems correlating the correct rows and columns.
Does ChatPDF use GPT-4?
ChatPDF uses GPT 3.5 for now, which is the same as ChatGPT. We are looking at how to add GPT-4. But GPT-4 won't be available for all messages on the free plan because it costs too much.
What is the cancellation policy?
You can cancel your Plus subscription at any time. If you cancel, you will still have access to your Plus plan until the end of the current 30-day billing period. To cancel your subscription, click on My Account.
Can I chat with multiple PDF files in one chat?
Not yet, but we’re working on it.
Where do I post feature requests or bug reports?
Feedback is very welcome! You can post into our Discord, reach out on Twitter, or write an email to support@chatpdf.com.
Fascinating!