# Graphic wrongly placed in md file output from pymupdf4llm.to\_markdown **URL:** \ **Category:** PyMuPDF\ **Created:** [July 22, 2025, 1:53pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82 "2025-07-22T13:53:37Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Steen\_Larsen](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/steen_larsen/32/99_2.png) [@Steen\_Larsen](https://forum.mupdf.com/u/Steen_Larsen)\ **Post date:** [July 22, 2025, 1:53pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/1 "2025-07-22T13:53:37Z")
Sorry, it is me nitpicking again 😀 My PDF file has this content on page 61 ![image](https://canada1.discourse-cdn.com/flex004/uploads/mupdf/original/1X/ae672f2315e9e30e7ab6ee5b29e50097b888d440.png) But the resulting markup from pymupdf4llm.to\_markdown renders like this : ![image](https://canada1.discourse-cdn.com/flex004/uploads/mupdf/original/1X/f5b92b91e28b4c7e71821b71363398639ba2eba5.png) This is IMHO a small bug. Here is the wrong and a a proposed improved markup : ```auto # Current output # **Important information** ![](fiasp-epar-product-information_en.pdf-60-0.png) Pay special attention to these notes as they are important for correct use of the pen. # Corrected # ![](fiasp-epar-product-information_en.pdf-60-0.png) **Important information** \ Pay special attention to these notes as they are important for correct use of the pen. ``` The corrected markup gives this ![image](https://canada1.discourse-cdn.com/flex004/uploads/mupdf/original/1X/aa19225c252a32c7f5fc37d81e1a36fbc9cbd9ac.png) I note that in the input PDF file the Y coordinate of the text “Important information” is slightly bigger than the Y coordinate of the graphic. The issue happens the first time on page 61 in the PDF file below. PS. My use case which requires all this precision is to extract the text and graphics content from a PDF file to a markup file, use an LLM to edit it and then convert the changed markup file back to a PDF which looks as similar as possible to the original. PPS I use the demo code below. import pymupdf4llm md\_text = pymupdf4llm.to\_markdown(“/pleaflet/samples/fiasp-epar-product-information\_en.pdf”, write\_images=True, force\_text=False, image\_size\_limit=0)) import pathlib pathlib.Path(“noutput.md”).write\_bytes(md\_text.encode()) ---
**Author:** ![Steen\_Larsen](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/steen_larsen/32/99_2.png) [@Steen\_Larsen](https://forum.mupdf.com/u/Steen_Larsen)\ **Post date:** [July 22, 2025, 1:54pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/2 "2025-07-22T13:54:55Z")
The PDF file I used is can be found here www.ema.europa.eu/en/documents/product-information/fiasp-epar-product-information\_en.pdf (not typed as link because this system does not accept it) ---
**Author:** ![Jamie\_Lemon](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/jamie_lemon/32/5_2.png) [@Jamie\_Lemon](https://forum.mupdf.com/u/Jamie_Lemon)\ **Post date:** [July 22, 2025, 2:06pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/3 "2025-07-22T14:06:41Z")
Fair point - not sure why it doesn’t find the image first in that line and present the MD in the oder which you suggested. @HaraldLieder any insights into this one? ---
**Author:** ![HaraldLieder](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/haraldlieder/32/14_2.png) [@HaraldLieder](https://forum.mupdf.com/u/HaraldLieder)\ **Post date:** [July 22, 2025, 2:44pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/4 "2025-07-22T14:44:55Z")
When we write the page content, then before writing a text line we look at any images (or tables) that _ **end above** _ the text’s top coordinate. That image occurs here in the page: `Rect(70.89993286132812, 297.0294494628906, 82.99993133544922, 307.37945556640625)` The text line `Important information` has this bbox: `Rect(99.248779296875, 295.7320861816406, 207.26747131347656, 311.0114440917969)` So `y0 = 295.7320861816406` of the text is smaller than `y1 = 307.37945556640625` of the image. This case only **_seems_** to go wrong. The actual problem is your expectation … 😎 ---
**Author:** ![Steen\_Larsen](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/steen_larsen/32/99_2.png) [@Steen\_Larsen](https://forum.mupdf.com/u/Steen_Larsen)\ **Post date:** [July 22, 2025, 3:51pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/5 "2025-07-22T15:51:01Z")
Thanks a lot for explaining your algorithm. My expectation is at a higher level : That the markup renders as close to the PDF as possible. I note that the algorithm you describe will always ruin the layout of any graphics and text which are aligned on the same horizontal line. In particular when graphics are used as a bullet point for text. In documents reading from left to right where a reader of the PDF first sees the graphic and then the text, the described algorithm will result in an “inversed” markup document where the reader first sees the text and then the graphic. In the context of a medical document where the graphic represents a warning related to the following text this is quite serious. Perhaps it would be better to look at the Y-axis mid points of the two objects instead of the top and bottom. When these are close, the X coordinate could be considered according to the reading direction. Best regards Steen ---
**Author:** ![Steen\_Larsen](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/steen_larsen/32/99_2.png) [@Steen\_Larsen](https://forum.mupdf.com/u/Steen_Larsen)\ **Post date:** [July 22, 2025, 4:01pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/6 "2025-07-22T16:01:41Z")
Here is something else that looks strange, even taking the algorithm you explained into consideration. On page 62 there is a page number at the bottom of the page far below the graphic picture. However, in the markup, the page number is written before the graphic. See below (rendered markup is on the left and the PDF is on the right) ![image](https://canada1.discourse-cdn.com/flex004/uploads/mupdf/original/1X/6f846d229cf697489e65fd452e62a619f1d105ef.png) ---
**Author:** ![Steen\_Larsen](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/steen_larsen/32/99_2.png) [@Steen\_Larsen](https://forum.mupdf.com/u/Steen_Larsen)\ **Post date:** [July 22, 2025, 4:15pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/7 "2025-07-22T16:15:28Z")
Another example showing how the graphics in the markup (left) renders very differently to the original PDF (right). This is very disturbing to the reading of the document. ![image](https://canada1.discourse-cdn.com/flex004/uploads/mupdf/original/1X/2bba65c526b58f6d42891908b718f0a4dd109600.png) An LLM which has to summarize the important points could get these wrong due to this markup IMHO. ---
**Author:** ![HaraldLieder](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/haraldlieder/32/14_2.png) [@HaraldLieder](https://forum.mupdf.com/u/HaraldLieder)\ **Post date:** [July 22, 2025, 4:40pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/8 "2025-07-22T16:40:16Z")
I would make the point that there is no way to state an unambiguous / failsafe rule. At least I didn’t come across one yet. Neither comparing the top, bottom nor center point coordinates will always work. There is no nonsense that can _ **not** _ be found in PDFs. An idea could be to also look at the left-most coordinate and just confirm that the image / text vertical intervals overlap if `image.x0 < text.x0`. If `image.x1 > text.x1` (and the verticals overlap), then the image should also be written first. How about this? ---
**Author:** ![Steen\_Larsen](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/steen_larsen/32/99_2.png) [@Steen\_Larsen](https://forum.mupdf.com/u/Steen_Larsen)\ **Post date:** [July 22, 2025, 5:04pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/9 "2025-07-22T17:04:55Z")
> [@HaraldLieder](#): > > There is no nonsense that can _ **not** _ be found in PDFs. I am far from an expert, but yes, PDF is a crazy format which was definitely not made to be interpreted and reformatted outside a printer. 😂 > [@HaraldLieder](#): > > How about this? That sounds like an excellent idea which is better and simpler than my proposal. Perhaps the method to “sort” images and text can be an option. ---
**Author:** ![HaraldLieder](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/haraldlieder/32/14_2.png) [@HaraldLieder](https://forum.mupdf.com/u/HaraldLieder)\ **Post date:** [July 22, 2025, 5:18pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/10 "2025-07-22T17:18:16Z")
They **_are_** already sorted in all 4 directions and at zillions of places, believe me. ---
**Author:** ![HaraldLieder](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/haraldlieder/32/14_2.png) [@HaraldLieder](https://forum.mupdf.com/u/HaraldLieder)\ **Post date:** [July 22, 2025, 5:24pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/11 "2025-07-22T17:24:47Z")
All improvement ideas are prone to be perception-biased. Things can get very complicated when text in multi-column format is present. How do we make sure that images belong to one of the text columns - as opposed to interrupting the the multi-column structure. Plus, what do we do with images where text flows around it. … ---
**Author:** ![HaraldLieder](https://yyz2.discourse-cdn.com/flex004/user_avatar/forum.mupdf.com/haraldlieder/32/14_2.png) [@HaraldLieder](https://forum.mupdf.com/u/HaraldLieder)\ **Post date:** [July 22, 2025, 6:32pm UTC](https://forum.mupdf.com/t/graphic-wrongly-placed-in-md-file-output-from-pymupdf4llm-to-markdown/82/12 "2025-07-22T18:32:11Z")
This effect comes from the logic we already discussed. Images that were not picked up during writing of the text lines are collectively written at the end of the page. My changes that I am testing currently will prevent this, too.