{"id":1232,"date":"2013-06-28T12:39:53","date_gmt":"2013-06-28T12:39:53","guid":{"rendered":"http:\/\/garysieling.com\/blog\/?p=1232"},"modified":"2013-06-28T12:39:53","modified_gmt":"2013-06-28T12:39:53","slug":"scraping-pdf-text-with-python","status":"publish","type":"post","link":"https:\/\/www.garysieling.com\/blog\/scraping-pdf-text-with-python\/","title":{"rendered":"Scraping PDF text with Python"},"content":{"rendered":"<p>If you want to extract text from a PDF with Python, there is a library called<a href=\"http:\/\/www.unixuser.org\/~euske\/python\/pdfminer\/\"> PDFMiner<\/a> (beware: does not work in Python 3).<\/p>\n<p>This example will walk a directory structure, look for PDFs, and make a &#8220;.txt&#8221; file next to the PDF with a text rendition.<\/p>\n<pre lang=\"Python\">\nimport sys\nfrom pdfminer.pdfparser import PDFDocument, PDFParser\nfrom pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter, process_pdf\nfrom pdfminer.pdfdevice import PDFDevice, TagExtractor\nfrom pdfminer.converter import XMLConverter, HTMLConverter, TextConverter\nfrom pdfminer.cmapdb import CMapDB\nfrom pdfminer.layout import LAParams\nimport os\n\n# main\ndef main(outfile, fname):\n    # debug option\n    debug = 0\n    # input option\n    password = ''\n    pagenos = set()\n    maxpages = 0\n    # output option\n    outtype = 'text'\n    layoutmode = 'normal'\n    codec = 'utf-8'\n    pageno = 1\n    scale = 1\n    caching = True\n    showpageno = True\n    laparams = LAParams()\n    \n    PDFDocument.debug = debug\n    PDFParser.debug = debug\n    CMapDB.debug = debug\n    PDFResourceManager.debug = debug\n    PDFPageInterpreter.debug = debug\n    PDFDevice.debug = debug\n    \n    rsrcmgr = PDFResourceManager(caching=caching)\n    outtype = 'text'\n    if outfile:\n        outfp = file(outfile, 'w')\n    else:\n        outfp = sys.stdout\n    if outtype == 'text':\n        device = TextConverter(rsrcmgr, outfp, codec=codec, laparams=laparams)\n    \n    fp = file(fname, 'rb')\n    process_pdf(rsrcmgr, device, fp, pagenos, maxpages=maxpages, \n                password=password, caching=caching, \n                check_extractable=True)\n    fp.close()\n    device.close()\n    outfp.close()\n    return\n\nindir = \"00\"\nfor root, dirs, filenames in os.walk(indir):\n   for fname in filenames:\n       if (fname.endswith(\".pdf\")):\n            main(os.path.join(root, fname) + \".txt\", os.path.join(root, fname)) \n\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>If you want to extract text from a PDF with Python, there is a library called PDFMiner (beware: does not work in Python 3). This example will walk a directory structure, look for PDFs, and make a &#8220;.txt&#8221; file next to the PDF with a text rendition. import sys from pdfminer.pdfparser import PDFDocument, PDFParser from &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.garysieling.com\/blog\/scraping-pdf-text-with-python\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Scraping PDF text with Python&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4,5,11],"tags":[420,447,495],"aioseo_notices":[],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/posts\/1232"}],"collection":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/comments?post=1232"}],"version-history":[{"count":0,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/posts\/1232\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/media?parent=1232"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/categories?post=1232"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/tags?post=1232"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}