{"id":3882,"date":"2016-04-24T02:15:44","date_gmt":"2016-04-24T02:15:44","guid":{"rendered":"http:\/\/www.garysieling.com\/blog\/?p=3882"},"modified":"2016-04-24T02:15:44","modified_gmt":"2016-04-24T02:15:44","slug":"get-nested-text-within-tag-beautiful-soup","status":"publish","type":"post","link":"https:\/\/www.garysieling.com\/blog\/get-nested-text-within-tag-beautiful-soup\/","title":{"rendered":"Get all nested text within a tag with Beautiful Soup"},"content":{"rendered":"

The “beautiful soup” library in Python lets you parse HTML pages.<\/p>\n

It does some things a little weirdly if you’re used to Javascript. To filter the document, you can use “find”, which gives you a list of tags matching some condition. However, these values are text elements, not DOM nodes, so you have to do “parent” to get something that is actually useful.<\/p>\n

You can then do a “find” on the element you found, to filter to it’s child elements that are bits of text.<\/p>\n

\nfile = 'pages\/talk' + str(i) + '.html'\nsoup = BeautifulSoup(open(file), 'html.parser')\n\ndef getTexts():  \n  for hit in soup.find(attrs={\"class\": \"transcript-text-content\"}):\n    yield \"\".join(hit.parent.findAll(text=True))\n\nprint \"\".join(getTexts())\n<\/pre>\n
In my tests, the join gives you newlines between the elements, but this may be a coincidence based on my data.<\/p>\n","protected":false},"excerpt":{"rendered":"
How to get the text nested within a DOM element in Python<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4],"tags":[82,447,495],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t