{"id":2754,"date":"2015-12-16T02:58:23","date_gmt":"2015-12-16T02:58:23","guid":{"rendered":"http:\/\/www.garysieling.com\/blog\/?p=2754"},"modified":"2015-12-16T02:58:23","modified_gmt":"2015-12-16T02:58:23","slug":"script-to-get-values-from-wikiart-pages","status":"publish","type":"post","link":"https:\/\/www.garysieling.com\/blog\/script-to-get-values-from-wikiart-pages\/","title":{"rendered":"Script to get values from wikiart pages"},"content":{"rendered":"

The following script will pull data values from a wikiart page (an excellent index of paintings<\/a>) –<\/p>\n

\nimport glob\n\nindir = 'D:\\\\projects\\\\art\\\\'\nfor filename in glob.glob(indir + \"*.html\"):\n    print filename\n    spans = soup.select(\"span[itemprop]\")\n    ahrefs = soup.select(\"a[itemprop]\")\n    \n    file = open(filename, 'r')\n\n    soup = BeautifulSoup(file, 'html.parser')\n\n    spans = soup.select(\"span[itemprop]\")\n    arefs = soup.select(\"a[itemprop]\")\n    \n    [(v[\"itemprop\"], v.text) for v in spans + arefs]    \n<\/pre>\n
Since most of the data values are labelled, you get a data structure like so:<\/p>\n
[(u’birthDate’, u’31 March 1902′), (u’dearthDate’, u’24 August 1976′), (u’nation’, u’French’), (u’nation’, u’Russian’), (u’art movement’, u’Art Informel’), (u’art movement’, u’Tachisme’), (u’painting school’, u’\\xc9cole de Paris’), (u’genre’, u’abstract’)]
\nD:\\projects\\art\\wikiart\\www.wikiart.org\\en\\andre-pierre-arnal.html
\nD:\\projects\\art\\wikiart\\www.wikiart.org\\en\\andre-pierre-arnal.html
\n[(u’birthDate’, u’16 December 1939′), (u’nation’, u’French’), (u’art movement’, u’Art Informel’), (u’art movement’, u’Contemporary’), (u’painting school’, u’Supports\/Surfaces’), (u’genre’, u’abstract’)]
\nD:\\projects\\art\\wikiart\\www.wikiart.org\\en\\andrea-del-castagno.html
\nD:\\projects\\art\\wikiart\\www.wikiart.org\\en\\andrea-del-castagno.html
\n[(u’birthDate’, u’c.1421′), (u’dearthDate’, u’19 August 1457′), (u’nation’, u’Italian’), (u’art movement’, u’Early Renaissance’), (u’painting school’, u’Florentine School’)]<\/p>\n","protected":false},"excerpt":{"rendered":"
The following script will pull data values from a wikiart page (an excellent index of paintings) – import glob indir = ‘D:\\\\projects\\\\art\\\\’ for filename in glob.glob(indir + “*.html”): print filename spans = soup.select(“span[itemprop]”) ahrefs = soup.select(“a[itemprop]”) file = open(filename, ‘r’) soup = BeautifulSoup(file, ‘html.parser’) spans = soup.select(“span[itemprop]”) arefs = soup.select(“a[itemprop]”) [(v[“itemprop”], v.text) for v in … <\/p>\n
Continue reading “Script to get values from wikiart pages”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[4],"tags":[61,83,447,495],"aioseo_notices":[],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/posts\/2754"}],"collection":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/comments?post=2754"}],"version-history":[{"count":0,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/posts\/2754\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/media?parent=2754"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/categories?post=2754"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.garysieling.com\/blog\/wp-json\/wp\/v2\/tags?post=2754"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}