The following sample will extract the contents of a court case and attempt to recognize names and locations using entity recognition software from Stanford NLP. From the samples, you can see it’s fairly good at finding nouns, but not always at identifying the type of each noun.<\/p>\n
In this example, the entities I’d like to see are different – companies, law firms, lawyers, etc, but this test is good enough. The default examples provided let you choose different sets of things that can be recognized: {Location, Person, Organization}, {Location, Person, Organization, Misc}, and {Time, Location, Organization, Person, Money, Percent, Date}. The process of extracting PDF data and processing it takes about five seconds.<\/p>\n
For this text, selecting different options sometimes led to the classifier picking different options for a noun – one time it’s a person, another time it’s an organization, etc. One improvement might be to run several classifiers and to allow them to vote. This classifier also loses words sometimes – if a subject is listed with a first, middle, and last name, it sometimes picks just two words. I’ve noticed similar issues with company names.<\/p>\n
\nimport org.apache.tika.parser.pdf._\nimport org.apache.tika.metadata._\nimport org.apache.tika.parser._\nimport java.io._\nimport org.xml.sax._\nimport edu.stanford.nlp.ie.crf.CRFClassifier\nimport edu.stanford.nlp.ling.CoreAnnotations\n\nobject pdfHandler extends ContentHandler {\n val contents: StringBuffer = new StringBuffer()\n\n def characters(ch: Array[Char], start: Int, length: Int) {\n contents.append(new String(ch))\n }\n\n def endDocument() {\n }\n\n def endElement(uri: String, localName: String, qName: String) {\n }\n\n def endPrefixMapping(prefix: String) {\n }\n\n def ignorableWhitespace(ch: Array[Char], start: Int, length: Int) {\n }\n\n def processingInstruction(target: String, data: String) {\n }\n\n def setDocumentLocator(locator: Locator) {\n }\n\n def skippedEntity(name: String) {\n }\n\n def startDocument() {\n }\n\n def startElement(uri: String, localName: String, qName: String, atts: Attributes) {\n }\n\n def startPrefixMapping(prefix: String, uri: String) {\n }\n}\n\nobject pdf extends App {\n val file = \"\"\"e:\\data\\11-1285_i4dk.pdf\"\"\"\n\n val pdf: PDFParser = new PDFParser();\n\n val stream: InputStream = new FileInputStream(file)\n val handler: ContentHandler = pdfHandler\n val metadata: Metadata = new Metadata()\n val context: ParseContext = new ParseContext()\n\n pdf.parse(stream,\n handler,\n metadata,\n context)\n\n stream.close()\n\n val contents: String = pdfHandler.contents.toString()\n println(contents)\n\n val src = \"stanford-ner-2013-04-04\/classifiers\/\"\n val classifier1 = \"english.all.3class.distsim.crf.ser.gz\"\n val classifier2 = \"english.conll.4class.distsim.crf.ser.gz\"\n val classifier3 = \"english.muc.7class.distsim.crf.ser.gz\"\n\n val serializedClassifier = src + classifier1\n\n val classifier = CRFClassifier.getClassifierNoExceptions(serializedClassifier)\n val out = classifier.classify(contents)\n\n var words = 0\n for (i <- 0 to out.size() - 1) {\n val sentence = out.get(i)\n\n var foundWord = \"\"\n var oldWordClass = \"\"\n\n for (j <- 0 to sentence.size() - 1) {\n val word = sentence.get(j)\n val wordClass = word.get(classOf[CoreAnnotations.AnswerAnnotation]) + \"\"\n\n if (!oldWordClass.equals(wordClass)) {\n if (!oldWordClass.equals(\"O\") && !oldWordClass.equals(\"\")) {\n print(\"[\/\" + oldWordClass + \"]\")\n }\n }\n\n if (!wordClass.equals(\"O\") && !wordClass.equals(\"\")) {\n if (!oldWordClass.equals(wordClass)) {\n print(\"[\" + wordClass + \"]\")\n }\n }\n\n oldWordClass = wordClass\n\n words = words + 1\n print(word);\n print(\" \");\n\n if (words > 10) {\n words = 0\n println(\" \")\n }\n }\n }\n}\n<\/pre>\n\n11-1285 [ORGANIZATION]US Airways , Inc. [\/ORGANIZATION]v. \n[PERSON]McCutchen [\/PERSON]-LRB- 4\\\/16\\\/13 -RRB- 1 -LRB- \nSlip Opinion -RRB- OCTOBER TERM , \n2012 Syllabus NOTE : Where it \nis feasible , a syllabus -LRB- \nheadnote -RRB- will be released , \nas isbeing done in connection with \nthis case , at the time \nthe opinion is issued . The \nsyllabus constitutes no part of the \nopinion of the Court but has \nbeenprepared by the Reporter of Decisions \nfor the convenience of the reader \n. See [LOCATION]United States [\/LOCATION]v. [ORGANIZATION]Detroit \nTimber & Lumber Co. [\/ORGANIZATION], 200 \nU. S. 321 , 337 . \nSUPREME COURT OF THE [ORGANIZATION]UNITED STATES \nSyllabus US AIRWAYS [\/ORGANIZATION], INC. , \nIN ITS CAPACITY AS FIDUCIARY AND \nPLAN ADMINISTRATOR OF THE [LOCATION]US [\/LOCATION]AIRWAYS \n, INC. . EMPLOYEE BENEFITS PLAN \nv. [PERSON]MCCUTCHEN [\/PERSON]ET AL. . CERTIORARI \nTO THE [ORGANIZATION]UNITED STATES [\/ORGANIZATION]COURT OF \nAPPEALS FOR THE THIRD CIRCUIT No. \n11 -- 1285 . Argued November \n27 , 2012 -- Decided April \n16 , 2013 The health benefits \nplan established by petitioner [ORGANIZATION]US Airways \n[\/ORGANIZATION]paid $ 66,866 in medical expenses \nfor injuries suffered by respondentMcCutchen , \na [ORGANIZATION]US Airways [\/ORGANIZATION]employee , in \na car accident caused by athird \nparty . The plan entitled [ORGANIZATION]US \nAirways [\/ORGANIZATION]to reimbursement if \n[PERSON]McCutchen [\/PERSON]\n<\/pre>\n","protected":false},"excerpt":{"rendered":"The following sample will extract the contents of a court case and attempt to recognize names and locations using entity recognition software from Stanford NLP. From the samples, you can see it’s fairly good at finding nouns, but not always at identifying the type of each noun. In this example, the entities I’d like to … <\/p>\n