Using ORM layers to ease Support

One of the challenges in supporting complex applications is to answer questions like “Why doesn’t my document show up in search results,” or “why isn’t this event in my report?” Even though database operations are often modeled with set theory, returning the inverse of a query is not sufficient. Consider, however, a hypothetical ORM layer, […]

Base64 Encoding and Decoding Example in Scala

This fairly simple example makes use of the core Java APIs to read files, demonstrating how to construct a base-64 encoded version, and then generating the original file again. package com.garysieling { import java.io.{File, FileInputStream, FileOutputStream} import sun.misc.{BASE64Encoder, BASE64Decoder}   object Base64 extends App { val filename = """C:\IdeaProjects\base64\tests.pdf""" Console.println("Testing " + filename)   val […]

Fixing Scala Error: Main method is not static in class X

If you set up a Scala HelloWorld example naively, it is easy to get the following perplexing error message: :runError: Main method is not static in class com.garysieling.HelloWorld, please define the main method as: public static void main(String[] args) This happens if the code looks like the following: package com.garysieling { class HelloWorld extends App […]

Improving Performance of Gradle + Scala Build with a compiler daemon

If you find the build times of Scala programs in gradle unacceptably slow, you can add the following to build.gradle. This took my simple build from 25 seconds to five. compileScala { scalaCompileOptions.useCompileDaemon = true // optionally specify host and port of the daemon: scalaCompileOptions.daemonServer = “localhost:4243” } This doesn’t seem to be stable all […]

Fixing Gradle Error: Cannot infer Scala class path because no Scala library Jar was found on class path: configuration ‘:compile’

When you get this error: Cannot infer Scala class path because no Scala library Jar was found on class path: configuration ‘:compile’ This indicates that you need to tell gradle which version of Scala you’re interested in, so that it can download it. To fix this, add the following to build.gradle: dependencies { compile ‘org.scala-lang:scala-library:2.10.1’ […]

Querying Multiple Postgres Full-Text Indexes

Setup I’ve set up several scenarios to see how Postgres handles querying multiple full-text indexes. To demonstrate these scenarios I set up the following table and indexes, which has 1.1 million records: CREATE TABLE data2 ( author CHARACTER VARYING, id CHARACTER VARYING, email CHARACTER VARYING, company CHARACTER VARYING, DATE CHARACTER VARYING, message CHARACTER VARYING, github […]

GIN vs GiST For Faceted Search with Postgres Full Text Indexes

For this test, I set up a 1.1 million row table using data from github. If you’re not familiar with full-text in Postgres, they use the @@ operator to apply a query to a document, which looks like this: SELECT to_tsvector(’fat cats ate fat rats’) @@ to_tsquery(’fat & rat’); The order of the conditions doesn’t […]

Fixing Issues Where Postgres Optimizer Ignores Full Text Indexes

When I attempted to implement faceted search against a Postgres full-text index, I found I had issues getting Postgres (9.2) to pick up the index: CREATE INDEX search_idx2 ON data2 USING gin(to_tsvector(’english’, SEARCH));   SELECT * FROM ( SELECT author, COUNT(*) c FROM data2 WHERE SEARCH @@ to_tsquery(’linux’) GROUP BY author ) counts ORDER BY […]

Fixing Solr error “no segments* file found in NRTCachingDirectory”

If you try to run a Solr DataImportHandler by running HTTP commands (e.g. from a build script), you may hit an error like this: “no segments* file found in NRTCachingDirectory” It appears that this indicates that Solr didn’t initialize properly. You can fix this by adding a second request, to hit the core directly: http://localhost:8080/solr/#/core/dataimport//dataimport