Update data in Solr from Scala

Updating or adding fields in Solr is a little trickier than inserting new documents.

When you add documents to Solr through the API, it can create all the fields you send automatically – these will end up in a file called “managed-schema”. This is a neat feature, but it doesn’t occur by default when you update so you’ll have to add any fields you want manually:

The API I’ve been using (jp.sf.amateras.solr.scala) doesn’t support updating fields, so we’ll have to use HTTP directly, by doing a POST to the update endpoint:

http://localhost:8983/solr/ssl_certificates/update?commit=true

Normally you can post new documents to this API. Be careful not to do this, because it overwrites whatever was there If you’re doing this in development, you might want to take a backup of your Solr core before you do this.

To modify fields, you can set the value of the attribute to an object, which is in the format “attribute: {“set”: “value”}”. The “set” command will set the value of the specified attribute. There are also commands to append to a list, remove a value, and increment a number¹

According to the docs, the “set” command creates a new document, which replaces the old one. This is important: if you don’t store fields it wouldn’t be able to pull them out to create the new document.

Here’s how we can make the JSON:

val solrUpdate =
  Json.arr(
    Json.obj(
      "id" -> id,
      "short_certificate" ->
        Json.obj(
          "set" -> chain(0).toString
        )
      )
    )

And for completeness, here is the code to do the actual update:

val updateUrl = "http://localhost:8983/solr/ssl_certificates/update"

val response =
  Http.apply(updateUrl)
      .header("Content-type", "application/json")
      .postData(solrUpdate.toString)
      .asString
      .body

println(response)

The output of this is useful, because this is where any Solr errors will be present.

If you want to do a lot of individual updates, you don’t want to commit each time, or it will take forever. You can also do giant batch updates, depending on the nature of your situation.

1. http://solr.pl/en/2012/07/09/solr-4-0-partial-documents-update/ [↩]