Unstaging a list of files in git

When merging I find it helpful sometimes to un-stage all changes with conflicts. This allows you to commit changes without conflicts, and then resolve the conflicts as a separate step. If you do a cherry pick: git cherry-pick …insert sha here… You will see changes that have conflicts, but the parts of the files that […]

Fixing solr error: /dev/null: Permission denied

If you try to start solr on Ubuntu (service solr start) you can hit the following error: service solr start /opt/solr/bin/solr: line 120: /dev/null: Permission denied Java not found, or an error was encountered when running java. A working Java 7 or later is required to run Solr! Please install Java or fix JAVA_HOME before […]

Fixing solr error: The currently defined JAVA_HOME refers to a location where Java could not be found

If you uninstall Java, or it is installed in a way that the solr service can’t see, you can get the following error: The currently defined JAVA_HOME (/usr/lib/jvm/java-8-oracle) refers to a location where Java could not be found. Aborting. Either fix the JAVA_HOME variable or remove it from the environment so that the system PATH […]

Analyzing SSL Certificates in Scala with Akka

This example shows how to use Akka to get SSL certificate information (i.e. public keys) from a large list of domains. To do this, we’ll need to create a buil.sbt that pulls in Akka, a CSV reader, and a solr client (this is where we’ll store the output). libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.2.1" […]

Write CSV files in Scala

There is a convenient library for reading/writing CSVs in Scala. To include it in your project, add this to your build.sbt: libraryDependencies += "com.github.tototoshi" %% "scala-csv" % "1.3.0" And import it: import com.github.tototoshi.csv._ Then say you create some data in a loop: results += List( domain, conn.getCipherSuite, cert.hashCode, cert.getPublicKey().getAlgorithm, cert.getPublicKey().getFormat, cert.getSigAlgName, cert.getSigAlgOID, cert.getIssuerDN, cert.getSerialNumber, cert.getSubjectDN, […]

Scala: concatenate ArrayBuffer

If you use a mutable collection in scala, you can append to it with += and ++=, like you might expect. E.g., here is adding an entry to a list of lists: val results = ArrayBuffer[List[Any]]() results += List( domain, conn.getCipherSuite, cert.hashCode, cert.getPublicKey().getAlgorithm, cert.getPublicKey().getFormat, cert.getSigAlgName, cert.getSigAlgOID, cert.getIssuerDN, cert.getSerialNumber, cert.getSubjectDN, cert.getVersion, cert.getNotAfter, cert.getNotBefore, cert.getPublicKey.getFormat, cert.getPublicKey.getAlgorithm, cert.getIssuerDN.getName […]

Scala: wget/curl example

The following example shows how to implement the bones of a wget or curl style application in Scala, including checking for HTTPS and obtaining the certificate chain. val testUrl = "https://" + domain val url = new URL(testUrl) val conn: HttpsURLConnectionImpl = url.openConnection() match { case httpsConn: HttpsURLConnectionImpl => httpsConn case conn => { println(conn.getClass) […]

Handling duplicate query string arguments in Express.js

Duplicate query string entries are somewhat common, for example if you use Solr, you can do something like this: /execute?facet.field=stream_size&facet.field=content_type If you make an Express.js endpoint, you get can this data through the “query” option: app.use(‘/execute/’, function(req, res) { console.log(req.query) } ) The “req.query” is a Javascript object, which presents difficulties with duplicated arguments, which […]