Convert scaladoc output to Markdown and Jekyll

Scaladoc generates a listing page for every class you run it on.

In this post, I’ll show how to convert the output of the entire API documentation (on the official scala site) to Markdown that is compatible with Jekyll (so that you can add / modify formatting as desired).

I got the main scala library, forked it, and built from source. To do this, you will need some pre-requisites:

brew update
brew install ant
brew install gem
gem install jekyll

There is an ant target that lets you generate all of the scala documentation from source (this is what you’d need if you want to keep up with the nightly builds).

git clone https://github.com/garysieling/scala.git
cd scala
ant docs

This took ~40 minutes for me.

To convert this to markdown, there is a handy library called “nyandoc”, but you need to install a scala dependency tracking thing called “conscript” first (this seems to be a wrapper around SBT).

There are a couple ways to do this, and I had to fiddle with it to get it to work.

You can either download the JAR and install or yolo it and get code directly from Github:

curl https://raw.githubusercontent.com/n8han/conscript/master/setup.sh | sh

I had problems with both methods, but eventually got this to work.

Once conscript installs it’s not on your path by default, but it’s not a big deal. You then install nyandoc with this command:

/Users/gary/bin/cs todesking/nyandoc
sbt launcher version 0.13.7
 
Conscripted todesking/nyandoc to /Users/gary/bin/nyandoc

This tool is also not on your path.

To convert to markdown, go into the folder where you build scala, and there will be a “library” folder that has the compiled scaladoc output.

cd scala/build/scaladoc
/Users/gary/bin/nyandoc library library-markdown

The above command should only take a few minutes.

Now, copy the output over to your jekyll site.

cp -R library-markdown ../../../scaladoc-jekyll/_posts/
cd ../../../scala-jekyll/_posts

Scaladoc output is a deeply nested folder structure. For this example the easiest thing to do is to flatten this:

find */ -type f -exec bash -c 'file=${1#./}; echo mv "$file" "${file//\//_}"' _ '{}' \; > move_all.sh
chmod +x ./move_all.sh 
mv *.md ..

You still have a problem, because Jekyll expects a blog, with numbered blog posts.

So, we can cheat the system by adding this:

ls | sed "s/\(.*\)/mv \1 2016-02-15-\1/" > move_all.sh
chmod +x move_all.sh 
./move_all.sh

Now, you will have a Jekyll site with all the scala docs (one page per article).

Interested in Scala? I send out weekly, personalized emails with articles and conference talks. Click here to see an example and subscribe.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *