{"id":4628,"date":"2016-07-04T14:43:44","date_gmt":"2016-07-04T14:43:44","guid":{"rendered":"http:\/\/www.garysieling.com\/blog\/?p=4628"},"modified":"2016-07-04T14:43:44","modified_gmt":"2016-07-04T14:43:44","slug":"import-set-json-files-rethinkdb","status":"publish","type":"post","link":"https:\/\/www.garysieling.com\/blog\/import-set-json-files-rethinkdb\/","title":{"rendered":"How to import a set of JSON files into RethinkDB"},"content":{"rendered":"

RethinkDB ships with utilities for doing imports and exports. There are two purposes in this; database backup and restore, and the import of new data.<\/p>\n

Importing new data is probably a more interesting challenge, since you have to get your import process to map to what RethinkDB wants.<\/p>\n

If you haven’t done this before, it requires a python script for RethinkDB:<\/p>\n

\napt-get install -y python-pip\npip install rethinkdb\n<\/pre>\n
Once you do this, you’ll need to define a database in RethinkDB – my example is a series of JSON exports from the Watson API so I’ve called this “Watson”.<\/p>\n
This will let you import a single JSON file:<\/p>\n
\nrethinkdb import -f \\\n  .\/watson\/transcript_s_TextGetEmotion_1608.json \\\n  --table Watson.transcript_s_TextGetEmotion\n<\/pre>\n
When you run this, it creates the table automatically. It seems to treat the file as a row (possibly because mine contains one object). If you import it again, you will need to use “–force” because it’s not sure how to reconcile it with the existing table. The “–force” option will put the new data in as new rows.<\/p>\n
In my case I have a folder that has all the JSON files, named based on the originating ID and the API they are exporting.<\/p>\n
Thus, to import an entire folder, I can do this:<\/p>\n
\ncd watson\n\nfor f in *\ndo\n  table=$(echo $f | sed \"s\/\\(.*\\)_[0-9]\\+.json\/\\1\/g\")\n  table=$(echo $table | sed \"s\/-\/_\/g\")\n\n  rethinkdb import -f $f --table Watson.$table --force\ndone\n\ncd ..\n<\/pre>\n
Note that you can’t use “-” in a RethinkDB table name, so you’ll want to replace those with underscores if you have them in your source file names.<\/p>\n","protected":false},"excerpt":{"rendered":"
How to import a folder of JSON files into RethinkDB<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"footnotes":""},"categories":[4],"tags":[466],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t