Some handy sed commands for formatting a large concatenated json file

More on Salt later in a separate post, but I am using Salt to pull salt grains from a number of servers so that I can extrapolate out the gain items (in my case, serial number, operating system and so forth).

I ran into an issue where Salt concatenates the json file as a bunch of continguous blocks of json.

When you try to load this into a json parser, it fails.

So in researching how to split this file, I ran into one particularly clever person on the web who said, "don't SPLIT the file! just make it an array of json elements".

I wonder if this guy knew how much wisdom he expelled.

So - I needed some sed to "prepare" this file.

And here it is:

#!/bin/bash

# This script will take the huge json file from Salt and make it something we can parse by
# making each json block an array element.

# Step 1 - add commas between blocks
sed -i 's/^}/},/g' saltgraininfo.json

# Step 2 - remove that last comma out which I could not figure out how to do in Step 1 in same sed command.
sed -i '$s/,//g' saltgraininfo.json

# Step 3 - put a bracket in beginning of the file
sed -i '1s/^/[\n/' saltgraininfo.json

# Step 4 - put a bracket at the end of the file
sed -i '$s/}/}\n]/g' saltgraininfo.json

After I did this, I used Python and did a json.load on the file, and voila'! It loads!

On to the parsing now....