DataWeave: Collapsing Keys

Collapse the keys of an Object to flatten a nested Object.

DataWeave: Collapsing Keys

Intro

Note: I'm experimenting with a shorter format for these posts. Please let me know in the comments if you prefer this or something more long-form like I've done in the past.

I was working out a problem with a colleague to get nested data into a CSV and created a useful utility function on the way to the solution, collapseKeys. This function can be used to take this:

{
  "Person" : {
    "Name"      : "Jerneydude",
    "Profession": "Software Dev"
  },
  "Address" : {
    "Line1" : "1234 Mayne Street",
    "City"  : "Denver",
    "State" : "CO"
  }
}

And flatten it to this:

{
  "PersonName"       : "Jerneydude",
  "PersonProfession" : "Software Dev",
  "AddressLine1"     : "1234 Mayne Street",
  "AddressCity"      : "Denver",
  "AddressState"     : "CO"
}

Let's check it out.

The Code

We already know the function has to concatenate keys for different levels of nesting. It would be nice to also include a string that we can use to join to keys:

collapseKeys(data, "-")

So in the intro example we would get "Person-Name" for the first key instead of "PersonName".

Here's the code:

import fail from dw::Runtime

fun collapseKeys(object, joiner="", keys=[]) =
    object match {
        case is Object -> object mapObject (v, k) -> 
                              collapseKeys(v, joiner, keys + k)
        else           -> if (isEmpty(keys))
                              // If an object was passed, we should have at least one key 
                             fail("collapseKeys: Did not receive Object") 
                          else
                             {(keys joinBy joiner): object}
    }

The function uses recursion and pattern matching to deal with nested Objects. In the case that the input is an Object, the function passes the current value, the original joiner string, and the keys Array with the current key appended to itself. In the case that the input is anything besides an Object, the function does a test. If the keys Array is empty, it crashes, complaining that it did not receive an Object. This is because if the function was passed an object, the keys parameter should contain at least one item by the time it reaches this branch. If the keys Array is not empty it returns an Object where the key is all the Strings in the keys array joined by the joiner string, and the value was whatever was passed as the first argument to the function.


Thanks to Bera Aksoy for helping me "see" this function.