Javascript: JSON Data Manipulation

Intro

If you are looking for methods to transform, merge and format JSON data this is the post for you. Many of the methods here make heavy use of the vastly underappreciated Map object.

📘 MDN: Map

So take a moment to get acquainted with that if you aren't already before continuing.

All of these methods use plain javascript with no libraries.

Formatting

Build New Array Using Accessor

One common JSON scenario is that you pull a dataset that contains a set of ID's, which you then need to use to fetch another set of data. When that happens it is useful to be able to create an array of those ID's to include in the next request.

// returns array of unique values matching accessor
export const buildNewArrayUsingAccessor = (inputArray, accessor) => {
  let newArray = [];
  // push only keys matching accessor into new array
  inputArray.forEach((d) => {
    newArray.push(accessor(d));
  });

  // remove duplicates
  let unique = [...new Set(newArray)];

  return unique;
};

The accessor input can be a bit confusing so here is a sample call of this function:

const adGroupsIdArray = buildNewArrayUsingAccessor(
  adgroupsArray,
  (d) => d.adGroupId
);

Add Fixed Key Value Pair To Each Object

Adds a fixed Key and Value to each object in the array. IE status: "archived"

// adds a specific key:value pair to each object in the array
export const addFixedKeyValueToEachObject = (array, key, value) => {
  const newArray = array.map((object) => {
    return { ...object, [key]: value };
  });

  return newArray;
};

Delete All Instances of Key

Whatever key is input will be removed from all objects in the array.

export const deleteAllInstancesOfKey = (arr, key) => {
  let newArray = [];
  arr.forEach((d) => {
    const { [key]: foo, ...newObject } = d;
    newArray.push(newObject);
  });

  return newArray;
};

Rename Key

Very useful to format datasets before merging them. Typically will be used to make sure merge keys are matching. Sometimes to prevent items from having identical keys in different sets. Generally useful.

export const renameKey = (arr, oldKey, newKey) => {
  let newArray = [];

  arr.forEach((obj) => {
    let newObj = {};
    const keys = Object.keys(obj);
    keys.forEach((key) => {
      if (key === oldKey) {
        Object.assign(newObj, { [newKey]: obj[oldKey] });
      } else {
        Object.assign(newObj, { [key]: obj[key] });
      }
    });
    newArray.push(newObj);
  });

  return newArray;
};

Change Key Value Format

Sometimes you get a set of ID's coming in as a string when they should be integers. Fairly straightforward.

// convert id's to numbers for merge
let athenaSearchTermsDataIntParsed = athenaSearchTermsData.map((term) => ({
  ...term,
  adGroupId: parseInt(term.adGroupId),
}));

Merging

Merge Parent Array Into Child Array

Say that we have two arrays full of objects, and we want to merge the objects in each array with each other based on a matching key:value pair and output a single new array. Here is a great way to get this done.

First let us briefly define what we mean by parent array and child array.

When looking to merge two arrays the parent array will typically be the shorter array (fewer items) and the child array will be longer (more items). Each object in both arrays will have a common key such as parentId which is what relates the two objects to each other.

The key to accomplishing this is that we need to have a common key in each object, which we will refer to as the mergeKey.

In this scenario we want the output array to have the same length as the input child array. We just want to output each of the original child objects, with the additional data from the object in the parent array that matches the child objects merge key value.

It is also important in this scenario that each object in the Parent Array has a unique mergeKey value.

The objects in the child array do not need to have any unique ID's. Extremely useful, as they often do not. We accomplish this by using the index of each object in the child array as a temporary unique key in the Map.

export const mergeParentArrayIntoChildArray = (
  childArray,
  parentArray,
  mergeKey
) => {
  const childMap = new Map();
  const parentMap = new Map();
  // place all original children in map using index as unique key
  childArray.forEach((item) => childMap.set(childArray.indexOf(item), item));
  // place all parent items in map by merge key
  parentArray.forEach((item) => parentMap.set(item[mergeKey], item));
  // for each item in child map, add the item from parent map that matches merge key
  childArray.forEach((item) => {
    childMap.set(childArray.indexOf(item), {
      ...parentMap.get(item[mergeKey]),
      ...item,
    });
  });

  // convert back to array
  const merged = Array.from(childMap.values());

  return merged;
};

Your instinct may be to use loops to accomplish this task, but taking advantage of Maps is much more efficient. This methods will merge arrays full of thousands of objects very quickly.

Comments

Recent Work

Basalt

basalt.software

Free desktop AI Chat client, designed for developers and businesses. Unlocks advanced model settings only available in the API. Includes quality of life features like custom syntax highlighting.

Learn More

BidBear

bidbear.io

Bidbear is a report automation tool. It downloads Amazon Seller and Advertising reports, daily, to a private database. It then merges and formats the data into beautiful, on demand, exportable performance reports.

Learn More