Critique my JSON-optimizing Python script

    digest = defaultdict(lambda: defaultdict(set))

It's pretty rare that we have a good reason for such a nested data structure in python.

    for element in data:
        if not isinstance(element, dict) or len(element) != 2:
            break
        for key, value in element.items():
            digest[key][type(value)].add(repr(value))

Why are we adding the repr of the value? I assume you do it to avoid taking the hash of unhashable objects. But what if somebody puts some strange object in your list that has a repr which always returns the same thing. Besides you are only interested in strings.

    else:
        if len(digest) == 2:

This is an unintuive way to check for your requirement. Your requirement is actually that all elements have the same keys. Check for that instead.

            pool = {k for k in digest if len(digest[k][str]) == len(data)}

Your actually checking to see if that key was always a string. Perhaps you should check that more directly.

            if pool:
                if len(pool) > 1:
                    meta = input("Which key (%s)? " % "|".join(pool))

Its deeply suspicious if you need to ask the user for instructions.

                else:
                    meta = next(iter(pool))
                infra = (digest.keys() - {meta}).pop()
                return {d[meta]: optimize(d[infra]) for d in data}
    return [optimize(x) for x in data]    

Here is my reworking of it.

def optimize(data):
    """
    Dispatcher, figure out which algorithm to use to compress the json
    """
    if isinstance(data, list):
        if len(data) == 1:
            return optimize(data[0])
        else:
            keys = determine_key(data)
            if keys is None: # unable to determine tkeys
                return [optimize(element) for element in data]
            else:
                return optimize_list(data, keys)
    elif isinstance(data, dict):
        if len(data) == 1:
            return optimize(next(iter(data.values())))
        else:
            return optimize_dict(data)
    else:
        return data

def optimize_dict(data):
    """
    Optimize a dict, just optimizes each value
    """
    return {key : optimize(value) for key, value in data.items()}

def can_be_key(data, key):
    """
    return true if key could be a key
    """
    if not all( isinstance(element[key], str) for element in data):
        return False
    return len(set(element[key] for element in data)) == len(data)

def determine_key(data):
    """
    Determine the key, value to compress the list of dicts
    or None if no such key exists
    """
    for element in data:
        if not isinstance(element, dict):
            return None
        if len(element) != 2:
            return None
        if element.keys() != data[0].keys():
            return None

    key1, key2 = data[0].keys()
    key1_possible = can_be_key(data, key1)
    key2_possible = can_be_key(data, key2)

    if key1_possible and key2_possible:
        meta = input("Which key (%s|%s)? " % (key1, key2))
        if meta == key1:
            return key1, key2
        else:
            return key2, key1
    elif key1_possible:
        return key1, key2
    elif key2_possible:
        return key2, key1
    else:
        return None

def optimize_list(data, keys):
    key_key, key_value = keys
    return {element[key_key]: optimize(element[key_value]) for element in data}

But... I must question the wisdom of the entire approach. The function is actually rather useless. Suppose that you use it on web service that returns a json object. It gives you

{"results" : [
     {"name" : "foo", "age": 42}
     {"name" : "bar", "age": 36}
}

Which optimize to

{"foo": 42, "age": 36}

So you write:

for name, age in optimize(the_json).items():
    print(name, age)

But then in another request, the server returns

{"results" : [
     {"name" : "foo", "age": 42}
}

which optimizes to:

42

So when you run:

for name, age in optimize(the_json).items():
    print(name, age)

You get an error.

Or the server adds extra data to the json:

{
 "salinity" : "high",
 "results" : [
     {"name" : "foo", "age": 42, "status" : "sad"}
     {"name" : "bar", "age": 36, "status" : "happy"}
}

Which should be backwards compatible using json, but will break your code or anything doing the same job.

This is not to say that can't or shouldn't simplify the json. But don't try to do it by analyzing the data like this. You should do it by some external record of the schema.

class Fetch(object):
    """
    Fetch one element from a dictionary
    """
    def __init__(self, name, inner):
        self._name = name
        self._inner = inner

    def __call__(self, data):
        return self._inner(data[self._name])

class KeyValue(object):
    """
    Given a list of dicts, convert to a dictionary
    with each key being taken from the `key` on each element
    and each value being taken from the `value`
    """
    def __init__(self, key, value, inner):
        self._key = key
        self._value = value
        self._inner = inner

    def __call__(self, data):
        return {element[self._key] : self._inner(element[self._value]) for element in data}

class Literal(object):
    def __call__(self, data):
        return data

schema = Fetch("recipes",
    KeyValue("name", "ingredients",
        KeyValue("ingredient", "quantity", Literal())
    )
)

That's actually simpler then what you were doing, and its much more robust. Its also very flexible as you can write code to perform just about any transformation on the data.