pythonfor-loopgenerator-expression

for x, y in function(): - Explain the python for loop structure


I want to know what the for loop does on es_questions and the explanation for line of code below es_question.

Anyone please explain the line of code:

    es_question = (q.as_elasticsearch_dict() for q in questions)
    for ok, result in streaming_bulk(
        get_client(),
        es_question,
        index = settings.ES_INDEX,
        raise_on_error = False
    ):
        if not ok:
            all_ok = False
            action, result = result.popitem()
            logger.error(FAILED_TO_LOAD_ERROR.format(result['_id'], result))

Solution

  • Part 1 - Generator Expression

    es_question = (q.as_elasticsearch_dict() for q in questions)
    

    Is not a true for loop and is what is called a list comprehension. In this case, because it is in parentheses and not in brackets, it is actually a generator expression.

    Resources for these can be found at: List comprehension and generator expression.

    Part 2 - For Loop

    Your question is on the line of code which is:

        for ok, result in streaming_bulk(
            get_client(),
            es_question,
            index = settings.ES_INDEX,
            raise_on_error = False
        ):
    

    Let's look at a simpler example first:

    my_list=["a","b","c","d"]
    for x in my_list:
        print(x)
    

    This will loop over the values in my_list assigning those values to the variable x. Now let's look at a slightly more complicated example.

    my_list=[("a",1),("b",2),("c",3),("d",4)]
    for x,y in my_list:
        print(x)
        print(y)
    

    In this case, the for loop is still looping over each of the 4 items in my_list, but it is also doing a tuple unpacking, so the variable x gets the value of the first item in the tuple (the letter) and the variable y gets the value of the second item in the tuple (the number).

    Finally, one other thing to look at:

    streaming_bulk(
            get_client(),
            es_question,
            index = settings.ES_INDEX,
            raise_on_error = False)
    

    This is simply a function call (Spread out over multiple lines). This function either returns a list/tuple of lists/tuples (which have two values each) similar to my last example, or this function is a generator function which yields two values each time through the loop.