excelexcelquery

Excel, Power Query: Add multiple columns from query2 into query1


A continuation of my question here: Excel, query: How to add specific info from query 2 into query 1 with lookup

I have 2 queries, "Overview" and "Data". And I want to add the information from "Data" in the "Overview" query on the correct place. The information of columns ITEM, ITEM 2 and Y/N needs to be added to the correct row, determined by the value in NAME.1 and NAME.2.

These are the two queries. (Slightly different from my last question)

Overview

NAME.1 NAME.2 NUMBER RANDOM
Name11 243324 qwsa
Name22 6747 dsfsdf
Name13 455 yyu
Name14 908098 hfhn
Name25 34 ertew
132 uil
Name17 Name27 64 tgvc
Name18 Name28 876 iorts

Data

ITEM ITEM 2 Y/N NAME.1 NAME.2
123123 AA Y Name25
234324 BB Y Name11
345345 CC N Name17
456456 AA Y Name12 Name22
567567 AA N
678678 DD N Name13 Name23
789789 DD Y Name16
890890 AA N Name14

I want to add information ITEM, ITEM 2 and Y/N from query "Data" to query "Overview" on the correct row. So, the first row of "Overview" will be:

NAME.1 NAME.2 NUMBER RANDOM ITEM ITEM 2 Y/N
Name11 243324 qwsa 234324 BB Y

The code I got from Ron in my previous question is:

let
    Source = Excel.CurrentWorkbook(){[Name="Overview3"]}[Content],
    #"Overview" = Table.TransformColumnTypes(Source,{
        {"NAME.1", type text}, {"NAME.2", type text}, {"NUMBER", Int64.Type}, {"RANDOM", type text}}),
  
//Add index column to be able to go back to original table order
    #"Added Index" = Table.AddIndexColumn(Overview, "Index", 0, 1, Int64.Type),

//Create a Name column that we will expand to all names in a single column
    #"Added Custom" = Table.AddColumn(#"Added Index", "Name", each List.RemoveNulls({[NAME.1]} & {[NAME.2]})),
    #"Expanded Name" = Table.ExpandListColumn(#"Added Custom", "Name"),

//Since Data table could also have Names in both NAME columns, we can unpivot (and delete the unneeded columns)
    #"Unpivot Data" = Table.UnpivotOtherColumns(Table.RemoveColumns(Data3,"Y/N"),{"ITEM"},"Attribute","Name"),
    #"Removed Columns" = Table.RemoveColumns(#"Unpivot Data",{"Attribute"}),

//Now we can merge the tables and return the ITEM
    #"Add Item" = Table.TransformColumns(
        Table.NestedJoin(#"Expanded Name","Name",#"Removed Columns","Name","ITEM",JoinKind.LeftOuter),
        {"ITEM", each try [ITEM]{0} otherwise null, type text}),
    #"Removed Columns1" = Table.RemoveColumns(#"Add Item",{"Name"}),

//Group by Index to return initial table in order
    #"Grouped Rows" = Table.Group(#"Removed Columns1", {"Index"}, {

    //If two names in the same row of Overview have different ITEMs, we will returned a concatenated string
        {"All", each  Table.Distinct(Table.TransformColumns(_, {"ITEM", (c)=>Text.Combine(_[ITEM], ", ")}))  , 
        type table [NAME.1=nullable text, NAME.2=nullable text, NUMBER=nullable number, RANDOM=nullable text, Index=number, ITEM=text]}}),


    #"Removed Columns2" = Table.RemoveColumns(#"Grouped Rows",{"Index"}),
    #"Expanded All" = Table.ExpandTableColumn(#"Removed Columns2", "All", {"NAME.1", "NAME.2", "NUMBER", "RANDOM", "Index", "ITEM"}),
    #"Removed Columns3" = Table.RemoveColumns(#"Expanded All",{"Index"})
in
    #"Removed Columns3"

That worked for that question. However, I can't get it working when I try to add more columns in an efficient/clean way. (add all the columns at once instead of repeating the code 3 times. Repeating the code 3 times doesn't work so far, but I think I made a minor mistake somewhere. However, I think it should be possible to do it directly for 3 columns at once. In my actual file where I want to use it, I need to add multiple columns. So a more efficient method would be much better. Both for usage and for other people when someone wants to use/edit my file)


Additional information for reply on comment Ron Rosenfield:

When I use this code in my actual file, it works till #"Expanded Fields". However, it gives an error at #"Grouped Rows". It says: Expression.SyntaxError: Invalid identifier. And it points to [Sub-supplier]

    #"Expanded Fields" = Table.ExpandRecordColumn(#"Removed Columns1", "Fields", {"Report Number", "Sub-supplier", "text text text text.", "text text text", "text text text text text", "text text"}),
    #"Grouped Rows" = Table.Group(#"Expanded Fields", {"Index"}, {{"all", 
        each Table.TransformColumns(_, {
            {"Report Number", (c)=>Text.Combine(_[Report Number], ", ")},
            {"Sub-supplier",(c)=>Text.Combine(_[Sub-supplier], ", ")}, 
            {"text text text text.",(c)=>Text.Combine(_[text text text text.], ", ")}, 
            {"text text text",(c)=>Text.Combine(_[text text text], ", ")}, 
            {"text text text text text",(c)=>Text.Combine(_[text text text text text], ", ")}, 
            {"text text", (c)=>Text.Combine(_[#"text text"], ", ")}}
        ){0}}}),

Solution

  • Given your data, the following produces the result you show:

    let
        Source = Excel.CurrentWorkbook(){[Name="Overview"]}[Content],
        #"Overview" = Table.TransformColumnTypes(Source,{
            {"NAME.1", type text}, {"NAME.2", type text}, {"NUMBER", Int64.Type}, {"RANDOM", type text}}),
      
    //Add index column to be able to go back to original table order
        #"Added Index" = Table.AddIndexColumn(Overview, "Index", 0, 1, Int64.Type),
    
    //Create a Name column that we will expand to all names in a single column
        #"Added Custom" = Table.AddColumn(#"Added Index", "Name", each List.RemoveNulls({[NAME.1]} & {[NAME.2]}), type {text}),
        #"Expanded Name" = Table.ExpandListColumn(#"Added Custom", "Name"),
    
    //Use same technique on Data table
    //Be certain the ITEM column is of "type text" if not set that way initially
        #"Added Custom2" =Table.AddColumn(Data,"dataName",each {[NAME.1],[NAME.2]}, type {text}),
        #"Expanded dataName" = Table.ExpandListColumn(#"Added Custom2", "dataName"),
    
    //Extract these fields from the Data table
        #"Data Fields" = List.FirstN(Table.ColumnNames(#"Expanded dataName"),3),
    
    //Merge the tables and return all the columns from the DATA table
        #"Merge Overview/Data" = Table.NestedJoin(#"Expanded Name","Name",#"Expanded dataName","dataName","Join",JoinKind.LeftOuter),
    
    //Extract the desired fields
        #"Add Custom3" = Table.AddColumn(#"Merge Overview/Data", "Fields", each if [Name]=null then null
                else if Table.RowCount([Join]) = 0 then null
                else Record.SelectFields([Join]{0}, #"Data Fields"),type record),
        #"Removed Columns" = Table.RemoveColumns(#"Add Custom3",{"Name", "Join"}),
        #"Expanded Fields" = Table.ExpandRecordColumn(#"Removed Columns", "Fields", {"ITEM", "ITEM 2", "Y/N"}),
        #"Grouped Rows" = Table.Group(#"Expanded Fields", {"Index"}, {{"all", 
            each Table.TransformColumns(_, {
                {"ITEM", (c)=>Text.Combine(_[ITEM], ", ")},
                {"ITEM 2",(c)=>Text.Combine(_[ITEM 2], ", ")}, 
                {"Y/N", (c)=>Text.Combine(_[#"Y/N"], ", ")}}
            ){0}}}),
           
        #"Removed Columns1" = Table.RemoveColumns(#"Grouped Rows",{"Index"}),
        #"Expanded all" = Table.ExpandRecordColumn(#"Removed Columns1", "all", {"NAME.1", "NAME.2", "NUMBER", "RANDOM", "Index", "ITEM", "ITEM 2", "Y/N"}),
        #"Removed Columns2" = Table.RemoveColumns(#"Expanded all",{"Index"}),
        #"Changed Type" = Table.TransformColumnTypes(#"Removed Columns2",{
            {"NAME.1", type text}, {"NAME.2", type text}, {"NUMBER", Int64.Type}, 
            {"RANDOM", type text}, {"ITEM", type text}, {"ITEM 2", type text}, 
            {"Y/N", type text}})
    in
        #"Changed Type"
    

    Overview
    enter image description here

    Data
    enter image description here

    Results
    enter image description here