[SOLVED] Iterate through all tr in html using selenium library in python

Iterate through all tr in html using selenium library in python

<table class="flush-table sticky-table" id="myTable">
    <thead class="sticky-thead">
     <tr>
      <th>A Number</th>
      <th>B</th>
      <th>Pkts</th>
      <th>ID1</th>
      <th>ID2</th>
      <th>Type1</th>
      <th>Type2</th>
      <th>C</th>
      <th>D</th>
      <th>E</th>
    </tr>
    <tr>
      <td>1</td>
      <td>0 days, 20h:55m:18s</td>
      <td>0</td>
      <td>5</td>
      <td>7</td>
      <td>cast</td>
      <td>Mul</td>
      <td>5.0</td>
      <td>5.0</td>
      <td>160.0</td>
    </tr>
    <tr>
      <td>2</td>
      <td>0 days, 20h:55m:18s</td>
      <td>0</td>
      <td>6</td>
      <td>9</td>
      <td>Un</td>
      <td>Mu</td>
      <td>5.0</td>
      <td>5.0</td>
      <td>160.0</td>
    </tr>
   </thead>
  </table>

table = driver.find_element(By.XPATH, "//thead[@class = 'sticky-thead']")
print(table.text)

for row in table.find_elements(By.XPATH, ".//tr"):
    print(row.text)
    print([td.text for td in row.find_elements(By.XPATH, ".//td")])

If i run this script, the elements "A B Pkts ID1 ID2 Type1 Type2 C D E"(i.e. Table header) is captured in list. However the i could not capture table data. Please help me to understand how can i capture the table data as well.

Solution

XPATH is an absolute path and is being used only once. You need to first find all th tags and then all remaining tr tags.

cols = table.find_element(By.TAG_NAME, "tr")
print([th.text for th in cols.find_elements(By.TAG_NAME, "th")])

for row in table.find_elements(By.TAG_NAME, "tr")[1:]:  # Skip first
    print([td.text for td in row.find_elements(By.TAG_NAME, "td")])

Which results to:

['A Number', 'B', 'Pkts', 'ID1', 'ID2', 'Type1', 'Type2', 'C', 'D', 'E']
['1', '0 days, 20h:55m:18s', '0', '5', '7', 'cast', 'Mul', '5.0', '5.0', '160.0']
['2', '0 days, 20h:55m:18s', '0', '6', '9', 'Un', 'Mu', '5.0', '5.0', '160.0']