Python selenium invalid Xpath expression

So I am trying to use selenium Chrome webdriver to select an element based on the Xpath.

browser.find_element('xpath', "div[dyc-date='19990301']").click()

I am getting the following error:

Blockquote Failed to execute 'evaluate' on 'Document': The string '[dyc-date='19990301']' is not a valid XPath expression."}

It seems like it needs me to specify the starting node to be considered a valid expression? i.e.)

browser.find_element('xpath', "//*div[dyc-date='19990301']").click()

This no longer gives me an error, but now it can't find the expression.

This seems odd to me since (based on my understanding) //* should search for any node that matches from the current node, no matter where they are in the document.

So //* will search all nodes below your current node no matter where they are?

My goal is to fix my Xpath expression to find/click on the element in question. Thanks!

The error message you've reported doesn't match the XPath you've used. I think you have tried different XPath expressions, and in your question you've mixed up one of those XPath expressions and the error message from a different XPath expression.

This is not a valid XPath expression:

//*div[dyc-date='19990301']

And neither is this:

[dyc-date='19990301']

This, however, is a valid expression

//div[dyc-date='19990301']

NB the leading // is shorthand which expands to /descendant-or-self::node()/, so the expression means:

/descendant-or-self::node()/div[dyc-date='19990301']

So the expression returns any div element which has a dyc-date element with the string value "19990301" as one of its children), where that div is a child of the document node (/); or any of the descendant nodes of the document node.

So despite being a valid expression, it's not correct for your purpose because it is looking for an element named dyc-date rather than an attribute with that name.

Try this:

//div[@dyc-date='19990301']

By the way, regarding your comment:

This seems odd to me since (based on my understanding) //* should search for any node that matches from the current node, no matter where they are in the document.

So //* will search all nodes below your current node no matter where they are?

That's not actually quite correct. That XPath //* is equivalent to /descendant-or-self::node()/*, which is equivalent to /descendant::*; i.e. it will search for all elements which are descendants of the document node /, i.e. all elements anywhere in the document, no matter what the "current node" happens to be.

What you call the "current node" is more properly known as the "context node", and is represented by the expression . in XPath.

To search for all descendant elements of the context node, you could use .//* or ./descendant::*.

Of course if your context node is the document node /, then .//* and //* will indeed return the same result. But if you have a variable in your webdriver client code which is a reference to an element in the document, and you call find_element with that as the context node, then .//* will return all descendant elements of that node, while //* will return all the elements in the document, without regard to the context node, because the XPath begins with a /, referring to the document root. That makes it an "absolute" XPath, rather than relative to the context node.